JP5919752B2 - Minutes creation system, minutes creation device, minutes creation program, minutes creation terminal, and minutes creation terminal program - Google Patents

Description

  The present invention relates to a minutes creation system, a minutes creation device, a minutes creation program, a minutes creation terminal, and a minutes creation terminal program.

  In order to automate the creation of meeting minutes, the content of speech uttered by participants in the meeting is converted into text information, the speaker is identified from the voice, and the minutes are created by associating the text information with the speaker. Techniques that can be known are known.

  In Patent Document 1, a microphone ID for identifying a microphone is associated with input information of an input person who inputs sound using the microphone, and the voice information input from the microphone and the microphone ID of the microphone A technique for creating minutes data using associated input person information is described.

  Patent Document 2 includes a participant information acquisition unit that acquires user identification information related to a conference participant, and a unit that specifies a speaker whose content is spoken using the acquired user identification information. The technology to create a minutes by associating a speaker with a speaker is described.

  However, although it was possible to identify the speaker and to create a minutes in which the text information and the speaker were associated, there was a problem that it was not known to which material each utterance was related. .

  The present invention has been made in view of the above problems, and an object of the present invention is to make it possible to understand which material the utterance content recorded in the minutes of the meeting relates to.

  In order to solve the above-described problems and achieve the object, the minutes creation system according to the present invention includes a minutes creation device for creating minutes by converting the utterance contents of users participating in the meeting into character information, and electronic data. A minutes creation system having an information terminal for displaying a certain document, wherein the minutes creation device requests a document information from the information terminal used by the user who uttered when the user utters Means for creating a minutes in which the contents of the user's utterance and the information of the material are associated with each other, and the information terminal accepts a request from the material information requesting means. Receiving means, material information acquiring means for acquiring information of the material being displayed by the terminal when the material information request receiving means receives the request, and It has a document information transmission means for transmitting broadcast in the proceedings preparation device.

  In addition, the minutes creation system according to the present invention is mounted on a minutes creation device for creating minutes by converting the utterance contents of users participating in the meeting into text information, and an information terminal for displaying materials as electronic data. A minutes creation system comprising: a material information requesting means for requesting material information from an information terminal used by a user who uttered when the user speaks; A minutes creation means for creating a minutes that associates the utterance content with the information on the material, and the program receives a request from the material information requesting means to the computer, A material information acquisition step for acquiring information of the material being displayed by the terminal when the request is received in the material information request reception step; And article information transmission step of transmitting the information of the article in the proceedings preparation apparatus, to the execution.

  The minutes creation device according to the present invention is a minutes creation device that creates a minutes by converting the utterance content of a user who participates in a meeting into character information, and when the user speaks, the user who spoke A material information requesting means for requesting information on the material to the information terminal used by the information terminal, and a minutes creating means for creating a minutes associating the utterance content of the user with the information on the material received from the information terminal, Have.

  In addition, the minutes creation program in the present invention is a minutes creation program installed in a minutes creation device for creating minutes by converting the utterance content of users participating in the meeting into character information, When a user utters, the material information request stage for requesting material information to the information terminal used by the user who uttered, the content of the user's utterance, and the information of the material received from the information terminal The minutes creation stage for creating the minutes is executed.

  Further, the minutes creation terminal in the present invention is capable of displaying materials that are electronic data, and communicates with the minutes creation device that creates the minutes by converting the utterance content of the user participating in the meeting into character information, The user's minutes creation terminal, and when the user speaks, the material information request acceptance means for accepting a request for information on the material from the minutes creation apparatus, and the material information request acceptance means accepts the request. In this case, material information acquisition means for acquiring information on the material being displayed by the terminal, and the minutes creation device transmits the information to create the minutes in association with the user's utterance content to the minutes creation device. Material information transmitting means.

  Further, the minutes creation terminal program according to the present invention is capable of displaying materials that are electronic data, and communicates with the minutes creation device that creates the minutes by converting the utterance content of the user participating in the meeting into character information. A minutes creation terminal program installed in the user's minutes creation terminal, and when the user speaks to a computer, a material information request acceptance stage for receiving a request for information on materials from the minutes creation device; A material information acquisition step of acquiring information of the material being displayed by the terminal when the request is received in the material information request reception step, and a minutes associated with the utterance content of the user by the minutes generation device A material information transmitting step of transmitting the information for generating the minutes to the minutes generating device.

  According to the present invention, it is possible to know which material the utterance content recorded in the minutes of the meeting relates to.

1 is a schematic diagram of a minutes creation system in an embodiment of the present invention. The hardware block diagram of the minutes production apparatus in one Embodiment of this invention. The hardware block diagram of the information terminal in one Embodiment of this invention. The functional block diagram of the minutes creation apparatus in one Embodiment of this invention. The functional block diagram of the information terminal in one Embodiment of this invention. The sequence diagram showing the whole flow of the minutes creation processing. The sequence diagram showing the processing flow before the start of minutes preparation. The figure showing the example of the table which associates a user and an information terminal. The sequence diagram showing the processing flow of minutes preparation. The figure showing the example of the table which links | relates and stores the character information and document information of minutes. The figure showing the example of the table which links | relates and stores the character information of a minutes, and a user. The figure showing the example of an output of the minutes in one embodiment of the present invention. The figure showing the example of the table which correlates document information and format information. The figure showing the example of the table which associates affiliation department information and format information. The figure showing the example of the table which stores speaker information. The schematic diagram showing the example of an output of the minutes colored for every material in one embodiment of the present invention. The schematic diagram showing the example of an output of the minutes colored for every page of the material in one embodiment of the present invention. The schematic diagram showing the example of output of the minutes colored for every department to which a user belongs in one embodiment of the present invention. The figure showing the output example of the minutes which added the format information and user affiliation information in one Embodiment of this invention. The figure showing the output example of the minutes which added the format information in one Embodiment of this invention, and was sorted by the user's affiliation information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

1. Overview 2. 2. Hardware configuration Function 4. 4. Outline of minutes creation process User information registration processing 6. Details of the minutes creation process Minutes output example
7.1 Basic output example 7.2 Output example using format information and affiliation information 7.3 Output example by sorting

(1. Overview)
FIG. 1 is a schematic diagram of a minutes creation system 1 according to an embodiment of the present invention. The minutes creation system 1 includes a minutes creation apparatus 100 that creates the contents of speech of a user who is a participant of a meeting in association with the material, minutes data 102 that records the minutes in which the content of speech is associated with the material, and the user's A file server 110 for sharing materials related to the utterance content with other users, an information terminal 130 for users participating in the conference to display the materials 132, and other users participating in the conference displaying the materials 132 respectively. Information terminals 140 and 150, a voice acquisition device 160 for collecting voice uttered by the user, a projector 170 and minutes recording device 100 for outputting the material 132 to the outside, and the information terminals 130, 140 and 150 and voice A network 120 connected to the acquisition device 160, and the utterance content described in the minutes of the conference is Of identifying whether been made with respect to portion, it is possible to create the minutes that associates the spoken content and documentation. Here, the material 132 is a material in an electronic data format that can be displayed on the information terminals 130, 140, and 150. The materials 132 include conference materials for presentation that are displayed by the conference presenter using the information terminal 130, and materials that are displayed by the other conference participants using their information terminals 140 and 150, and the like.

  The minutes creation device 100 obtains the identification information of users who participate in the conference and the identification information of the information terminals 130, 140, and 150 used by each user before the creation of the minutes, that is, before the start of the conference. Register in association. The minutes creation device 100 acquires the utterance contents converted into character information from the voice acquired through the voice acquisition device 160 during the conference, and further identifies the speaker.

  Here, it is assumed that the user of the information terminal 130 displays the material 132 on the information terminal 130 and speaks. The minutes creation device 100 identifies the user from the spoken voice, and requests the information 132 related to the information 132 displayed on the information terminal 130 when the utterance is made to the associated information terminal 130. . The information on the material 132 includes, for example, the file name and title of the material 132. Further, the information of the material 132 includes detailed information such as the number of pages and the line number indicating the position in the material. Then, the received information on the material 132 and the character information on the utterance content are associated with each other and stored as the minutes data 102. After the meeting is over, the minutes can be created and output in response to a request from the user. At this time, when the material 132 is shared with other users, information necessary for uploading the material 132 to the file server 110 is transmitted to the information terminal 130.

  The minutes data 102 is data in which the utterance contents converted into character information, the information of the material 132 corresponding to the utterance contents, the information of the related utterers, and the like are recorded. For example, the minutes data 102 is configured as a database accessible from the minutes creation apparatus 100.

  The file server 110 is a server used when sharing the material 132 referred to in the minutes created by the minutes creation device 100.

  The network 120 connects the minutes creation apparatus 100, the file server 110, the information terminals 130, 140, 150, and the voice acquisition apparatus 160, and is an intranet in a company, for example.

  The information terminals 130, 140, and 150 are information terminals used by users who participate in the conference, respectively, and are generally PCs. Each user proceeds with the conference by referring to the materials displayed on the information terminals 130, 140, and 150, respectively. For example, when the user of the information terminal 130 utters while displaying the material 132 on the information terminal 130 during the conference, the information terminal 130 displays the material displayed when the utterance is made from the minutes creation device 100. A request for information 132 is accepted. The information terminal that has received the inquiry identifies the application displaying the material 132, acquires the information of the material 132 including the file name, title, number of pages, and line number of the material 132, and records the minutes 100. Send to.

  The material 132 is a material in an electronic data format that can be displayed on the information terminals 130, 140, and 150, and is generally a presentation file.

  The voice acquisition device 160 is a device that collects voices of users who participate in the conference, and is generally a microphone. The microphone is connected to the network 120, converts the user's voice into an electronic voice signal, and transmits it to the minutes creation apparatus 100.

  The projector 170 is connected to the information terminal 130 and can display the material 132.

  As described above, the minutes creation system 1 according to the embodiment of the present invention specifies which part of the material the utterance content described in the meeting minutes is associated with, and associates the utterance content with the material. A record can be created.

  The configuration of the minutes creation system 1 is arbitrary, and the minutes creation apparatus 100 may be incorporated in the projector 170, for example, or may be incorporated in any of the information terminals 130, 140, and 150.

  In addition, the minutes creation apparatus 100 can perform the same processing for the utterances of the users of the information terminals 140 and 150. Therefore, even when a plurality of participants display different materials on their respective information terminals and hold a meeting, the same minutes can be created.

  The information terminals 130, 140, and 150 can include any terminal capable of reproducing electronic data format materials, including a smartphone and a tablet PC, in addition to a desktop PC and a laptop PC. Thereby, it can respond to the format of various meetings.

  The material 132 may be any electronic data that can be used in a meeting, such as a document format file such as a text file, an image file, and a moving image file. Thereby, even when various types of materials 132 are used in the conference, the location of the materials corresponding to the minutes can be specified.

  The voice acquisition device 160 may be replaced by a microphone provided in each information terminal 130, 140, 150. In this case, voice is transmitted to the minutes creation apparatus 100 through the microphones of the information terminals 130, 140, and 150. As a result, since the speaker and the information terminals 130, 140, and 150 are associated one-to-one, the speaker can be easily identified.

(2. Hardware configuration)
FIG. 2A shows an example of the hardware configuration of the minutes creation apparatus 100 according to an embodiment of the present invention. The minutes creation device 100 has a CPU 200, ROM 202, RAM 204, HDD / SSD 206, communication control unit 208, communication device 210, and bus 212, and acquires utterance contents converted into character information from the voices of users participating in the conference. Further, the user who is the speaker is specified, the information of the material 132 corresponding to the utterance content is acquired from the information terminal 130, and the minutes in which the utterance content is associated with the information such as the number of pages of the material 132 are created. Output.

  The CPU 200 controls the operation of the minutes creation apparatus 100 and the minutes creation program in one embodiment of the present invention. The ROM 202 is for storing a minutes creation program executed by the CPU 200, programs and data necessary for the operation of the apparatus, and the like. The RAM 204 is for configuring a work area of the CPU 200 and the like. A HDD / SSD (Solid State Drive) 206 is used to store a minutes creation program and programs and data necessary for the operation of the apparatus. The communication control unit 208 executes communication control processing for performing communication with the information terminals 130, 140, 150 and the voice acquisition device 160 via the network 120 via the communication device 210. The bus 212 is used for mutually connecting the above-described devices constituting the minutes creation device 100 to exchange data.

  With the above configuration, the minutes generating apparatus 100 according to an embodiment of the present invention acquires the utterance content converted into character information from the voice of the user participating in the conference, further specifies the user who is the speaker, and the utterance content. The information of the corresponding material 132 can be acquired from the information terminal 130, and the minutes in which the utterance contents and the material 132 are associated can be created and output.

  The minutes creation device 100 may include a display control unit 214 and a screen display device 216 (not shown), and may display a screen according to the processing content of the minutes creation device 100.

  The minutes creation apparatus 100 may include an input control unit 218 and an input device 220 (not shown), and may accept input from the user to the minutes creation apparatus 100.

  The HDD / SSD 206 may be any storage device including a tape drive.

  FIG. 2B shows an example of the hardware configuration of the information terminal 140 according to an embodiment of the present invention. The information terminal 140 includes a CPU 250, a ROM 252, a RAM 254, an HDD / SSD 256, a communication control unit 258, a communication device 260, a display control unit 262, a screen display device 264, an input control unit 266, an input device 268, and a bus 270. The material 132 is displayed on the screen, and the information of the material 132 is transmitted in response to a request from the minutes creation apparatus 100.

  The CPU 250 controls the operation of the information terminal 140 and a program executed on the information terminal 140 according to an embodiment of the present invention, and the ROM 252 stores a program executed by the CPU 250, data, and the like. Yes, the RAM 254 is for configuring a work area of the CPU 250 and the like. An HDD / SSD (Solid State Drive) 256 is used to store a minutes creation program and programs and data necessary for the operation of the terminal. The communication control unit 258 performs a communication control process for performing communication with the minutes creation apparatus 100 via the network 120 via the communication apparatus 260. The display control unit 262 controls the content displayed on the screen display device 264 according to the processing content in the information terminal 140. The input control unit 266 processes a signal from the input device 268 that receives input from the user. Examples of the input device 268 include a keyboard, a mouse, a touch panel, buttons, and a dial. The bus 270 is for connecting the above devices constituting the information terminal 140 to each other and exchanging data.

  With the above-described configuration, the information terminal 140 according to an embodiment of the present invention can display the material 132 on the screen during the meeting, and transmit information on the material 132 in response to a request from the minutes creation apparatus 100.

  Although an example of the hardware configuration of the information terminal 140 has been described above, the information terminal 130 connected to the projector 170 includes an external output control unit (not shown) and controls output to the projector 170.

  The information terminal 140 may include a voice input control unit and a voice input device (not shown), and may be configured to process a user's voice input through the voice input device as a voice signal by the voice input control unit. Thereby, the acquired audio signal is transmitted to the minutes creation apparatus 100 through the communication apparatus 260. As a result, the uttered user and the information terminal 140 are associated with each other, and the user can be easily specified in the minutes creation apparatus 100.

  The HDD / SSD 206 may be any storage device including a flash memory.

(3. Function)
FIG. 3A is a functional block diagram of the minutes creation apparatus 100 according to an embodiment of the present invention. In the following description, it is assumed that the user of the information terminal 130 speaks about the material 132 displayed on the information terminal 130 unless otherwise specified.

  The minutes creation apparatus 100 includes a communication unit 300 for communicating with the information terminals 130, 140, 150, a user information registration unit 302 for registering the user participating in the conference and the information terminals 130, 140, 150 used by the user, The user information storage means 303 that stores the identification information of the user participating in the conference and the identification information of the information terminals 130, 140, and 150 used by the user and stores it as user information 304, and the voice of the user uttered in the conference Voice acquisition means 306 to acquire, correlation means 308 for associating the acquired voice with the information of the material 132, voice conversion means 310 for converting the acquired voice into character information, utterance for identifying a speaker from the acquired voice feature information Requester 312 and the information terminal 130 of the speaker are requested for information on the material 132 corresponding to the utterance content. Material information requesting means 314, material information receiving means 316 for receiving information on the material 132 from the information terminal 130, minutes holding means 318 for holding the associated utterance content and information on the material 132 as the minutes data 102, minutes Minutes creation means 322 for creating data 102, upload means 324 for transmitting access information necessary for uploading material 132 to file server 110, and format information addition means 326 for adding format information 328 to the created minutes. And a sort means 332 that sorts the affiliation information 330 that is information related to the user's affiliation and the minutes to be created, and creates minutes in which the utterance contents during the meeting are associated with the information of the corresponding material 132. Output.

  The communication unit 300 is used to communicate information with the file server 110, the information terminals 130, 140, 150, and the voice acquisition device 160.

  The user information registration unit 302 registers the user who participates in the conference and the information terminals 130, 140, and 150 used by each user before the creation of the minutes, that is, before the start of the conference. The user information registration unit 302 acquires user identification information and identification information of the information terminals 130, 140, and 150 from the information terminals 130, 140, and 150, and passes these identification information to the user information storage unit 303. The user identification information is, for example, a user identifier that can uniquely identify the user. The identification information of the information terminal is, for example, an IP address that is uniquely assigned to the information terminal.

  The user information storage unit 303 associates the user identification information acquired through the user information registration unit 302 with the identification information of the information terminals 130, 140, and 150 and stores them as user information 304.

  The user information 304 is data that associates identification information of users who participate in the conference with identification information of the information terminals 130, 140, and 150 used by each user. The user information 304 is stored in, for example, a database connected to the user information storage unit 303 and stored with identification information such as a conference name for each conference. FIG. 6 is an example of a table that stores user information 304.

  The voice acquisition unit 306 is used to acquire the voice uttered by the user in the conference through the voice acquisition device 160. The voice acquisition unit 306 passes the voice signal acquired through the voice acquisition device 160 to the association unit 308.

  The associating unit 308 uses the audio signal acquired by the audio acquiring unit 306 as audio information divided into predetermined units, adds an audio identification number for uniquely identifying the audio information, Association with the information of the document 132 is performed. The predetermined unit in which the audio signal is divided may be, for example, a unit of the audio signal divided in a silent section having a predetermined length. The voice information divided into predetermined units is transferred to the voice conversion means 310 for conversion into character information. The voice information divided into predetermined units is passed to the speaker specifying means 312 in order to specify the user who uttered the voice. When the uttered user is specified, information on the material 132 corresponding to the sound information is acquired from the information terminal of the user through the material information requesting unit 314 using the voice identification number as a key. At this time, the voice identification number is separately shared with the information terminals 130, 140, and 150, and for example, the time when the voice is emitted can be specified. Then, the voice information and the information of the material 132 are associated with the voice identification number added first, and stored in the minutes holding means 318 as the minutes data 102.

  The voice conversion unit 310 converts the voice information passed from the association unit 308 into character information and returns it to the association unit 308. A known speech recognition technique can be used to convert the speech information into character information.

  The speaker specifying unit 312 extracts feature information related to the audio signal from the audio information passed from the associating unit 308, and determines which of the users registered through the user information registration unit 302 is the audio. Identify. Known speaker identification techniques can be used to identify the speaker from the audio signal. Here, the user's voice feature information (template) necessary for specifying the user is stored in advance in, for example, the minutes creating apparatus 100. The identified speaker information is passed to the associating means 308.

  The material information requesting unit 314 inquires of the user information terminal specified by the associating unit 308 about information on the material 132 corresponding to the voice information. The information of the material 132 includes, for example, detailed information such as the number of pages and the number of lines in addition to the file name and title of the material 132.

  The material information receiving unit 316 receives information on the material 132 requested by the material information requesting unit 314 from the user information terminal. The received information is transferred to the associating means 308 and stored as minutes together with the corresponding voice information and voice identification number.

  The minutes holding unit 318 holds the voice information and the information of the material 132 associated with each other by the associating unit 308 as the minutes data 102. The minutes data 102 is registered in, for example, a database connected to the minutes holding means 318 and stored with identification information such as a meeting name for each meeting.

  The minutes creation means 322 creates and outputs minutes in a format that can be browsed from the minutes data 102. The browsable format is, for example, HTML, rich text format, and the like, and the created minutes can be browsed by the user through the information terminal 130. Creation of the minutes can be performed at a predetermined timing such as after the meeting is finished, or at a timing when an instruction is given from the user. 9, 12A, and 12B are output examples of the created minutes. Details of the output example will be described later.

  The upload unit 324 transmits access information (address information, login information, etc.) to cause the information terminal 130 to upload the material 132 to the file server 110. The uploaded material 132 is associated with the minutes data 102. Note that since the material 132 may not be shared with the outside, the upload process is arbitrarily performed according to the determination of the user who participated in the conference.

  The format information adding unit 326 adds a format to the minutes created by the minutes creating unit 322 according to a predetermined condition defined as the format information 328. For example, a different color or font is designated for each material of the material 132, each page, or each department to which the user belongs. The format information adding method differs depending on the format output by the minutes creation means 322. For example, when the format of the minutes is HTML, the format can be set using a FONT tag, CSS (Cascading Style Sheets), or the like. Further, the contents of the format information and applicability can be arbitrarily changed by a predetermined setting or an instruction from the user.

  The format information 328 defines the format information added to the minutes by the format information adding means 326. The format information 328 is registered in, for example, a database connected to the format information adding unit 326, and is managed with identification information such as a conference name for each conference. The format information can be appropriately designated by the user, for example. 10A and 10B are examples of tables that define the format information 328.

  The affiliation information 330 is a database that stores user affiliation information and is prepared in advance, such as personnel data. The affiliation information 330 is associated with the user information 304 of the user who participates in the conference, and is used when creating the minutes using the affiliation information 330. FIG. 11 is an example of a table that defines the affiliation information 330.

  The sorting unit 332 is used to sort items in the minutes created by the minutes creating unit 322 according to a predetermined condition. The sort condition includes, for example, a voice identification number, a spoken user identifier, and a user affiliation included in the minutes data. The sort condition can be arbitrarily changed by a predetermined setting or an instruction from the user.

  With the above function, the minutes creation apparatus 100 according to an embodiment of the present invention can create and output the minutes in which the utterance contents during the meeting are associated with the information of the corresponding material 132.

  Note that the user information 304 may be stored in a database external to the user information storage unit 303. Also, the user information 304 can be stored in any format such as text data or other binary formats.

  Further, the voice identification number assigned by the associating means 308 may be a time stamp value indicating the time when the voice is spoken. By using the time stamp value as the voice identification number, it is not necessary to share the voice identification number and the time when the voice was spoken in advance between the minutes creation apparatus 100 and the information terminals 130, 140, and 150.

  Further, the voice conversion unit 310 can perform processing simultaneously with the speaker specifying unit 312. Thereby, the accuracy can be further improved by performing speech recognition reflecting the characteristics of the speech for each identified speaker.

  In addition, when the user information registration unit 302 finds that a user who has not registered a voice template for speaker identification participates in the conference, the speaker specifying unit 312 determines the template before starting the conference. Registration processing may be performed. For this reason, the minutes creation apparatus 100 displays a predetermined instruction on the screen display device 264 of the information terminals 130, 140, and 150 of users participating in the conference, and urges the user to register the template through the voice acquisition device 160, for example. Thereby, even when a user whose template is not registered participates in the conference, the minutes can be created by associating the voice information of the user with the information of the material 132.

  The minutes data 102, the format information 328, and the affiliation information 330 may be stored in another device on the network 120, and accessed from the minutes creation device 100 as necessary. As a result, the storage capacity required by the minutes creation apparatus 100 is reduced.

  In addition, when the affiliation information 330 is not provided in advance, it may be input as appropriate through the information terminals 130, 140, and 150 of users who participate in the conference before the start of the conference. For this reason, the minutes creation device 100 displays a predetermined instruction on the screen display device 264 of the information terminals 130, 140, and 150 of the users participating in the conference, and prompts the user to enter the affiliation information. The input affiliation information is stored as affiliation information 330. Thereby, even when the affiliation information of the user who participates in the conference is not registered in advance, the minutes can be created using the affiliation information 330.

  FIG. 3B is a functional block diagram of the information terminal 130 in one embodiment of the present invention. The information terminal 130 communicates with an input unit 350 that receives an input from a user, a control unit 352 that controls an application that operates on the information terminal 130, a display unit 354 that displays a material 132, and the minutes creation apparatus 100. Communication means 356, material information request accepting means 358 for accepting a request for information on the material 132 from the minutes creating apparatus 100, application identifying means 360 for identifying the application displaying the material 132, and the material 132 being displayed It has material information acquisition means 362 for acquiring information such as pages, and material information transmission means 364 for transmitting information on the acquired material 132 to the minutes creation apparatus 100, and information on the material 132 corresponding to the utterance content during the meeting. Is transmitted to the minutes creating apparatus 100.

  The input means 350 is used for receiving input from the user. In particular, it is used to input user identification information and identification information of the information terminal 130 transmitted to the user information registration means 302 of the minutes creation apparatus 100 before the start of the conference. Further, it is used for operating the material 132 during the meeting. Further, it is also used for inputting information such as a meeting name required when the minutes generating apparatus 100 is requested to output the minutes after the meeting.

  The control unit 352 controls a program (application) that runs on the information terminal 130. In particular, the application for displaying the material 132 is controlled according to the user input input through the input unit 350 and is displayed on the display unit 354.

  The display unit 354 displays the material 132 according to the control unit 352.

  The communication unit 356 is used for communicating with the minutes creation apparatus 100 or the file server 110.

  The material information request accepting unit 358 accepts a request for information on the material 132 corresponding to the audio information from the material information requesting unit 314 of the minutes creation apparatus 100.

  When the material information request receiving unit 358 receives a request for information on the material 132, the application specifying unit 360 specifies an application displaying the material 132 in order to specify information on the material 132 corresponding to the audio information. In general, the material 132 used during the meeting is displayed in the foreground window of the display means 354 on the information terminal 130. Therefore, the application specifying unit 360 specifies the application of the window displayed on the foreground of the display unit 354. In order to specify an application, for example, a window name or an application name related to a window that is a foreground can be acquired and used by using a general window system API provided in the OS.

  The material information acquisition unit 362 acquires information on the material 132 from the application specified by the application specifying unit 360. The information of the material 132 includes, for example, the number of displayed pages and the number of lines where the cursor is located, in addition to the file name and title of the material 132. Such information can be acquired by using an API provided from the OS or an application.

  The material information transmission unit 364 transmits the information on the material 132 acquired by the material information acquisition unit 362 to the minutes creation apparatus 100. The information of the material 132 can be described and transmitted in XML format, for example.

  With the above function, the information terminal 130 according to the embodiment of the present invention can transmit information such as the page of the material 132 corresponding to the utterance content during the meeting to the minutes creation apparatus 100.

  Note that the user information and the identification information of the information terminal 130 may not be input one by one in the input unit 350, and may be automatically transmitted based on the information stored in the information terminal 130. Thereby, a user's effort can be saved.

  Further, the application specifying unit 360 may specify that an application that occupies the entire screen of the display unit 354 is an application that displays the material 132. Alternatively, a predetermined application designated in advance by the user may be specified as an application for displaying the material 132.

  Further, the application identification unit 360 and the material information acquisition unit 362 may continue to execute the process during the meeting regardless of the request from the material information request receiving unit 358. At this time, the application identification unit 360 and the material information acquisition unit 362 store the acquisition result together with the time information. Thereby, even if the request from the minutes creating apparatus 100 is delayed, the time can be specified based on the voice identification number and the corresponding acquisition result can be transmitted.

  Further, the information of the material 132 transmitted by the material information transmitting unit 364 can use not only the XML format but also any format including a text format and a binary format.

(4. Outline of minutes creation process)
With reference to FIG. 4, an outline of a process flow for creating minutes by the minutes creation system 1 according to the embodiment of the present invention will be described. Details of the processing flow will be further described later with reference to FIGS.

  FIG. 4 shows an overview of the sequence of the entire minutes creation process, and shows the processes among the voice acquisition device 160, the minutes creation device 100, the information terminals 130 and 140, and the file server 110. Here, each user who uses the information terminals 130 and 140 displays a different material on each information terminal 130 and 140 to hold a meeting, and the minutes of the user's voice acquired by the minutes creation apparatus 100 through the voice acquisition apparatus 160 Create minutes based on Finally, the material is uploaded to the file server.

  First, before starting a conference, users who participate in the conference register user information with the minutes creation apparatus 100 using the information terminals 130 and 140 used by the users (steps S401 and S402). At this time, the conference name may be registered.

  The voice acquisition device 160 that has acquired the voice spoken by the user during the conference transmits the voice as a voice signal to the minutes creation device (step S411). The minutes creation apparatus 100 identifies the user who uttered the voice from the voice signal, and requests information on the material displayed on the screen when the voice is spoken to the information terminal 130 of the user. (Step S412). The information terminal 130 that has received the request transmits information on the corresponding material (step S413). As described above, the information on the material includes detailed information such as the number of pages in addition to the file name and title of the material. The same processing is performed on the voice uttered by the user of the information terminal 140 (steps S414 to S416). During the progress of the meeting, the minutes creation device 100 repeats the above operation to create and store the minutes data 102.

  The minutes creation apparatus 100 transmits information such as the address of the file server 110 and account information to the information terminal 130 in order to upload the material to the file server 110 (step S421). Then, the information terminal 130 uploads the material to the file server 110 (step S422). Similarly, the information terminal 140 is also made to upload the material (steps S423 and S424).

  As described above, the user can specify which part of the material the utterance content described in the minutes of the meeting is made and can make the minutes.

  The user information 304 and the minutes data 102 are managed for each meeting by a meeting name or the like separately designated by the user or the like. The conference name may be specified at the same time as the user information registration as described above, or may be automatically generated from information on the names, number of people, and date / time of the participating users.

  In addition, the upload process of the material 132 performed in steps S421 to S424 is arbitrary. For example, when the material 132 includes confidential information that cannot be shared, the material 132 may not be uploaded. In such a case, only information on which part of which material 132 the utterance content was made can be acquired from the minutes.

(5. User information registration process)
Next, using FIG. 5 and FIG. 6, user information registration, which is performed before starting the conference, registers the users participating in the conference and the information terminals 130 and 140 used by each user in the minutes creation apparatus 100. A processing sequence will be described. FIG. 5 shows a sequence of user information registration processing in the minutes creation system 1. This sequence corresponds to step S401 or S402 in FIG.

  First, the information terminal 130 participating in the conference uses the communication unit 354 to connect to the communication unit 300 of the minutes creation apparatus 100 to display a user registration screen, and to the user information registration unit 302, the information terminal 130. The identification information of the user who uses the ID and the identification information of the information terminal 130 are transmitted (steps S501 and S502). The user identification information is a user identifier for identifying the user, such as a user ID or an employee number. The identification information of the information terminal 130 is, for example, an IP address allocated to the information terminal 130.

  FIG. 6 shows a specific example of data transmitted in step S501. FIG. 6 includes items of a user identifier and an IP address. The user identifier is used to uniquely identify a user who participates in the conference. The IP address is used for the minutes creation apparatus 100 to communicate with the information terminals 130 and 140. Here, the IP address “192.168.0.2” is registered for the user identifier “UserA” of the user of the information terminal 130.

  Next, the user information registration unit 302 of the minutes creation apparatus 100 passes the received user identification information and identification information of the information terminal 130 to the user information storage unit 303 (step S503). The user information storage unit 303 stores the received user identification information and the identification information of the information terminal 130 in association with each other as user information 304 (steps S504 and S505). The user information 304 is stored in the form of a table as shown in FIG. 6, for example.

  Similarly, the process is repeated for the information terminals 140 of other users participating in the conference (steps S511 to S515). As a result, in FIG. 6, the IP address “192.168.0.3” is registered for the user identifier “UserB” of the user of the information terminal 140. Further, when there is a user who participates in the conference, the same processing is repeated.

  As described above, the user identification information of the users participating in the conference and the identification information of the information terminal used by each user are registered as the user information 304.

  In the examples of FIGS. 5 and 6, a user identifier that can uniquely identify the user is used as the user identification information. However, a temporary identifier that is effective only in a certain meeting may be used, for example.

  In the examples of FIGS. 5 and 6, the IP address assigned to the information terminal is used as the identification information of the information terminals 130 and 140 used by the user. However, in order to communicate with the minutes creation apparatus 100 belonging to the same network. Any identification information that can be used is used.

(6. Details of minutes creation process)
Next, a processing sequence for creating minutes will be described with reference to FIG. The configuration of the minutes creation system 1 is the same as that shown in FIGS. Here, a sequence until the voice uttered by the user of the information terminal 130 is stored as the minutes data 102 in association with the information of the material 132 will be described.

  The voice uttered by the user during the conference is acquired as a voice signal by the voice acquisition device 160. The voice acquisition unit 306 of the minutes creation apparatus 100 acquires the voice signal via the communication unit 300 (steps S701 and S702). Further, the audio signal is passed to the associating means 308 (step S703). The associating unit 308 converts the audio signal into audio information divided by a predetermined unit, and attaches an audio identification number (step S704). Hereinafter, the voice information is specified by the voice identification number.

  The associating unit 308 passes the audio information to the audio converting unit 310 and requests conversion of the audio information into character information (step S705). The voice conversion unit 308 returns the voice information to the character information, and transmits the result to the association unit 308 (step S706).

  The associating unit 308 further passes the audio information to the speaker specifying unit 312 and requests the identification of the user who uttered the audio related to the audio information (step S707). The speaker specifying unit refers to the user information 304 stored in the user information storage unit 303, and acquires the user identifier of the user participating in the conference (steps S708 and S709). Then, speaker identification is performed using the templates of those users, and the user identifier of the user who uttered the voice is passed to the association means (step S710).

  The associating unit 308 designates the voice identification number and the user identifier, and requests the material information requesting unit 314 for information on the material 312 corresponding to the sound information (step S711). The material information requesting unit 314 requests material information from the information terminal 130 through the communication unit 200 using the IP address of the information terminal 130 corresponding to the user identifier (step S712).

  The material information request accepting means 358 of the information terminal 130 receives the material information request including the voice identification number from the minutes creating apparatus 100 through the communication means 356 (step S713). The application specifying unit 360 specifies the application displaying the material 132 on the information terminal 130 when the voice related to the voice information corresponding to the received voice identification number is uttered (step S714). As described above, for example, the application displaying the window in the foreground is specified as the application displaying the material 312. Then, the material information acquisition unit 364 acquires which part of the material 132 is displayed (step S715). As described above, the file name, title, page number, line number, etc. of the material 312 displayed on the application are acquired. The information of the material 132 acquired here is transmitted to the minutes creation apparatus together with the voice identification number by the material information transmitting means 362 (step S716).

  The material information receiving unit 316 receives the information of the material 132 and the voice identification number from the information terminal 130 through the communication unit 300 (step S717) and passes them to the associating unit 308 (step S718). The associating unit 308 associates the information of the received material 132 with the character information based on the voice identification number (step S719), and stores it as the minutes data 102 in the minutes holding unit 318 (step S720).

  FIGS. 8A and 8B are examples of tables for storing the associated voice identification number, character information, and information of the material 312 that are stored as the minutes data 102. The table shown in FIG. 8A includes items of an audio identification number, character information, and material information. The voice identification number is the voice identification number assigned to each piece of voice information by the associating means in step S704. The character information is obtained by converting the voice information into the character information by the voice conversion unit 310, and corresponds to the utterance content. The material information represents information of the material 312 corresponding to the character information. Further, the table shown in FIG. 8B has a voice identification number and a user identifier, and associates the voice identification number with the user who is the speaker.

  The above steps S701 to S720 are repeated for each voice uttered by the user during the meeting, and the minutes data 102 in which the utterance contents and the material 132 are associated is created.

  Note that the application specifying unit 360 and the material information acquiring unit 362 of the information terminal 130 may continue to execute the process at all times during the meeting, and store the acquisition result together with the time information. Thereby, it is possible to specify the displayed material 132 at the past time point, and it is possible to cope with a case where a request from the minutes creation apparatus 100 is delayed.

  Further, the tables shown in FIGS. 8A and 8B stored as the minutes data 102 are an example, and may be configured together in one table or may be configured in a more detailed manner. Further, as described above, the table may be configured to include time information when the voice is spoken, information on the user's location, and the like.

(7. Minutes output example)
Next, an example of the output of minutes will be described with reference to FIGS.

  The information terminal 130 transmits information that can specify the conference, such as the conference name, designated by the user to the minutes creation device 100 and requests creation of the minutes of the meeting. The minutes creation means 322 of the minutes creation apparatus 100 receives this request through the communication means 300 (step S751).

  Next, the minutes creation means 322 first acquires the minutes data 102 of the designated meeting from the minutes holding means 318 (steps S752, S753). For example, examples of the minutes data 102 to be stored are shown in FIGS. 8A and 8B.

  When the minutes data 102 is acquired, the minutes creation unit 322 next requests the format information addition unit 326 to add the format information, and adds the format information 328 corresponding to the minutes data 102 (step S754). , S755). Then, the sorting means 332 sorts the minutes data 102 (steps S756 and S757), and then creates and outputs the minutes (step S758). The minutes are formatted using a format that can be viewed on the information terminal 130 or the like at any stage from the acquisition of the minutes data 102 (S752) to the output of the minutes (S758).

  The order of the minutes data request (steps S752, S753), the addition of format information 328 (steps S754, S755), and the sorting (steps S756, S757) is arbitrary. Further, the addition of the format information 328 (steps S754 and S755) and the sort (steps S756 and S757) can be arbitrarily executed by a predetermined setting or an instruction from the user.

  Below, it explains with a specific output example.

(7.1 Basic output example)
FIG. 9 shows an output example of the minutes. Here, the processing by the format information adding unit 326 and the sorting unit 332 is not executed. The minutes shown in FIG. 9 include the title (for example, meeting name) indicated by “XX minutes” and the utterance content expressed in the form of “user identifier: character information” output in the order of voice identification numbers. And material information corresponding to the utterance content. Elements of the user identifier, character information, and material information are extracted from the minutes data 102, created by a minutes creation unit 322 in a predetermined format such as HTML, and output. In FIG. 9, in the minutes data 102 shown in FIG. 8A and FIG. 8B, as the utterance content of the voice identification number 1, “UserA”, which is the user identifier of the uttered user, and the text information of the utterance content “this” Is the relation table. ”, The material name“ material 1 ”of the material 132 displayed on the information terminal 130 of“ UserA ”when this utterance is made is output. As described above, a user who refers to the minutes later can clarify the relationship between the materials used in the meeting and the utterance contents and understand the contents of the minutes.

(7.2 Example of output using format information and affiliation information)
12A to 12C are schematic diagrams showing examples of output of minutes using format information and user affiliation information. In FIGS. 12A to 12C, the title, the information of the user who spoke, and the like are omitted. In FIG. 12A, the utterance contents in the minutes are colored for each of the documents “document 1” and “document 2”. In FIG. 12B, the utterance contents in the minutes are colored for each page “page 1” and “page 2” of the material. In FIG. 12C, the utterance contents in the minutes are colored for each department “development department” and “planning department” of the user who is the speaker. As described above, according to an embodiment of the present invention, the minutes can be output by setting the format information for each material, each page of the material, or each department to which the user belongs. This will be described in detail below using detailed output examples.

  FIG. 13A is an example of output of the minutes in which different format information is set for each material and page and the user's affiliation information is used. Unlike the output example of FIG. 9, the name of the participant is output together with the department information, title, and contact information after the title. Also, the utterance content is displayed together with the name and title instead of the user identifier. Further, each material information corresponding to the utterance content is displayed with a format of a different color (displayed in italics in the figure). In FIG. 12A, the user identifiers “UserA” to “UserD” in FIG. 6 are associated with employee numbers “403234” to “233431” of the affiliation information 330 shown in FIG. Based on the format information 328 shown in FIG. 10A, the minutes regarding “Document 1: Page 2” are indicated by “character color: blue”, and the minutes regarding “Document 2: Page 3” are indicated by “Character color: Output in red (shown in italics).

  Here, an example of the format information 328 will be described with reference to FIGS. 10A and 10B.

  FIG. 10A defines format information to be set for each piece of material information included in the information of the material 132 with respect to the minutes data 102 of FIG. 8A. FIG. 10A has items of material information and format information, and the contents related to the material information specified here in the minutes indicate that the specified format information should be applied. Specifically, the example of FIG. 10A defines the “character color” of the utterance content corresponding to “page 2” of “document 1” to “blue” with respect to the minutes data of FIG. 8A. . These settings are predefined in the minutes creation device 130 or can be arbitrarily set by the user.

  On the other hand, FIG. 10B defines format information to be set for each department information of the user in the minutes data 102 of FIG. 8A. FIG. 10B has items of affiliation department information and format information, and the contents related to the affiliation department information specified here in the minutes indicate that the specified format information should be applied. . Specifically, the example of FIG. 10B defines that the “character color” of the utterance content of the user belonging to the “development department” is “red”.

  FIG. 11 shows user affiliation information 330, and includes items of employee number, name, affiliation department information, title, and contact information. The affiliation information 330 is, for example, a personnel database prepared in advance, and can be referred to by the minutes creation system 1 as necessary. When the affiliation information 330 is available, the user information storage unit 303 of the minutes creation apparatus 100 associates the user registered in the user information 304 with the affiliation information 330. Here, “UserA” in FIG. 6 is associated with the employee of the employee number “403234” in FIG. The information necessary for the association may be acquired from the user in step S501 of FIG. 5, which is the stage of user information registration, or the user identifier may correspond to the employee number in advance.

  By the above, users who refer to the minutes later can understand the affiliation of the users who participated in the meeting, clarify the relationship between the materials used in the meeting and the utterance contents, and understand the contents of the minutes it can. In addition, the readability is improved by the utterance contents color-coded for each material.

(7.3 Output example by sorting)
FIG. 13B shows an output example in which the sorting processing by the sorting unit 332 is further performed on the output result of FIG. 13A. Unlike FIG. 12A, the utterance contents are sorted and output for each department to which the user belongs, such as “development of development department” and “utterance of planning department”. Different departments are displayed with different colors (shown in italics in the figure). The format information 328 is defined in FIG. 10B. That is, the utterance content of the user belonging to the “development department” is output in “character color: red” (indicated in italics). The sorting condition can be arbitrarily changed afterwards according to the user's designation for displaying the minutes. For example, an item can be selected from a menu or the like on the screen for displaying the minutes, and the sorting unit 332 performs sorting based on the selected item, and the minutes creation unit 322 creates and outputs the minutes. be able to.

  As described above, a user who refers to the minutes later can sort and output the utterance contents for each predetermined item in the minutes, and can understand the contents of the minutes from a specific aspect.

  9, 13A, and 13B show examples of the output of the minutes. The date and time of the meeting, the meeting time, the location, the number of participants, the date and time of the minutes, the data size of the minutes, and the content of the utterance Data relating to the meeting or minutes, such as the number of cases and the number of materials, may be output to an arbitrary place.

  Further, the position for displaying the title, utterance content, and information of the material can be arbitrarily changed. In particular, the information of the material does not necessarily have to be written together with the content of the utterance. For example, when the user points on the content of the utterance on the information terminal 130 displaying the minutes, the information of the material is displayed in a pop-up. Also good. As a result, when the information of the material increases, it can be avoided that the output of the minutes is complicated.

  In addition to the color, the format set in the format information 328 can use various attributes such as font type, font size, bold, italic, and hyperlink.

DESCRIPTION OF SYMBOLS 100 Minutes preparation apparatus 130 Information terminal 300 Communication means 302 User information registration means 303 User information storage means 304 User information 306 Voice acquisition means 308 Association means 310 Voice conversion means 312 Speaker identification means 314 Material information request means 316 Material information reception means 318 Minutes holding means 320 Minutes data 322 Minutes creation means 324 Upload means 326 Format information addition means 328 Format information 330 Affiliation information 332 Sort means 350 Input means 352 Control means 354 Display means 356 Communication means 358 Document information request acceptance means 360 Application identification means 362 Material information acquisition means 364 Material information transmission means

JP 2009-288567 A JP 2009-69172 A

Claims (13)

A minutes creation system having a minutes creation device for creating minutes by converting the utterance content of users participating in a meeting into character information, and an information terminal for displaying materials that are electronic data,
The minutes creation device
A material information requesting means for identifying a user who has spoken and requesting information on the material being displayed when speaking to an information terminal used by the user who has spoken ;
Minutes creation means for creating a minutes in which the user's utterance content and information on the material are associated with each other;
Have
The information terminal
Material information request accepting means for receiving a request from the material information requesting means;
Material information acquisition means for acquiring information of the material being displayed by the information terminal when the material information request reception means receives the request;
Material information transmitting means for transmitting the information of the acquired material to the minutes creation device;
A minutes creation system. This is a minutes creation system that has a minutes creation device that converts the utterances of users participating in a meeting into text information and creates a minutes, and a program installed in an information terminal that displays materials that are electronic data. And
The minutes creation device
A material information requesting means for identifying a user who has spoken and requesting information on the material being displayed when speaking to an information terminal used by the user who has spoken ;
Minutes creation means for creating a minutes in which the user's utterance content and information on the material are associated with each other;
Have
The program is stored in the information terminal .
A material information request receiving stage for receiving a request from the material information requesting means;
A material information acquisition step of acquiring information of the material being displayed by the information terminal when the request is received in the material information request reception step;
A material information transmission stage for transmitting the information of the acquired material to the minutes creation device;
A system for creating minutes. A minutes creation device for creating minutes by converting the utterance content of users participating in a meeting into text information,
A material information requesting means for identifying a user who has spoken and requesting information on the material being displayed when speaking to an information terminal used by the user who has spoken ;
Minutes creating means for creating a minutes that associates the user's utterance content with the information of the material received from the information terminal;
A minutes creation device. The minutes creation device according to claim 3, further comprising user information storage means for storing the user identification information and the identification information of the information terminal in association with each other. The minutes creation apparatus of Claim 3 or 4 which has the speaker identification means which identifies the said user who spoke. The minutes creation device according to any one of claims 3 to 5, further comprising: format information addition means for adding format information to the text information in the minutes based on the information of the material. The format information adding means attaches color information to the character information in the minutes.
The minutes preparing device according to claim 6. The format information adding means adds different format information to the character information in the minutes according to the affiliation of the user.
The minutes preparing device according to claim 6 or 7. The information of the material includes page information of the material,
The format information adding means adds different format information to the character information in the minutes according to the page information.
9. A minutes creation device according to claim 6. The minutes creation device according to any one of claims 3 to 9, further comprising: a sorting unit that sorts the character information in the minutes based on information of the material. A minutes creation program installed in a minutes creation device that converts the contents of utterances of users participating in a meeting into character information to create minutes, and in the minutes creation device ,
A material information request stage for identifying a user who has spoken and requesting information on a material being displayed when speaking to an information terminal used by the user who has spoken ,
Minutes creation stage for creating minutes in association with the user's utterance content and information of the material received from the information terminal;
Minutes creation program that runs The user's minutes creation terminal that can display a document that is electronic data and communicates with the minutes creation device that creates the minutes by converting the utterance content of the user participating in the meeting into text information,
A material information requesting means for identifying a user who has spoken and requesting information on the material being displayed when speaking to an information terminal used by the user who has spoken ;
And article information acquiring means the proceedings preparation terminal obtains information of the article being displayed when the proceedings preparation terminal has accepted the request,
Material information transmitting means for transmitting the information for creating the minutes in association with the utterance contents of the user to the minutes creating device,
A minutes creation terminal. Minutes installed in the user's minutes creation terminal that can display materials that are electronic data, and communicate with the minutes creation device that creates the minutes by converting the utterances of users participating in the meeting into text information It is a record creation terminal program, in the minutes creation terminal ,
A material information request stage for identifying a user who has spoken and requesting information on a material being displayed when speaking to the minutes creation terminal used by the user who spoke .
And article information acquiring step of the proceedings preparation terminal obtains information of the article being displayed when the request has been accepted in the proceedings preparation terminal,
A material information transmitting step for transmitting the information for creating the minutes in association with the utterance content of the user to the minutes creating device;
Minutes creation terminal program that executes.

Images