JP2021136606A - Information processing device, information processing system, information processing method, and information processing program - Google Patents

Abstract

To generate a minute book according to a progress state of a conference.SOLUTION: A video conference server (4) includes acquisition units (402, 404, 406) for acquiring transmission destination information showing transmission information transmitted from a conference terminal, identification information of a conference participant, and transmission destination information showing a transmission destination, a generation unit (400) for generating minute book information including the transmission information and the transmission destination information as transmission information of the conference participant on the basis of the acquired information, and a minute book record part (420) for performing control to record minute book information in the storage unit each time the minute book information is generated.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to information processing devices, information processing systems, information processing methods, and information processing programs.

Conventionally, in a conference system in which a plurality of terminals such as a computer are connected to a communication line to hold a conference, an information processing device that converts voice data into text and creates minutes has been known (for example, Patent Document 1). See). This information processing device records the status of the meeting and converts the recorded voice signal into text data after the meeting to create minutes and the like. In addition, an information processing device capable of efficiently extracting the information necessary for creating the minutes and creating the optimum minutes of a remote conference using a communication system is also known.

JP-A-2019-164327

By the way, in a meeting involving a plurality of attendees, as the proceedings progress, some attendees of the plurality of attendees intentionally confirm the contents of the proceedings and related contents among the some attendees. May be exchanged.

On the other hand, by converting the voice uttered by the attendee into voice text, it is possible to convert the content of the remarks made by the attendee into text that can be recorded in the minutes. However, when the minutes converted into text are generated using the recorded recorded data, the conversion accuracy is lowered due to the overlap of the voices of a plurality of speakers. In addition, the text-converted minutes will be uniformly text-converted minutes even if some attendees intentionally exchanged information indicating confirmation or the like. For this reason, the minutes intentionally exchanged between some attendees according to the progress of the meeting are not reflected in the minutes, and the minutes are faithfully generated according to the progress of the meeting. Has room for improvement.

In this disclosure, an information processing device, an information processing system, an information processing method, which can generate minutes according to the progress of a meeting, as compared with the case where the minutes of a meeting are generated only by text data obtained by converting voice data. And to provide an information processing program.

The first aspect of the present disclosure is transmission information transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals, and a first conference that operates the first conference terminal. Acquire identification information indicating a participant and destination information indicating a second conference participant in which the first conference participant operates one or more second conference terminals indicating a destination of the transmission information. The acquisition unit, the generation unit that generates the minutes information in which the acquired transmission information, the identification information, and the destination information are used as the remark information of the first conference participant, and the minutes information are generated. It is an information processing device having a control unit that controls to store the generated minutes information in the storage unit each time.

The second aspect of the present disclosure is an information processing system including a plurality of conference terminals and the information processing device according to the first aspect.

A third aspect of the present disclosure is an information processing method in an information processing device, in which the information processing device is transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals. Transmission information, identification information indicating a first conference participant who operates the first conference terminal, and one or more second conference terminals in which the first conference participant indicates a destination of the transmission information. The acquisition step of acquiring the destination information indicating the second conference participant who operates the above, and the acquired transmission information, the identification information, and the destination information are used as the remark information of the first conference participant. It is an information processing method having a generation step of generating minutes information and a control step of controlling to store the generated minutes information in a storage unit each time the minutes information is generated.

A fourth aspect of the present disclosure is to operate a computer, transmission information transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals, and the first conference terminal. Identification information indicating one conference participant, and destination information indicating a second conference participant in which the first conference participant operates one or more second conference terminals indicating a destination of the transmission information. The acquisition unit, the generation unit that generates the minutes information in which the acquired transmission information, the identification information, and the destination information are used as the remark information of the first conference participant, and the minutes information is generated. It is an information processing program for functioning as a control unit that controls the storage of the generated minutes information in the storage unit each time it is performed.

According to the present disclosure, it is possible to obtain the effect that the minutes can be generated according to the progress of the meeting, as compared with the case where the minutes of the meeting are generated only by the text data obtained by converting the voice data.

It is a figure which shows the outline of the information processing system of 1st Embodiment. It is a figure which shows an example of the structure of the video conferencing server of 1st Embodiment. It is a figure which shows an example of the structure of the voice-to-text conversion apparatus. It is a figure which shows an example of the structure of a conference terminal. It is a figure which shows an example of the functional structure of a video conferencing server. It is a figure which shows an example of the table which recorded the presence or absence of the right to speak. It is a flowchart which shows an example of the flow of the minutes recording process concerning voice. It is a flowchart which shows an example of the flow of the minutes record processing about a chat. It is a figure which shows an example of the minutes which recorded voice and chat. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon. It is a flowchart which shows an example of the flow of the minutes provision processing executed in the video conference server 4. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon. It is a figure which shows an example of the minutes which recorded voice and chat. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon. It is a figure which shows an example of the structure of the video conferencing server of 2nd Embodiment. It is a figure which shows an example of the minutes which represented the minutes memo with a typical icon.

Hereinafter, an example of the information processing apparatus according to the embodiment of the present invention will be described in detail with reference to the drawings. It should be noted that components and processes in which movement, action, and function have the same function may be given the same sign throughout the drawings, and duplicate explanations may be omitted as appropriate. Each drawing is only schematically shown to the extent that the present invention can be fully understood. Therefore, the present invention is not limited to the illustrated examples. Further, in the present embodiment, description may be omitted for configurations that are not directly related to the present invention and well-known configurations.

[First Embodiment]

FIG. 1 shows an example of the configuration of the information processing system 1 of the present disclosure.
As shown in FIG. 1, the information processing system 1 includes a plurality of conference terminals 2, a video conference server 4, and a voice-text conversion device 5.

In the example shown in FIG. 1, the plurality of conference terminals 2 include a conference terminal 2A, a conference terminal 2B, a conference terminal 2C, and a conference terminal 2D. The conference terminal 2A is a terminal device for a video conference that is operated by the operation of the attendee A of the conference and performs processing such as attending a video conference held by the video conference server 4. Each of the conference terminals 2B, 2C, and 2D is a terminal device operated by the operations of attendee B, attendee C, and attendee D, similarly to the conference terminal 2A. Each of the attendees A to D who operate each of the conference terminals 2A from the conference terminal 2A may be an individual user or a group user. Each of the conference terminal 2A to the conference terminal 2D can be realized by a computer having a communication function.

In the example shown in FIG. 1, four terminals from the conference terminal 2A to the conference terminal 2D are shown as an example as terminal devices for the attendees to attend the video conference, but the conference terminal is an example shown in FIG. 1 (for example, 4). The number is not limited to 3 or more. In the following description, it is not necessary to individually explain each of the conference terminal 2A to the conference terminal 2D, and when the matters common to the conference terminals are explained, the conference terminal 2 may be referred to. Further, it is not necessary to individually explain the attendees A to D who operate each of the conference terminals 2A from the conference terminal 2A, and when explaining common matters, they are referred to as attendees without assigning a code. There is.

The video conferencing server 4 is a device that performs processing for executing management of the conferencing system, including processing for holding and proceeding with video conferencing.

The voice-to-speech conversion device 5 is a device having a function of mainly converting input voice data into text data and outputting the data. The voice-to-speech conversion device 5 is not limited to the configuration of individually configured devices. For example, the video conferencing server 4 may be provided with the functions of the voice-to-text converter 5.

The plurality of conference terminals 2, the video conference server 4, and the voice-to-text converter 5 are connected to each other via the network 6. As the network 6, at least one network of a wireless network by a wireless connection and a wired network by a wired connection can be applied.

The network 6 is an example of a device-to-device connection as a concept, and may include wired and wireless communication lines, and is connected via a plurality of network-to-network communications, for example, a so-called intranet and the Internet. May include doing.

The information processing system 1 is an example of an information processing system of the technology of the present disclosure. The video conferencing server 4 is an example of an information processing device of the technology of the present disclosure. The conference terminal 2 is an example of a conference terminal of the technology of the present disclosure, and any one of the conference terminals 2A to 2D of the conference terminal 2D is an example of the first conference terminal of the technology of the present disclosure. The other conference terminal 2 other than the conference terminal 2 from 2A to the conference terminal 2D is an example of the second conference terminal of the technique of the present disclosure.

(Video conferencing server 4)
Next, with reference to FIG. 2, an example of the configuration of the video conferencing server 4 capable of realizing the information processing apparatus of the present disclosure will be described.

The video conferencing server 4 includes a computer main body 40, and the computer main body 40 includes a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, a ROM (Read-Only memory) 43, and an input / output port (I). / O) 44, which are connected to each other via a bus 46. An auxiliary storage device 45 that can be realized by an HDD (Hard Disk Drive), a non-volatile flash memory, or the like is connected to the bus 46. Further, a communication I / F 47 that communicates with an external device is connected to the I / O 44. Although omitted in the example shown in FIG. 2, the I / O 44 is configured to be connectable to an operation display unit such as a display, a mouse, and a keyboard.

The auxiliary storage device 45 stores a conference system program 45M having a function of executing management of the conference system including a process of holding and proceeding a video conference. The CPU 41 of the video conferencing server 4 reads the conferencing system program 45M from the auxiliary storage device 45, expands it into the RAM 42, and executes processing. As a result, the video conferencing server 4 that executes the conferencing system program 45M operates as a device that manages the conferencing system and has a function of a multipoint connection device (MCU: Multi-point Control Unit). Since the video conferencing server 4 having a function of executing the management of the conferencing system is a known technique, detailed description thereof will be omitted.

Further, the auxiliary storage device 45 stores various data 45D used in each of the conference system program 45M and the information processing program 45P described later. For example, minutes information indicating the minutes of the meeting held by executing the meeting system program 45M is stored (details will be described later).

The data 45D stored in the auxiliary storage device 45 is not limited to being stored in the video conferencing server 4. For example, it may be stored in an external storage (not shown) so that the video conferencing server 4 reads and writes to the external storage (not shown) by communication.

Further, the auxiliary storage device 45 stores an information processing program 45P for making the video conferencing server 4 function as the information processing device of the present disclosure. The information processing program 45P is a program that mainly executes a process of storing the minutes information indicating the minutes of the conference held by executing the conference system program 45M in the data 45D. Specifically, the video conferencing server 4 receives remarks from attendees at the conference being held and executes a process of storing the minutes information. That is, the CPU 41 of the video conferencing server 4 reads the information processing program 45P from the auxiliary storage device 45, expands it into the RAM 42, and executes the process. As a result, the video conference server 4 that executes the information processing program 45P operates as the information processing device of the present disclosure.

Although the details will be described later, the information processing program 45P includes a process of executing each of the minutes recording process and the minutes providing process.

Further, in the present embodiment, the case where the information processing program 45P is stored in the auxiliary storage device 45 of the video conference server 4 and executed by the video conference server 4 will be described. The information processing program 45P is the communication I / F 47. It may be read out from an external device and a recording medium and acquired via.

(Voice-text converter 5)
Next, an example of the configuration of the voice-to-text conversion device 5 will be described with reference to FIG.
As shown in FIG. 3, the voice-text conversion device 5 includes a device main body 50, and the device main body 50 is realized by a computer equipped with a CPU 51, a RAM 52, a ROM 53, and an I / O 54.

The voice-text conversion device 5 includes a device main body 50, which includes a CPU 51, a RAM 52, a ROM 53, and an I / O 54, which are connected to each other via a bus 56. A communication I / F 57 that communicates with an external device is connected to the I / O 54. Further, an auxiliary storage device 55 is connected to the bus 56. The voice-text conversion program 55P is stored in the auxiliary storage device 55. The voice-text conversion program 55P is a program that executes a process for converting voice data from the conference terminal 2 into text data. Specifically, the voice-text conversion device 5 receives voice data and executes a process of converting the received voice data into text data. That is, the CPU 51 of the voice text conversion device 5 reads the voice text conversion program 55P from the auxiliary storage device 55, expands it into the RAM 52, and executes the process. As a result, the voice-text conversion device 5 that executes the voice-text conversion program 55P operates as a device that converts the received voice data into text data and outputs the data. Since the function of converting voice data that can be realized by the voice-text conversion device 5 into text data and outputting it is a known technique, detailed description thereof will be omitted.

(Conference terminal 2)
Next, with reference to FIG. 4, the configuration of the conference terminal 2 which is an example of the terminal device for video conferencing will be described.

The conference terminal 2 includes a computer main body 20, which includes a CPU 21, a RAM 22, a ROM 23, and an I / O 24, which are connected to each other via a bus 26. Further, the I / O 24 includes a communication I / F 27 that communicates with an external device, a camera 28C that captures an attendee who operates the conference terminal 2, a microphone 28M that collects the sound of the attendee, and the first attendee. An operation input unit 28K such as a mouse and a keyboard to be operated, and a display unit 29 such as a display are connected.

It is assumed that the terminal program (not shown) for making the conference terminal 2 function as a terminal device for video conferencing is stored in the ROM 23 in advance. The conference terminal program (not shown) may be read by equipping the conference terminal 2 with an auxiliary storage device (not shown) and storing the conference terminal program (not shown) in the auxiliary storage device (not shown). It may be stored in an external storage (not shown) and read by communication.

Since each of the conference terminal 2A to the conference terminal 2D has substantially the same configuration, individual description thereof will be omitted.
The conference terminal 2 may be incorporated into an electronic device that internally performs data processing, and the incorporated electronic device may function as the conference terminal 2. For example, the conference terminal 2 may function as the conference terminal 2 by introducing a terminal program (not shown) into an independent computer such as a personal computer (PC). Further, instead of the PC, it may be applied to a portable device such as a PDA (Personal Digital Assistant) or a mobile phone.

The information processing program 45P is an example of an information processing program of the technique of the present disclosure. Further, the video conference server 4 on which the information processing program 45P is executed is an example of an information processing device of the disclosed technology. In addition, voice data, chat data, and text data are examples of transmission information of the present disclosure.

(Video conferencing server 4)
Next, the function of the video conference server 4 will be described.

By the way, it is preferable that the minutes of a meeting attended by a plurality of attendees record the remarks of each of the plurality of attendees. However, when multiple remarks are made by multiple attendees at the same time or at the same time, each of the multiple remarks is faithful only by converting the text for the minutes using the recorded data of the remarks at the meeting. It may be difficult to record in.

In view of this fact, in the present embodiment, it is possible to faithfully add each of the statements made by the plurality of attendees during the meeting to the minutes.

In a meeting attended by a plurality of attendees, for example, during the meeting, inquiries such as questions and confirmations may be made or instructed among some attendees (for example, between two parties), which is less than all the attendees of the meeting. Information may be exchanged (information exchange) to request such things. It is preferable that the exchange of information (transfer of information) between some of the attendees is also recorded in the minutes. However, the remarks in the conference system are generally open to all participants, and it is difficult to record them in the minutes as information exchanged among some attendees.

In view of this fact, in the present embodiment, it is possible to add information exchanged between some attendees during the meeting to the minutes.

On the other hand, in the exchange of information between some attendees, which is less than all the attendees of the meeting, during the meeting, the party exchanging the information is open to other attendees outside the party. You may not want it, for example, you may intentionally want it to be concealed.

Therefore, in the present embodiment, it is possible to add information to the minutes while reflecting the intentions desired by the parties regarding the exchange of information between some attendees, which is less than all the attendees of the meeting, during the meeting.

In addition, in order to simplify the following explanation, in the present embodiment, when it is possible to speak to the conference by voice and to chat among a plurality of attendees by the conference terminal 2 operated by the attendees. Will be explained. Here, chat means exchanging a message (for example, text information) in real time between a plurality of conference terminals 2.

Further, in the present embodiment, a case where the conference terminal 2 is used to speak to the conference by voice and information is exchanged between some of the attendees among a plurality of attendees by chat will be described. do. In this case, the voice statement to the meeting shall be a statement made to all attendees, and the exchange (transmission) of information to some attendees by chat shall be made to all attendees or the attendees at the destination. The chat will be held.

FIG. 5 shows an example of the functional configuration of the video conferencing server 4 in the present embodiment.
As shown in FIG. 5, the video conferencing server 4 includes a minutes generation unit 400 and a functional unit that functions as the minutes recording unit 420. The minutes generation unit 400 and the minutes recording unit 420 make it possible to record the remarks made by each attendee and the exchange of information among some attendees as minutes memos in the minutes.

Each functional unit of the minutes generation unit 400 and the minutes recording unit 420 includes minutes memo generation units 400A to 400D and minutes memo recording units 420A to 420D for each of the plurality of conference terminals 2.

The minutes memo generation unit 400A and the minutes memo recording unit 420A record the remarks of attendee A input by the conference terminal 2A and the exchange of information by attendee A with other attendees as minutes memos. Record to 45G.

The minutes memo generation unit 400A includes an instruction acquisition unit 402, a voice acquisition unit 404, a chat acquisition unit 406, a qualification determination unit 408, a voice / text conversion unit 410, and a mode confirmation unit 412.

The instruction acquisition unit 402 acquires instruction data indicating the attendees at the transmission destination for the chat data from the conference terminal 2A and an instruction as to whether or not to keep the chat data confidential, and outputs the acquired instruction data to the mode confirmation unit 412. It is a functional part to do. The instruction data from the conference terminal 2A includes information indicating attendees at the transmission destination, for example, IDs indicating attendees (in FIG. 6, they are indicated as ID-A to ID-D). That is, the instruction acquisition unit 402 acquires the information indicating the instruction of the attendee at the transmission destination by acquiring the ID from the conference terminal 2A. Further, the instruction data from the conference terminal 2A includes information indicating an instruction as to whether or not to keep the chat data confidential. That is, the instruction acquisition unit 402 acquires an instruction as to whether or not to keep the chat data confidential by acquiring information indicating whether or not to keep the chat data confidential from the conference terminal 2A. The instruction acquisition unit 402 has a function of acquiring time data indicating the date and time when the instruction data was acquired and associating the instruction data with the instruction data. This makes it possible to grasp the instruction data in chronological order.

The ID indicating the attendee is an example of the identification information and the destination information of the present disclosure. Further, the information indicating the attendees of the destination included in the instruction data from the conference terminal 2A, for example, the ID indicating the attendees is an example of the destination information of the present disclosure. The attendees at the transmission destination for the chat data indicated by the instruction data and the information indicating the instruction as to whether or not to keep the chat data confidential are an example of the confidential identification information of the present disclosure.

Further, the instruction acquisition unit 402, the voice acquisition unit 404, and the chat acquisition unit 406 are examples of the acquisition units of the present disclosure. The minutes generation unit 400 and the minutes memo generation unit 400A are examples of the generation unit of the present disclosure. The minutes recording unit 420 and the minutes memo recording unit 420A are examples of the control unit of the present disclosure.

The voice acquisition unit 404 is a functional unit that acquires voice data transmitted from the conference terminal 2A and outputs the acquired voice data to the qualification determination unit 408. The voice acquisition unit 404 has a function of acquiring time data indicating the date and time when the voice data was acquired and associating the voice data with the voice data. This makes it possible to grasp the voice data in chronological order.

The chat acquisition unit 406 is a functional unit that acquires the chat data transmitted from the conference terminal 2A and outputs the acquired chat data to the mode confirmation unit 412. The chat acquisition unit 406 has a function of acquiring time data indicating the date and time when the chat data was acquired and associating the chat data with the chat data. This makes it possible to grasp the chat data in chronological order.

The qualification determination unit 408 is a function unit that determines whether or not there is a right to speak with respect to the voice data acquired by the voice acquisition unit 404. The qualification determination unit 408 determines using the table 45T (FIGS. 5 and 6) stored in the data 45D of the auxiliary storage device 45 in which the presence or absence of the right to speak is recorded. That is, the qualification determination unit 408 outputs voice data to the voice / text conversion unit 410 when the attendees have the right to speak recorded in the table 45T (FIGS. 5 and 6). On the other hand, when the attendees have no voice recorded in the table 45T (FIGS. 5 and 6), the qualification determination unit 408 does not output the voice data to the voice / text conversion unit 410, for example, noise. Discard as. If other attendees are currently speaking, the attendee A may be made to stand by.

The voice / text conversion unit 410 is a functional unit that converts voice data input via the qualification determination unit 408 into text and outputs it to the minutes memo recording unit 420A. In the present embodiment, the voice / text conversion unit 410 transmits the input voice data to the voice / text conversion device 5, and acquires the text-converted text data as a response.

The voice / text conversion unit 410 adds "speech" indicating that the voice data is a remark to all the attendees and information indicating "all" to the text-converted voice text data and outputs the data. Specifically, the voice / text conversion unit 410 sets "voice", "speech", and "speech" as attribute information according to each of the "medium", "mode", and "destination" of the information to be recorded in the minutes. Outputs the minutes memo data in which the attribute information indicating each of "everyone" is added to the voice text data.

The mode confirmation unit 412 confirms the mode for the chat data acquired by the chat acquisition unit 406 using the instruction data acquired by the instruction acquisition unit 402, and adds information indicating the mode of the confirmation result to the chat data. It is a functional unit that outputs to the minutes memo recording unit 420A. Whether the mode to be confirmed by the mode confirmation unit 412 is the chat data of "speaking" to all the attendees or the chat data of "confidential" with the instructed attendees. Confirmation mode can be mentioned. When the mode confirmation unit 412 indicates that the instruction data is chat data of "speaking" to all the attendees, the mode is set to "speaking" and the destination is set to "all". Further, when the mode confirmation unit 412 indicates that the instruction data is "confidential" chat data with the attendees, the mode is set to "confidential" and the destination is the "attendee" instructed by the instruction data. And. Specifically, the mode confirmation unit 412 indicates "chat", "speech" or "confidentiality", and "everyone" or an instruction as attribute information by each of "medium", "mode", and "destination". Outputs the minutes memo data in which the attribute information indicating each of the "attendees" indicated by the data is added to the chat data.

The minutes memo recording unit 420A is a functional unit that adds the minutes memo data generated by the minutes memo generation unit 400A to the minutes 45G of the meeting. That is, the remarks of the attendee A input by the conference terminal 2A and the exchange of information by the attendee A with other attendees are recorded in the minutes 45G as the minutes memo.

Therefore, the voice data of the attendee A is converted into text data by the voice / text conversion unit 410, and recorded (added) in the minutes 45G by the minutes memo recording unit 420A. Attribute information by "medium" = "voice" / "mode" = "speech" or "secret story" / "destination" = "everyone" or "attendee" is added to the text data.

The chat data of attendee A is added with attribute information by "medium" = "chat" / "mode" = "speech" or "secret story" / "destination" = "everyone" or "attendee", and the minutes. It is recorded (added) in the minutes 45G by the memo recording unit 420A.

Since each of the minutes memo generation units 400B to 400D and the minutes memo recording units 420B to 420D has the same configuration as the minutes memo generation unit 400A and the minutes memo recording unit 420A, detailed description thereof will be omitted.

With the above configuration, the voice data of each of the attendees A to D is recorded (added) in the minutes 45G by adding the attribute information to the text data converted into the character information. In addition, the chat data of each of the attendees A to D is also recorded (added) in the minutes 45G by adding the attribute information to the chat data.

In this way, by converting the voice data of each attendee into text in the video conference server 4, each voice of the conference terminal 2 can be individually converted into text, and the conversion accuracy can be improved. That is, it is possible to faithfully record each of the statements made by a plurality of attendees during the meeting in the minutes.

In addition, information exchanged between some attendees during the meeting can also be recorded in the minutes. Furthermore, regarding the exchange of information between some attendees during the meeting, the intention of the parties is reflected, for example, the exchange of information is recorded in the minutes of the meeting in a concealed state (mode). Is possible.

Next, as the information processing method of the present disclosure, the minutes recording process of the information processing executed by the video conference server 4 will be described.

FIG. 7 shows an example of the flow of the minutes recording process executed by the video conference server 4. FIG. 7 shows an example of the flow of the minutes recording process when the minutes related to the voice of the attendees are recorded on the conference terminal 2.

When the video conferencing server 4 receives the audio data of the attendees output from the conferencing terminal 2, the CPU 41 executes the processing routine shown in FIG. 7.

In step S100, the CPU 41 acquires the audio data of the attendees output from the conference terminal 2. The process of step S100 corresponds to the function of the voice acquisition unit 404 (any of the minutes memo generation units 400A to 400D) shown in FIG. Next, the CPU 41 confirms in step S110 that the attendees of the conference terminal 2 have the right to speak, and synthesizes the confirmed voice data acquired from the conference terminal 2 in step S100 as the voice data of the speaker. Judgment of whether to do (that is, perform a synthesis judgment). In step S110, if there is a right to speak, the process shifts to step S120, and if there is no right to speak, the process returns to step S100. The process of step S110 corresponds to the function of the qualification determination unit 408 (any of the minutes memo generation units 400A to 400D) shown in FIG. Next, in step S120, the CPU 41 performs voice-to-text conversion on the voice data to be synthesized in step S110. The process of step S120 corresponds to the function of the voice / text conversion unit 410 (any of the minutes memo generation units 400A to 400D) shown in FIG. Next, in step S130, the CPU 41 records the minutes and ends the processing routine. The process of step S130 corresponds to the function of the minutes recording unit 420 (any of the minutes memo recording units 420A to 420D).

FIG. 8 shows an example of the flow of the minutes recording process when the minutes related to the chat by the attendees are recorded on the conference terminal 2.

When the video conferencing server 4 receives the chat data from the attendees from the conferencing terminal 2, the CPU 41 executes the processing routine shown in FIG.

In step S200, the CPU 41 acquires the chat data by the attendees output from the conference terminal 2. The process of step S200 corresponds to the function of the chat acquisition unit 406 (any of the minutes memo generation units 400A to 400D) shown in FIG. Next, in step S210, the CPU 41 acquires the instruction data from the conference terminal 2 acquired by the instruction acquisition unit 402 from the instruction acquisition unit 402, and determines whether or not the chat data is in a mode indicating confidentiality. .. In step S210, if the instruction data from the conference terminal 2 indicates the confidentiality of the chat data, it is affirmed, and the process shifts to step S230. On the other hand, if the instruction data from the conference terminal 2 indicates the remark of the chat data, it is denied, and the process proceeds to step S220. The process of step S210 corresponds to the function of the mode confirmation unit 412 (any of the minutes memo generation units 400A to 400D) shown in FIG. In step S220, the CPU 41 generates a minutes memo in which the chat data is used as speech data (data to which attribute information including "chat", "confidential", and "attendee" instructed by the instruction data is added). , The process shifts to step S240. Further, in step S230, the CPU 41 generates a minutes memo in which the chat data is concealed data (data to which attribute information including "chat", "speech", and "everyone" is added), and processes the chat data in step S240. To migrate. The processing of steps S210 to S230 corresponds to the function of the mode confirmation unit 412 (any of the minutes memo generation units 400A to 400D) shown in FIG. Next, in step S240, the CPU 41 records the minutes by adding the minutes memo generated in step S220 or step S230 to the minutes. The process of step S240 corresponds to the function of the minutes recording unit 420 (any of the minutes memo recording units 420A to 420D) shown in FIG.

FIG. 9 shows an example of minutes in which voice data and chat data by attendees from the conference terminal 2 are recorded.

FIG. 9 shows an example of the minutes in which all the minutes memos are arranged in chronological order for the voice data and chat data by each of the attendees A to D. For example, the minutes memo 70 regarding the remark made by attendee A at 10:00 is the time data of "10:00" triggered by the remark time and "the meeting is ready". The minutes memo consisting of the text data of the above and the attribute information is recorded in the time zone of the meeting "10: 00: 00-10: 14: 59". In the attribute information of the minutes memo, information indicating "medium" = "voice" / "mode" = "speech" / "destination" = "everyone" is stored.

Further, in the example shown in FIG. 9, the area of the time zone including the minutes memo in which the minutes memo is kept secret by the instruction from the attendee A is surrounded by the dotted line 72. For example, the minutes memo 71, which was spoken by attendee A to attendee C at 10:12:30, includes time data of "10:12:30" triggered by the time of speech and "current task is The minutes memo 71 consisting of the text data of "Issue 2" and the attribute information is recorded in the time zone "10: 00: 00-10: 14: 59". In the attribute information of the minutes memo 71, information indicating "medium" = "chat" / "mode" = "secret story" / "destination" = "C" is stored.

FIG. 10 shows an example of the case where the minutes shown in FIG. 9 are represented by the minutes memo with a typical icon.

That is, in the example shown in FIG. 10, for the voice data and chat data for each attendee, an example of the minutes in which all the minutes memos are expressed in a time-series manner or a secret statement can be displayed is shown. There is. For example, the minutes memo 70 shown in FIG. 9 is arranged at the time “10:00:00” as an icon 700 surrounding the “departure” symbol. The text data of "There is a problem of K in the product" is associated with this icon 700. Further, as the attribute information of the text data, information indicating "medium" = "voice" / "mode" = "speech" / "destination" = "everyone" is also associated.

Further, in the example shown in FIG. 10, the minutes memo 71 shown in FIG. 9 is arranged at the time “10:12:30” as an icon 710 surrounding the “secret” symbol. The icon 710 is associated with an icon 712 (an icon surrounded by a dotted line in the example shown in FIG. 10) in which attribute information is briefly described. In FIG. 10, the destination is stored as the attribute information briefly described by the icon 712. From these icons 710 and 712, it is possible to intuitively grasp that the attendee A has sent a secret message (secret story) to the attendee C.

Next, the minutes provision process of the information processing executed by the video conference server 4 will be described.

FIG. 11 shows an example of the flow of the minutes providing process executed by the video conference server 4.

When the video conferencing server 4 receives the data from the conferencing terminal 2, the CPU 41 executes the processing routine shown in FIG.

In step S300, the CPU 41 determines whether or not the data from the conference terminal 2 is the request data for requesting the provision of minutes, and if the received data is not the request data, makes a negative determination and ends this processing routine. do. On the other hand, if a positive judgment is made in step S300, the process proceeds to step S310. In step S310, the CPU 41 acquires the information indicating the conference terminal 2 or the information (ID) indicating the attendees included in the request data, and identifies the attendees who requested the provision of the minutes. Here, any attendee from attendee A to attendee D is identified. Next, in step S320, the CPU 41 generates minutes for presentation to the attendees specified in step S310, that is, the attendees who requested the provision of minutes. Then, in the next step S330, the minutes for presentation generated in step S320 are transmitted, and the present processing routine is terminated.

For example, if the attendee who requested the provision of minutes is attendee A, the minutes are generated so that the minutes memo to which a confidential message (secret story) is sent / received about attendee A can be displayed, and the minutes of attendee A can be displayed. Send to the conference terminal 2A. In this case, the minutes shown in FIG. 9 or 10 are transmitted.

When the minutes representing the minutes memo with the representative icon shown in FIG. 10 are transmitted, the detailed data of the minutes memo associated with the representative icon may be linked to the icon so as to be displayed.

FIG. 12 shows an example of the minutes linked to the icon so that the detailed data of the minutes memo associated with the typical icon can be displayed.

For example, the minutes memo data is associated with the icon 700 indicating the remark by the attendee A. Therefore, the video conference server 4 convenes so that when the icon 700 is instructed by the conference terminal 2A, the detailed data of the minutes memo 70 is displayed as the detailed data of the icon 700 as shown in FIG. The minutes are provided including the data 700A that can be displayed on the terminal 2A. This makes it possible to provide the minutes that enable the detailed data of the minutes memo to be confirmed from the minutes that are simply displayed by a typical icon.

In addition, icons 710 and 712 (FIG. 10) indicate that attendee A has sent a confidential message (secret story) to attendee C. When these icons 710 and 712 are instructed, as shown in FIG. 12, a meeting is displayed so that a figure (a figure with a diagonal background in the example shown in FIG. 12) indicating that the message is a secret message (secret story) is displayed. The minutes can be provided including the data 710A that can be displayed on the terminal 2A. By doing so, the attendee A can clearly grasp that it is a secret message (secret story).

On the other hand, when the attendee who requested the provision of the minutes is attendee B, it is preferable to hide the minutes memo to which a confidential message (secret story) is sent / received about attendee A. Therefore, the video conference server 4 generates the minutes in which the minutes memo in which the attendee B is not involved is hidden, and transmits the minutes to the conference terminal 2B of the attendee B. In this case, the method of hiding the minutes memo that attendee B is not involved in is a non-display mode that generates the minutes with the minutes memo deleted so that the minutes are not displayed, and other predetermined figures. There are other graphic display modes for embedding.

FIG. 13 shows an example of the minutes in which the minutes memo in which the attendee B is not involved is hidden by the other figure display mode in which another predetermined figure is embedded.

In the example shown in FIG. 13, instead of the icons 710 and 712, an example of the icon 714 in which the “?” Symbol is surrounded by a dotted line is shown. In order to set the non-display mode, the icon 714 may be deleted so as not to be displayed. By displaying the icon 714, the attendee B can confirm that some kind of interaction has occurred during the meeting.

In the example shown in FIG. 13, when the icon 714 is instructed by the conference terminal 2B, as shown in FIG. 14, a figure (FIG. 14) indicating that the icon 714 is a secret message (secret story) without displaying the minutes memo. In the example shown in, the minutes can be provided so that the diagonal background figure) is displayed. By doing so, attendee B can at least confirm that some kind of interaction has occurred during the meeting.

Further, when generating the minutes corresponding to the minutes shown in FIG. 9, the data displayed in the dotted line 72 is deleted as shown in FIG. 15 (in the example shown in FIG. 15, the minutes are deleted by inserting the diagonal line figure. Do).

As described above, according to the video conferencing server 4 that functions as an information processing device in the information processing system 1 of the present embodiment, the audio data of each attendee is converted into text, and the audio of each of the conference terminals 2 is converted. Can be individually converted to text, and conversion accuracy can be improved. That is, each of the statements made by a plurality of attendees during the meeting can be faithfully recorded in the minutes.

Further, according to the video conferencing server 4 of the present embodiment, information exchanged between some attendees during the conference can also be recorded in the minutes. In addition, regarding the exchange of information between some attendees during the meeting, the minutes should be recorded to reflect the intentions of the parties, for example, to indicate that the exchange of information is concealed (mode). be able to.

In the present embodiment, the case where the voice data is converted into text data for each of the conference terminals 2A to 2D and added to the minutes and stored is described, but the technique of the present disclosure is a text in which the voice data is converted. The data is not limited to being recorded in a file such as minutes. For example, as shown in FIG. 16, the converted text data may be video-synthesized in real time and provided to the conference terminal.

Further, each of the conference terminals 2A to 2D in the present embodiment may be any terminal capable of participating in the conference, and of course, it may be a conference terminal installed at a distant base.

[Second Embodiment]

In the first embodiment, regarding the exchange of information between some attendees during the meeting, chat data is used to reflect the intention desired by the parties, for example, the exchange of information is concealed (mode). I explained the case of recording in the minutes to show that. The second embodiment records the audio in the minutes, reflecting the intentions of the parties. Since the second embodiment has the same configuration as the first embodiment, the same parts are designated by the same reference numerals and detailed description thereof will be omitted.

FIG. 17 shows an example of the functional configuration of the video conferencing server 4 in the present embodiment.
The difference between the functional configuration of the video conferencing server 4 shown in FIG. 17 and the functional configuration of the video conferencing server 4 shown in FIG. 5 is mainly a state (mode) in which the audio data from the audio acquisition unit 404 is concealed. The point is to show that and record it in the minutes. Specifically, instead of the mode confirmation unit 412 in which the chat data from the chat acquisition unit 406 is input, the mode confirmation unit 413 in which the voice data from the voice acquisition unit 404 is input is provided.

Here, since the chat acquisition unit 406 is a processing unit that handles chat data (chat data), the mode confirmation unit 413 further includes a voice / text conversion unit 414. The voice / text conversion unit 414 is a functional unit that converts voice data input via the qualification determination unit 408 into text and outputs it to the minutes memo recording unit 420A. However, unlike the voice / text conversion unit 410, the voice / text conversion unit 414 adds "speech" and "everyone" indicating that the voice data is a remark to all the attendees in the text-converted voice text data. It does not have a function to add the indicated information and output it.

In the present embodiment, the audio data acquired by the audio acquisition unit 404 is input to the qualification determination unit 408 of the video conferencing server 4 via the mode confirmation unit 413.

Each functional unit of the minutes generation unit 400 and the minutes recording unit 420 includes minutes memo generation units 400A to 400D and minutes memo recording units 420A to 420D for each of the plurality of conference terminals 2.

The minutes memo generation unit 400A and the minutes memo recording unit 420A record the remarks of attendee A input by the conference terminal 2A and the exchange of information by attendee A with other attendees as minutes memos. Record to 45G.

The minutes memo generation unit 400A includes an instruction acquisition unit 402, a voice acquisition unit 404, a qualification determination unit 408, a voice / text conversion unit 410, and a mode confirmation unit 413.

The instruction acquisition unit 402 acquires instruction data indicating the attendees at the transmission destination for the voice data from the conference terminal 2A and an instruction as to whether or not to keep the voice data confidential, and outputs the acquired instruction data to the mode confirmation unit 413. do.

The voice acquisition unit 404 acquires the voice data transmitted from the conference terminal 2A, and outputs the acquired voice data to the mode confirmation unit 413.

The mode confirmation unit 413 confirms the mode for the voice data acquired by the voice acquisition unit 404 using the instruction data acquired by the instruction acquisition unit 402, and adds information indicating the mode of the confirmation result to the voice data. This is a functional unit having a function of outputting the minutes memo recording unit 420A and a function of outputting voice data to the qualification determination unit 408.

Specifically, when the mode confirmation unit 413 includes information indicating that the instruction data from the instruction acquisition unit 402 keeps the voice data confidential, the mode confirmation unit 413 adds attribute information to the voice data and adds the minutes memo recording unit 420A. Output to. In this case, attribute information indicating each of "voice", "secret story", and, for example, "attendee C" is added to the voice text data as attribute information by each of "medium", "mode", and "destination". The minutes memo data is output. On the other hand, when the instruction data does not include information indicating that the voice data is kept secret, the mode confirmation unit 413 outputs the voice data to the qualification determination unit 408.

Since each of the minutes memo generation units 400B to 400D and the minutes memo recording units 420B to 420D has the same configuration as the minutes memo generation unit 400A and the minutes memo recording unit 420A, detailed description thereof will be omitted.

With the above configuration, the audio data of each of the attendees A to D is recorded (added) in the minutes 45G, reflecting the intention desired by the parties (see also FIG. 18).

FIG. 18 shows an example of the case where the minutes memo is represented by the same typical icon as the minutes shown in FIG. 9 according to the present embodiment. As shown in FIG. 18, it is possible to record the voice data in the minutes as well, indicating that the exchange of information is concealed (mode).

As described above, according to the present embodiment, the voice text data in which the voice data of each attendee is converted into text in the video conference server 4 is recorded in the minutes 45G by reflecting the intention desired by the parties. Is possible. Specifically, regarding the voice data exchanged by voice between some attendees during the meeting, it reflects the intention desired by the parties, for example, the state (mode) in which the exchange of information is concealed. It can be displayed and recorded in the minutes.

[Other forms]

In the above, the technique of the present disclosure has been described in detail for a specific embodiment, but the technique of the present disclosure is not limited to such an embodiment, and various other techniques are within the scope of the technique of the present disclosure. It is possible to take embodiments.

Further, in the above-described embodiment, the processing performed by executing the program stored in the auxiliary storage device has been described, but the processing of the program may be realized by hardware.

Further, the processing in the above embodiment may be stored as a program in a storage medium such as an optical disk and distributed.

In the above embodiment, the processor refers to a processor in a broad sense, and is a general-purpose processor (for example, CPU: Central Processing Unit, etc.) or a dedicated processor (for example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA). : Field Programmable Gate Array, programmable logic device, etc.).
Further, the operation of the processor in the above embodiment is not limited to one processor, but may be performed by a plurality of processors existing at physically separated positions in cooperation with each other. Further, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

1 Information processing system 2 Conference terminal (2A, 2B, 2C, 2D)
4 Video conference server 5 Voice-to-text converter 6 Network 20 Computer body 26 Bus 28C Camera 28K Operation input unit 28M Microphone 29 Display unit 40 Computer body 45 Auxiliary storage device 45D Data 45G Minutes 45M Conference system program 45P Information processing program 45T Table 46 bus

Claims (8)

Transmission information transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals, identification information indicating a first conference participant who operates the first conference terminal, and , An acquisition unit that acquires destination information indicating a second conference participant in which the first conference participant operates one or more second conference terminals indicating a destination of the transmission information.
A generation unit that generates minutes information using the acquired transmission information, identification information, and destination information as remark information of the first conference participant.
A control unit that controls storage of the generated minutes information in the storage unit each time the minutes information is generated.
Information processing device with. The information processing device according to claim 1, wherein the transmission information includes at least one of sound information and character information input to the first conference terminal. The acquisition unit is a secret identification indicating that the transmission information is transmitted from the first conference terminal only to the second conference terminal to which the first conference participant is the transmission destination of the transmission information. Get information,
The generation unit generates information including the transmission information, the identification information, the secret identification information, and the destination information as the minutes information as the remark information of the first conference participant, claim 1 or The information processing apparatus according to claim 2. The control unit
Receives an instruction to output the minutes information stored in the storage unit from any of the conference terminals of the plurality of conference terminals.
When the conference terminal that has received the instruction is included in the first conference terminal and the second conference terminal, the minutes information is controlled to be output as information that can be displayed by any of the conference terminals. ,
When the conference terminal that has received the instruction is not included in the first conference terminal and the second conference terminal, the minutes information is predetermined different from the information that can be displayed by any of the conference terminals. Controls the output of information,
The information processing device according to claim 3. The control unit
Receives an instruction to output the minutes information stored in the storage unit from any of the conference terminals of the plurality of conference terminals.
When the conference terminal that has received the instruction is included in the first conference terminal and the second conference terminal, the minutes information is controlled to be output as information that can be displayed by any of the conference terminals. ,
Control to output the minutes information as information that cannot be displayed by any of the conference terminals when the conference terminal that has received the instruction is not included in the first conference terminal and the second conference terminal. I do,
The information processing device according to claim 3. With multiple conference terminals
The information processing device according to any one of claims 1 to 5.
Information processing system equipped with. It is an information processing method in an information processing device.
The information processing device
Transmission information transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals, identification information indicating a first conference participant who operates the first conference terminal, and , An acquisition step of acquiring destination information indicating a second conference participant in which the first conference participant operates one or more second conference terminals indicating a destination of the transmission information.
A generation step of generating minutes information using the acquired transmission information, identification information, and destination information as remark information of the first conference participant, and
A control step that controls storage of the generated minutes information in the storage unit each time the minutes information is generated, and
Information processing method having. Computer,
Transmission information transmitted from a first conference terminal among a plurality of conference terminals that exchange information with other conference terminals, identification information indicating a first conference participant who operates the first conference terminal, and An acquisition unit that acquires destination information indicating a second conference participant in which the first conference participant operates one or more second conference terminals indicating the destination of the transmission information.
A generation unit that generates minutes information using the acquired transmission information, identification information, and destination information as remark information of the first conference participant.
A control unit that controls storage of the generated minutes information in the storage unit each time the minutes information is generated.
An information processing program to function as.

Images