JP2019153099A - Conference assisting system, and conference assisting program - Google Patents

Abstract

To collect and store data with high quality that is appropriate for analyzing post a conference without impacting real-time processing during the conference, concerning a communication conference held over communications amongst a plurality of locations.SOLUTION: A conference assisting system (30) embedded in an edge server 20 and the like comprises: tagging means for attaching a tag including identification information for identifying the location of a source to information history data in at least one format (audio and the like); adoption means for adopting the location of an information source to be integrated for each data having a different source to which the tag is attached by the tagging means; and integration means for chronologically integrating the saved data of the information source adopted by the adoption means, creating integrated history data in one format therefrom, and saving thereof. Fundamentally, the location of the information source is deemed as the location of the source.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a conference support system and a conference support program.

A system that displays audio as text in a communication conference system that transmits and receives video, audio, and the like is useful for understanding the contents of the conference. It is used for the purpose of understanding the contents in order to prevent difficult to hear voice assistance and communication errors. It is also effective in making up for lack of language skills, especially when holding meetings overseas. Recently, the number of remote work has increased, the number of conferences such as web conferences has increased from the conference range that was held in domestic conference rooms and conference rooms, and the choice of communication terminals such as smartphones and tablets has increased. It is supposed to be performed not only under the network, but also connected to any environment and any device.
In addition, with the development of AI and other means, conference support systems that store and analyze conference information in a lump and analyze it and make it useful for subsequent conferences become realistic, and efforts to improve the productivity of conferences themselves are progressing. It is out.
Japanese Patent Laid-Open No. 2004-151858 describes that processing is received, extracted and analyzed with respect to the content of a conference, and metadata of the conference is created and included in the report.

JP 2017-91536 A

There are two points that must be realized as a conference in the background of such communication conference technology.
The first point is that the meeting at the meeting proceeds smoothly. It is desirable that the conference be established without interruptions such as video, audio and text. The quality of speech is greatly affected by the network environment and may deteriorate on the listener side, so it is required to convert it to text where there is a speaker. If a large amount of processing is performed, the load on the terminal side increases, and the real-time property of the conference itself may be lost.
Second, considering the post-conference analysis, it is desirable that the quality of the data used for the analysis is high. Good analysis is not possible even if it is difficult to hear speech or text documents using the data.
If emphasis is placed on real-time performance, the quality of the data will be degraded, making it impossible to use for analysis, and if trying to obtain optimal data, the real-time performance of the conference cannot be ensured.

  In Patent Document 1, processing is performed step by step for the purpose of analysis, and in particular, it can be seen that processing is divided according to the concept of real-time processing and post-processing. There is a problem. Real-time processing needs to ensure real-time performance, but post-processing requires analysis based on higher-quality data in order to improve the accuracy of analysis results. Processing accuracy will be reduced.

  The present invention has been made in view of the problems in the prior art described above, and in a communication conference conducted via communication between a plurality of bases, it is possible to perform analysis after the conference without affecting real-time processing at the time of the conference. The task is to collect and store suitable high-quality data and to improve the analysis accuracy of the content of the meeting.

The invention according to claim 1 for solving the above-described problem is a conference support system for supporting a communication conference performed by communicating information between a plurality of bases.
Tagging means for adding a tag including identification information for identifying a source site to the history data of the information in at least one form;
Adopting means for adopting the base of the information source to be integrated for each data with a different source from which the tag is added by the tagging means;
A meeting support system comprising: integration means for creating and storing the integrated history data of the one form by integrating the storage data of the information source adopted by the adoption means in time series.

  The invention according to claim 2 has a server that collects history data from the plurality of bases, and the meeting support system according to claim 1, wherein the tagging unit, the selection unit, and the integration unit are mounted on the server. It is.

  A third aspect of the present invention is the conference support system according to the second aspect, wherein the server is an edge server provided at any of the bases, and includes a meeting content analysis unit based on the integrated history data.

  According to a fourth aspect of the present invention, the primary storage destination of the historical data targeted by the tagging means is the server or another storage device, and in the case of another storage device, from the other storage device 4. The conference support system according to claim 2, wherein the server collects history data stored in the other storage device.

The invention according to claim 5 comprises conversion means for converting the information from the first form to the second form,
The tagging means adds a tag to the history data of the first form and / or the second form, and the integrating means is the integrated history of the first form and / or the second form. The conference support system according to claim 1, wherein data is created and stored.

  The invention described in claim 6 is the conference support system according to claim 5, wherein the first form is voice and the second form is text.

  The invention according to claim 7 is the conference support system according to claim 5, wherein the tagging unit adds the tag to the text in units of speech when converting the text to the second form.

  According to an eighth aspect of the present invention, the conversion means for converting the voice that is the first form into the text that is the second form is an apparatus provided at a speaker-side base from which the voice is collected, The conference support system according to claim 6 or 7, wherein, among the devices provided in the listener side base, in consideration of the processing capability and communication state of the device, the device of the base device having the optimum conversion accuracy is applied.

  The invention according to claim 9 is the meeting support according to claim 6 or 7, wherein the conversion means is also mounted in the mounting device of the tagging means, and voice history data is collected in the device and then converted into text. System.

  A tenth aspect of the present invention is the conference support system according to the fifth aspect, wherein the first form is audio, image, video or text.

  The invention according to claim 11 is the conference support system according to claim 5, wherein the second form is a full text of a text, a translation or an excerpt thereof, a voice, an image or a video.

  The invention according to claim 12 is the invention according to any one of claims 1 to 11, wherein the history data targeted by the tagging means is collected at one time partially or sequentially during the meeting or after the meeting. The conference support system described.

  A thirteenth aspect of the present invention is the conference support system according to the twelfth aspect, wherein the collection timing is a predetermined timing or a timing determined by the environment.

  The invention according to claim 14 is the conference support system according to any one of claims 1 to 13, wherein the base of the information source is the base of the transmission source.

  According to the fifteenth aspect of the present invention, the history data targeted by the tagging unit is a voice, and the tagging unit identifies the base of the transmission source by identifying the input microphone of each speech voice, and the tag The conference support system according to any one of claims 1 to 14, to which is added.

  A sixteenth aspect of the present invention provides the conference support system according to any one of the first to fifteenth aspects, wherein the integration by the integration means is executed partially or collectively at a time during or after the conference. It is.

  The invention according to claim 17 is the conference support system according to claim 16, wherein the integration timing is a predetermined timing or a timing determined by an environment.

  The invention according to claim 18 is the conference support system according to any one of claims 1 to 17, wherein the tagging unit adds the tag to history data collected from at least one site. It is.

  The invention described in claim 19 is a conference support program for causing a computer to function as the conference support system according to any one of claims 1 to 18.

  According to the present invention, high-quality data suitable for post-conference analysis is collected and stored without affecting real-time processing at the time of communication in a communication conference performed via communication between a plurality of bases. The analysis accuracy of the meeting contents can be improved.

It is a schematic diagram which shows the outline | summary of the video conference system which the conference assistance system which concerns on one Embodiment of this invention supports. It is a functional block diagram which shows the principal part of the meeting assistance system which concerns on one Embodiment of this invention. It is a flowchart which shows the process from the meeting start by the meeting assistance system which concerns on one Embodiment of this invention to meeting log collection. It is a flowchart which shows the process from tagging by the meeting assistance system which concerns on one Embodiment of this invention to integrated historical data preparation. It is a schematic diagram of history data (log) of each base integrated by the conference support system according to an embodiment of the present invention, and integrated history data.

  An embodiment of the present invention will be described below with reference to the drawings. The following is one embodiment of the present invention and does not limit the present invention.

  As shown in FIG. 1, a meeting of two people in the meeting room of the base A, three persons in the meeting room of the base B, and one person in an open space outside the base C is assumed. The site A is a host, and the sites A and B are connected to the TV conference from the TV conference system in the conference room via the in-house network. The base C uses a public line with a smartphone to connect to the conference system and participates in the conference. The communication terminal 11 at the base A, the communication terminal 12 at the base B, and the communication terminal 13 at the base C are each provided with information input / output devices such as a video display device, a speaker, a camera, a microphone, a keyboard, and a mouse. Each of the communication terminal 11, the communication terminal 12, and the communication terminal 13 inputs information (video, audio, key input text, image file, document file, etc.) and transmits it to other terminals. Output. Thereby, a communication conference is performed by communicating information between the plurality of bases A, B, and C.

There is an (edge) server 20 at the site A, and the host conference support system 30 is incorporated in the server 20 (a conference support program is installed). As shown in FIG. 2, the conference support system 30 includes a conference support processing unit 40 and a conference analysis processing unit 50. The conference support processing unit 40 includes an audio text conversion unit (conversion unit) 41 that converts the audio of the conference into text. In addition, a process determination unit 42 that acquires the state of the conference and determines at which base the text conversion is executed, and a conference log collection unit 43 that collects the conference log of each base in the server 20 are included.
In addition, the conference analysis processing unit 50 includes a conference log integration unit (tagging unit, selection unit, integration unit) 51 that integrates collected conference logs into one, and a conference log analysis unit that analyzes a conference from the conference log after the conference (Analysis means) 52.
The conference support system for clients is a system that receives a command from the host and supports the conference. The conference support system collects conference logs and can convert voice texts depending on the case. The application is installed and working.

3 and 4 show flowcharts of the processing.
At the start of the conference, each site starts to acquire the conference log (accumulation of history data) (S1), and the process determination unit 42 of the conference support processing unit 40 determines the status of each site conference (network speed, capability of the processing device at the site). ) Is confirmed (S2).
When the TV conference system is used, the voice text conversion process is performed in the base (for example, equivalent to the case of S3 → S4 for the base B), and the text is distributed to various places almost simultaneously with the voice.
When connected from a PC, tablet, or smartphone, check the network status of the Wi-Fi environment or 3G or 4G communication and the communication speed at that time, check the processing capability of the device, and there will be no time lag Thus, it is determined whether the processing is performed at the base A or the base C (S3). Further, in order to reduce the processing load, the site C can reject the text conversion process by itself and entrust it to the other party (site A) (S3 → S5). That is, the conversion means for converting the voice of the first form into the text of the second form is a device provided at the speaker side base (C) from which the voice is collected, and other listener side bases (A, Among the devices provided in B), the device at the base with the optimum conversion accuracy is applied in view of the processing capability and communication state of the device.

  Here, the conference log will be described with an example including the audio during the conference and the text information, that is, an example including the audio history data as the first form and the text history data as the second form. Logs are not limited to this, but also include conference videos, images, minutes, and translations and edits thereof. The information in the first form corresponds to at least sound, image, video or text. The information in the second form corresponds to at least the whole text of the text, translation or an excerpt (summary) thereof, sound, image or video.

  After the conference ends (YES in S6), the conference log collection unit 43 collects the conference log (history data) stored at each site on the server 20 at the site A (S7). The collection from the base C is executed when the base C is in the Wi-Fi environment. Depending on the environment, it may be collected simultaneously during the meeting, or the collection timing may be changed for each site. The historical data targeted by the tagging means may be collected at a time during the meeting or after the meeting partially sequentially or collectively. The collection timing may be a predetermined timing, or may be a timing determined by the environment each time.

Reference is also made to FIG.
When the history data of all the bases is prepared, the conference log integration unit 51 performs the integration work. The tagging means of the conference log integration unit 51 first attaches a speaker base tag (a tag including identification information for identifying the base of the transmission source) T to each utterance voice with respect to the voice history data 100 of the base A. (S8). You can refer to the text data that keeps the utterance from the base as a way to attach the tag T. That is, when converting the text to the second form, add the tag to the text in the utterance unit. You may use this.
Further, it may be a method of distinguishing and storing the sound from the terminal microphone or the sound from the conference partner when recording the sound. That is, the form of the history data targeted by the tagging means is a voice, and the tagging means is a method of identifying a source site by adding an input microphone for each speech and adding a tag.

Here, the adoption means of the conference log integration unit 51 adopts the information source base to be integrated as the base of the transmission source for each data with different transmission sources to which the tag is added by the tagging means.
The integration means of the conference log integration unit 51 uses the portions 102, 105, and 108 with the tag T (B) of the base B in the voice history data 100 accumulated and created at the base A as the voice history data 201 of the base B. Replace with 202 and 203. Further, the integration means of the conference log integration unit 51 replaces the portions 103 and 106 with the tag T (C) of the site C with the audio history data 301 and 302 of the site C, and attaches the tag T (A) of the site A. The part is left as it is, and all voices are replaced with data acquired at the speaker's base (S9-S11) and integrated into one file (integrated history data 400) (S12). The conference log analysis unit 52 uses the integrated file 400 and performs analysis when necessary.

  The integration work described above may be executed partially or sequentially at the same time during or after the meeting, and the integration timing may be a predetermined timing or may be determined by the environment each time. Good.

The example in which the voice is replaced with the one acquired at each base (that is, the case where the information source is the transmission source) has been described, but a method of using the acquired log (history data) in a good state is also possible. .
An example of collecting and integrating logs (history data) at all sites has been explained. However, in the case of a conference with an outside customer who does not have the same system due to guest participation, collect the conference logs from all sites In that case, the log of the host site may be used, or the best state of the acquired voice may be judged (the data tagged with site C is used as the site). For example, when collecting from A). In addition, voice, image, and text data acquired by another method in the same base may be collected and integrated.
When it is determined that the quality of text used as a reference for tagging at the time of integration is poor, the text may be re-textured using high-quality audio information. In this case, in this embodiment, the conversion means (41) is also mounted on the server 20, which is a mounting apparatus of the tagging means. Therefore, after the voice history data is collected in the apparatus 20, the conversion means (41) Convert.

Although an example of collecting and integrating audio history data has been shown so far, it is also possible to collect and integrate images, videos, and text data. Audio integrated history data and text integrated history data may be created and stored as a set.
The text data does not need to include the entire meeting, and you can omit the digression at the beginning, the words before or after translation, and duplicate explanations.
As another example, even if a TV conference is used, such as in a meeting with an overseas base, if the equipment or network condition is poor, the text conversion is determined by the processing determination unit 42 in the TV conference system and the processing location is determined.
The form of the meeting may be a web meeting. Moreover, although the flow was demonstrated using the example of a video conference, the communication conference only of an audio | voice may be sufficient.

11 Communication terminal 12 Communication terminal 13 Communication terminal 20 Server 30 Conference support system 100 Audio history data 200 Audio history data 300 Audio history data 400 Integrated history data A, B, C Base T Tag

Claims (19)

A conference support system for supporting a communication conference in which information is communicated between a plurality of bases,
Tagging means for adding a tag including identification information for identifying a source site to the history data of the information in at least one form;
Adopting means for adopting the base of the information source to be integrated for each data with a different source from which the tag is added by the tagging means;
A meeting support system comprising: integration means for creating and storing the integrated history data of the one form by integrating the storage data of the information source adopted by the adoption means in time series. The conference support system according to claim 1, further comprising a server that collects history data from the plurality of bases, wherein the tagging unit, the selection unit, and the integration unit are mounted on the server. The conference support system according to claim 2, wherein the server is an edge server provided at any of the bases, and includes a meeting content analysis unit based on the integrated history data. The primary storage destination of the historical data targeted by the tagging means is the server or other storage device, and in the case of another storage device, the server is transferred from the other storage device to the other storage device. The conference support system according to claim 2 or 3, wherein the stored history data is collected. Conversion means for converting the information from the first form to the second form,
The tagging means adds a tag to the history data of the first form and / or the second form, and the integrating means is the integrated history of the first form and / or the second form. The conference support system according to claim 1, wherein data is created and stored. The conference support system according to claim 5, wherein the first form is voice and the second form is text. The conference support system according to claim 5, wherein the tagging unit adds the tag to the text in units of speech when converting the text to the second form. The conversion means for converting the voice of the first form into the text of the second form is an apparatus provided at a speaker side base from which the voice is collected, or an apparatus provided at another listener side base. The conference support system according to claim 6 or 7, wherein the apparatus of the base at the optimum conversion accuracy is applied in view of the processing capability and communication state of the apparatus. The conference support system according to claim 6 or 7, wherein the conversion means is also mounted on the mounting device of the tagging means, and voice history data is collected in the device and then converted into text. The conference support system according to claim 5, wherein the first form is audio, image, video, or text. The conference support system according to claim 5, wherein the second form is a full text of a text, a translation or an excerpt thereof, a voice, an image, or a video. The meeting support system according to any one of claims 1 to 11, wherein the history data targeted by the tagging unit is collected at one time during a meeting or after the meeting, partially sequentially or collectively. The meeting support system according to claim 12, wherein the collection timing is a predetermined timing or a timing determined by an environment. The conference support system according to claim 1, wherein the base of the information source is the base of the transmission source. The form of the history data targeted by the tagging unit is a voice, and the tagging unit identifies the source site by adding an input microphone for each speech and adds the tag. Item 15. The meeting support system according to any one of Items 14 to 14. The meeting support system according to any one of claims 1 to 15, wherein the integration by the integration unit is executed partly sequentially or collectively at a time during or after the meeting. The conference support system according to claim 16, wherein the integration timing is a predetermined timing or a timing determined by an environment. The meeting support system according to any one of claims 1 to 17, wherein the tagging unit adds the tag to history data collected from at least one site. A conference support program for causing a computer to function as the conference support system according to any one of claims 1 to 18.

Images