TWI795759B - Online meeting system - Google Patents

Abstract

The present invention aims to improve the convenience of the online conference system. The online conference system constructed by the server transmits video data including audio to each other between multiple terminals connected to the network to realize online conferences. In the meeting, as shown in Figure 6(a), a participant requests a direct dialogue with participant A. If the request is approved, connection 56 is displayed as shown in Figure 6(b), and the dialogue with participant A becomes unavailable for transmitting to other participants. In addition, in the conversation of all members, other participants can send stickers for the speaker, so that they can convey their reactions and feelings to the speaker without hindering the speech. Furthermore, after the meeting, analysis results such as speaking time and speaking ratio can be displayed. These functions can be used to enhance the convenience of the online conference system.

Description

Online Conference System

The present invention relates to an online conference system that is connected to a plurality of terminals through a network to realize an online conference among the aforementioned terminals.

In the online conference system, multiple terminals are connected through the Internet, and audio data and video data are transmitted between the terminals, thereby realizing online conferences. The video of other participants can be displayed on the terminal of each participant in the online meeting and the audio can be heard. By applying this system, conferences and conversations can be held between multiple people.

Various ideas have been proposed to improve the convenience of the online meeting system. For example, Patent Document 1 discloses a technique for introducing artificial intelligence into an online conference to perform voice recognition, report creation, sentiment analysis, and the like. Patent Document 2 discloses a technique for outputting sound from a direction corresponding to the position of a participant's icon on the screen by adjusting the left and right volume of stereo sound during an online conference. Patent Document 3 discloses a technique for urging the user to input text when the noise volume in the web conferencing system is above a certain level. [Prior Art Literature] [Patent Document]

[Patent Document 1] Japanese Patent Laid-Open No. 2008-63699 [Patent Document 2] Japanese Patent Application Laid-Open No. 6-175942 [Patent Document 3] Japanese Patent Laid-Open No. 2019-117998

[Problem to be Solved by the Invention]

However, there is still room for improvement in various aspects of the online conference system. The purpose of the present invention is to further improve the convenience of the online meeting system. [Means to solve the problem]

In the present invention, as a first configuration, it may be an online conference system, which is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, and includes a transmission control unit that communicates between the plurality of aforementioned terminals. Transmission of audio data and video data, among which, The aforementioned transmission control unit performs limited transmission when receiving a limited dialogue instruction in the aforementioned online conference, so as to prohibit the transmission of the audio data sent and received between the specific terminals in the state of being played by other terminals; the limited dialogue instruction is from any of the aforementioned A terminal designates a part of one of the aforementioned plurality of terminals including the terminal as a specific terminal, and a dialogue is performed only between the specific terminals.

In the previous online conference system, when multiple people hold an online conference, they can only choose to set it to a state where no one can hear their own speech, or set it to a state where all other participants in the conference can hear it. Although there is also a function for the host of an online meeting to divide into several conference rooms and have a conversation in each conference room, the speeches in each conference room are the same. However, in real meetings, there is not necessarily only a scene for all members to speak. For example, there are occasions when whispered consultations are conducted with nearby participants. In particular, in inter-company meetings, etc., there may be cases where only participants within the company speak after consultation. Previous online conference systems could not cope with this requirement. Also, recently, an online meeting system may be used to hold a meeting, but in an actual meeting using a non-online meeting system, several groups may be formed among participants and several conversations may be mixed. Participants can freely chat with various participants by joining one conversation and joining another conversation at will. The previous online conference system also cannot establish a dialogue with only a part of these participants according to the wishes of each participant. On the other hand, according to the present invention, in an online meeting, when any participant issues a limited conversation instruction from his own terminal, audio data sent and received between specific terminals will not be transmitted in a state that can be played by other terminals. That is, a session can be established only between specific terminals. Therefore, for example, in an online meeting, negotiations and the like can be performed only among some of the participants. In addition, at a party, etc., it is possible to divide into a plurality of groups and chat freely. Furthermore, in a real meeting or gathering, it is often limited to have a dialogue with people nearby, but the online conference system has the advantage of being able to establish a dialogue without being restricted by such a place. That is, according to the first configuration, it is possible to provide an advantage that actual meetings and gatherings do not have in that space restrictions can be avoided. Prohibition of transmission in a state where it can be played by other terminals includes the following methods: a method of prohibiting the transmission of audio data itself; a method of transmitting audio data with a tag, etc., that prohibits playback on the prohibited terminal. Also, when video data and audio data are integrated, audio-containing video data and audio-free video data may be generated and used separately.

In the first configuration, for example, limited transmission can be established at the point in time when one terminal transmits a limited session instruction to another terminal. In addition, the limited transmission may be established at the time of obtaining consent from the terminal as the recipient after the limited session instruction is transmitted.

In the first configuration, video data and audio data can also be limitedly distributed. In this way, it is possible to avoid the situation of the conversation conducted by the limited transmission from being seen by other participants. In addition, as another aspect, video data may be transmitted to all participants as usual. In this way, during the limited transmission, the screen to be transmitted to other participants will not disappear, so the limited transmission can be used for conversation without causing other participants to feel uncomfortable.

In the first configuration, various methods can be used to designate a specific terminal. For example, it may be a method of individually designating the participants participating in the online conference, that is, each terminal used by the participants.

In addition, the designation of a specific terminal can also be the following: Contains a conference database for storing group data pre-grouped with one of the aforementioned terminals, The transmission control unit, when receiving the limited session instruction targeting the group, regards a terminal belonging to the group as the specific terminal.

In this way, it is possible to easily perform limited dialog instructions on a group basis. In the above aspect, the limited transmission in the group may be established at the same time as the limited conversation instruction is transmitted, or the limited transmission may be established only when the consent of each participant in the group is obtained. In this case, limited transmission may be established when consent is obtained from all participants in the group, or limited transmission may be established only among participants who have obtained consent from some of them. In the above aspect, various groups can also be preset. In this way, a group can be selected for limited transmission for each scene of an online meeting.

In the first configuration, the following aspects may also be adopted: During the aforementioned limited transmission period, audio data from terminals other than the aforementioned specific terminal are also transmitted to at least a part of the aforementioned specific terminal.

In this way, sounds from other terminals can also be heard during limited transmission. That is, the overall progress of the online meeting can be grasped even in the limited transmission. The audio data in the above-mentioned aspect may be set to be the same as the volume between specific terminals, or may be set to be lower than this volume. Also, the volume can be adjusted on the side of the receiving terminal separately from the specific terminal.

In the first configuration, the following aspects may also be adopted: When the transmission control unit accepts a new limited session instruction by designating a part of the specific terminal from a terminal other than the specific terminal during the limited transmission period, it asks all the specific terminals to perform a process regardless of whether it is specified by the limited session instruction. Whether to approve this qualified dialog directive.

When limited transfers are in progress, new participants may arise wishing to participate. In this case, if the participation is approved inadvertently, there is a possibility that some participants in the transmission may continue the conversation without recognizing a new participant, and there may be a possibility that the contents of the conversation may be leaked to an unexpected person. On the other hand, in the above-mentioned aspect, when accepting a limited dialogue instruction from a new participant for a terminal that is performing limited transmission, it will ask all members in the limited transmission whether to approve participation, so it can be based on the recognition of all members. The above-mentioned problems can be avoided by admitting new participants on the Internet. In the above-mentioned aspect, the new entry may be admitted when approval is obtained from all terminals that have already made inquiries, or may be admitted at the stage of obtaining partial approval. Also, when only partial approval is obtained, the designation of the unapproved specific terminal can also be canceled.

In the first configuration, the following aspects may also be adopted: It includes a screen generating unit for generating a screen on which an indication icon for designating the terminal as the specific terminal is superimposed on the video data from the terminal.

In this way, images of other participants transmitted from other terminals are displayed on each person's terminal, and an indicator icon is displayed in the image at the same time. Therefore, by operating the instruction icon, it is easy to individually select participants, and it is easy to send a limited conversation instruction. The indication icon may be, for example, a button for selection. Also, by making the entire image clickable, it can be utilized as an indicator icon.

In the first configuration, the following aspects may also be adopted: It includes a screen generating unit that, when performing the limited transmission, pairs images from the specific terminal with each other on the screen displayed on the terminal, and simultaneously generates a screen displaying a cancel icon that can cancel the limited transmission. instruct.

In this way, images of other participants transmitted from other terminals are displayed on each person's terminal, and at the same time, images such as lines that pair specific terminals targeted for limited transmission are displayed, so that it is easy to identify who is being connected with Participants engage in conversations with limited teleportation. In addition, the image of the line and the like also displays a cancel icon for canceling the specific transfer, so by this operation, the limited transfer with the specific participant can be easily and accurately canceled, and the operability is improved. As an image for pairing specific terminals, for example, a line connecting these terminals may be used, or an aspect such as an outer frame surrounding these terminals may be used.

In the present invention, as a second configuration, it may be an online meeting system, which is connected to a plurality of terminals through a network to realize an online meeting between the aforementioned terminals, and includes: a transmission control unit, between the plurality of aforementioned terminals Mutual transmission of audio data and video data; and a screen generating unit superimposed on the video data of the aforementioned terminal, and displaying volume icons for individually specifying the volume of the terminal.

In the previous online conference system, when multiple people are in an online conference, the voices from other participants can only be heard at the same volume. However, online conference systems are sometimes used for various purposes, and recently, parties are sometimes held using online conference systems. In a real party, several groups are formed among the participants, and conversations on various topics are mixed. Also, when the self is interested in other topics while participating in any one topic, it is possible to participate in conversations on other topics. In the conventional online conference system, the conversations of other participants are heard at the same volume, so it is difficult to establish a conversation in this situation. On the other hand, according to the second configuration, it is possible to freely adjust and listen to the speeches of the participants in the online meeting. Therefore, it is possible to participate in conversations without hindrance even when various conversations are mixed. In addition, the second configuration also has an advantage in that it is only necessary to adjust the volume from other terminals in one's own terminal, and therefore it is not necessary to obtain permission from other terminals, which is very convenient in this point.

The second configuration can be realized in various aspects. For example, when the audio data from each terminal is not integrated in the online conference system but transmitted as independent data, it is only necessary to independently play the audio data from each terminal in each terminal. In this way, the volume of each sound data can be easily adjusted according to the indication in the volume icon. In the case of using a smart phone, tablet, computer, etc. as a terminal, sometimes the so-called operating system (OS, Operating System) provides the function of individually playing multiple sound data and adjusting the volume, so in this case, as In the second configuration, it is only necessary to provide the volume icon with a function of providing a volume instruction to the OS based on the operation result. Also, as another aspect, it may be played after synthesizing the audio data from each terminal at the volume indicated by the volume icon. This composition can be performed in the online conference system or in the receiving terminal.

In the present invention, as a third configuration, it may be an online conference system, which is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, which includes: a transmission control unit, between the plurality of aforementioned terminals Mutual transmission of audio data and video data; and a screen generation unit receiving a response from any of the aforementioned terminals to any other terminal, and displaying the response on the other terminal.

In a real meeting, a participant making a speech guesses whether or not he or she agrees with his or her speech from the facial expressions or attitudes of other participants. The same is true in the online conference system, and it is possible to infer from the facial expressions of other participants displayed on one's own terminal whether or not one agrees with one's speech. However, in the online conference system, there are limited things that can be grasped from the images of other participants displayed on the own terminal. In the past, in the online conference system, there was no way to effectively grasp whether or not other participants agreed with one's speech. On the other hand, according to the third configuration, each participant of the online meeting can individually transmit a "response" from one terminal to another terminal. In this way, the reaction of other participants to the speech during the speech is displayed on the terminal of the participant. Therefore, the speaker can easily and objectively grasp the reactions of other participants to his speech. Also, conversely, it can be said that the third configuration is also advantageous for participants other than the speaker. That is, in a real meeting or a conventional online meeting system, one's own "reaction" cannot be conveyed to the speaker who is speaking, but it can be realized according to the third configuration. For example, if you think a speech is good, you can show the response of "approval" to the speaker without interrupting the speech. From this point of view, it can also be said that the third configuration provides advantages unique to the online conference system.

In the third configuration, "reaction" can take various forms, including the following forms: The aforementioned responses are information represented by pre-prepared symbols or characters.

As a pre-prepared mark, for example, a mark expressing an affirmative opinion, a mark expressing disapproval, a mark expressing an unexpected or surprising reaction, etc. can be used. Various stickers applied to SNS etc. can also be utilized. As long as these symbols are applied, selecting any one can convey the reaction with simple operation. In addition, as long as it is set to be able to send text, specific opinions can be conveyed to the speaker.

In the third configuration, various aspects can be considered for the interface for transmitting logos or characters. For example, a list of prepared logos may be displayed on the speaker screen displayed on the own terminal, and the logo may be transmitted by selecting any one of them. In addition, a list of logos may be displayed outside the area where images of each terminal are displayed, and any one may be selected and dragged to the screen of the speaker.

In the third configuration, the following aspects may also be adopted: An emotion analysis unit is included, which analyzes the emotion of the terminal user according to the image data from the terminal; the aforementioned reaction is the result of the emotion analysis.

In this way, even if each participant does not perform any operation, a response can be obtained. As a method of emotion analysis, various techniques for analyzing emotions from facial expressions can be applied. The results of sentiment analysis can also be expressed by pre-prepared signs, characters, and scores such as approval and disapproval, for example. For example, it may also be expressed by using the brightness or color of the screen.

Various aspects can be considered in the content and utilization method of sentiment analysis. For example, when analyzing emotions from expressions, you can focus on their sight. It can evaluate whether the user's eyes leave the screen, and judge the user's favorability for the speech according to the number or frequency. The favorability of the speech is preferably fed back to the speaker in the above-mentioned various forms. For example, the line of sight can be judged by the position of the pupil in the image captured of the user's face. When the pupil is located within a predetermined range around the eyes, it is recognized as leaving the screen. In this case, the camera of the user terminal is not necessarily located at the center of the screen. Therefore, when starting an online conference, the pupil position of the center of the screen can be stored as a reference position in advance. A method of judging that the pupil is off the screen when the pupil moves away from the initial position by a predetermined distance may be employed. In addition, it is also possible to determine whether the user is dozing off based on the extent to which the user's eyes are opened or the shaking of the head. In this case, it can also be regarded as an indication that the speech is boring and fed back to the speaker, and a warning can also be issued to wake up the user. Furthermore, since such emotional analysis can be performed on all online meeting participants, for example, when the proportion of participants who are judged to be lacking in concentration such as dozing off increases, the moderator or all participants can be urged to take a break. display etc.

In the third configuration, the following aspects may also be adopted: The screen generating unit superimposes and displays responses from other terminals on the video data of the terminal user.

In this way, each end user can know how other participants react to them. In an online conference, not only one's own image but also images of each participant are displayed on one's own terminal, and reactions to each participant can be displayed on images of other participants. In this way, each participant's reaction to other participants can be compared with each participant's reaction to himself.

Also, in the third configuration, the following aspects may also be adopted: The above-mentioned screen generating unit superimposes and displays the response from other terminals to the above-mentioned terminal on the video of the other terminal of the sender of the response.

Display images of each participant on your own terminal. In the above aspect, when a participant sends a response to himself, the response is displayed on the participant's video. In this way, it is easy to know who sent the response to itself.

In the present invention, as a fourth configuration, it can be an online conference system, which is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, and includes: an analysis and processing unit for counting the users of each of the aforementioned terminals One or both of speaking time and silent time; and a screen generating unit that displays the aforementioned statistical results on the aforementioned terminal.

In a meeting, the speeches of the participants sometimes favor some of the participants. However, although the conventional online conference system can transmit audio and video among the participants to carry out the conference, it cannot objectively grasp the actual performance of the conference. On the other hand, according to the fourth configuration, one or both of speaking time and silent time can be counted for each terminal. Therefore, the degree of each participant's speech can be grasped. As a result, it can help to improve the quality of the meeting.

In the fourth configuration, only one's own statistical results may be displayed, or other people's statistical results may be displayed together. In addition, the display of statistical results can be performed during the online meeting, or as a review after the online meeting. The content of statistics may be, for example, the time of each utterance, the average utterance time, the total utterance time, and the like. The same applies to the time of silence.

In the fourth configuration, the following aspects may also be adopted: The analysis processing unit presents the time of each utterance of the terminal user.

In this way, each participant can judge the tendency of the length of his own speech. As a result, the content and quality of speech can be improved. Also, for example, as long as too long speeches can be shortened, other speakers can also be given a chance to speak, which helps to improve the overall quality of the meeting. In the fourth configuration, the speech time may be presented during the online meeting. In this way, the quality of speeches and the quality of online meetings can be improved at any time. In the fourth configuration, the speaking time for each time may be presented in a format that can be compared with other participants. In addition, the presentation may also be made by using the distribution form of the speaking time and the number of times of speaking. In this manner, it is possible to grasp in more detail the tendency of utterances, such as the tendency to repeat short utterances many times or the tendency to make fewer long utterances.

In the fourth configuration, the following aspects may also be adopted: The analysis and processing unit calculates the ratio of speaking time from each terminal in the online conference.

In this way, in the online meeting, the balance of the speeches of the participants can be grasped, which helps to improve the overall quality of the online meeting. The proportion of speaking time can be calculated in various forms. For example, the ratio of speaking time to the overall online meeting time can be obtained. In addition, it is also possible to obtain the ratio in the total speech time. Furthermore, in order to exclude simple replies, etc., only utterances with a predetermined length or longer may be used when calculating the utterance time.

In the present invention, as a fifth configuration, it may be an online conference system, which is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, and includes: a voice data input unit for inputting voice data from the aforementioned terminals ; a keyword extracting unit, extracting keywords contained in the audio data; and a screen generating unit, corresponding to the keywords and the audio data, and displaying them on the terminal.

In the past, in the online conference system, it was possible to record the utterances in the conference as audio data in advance. However, if it is desired to confirm such utterances later, it is necessary to listen to all of the contents again, and the burden is heavy. On the other hand, according to the fifth configuration, keywords can be extracted from the voice data of utterances, and can be displayed in association with the voice data. Therefore, the content of each utterance can be inferred from the keyword afterwards, and the efficiency of making meeting minutes or checking audio data can be realized. It is also possible to store the extracted keywords and audio data in correspondence with each other.

In the fifth configuration, keywords can be extracted by various methods. For example, artificial intelligence can be used. Also, as another method, the following method can also be used: voice data is converted into text data through voice recognition, words are extracted through morphological analysis, and keywords are extracted by sorting the extracted words according to predetermined rules. .

Keyword extraction can be performed at various times. For example, if keywords are extracted in an online meeting and presented to all participants, it will help to understand the content of the speech. Also, when keywords are extracted after the online meeting, more accurate extraction can be performed according to the time required for the processing.

In the present invention, as the sixth configuration, it can be an online conference system, which is connected to multiple terminals through the network to realize the online conference between the aforementioned terminals. The start time and end time of each speech of the terminal are used as conference data; and the screen generation unit, based on the conference data, displays a Gantt chart drawn by dividing the speech period according to each terminal one by one according to the time axis.

In the meeting, improving its efficiency and quality is often taken as a topic. As one of them, it is important to grasp the performance of the speeches of the participants in the meeting. However, in the previous online conference system, it did not take much consideration to grasp the performance of speeches in the conference. On the other hand, according to the sixth configuration, the start time and end time of each utterance are stored for each utterance, and it is possible to distinguish each terminal, that is, each participant using it, and display it in a Gantt chart. In this way, it is possible to visually grasp which participant in the conference speaks at what frequency and for how long. As a result, it is possible to urge a participant who has made a long speech to conclude a speech, or a participant who does not speak at all to make a speech, thereby improving the quality of the overall conference. In the sixth configuration, the above-mentioned Gantt chart may be displayed during the meeting or after the meeting.

In the sixth configuration, the following aspects may also be adopted: The screen generating unit may emphatically display a part of the speech of the other terminal that overlaps with the speech of the one terminal when another terminal starts to speak while the one terminal is speaking.

In this way, it is possible to visually and objectively grasp where the other person's speech starts before it ends. Speaking in this manner deprives others of the opportunity to speak and may prevent the smooth and fair conduct of the meeting. Therefore, by visualizing the inappropriate utterance, it is possible to improve the efficiency and quality of the meeting. Emphasis can be displayed in various ways, such as accenting with color, adding hatching and other patterns, forming a thick frame, and blinking.

In the sixth configuration, the following aspects may also be adopted: Responses from other terminals are paired with the utterances and stored in the conference database, and the screen generation unit displays the responses in pairs with the utterances.

In this way, it is possible to visually and objectively grasp the reactions from other participants when each presentation is made. Therefore, it is possible to determine whether each speech is approved or not, and there is an advantage that it is easy to extract important speeches when reviewing the online meeting. Regarding the reaction, as described above, various forms such as logos, characters, and results of sentiment analysis can be used.

In the sixth configuration, the following aspects may also be adopted: The said screen generation part associate|corresponds and displays the keyword of each speech specified by the said conference database, and this speech.

In this way, the content of each utterance can be inferred from keywords, and when reviewing an online meeting, it is easy to extract important utterances such as utterances for which audio data content should be confirmed. Various methods described above can be applied to the keyword extraction. Keyword extraction can be performed at the time point of drawing the Gantt chart, or before drawing the Gantt chart, and the results are stored in advance.

The present invention does not necessarily have all the above-mentioned various features, and some of them may be appropriately omitted or combined. In addition, the present invention may also be configured as an online conference method for realizing an online conference using a computer in addition to the configuration of the above-mentioned online conference system. In addition, the present invention may also be configured as a computer program for causing a computer to realize a function for realizing an online conference, and a computer-readable recording medium in which the computer program is recorded.

The online conference system as an embodiment of the present invention will be described in the following order. A. System composition: B. Direct dialogue function: C. Conference facilitation function: D. Meeting review function: E. Effect:

A. System composition: FIG. 1 is an explanatory diagram showing the configuration of an online conference system. The online conference system 100 of the embodiment is composed of a server connected to the Internet, and can transmit images and sounds between multiple terminals 10 also connected to the Internet, thereby realizing online conferences and online gatherings ( The following includes both referred to as online meetings) and so on. As the terminal 10, a tablet, a smart phone, a mobile phone, a personal computer, a notebook computer, and other devices capable of connecting to the Internet can be applied. The number of terminals participating in the online meeting is not limited.

The terminal 10 is a computer with a CPU and a memory inside, and the functions shown in the figure are constructed by software through computer programs. Part or all of the functions in the figure can also be constructed by hardware. The transceiver unit 11 functions to send and receive information through the Internet. In this embodiment, information such as video data and audio data in the online meeting is included. The voice input unit 12 inputs the voice of the user, that is, the participant of the online conference, using a microphone or the like provided in the terminal 10 . The image capturing unit 13 inputs images of its users, that is, participants of the online conference, using a camera or the like included in the terminal 10 . The command input unit 16 inputs various commands from a touch panel, a keyboard, and the like of the terminal 10 . Instructions include not only the image display mode of the online meeting, volume adjustment, etc., but also instructions related to the inherent functions in this embodiment. The display control unit 17 controls the screen display on the display of the terminal 10 . In the present embodiment, images and the like received from other terminals 10 are displayed during the online meeting. The audio output control unit 18 controls audio output from the terminal 10 . In the present embodiment, the audio and the like received from other terminals 10 during the online conference are output.

The terminal 10 is equipped with a browser 14, and the browser 14 executes the online meeting screen and other data generated by the online meeting system 100, thereby providing various functions for the online meeting. The browser 14 has a built-in volume adjustment unit 15 . The volume adjustment unit 15 provides a function of individually adjusting volumes received from other terminals 10 using an interface displayed on the browser 14 during an online conference. That is, when the user operates the volume adjustment interface displayed on the browser 14, the volume adjustment unit 15 functions to output volume adjustment instructions to the audio output control unit 18 according to the operation result. Instead of the browser 14 and the volume adjustment unit 15 , a dedicated application for online conferences may be installed in the terminal 10 .

The online conference system 100 is constituted by a server equipped with a CPU, a memory, etc., and each function shown in the figure is constructed with software by a computer program. Part or all of the functions in the figure can also be constructed by hardware. In addition, each function of the online conference system 100 is not limited to be provided by one server, but may also be provided in a distributed system by using a plurality of servers. Each function is explained below.

Three databases are provided in the online meeting system 100 . The structure of each database will be described in detail later. The user database 170 records information such as the user's name. Also record the participation history in the online meeting. The meeting database 171 is for making records for each online meeting, and recording the participants and other information. The video database 172 records the video data sent from each terminal 10 during the online meeting. In this embodiment, video data and audio data are integrated, so audio is also recorded in the video database 172 . The video data and the audio data can be composed of different files. In this case, the video data and the audio data are paired and recorded in the video database 172 .

The transceiver unit 110 functions to send and receive information through the Internet. In this embodiment, information such as video data and audio data in the online meeting is included. The video input unit 140 inputs video data from the terminal 10 . In this embodiment, audio data is also input together.

The transmission control unit 120 transmits data related to the online conference, such as video data and audio data, among the terminals 10 participating in the online conference. The transfer control unit 120 performs transfer control in two senses. In the embodiment, multiple online conferences may be held simultaneously, and multiple terminals 10 participating in each conference are connected. The first control process of the transmission control unit 120 is to classify the online conferences and control the transmission only between the terminals 10 participating in the online conferences. The second control process is the transfer control within each online conference. That is, basically, data is sent and received by all the terminals 10 participating in the online meeting, but in the characteristic function of this embodiment described below, the transfer between some terminals is prohibited. This control is performed as transfer control within the online meeting.

The sticker processing unit 150 functions to send and receive stickers between the terminals 10 . Stickers are graphics used to easily express opinions and impressions such as approval or disapproval of other participants' speeches in an online meeting. Textual information may also be included. The screen generator 130 functions to display a screen on the terminal 10 . Specifically, data for displaying a screen on each terminal 10 is generated in HTML or the like. Send this data to the terminal 10 and display it with the browser 14 .

The analysis processing unit 160 functions to analyze speeches and reactions of participants in the online meeting. Analysis can be performed in real-time during the online meeting, and can also be reviewed after the online meeting. As the content of the analysis, for example, the speech time, the speech rate of each participant, and the like are exemplified. Furthermore, the analysis processing unit 160 includes a sentiment analysis unit 161 and a keyword extraction unit 162 . The emotion analysis unit 161 analyzes the reaction to the utterance based on the expressions of the participants of the online meeting obtained from the terminal 10 . Analyzing emotions with facial expressions can be applied to various well-known techniques such as the use of artificial intelligence. In this sense, the sentiment analysis unit 161 can also be realized by using a service provided by a cloud hard disk, for example. The keyword extraction unit 162 analyzes utterances in the online meeting and extracts keywords. Various well-known techniques, such as the use of artificial intelligence, can also be applied to the keyword extraction. Therefore, the keyword extraction unit 162 is the same as the sentiment analysis unit 161, and can also be realized by using a service provided by a cloud hard disk, for example.

The functions of the online conference system 100 and the terminal 10 shown in FIG. 1 are merely examples, and various configurations can be realized. Part of the illustrated functions may be omitted or combined. In addition, it does not matter if a function other than that shown in the figure is provided.

Fig. 2 is an explanatory diagram showing the database structure. The data structure of the user database 170 is displayed on the upper right. The user database 170 creates a record for each user, and a user ID as unique identification information is attached to the record. The records record the user's name, email address and other contact information, as well as the resume of the online meeting attended by the user, etc. In this embodiment, the meeting ID is recorded in the history, and the meeting ID is the identification information attached to each meeting in the meeting database 171 .

The data structure of the image database 172 is shown at the bottom right. In the video database 172, video IDs, which are unique identification information, are attached to video data, respectively. Furthermore, it is also possible to pair and record user IDs, time and date, and the like.

The data structure of the conference database 171 is displayed on the left. The conference database 171 creates a record with unique identification information, that is, a conference ID, for each online conference. Each record records information such as the "title", date and time, start time, and end time of the online meeting. Regarding such information, the user hosting the online conference can register from the terminal 10 .

Also, the user ID of the user database 170 is recorded as the participant of the online meeting. Also considering the situation that multiple people use one terminal 10 in the online meeting, in this embodiment, only the users who log in the online meeting system 100 from each terminal 10 are recorded. In the following description, in order to avoid complicated expressions, it is assumed that one person uses one terminal 10 for description.

"Group" is the information grouping the users participating in the online meeting. Participants in the online meeting can freely register this information. In the example in the figure, two groups, group 1 and group 2, are registered, and the user IDs of the participants belonging to each group are recorded. It is also possible to allow one user to repeatedly register multiple groups. In this embodiment, as described below, there is a direct chat function in which only some participants of the online meeting can chat, and a group can be used to designate conversation partners and the like in this function.

"Speech history (log)" is the speech record of each participant in the online meeting. The time of speech, the user ID of the participant who made the speech and its image data are paired and recorded. Regarding the video data, the video ID of the video database 172 is recorded. As long as you refer to the speech history, you can play the speeches in the online meeting.

A "sticker journey" is a record of stickers sent and received between participants in an online meeting. Match and record the time, sender, receiver, and content of the stickers sent. Record the user ID of the sender and receiver respectively. As for the content of the sticker, it is only necessary to record an ID indicating the sticker. In the case where text information can be sent and received, the content of the text can also be recorded.

"Transmission data history" is the record of the transmission control results in the online meeting. As described below, in this embodiment, when video data is sent and received between a plurality of terminals 10 participating in an online meeting, not only all terminals can be transmitted, but only some terminals can be transmitted. As the data for controlling which terminals are to transmit video data, the transmission data is used in this embodiment. In the transmission data process, the transmission data and time are recorded in correspondence with each other in advance. In this way, the transmission status of image data at each time point in the online conference can be reproduced.

The data structure shown in FIG. 2 is only an example, and various structures may be adopted. Part of the data shown in FIG. 2 may be omitted, and other data may be recorded in each database.

B. Direct dialogue function: The direct dialogue function implemented in the online conference system of the embodiment will be described. The direct chat function refers to a limited transmission function that allows conversations between only some of the participants in an online conference. For example, when a direct conversation is performed between participant A and participant B, the conversation between AB and AB will not be transmitted to other participants. However, other participants are not restricted from conversations, and conversations between other participants can continue as usual. For example, consider the case where a party is held using an online conferencing system. In real gatherings, most of the conversations between some participants are carried out at the same time, but in the online conference system, the speech of one person is usually transmitted to all members, so it becomes difficult for multiple conversations to be carried out at the same time. In this case, by using the direct chat function, only some of the participants can chat, so the chat in the party can be carried out more smoothly. Also, a case where an online conference system is applied to negotiations between companies is considered. In this meeting, it may be necessary to answer the other party only after negotiating within the company in the middle. Also, even if it is not as exaggerated as the negotiation, during the meeting, sometimes it is desirable to confirm doubts among the partners in advance so as not to affect the progress. In a normal online conference system, such negotiation between peers cannot be performed, but this can be realized by using the direct chat function in this embodiment. The control processing for realizing the direct dialogue function will be described below.

Fig. 3 is a flowchart of transfer control processing. It is mainly the processing performed by the transmission control unit 120 (see FIG. 1 ), and is the processing performed by the server constituting the online conference system 100 on the hardware. This process is repeatedly performed in the online meeting.

When the processing is started, the server inputs video data including audio from each terminal 10 (step S10). Then, specify the delivery object according to the delivery data (step S11). The figure shows an example of transferring data. The transmission data is data defining which terminal the data received from which terminal is to be transmitted to. In the legend, each row represents the sending terminal, and each column represents the receiving terminal. In the row of sending terminal A, only receiving terminal D is "○", so the video data received from terminal A is sent only to terminal D. On the contrary, the image data received from terminal D is only transmitted to terminal A. That is, a state in which a direct dialogue is performed between the terminals AD is formed. During the direct dialogue, the dialogue of other terminals B and C is not restricted. Therefore, the image data received from terminal B is transmitted to terminal C. Also, the video data received from terminal C is transmitted to terminals B and D. FIG. Since terminals AD are in direct dialogue, the dialogue between AD is not transmitted to terminals B and C, but terminals A and D can hear the dialogue of other terminals according to the setting. Since it is set so that terminal D can also hear the dialogue of terminal C, even in the direct dialogue, the video data from terminal C is also transmitted to terminal D. In this way, according to the setting of the transmission data, it is possible to control the state of the normal conversation between all terminals, the state of direct conversation between some terminals, and the like.

According to the result of step S11, the server transmits the image data to each terminal (step S12), stores the transmitted data in the conference database, and updates the content of the database (step SS13). In the processing of the embodiment, the transmission of video data including audio is controlled, so in the direct dialogue, not only the transmission of audio but also video is restricted. That is, the participants in the direct conversation are in a state where the images of other participants are not displayed on the terminal. As an alternative thereto, it is also possible to apply the direct dialogue only to the audio material. That is, for image data without audio, it can be transmitted between all terminals at any time without relying on the transmission data, and only the audio data can be transmitted and controlled according to the transmission data. In the case of handling video data and audio data as separate data, it is only necessary to perform transmission control by transmitting data separately. On the other hand, in the case of integrally configuring data as video data including audio, two methods can be considered. One is a method in which audio data and video data are separated and separately controlled for transmission. The other method is to transmit both as a whole, but to a terminal that prohibits audio transmission in the transmission data, attach a label prohibiting audio playback to the video data and transmit it. The terminal receives video data that contains sound, but it is tagged, so the sound is deleted and only the video data is played. In this way, substantially the same effect as not transmitting sound can be obtained.

Fig. 4 is a flow chart of transmission data management processing. This process is a process for determining the contents of the transmission data to be controlled in accordance with the operation of the terminals 10 of the participants of the online conference. It is mainly the processing performed by the transmission control unit 120 (see FIG. 1 ), and is the processing performed by the server constituting the online conference system 100 on the hardware. This process is repeatedly performed in the online meeting.

When the process is started, the server detects whether or not the participant operates the screen displayed on each terminal 10 . In particular, in the case of no operation (step S20), there is no need to change the transmission data, so the server does not need to perform any processing to end the transmission data management process. On the other hand, when there is an operation (step S20), processing corresponding to each operation content is performed depending on the content.

When the content of the operation is a request for direct dialogue (step S20), that is, when any participant designates another participant and requests to start a direct dialogue, the following processing is performed. The server identifies the requester and the requested party based on the content of the operation. In this embodiment, there are two request methods, namely, a named request and a group request. A nomination request is a request to individually designate other participants. In this case, the requested party refers to the terminal 10 used by the designated participant. A group request is a request for specifying a group to which a participant belongs. Participants of the online meeting can form a group arbitrarily and register in advance (refer to "group" of the meeting database 171 in FIG. 2 ). A group request specifies anyone in the group and requests a direct conversation. For example, in a meeting between companies, as long as the participants of the same company are preset as a group, a request for direct conversation can be easily sent to the participants in the group without individual designation.

Next, the server adds the object in the direct dialogue to the requested party (step S22). For example, consider a case where a direct talk request is sent from the participant C to the participant A while the direct talk is in progress between the participants AB. At this time, in order for the participant C to participate in the direct conversation, the participant B's approval should also be obtained. From this point of view, even when the requestee of the direct conversation (the participant A in the above example) is already in the direct conversation, the server also adds the object (the participant B in the above example) to the requestee.

The server manages the transmission of direct chat and the result of approval as request data. Therefore, according to the above processing, the server adds a newly sent request, thereby updating the request data (step S23). Then, an approval request button is displayed on the screen of each terminal that is the sender of the request (step S24).

When the content of the operation is the approval of the request or the adjustment of the volume (step S20), the following processing is performed. Approval of the request means that the participant of the terminal displaying the "approval request button" described in the previous step S24 clicks this button and approves it. If the request is granted, the server must alter the send data to establish a direct session. In addition, the adjustment of the volume refers to an operation in which a participant individually adjusts the volume from other participants. If direct chat is set, the voice of other participants will not be transmitted temporarily. For example, when a direct conversation is started between participants AB, it is set so that the voice of other participants will not be transmitted to participant AB temporarily. However, in this embodiment, it is also allowed to listen to the conversation of other participants in the direct conversation. Therefore, when participant A or participant B in the direct conversation adjusts the volume to listen to the conversation of participant C, the server must change the transmission data to transmit participant C's voice. Described below is the processing for changing transmission data in this way.

In addition, when setting up a direct chat, it is also possible to continue sending without deleting the voices of other participants. Participants who have started a direct conversation can turn down or mute the volume of other participants on the endpoint if desired. In this case, there is no need to change the transmission data according to the volume adjustment, so the following processing can also be omitted.

If the processing starts, the server updates the request data according to the operation (step S30). That is, when the direct chat is approved, the record of the corresponding request is retrieved in the request data, and its status is changed to "approved". When adjusting the volume, it has nothing to do with the approval of the request, so there is no need to update the request data.

If the request has not been approved by all members (step S31), the direct dialogue will not be established only among the approved part of the participants, so the server does not perform special processing and ends the transmission data management process. This processing can also be divided into the following cases. For example, when a request is sent to a plurality of participants like a group request in a state where direct dialogue has not yet been established, it may be considered to partially establish direct dialogue at the stage of obtaining approval from a part thereof. On the other hand, when a request is made from a new participant while the direct conversation is already in progress, it can be said that approval must be obtained from all the participants of the existing direct conversation. Therefore, step S31 can also be processed according to the case where a direct dialogue has already taken place and the case where it is not.

When requesting the approval of all staff (step S31), the server updates the transmission data to establish an approved direct dialogue (step S32). When adjusting the volume, it is considered to have been approved by all staff and the same process is carried out. In this way, the transmission of sound is realized according to the operation.

When the content of the operation is to cancel or clear the authorization of the direct dialogue (step S20), the following processing is performed. Cancellation refers to the end of individual direct conversations, and clearing refers to the termination of all direct conversations related to oneself. For example, when participant A is having a direct chat with participants B and C, participant A can individually "dismiss" the direct chat with participant B or participant C, or "cancel" the direct chat with participants B and C. Clear" to end it all at once.

If the processing starts, the server firstly updates the request data according to the operation of canceling or clearing (step S40). In this embodiment, the request data corresponding to the canceled or cleared direct dialogue is deleted. Next, the server updates the transmission data (step S41). In the case of cancellation, the transmission data is updated so that the voice of the terminal related to the designated direct dialogue is also transmitted to other terminals. Also, in the case of clearing, the same process is performed for all related direct sessions. However, this is about the processing of the voice sent from the terminal that instructed cancel or clear, and the gist is not to cancel the direct dialogue between other terminals. In this way, the direct session is canceled or cleared, and a session involving other terminals can be performed.

Fig. 5 is an explanatory diagram showing the concept of transfer control. It shows an example of evolution of transmitted data and requested data. The transmission data D1 on the upper left indicates a normal state, that is, a state in which audio is transmitted between all terminals. In this state, a direct dialogue request is sent from terminal A to terminal D. In this way, the server generates the corresponding request data R1. A→D in the figure means that the sending terminal of the request is A, and the object is D.

Next, the request is approved in terminal D. According to this operation, the server changes the status of the request to "approved" as shown in the request data R2. "○" in the figure indicates approval.

Then, according to the approved request, the transmission data D1 is changed to the transmission data D2, so as to realize the direct dialogue between the terminals AD. That is, as shown in the data D2a, the sound from the terminal A is transmitted to the terminal D only. The columns of terminal B and terminal C become "×", indicating that transmission to these terminals is prohibited. Also, as shown in the material D2d, the sound from the terminal D is transmitted to the terminal A only. Also, when the direct dialogue between the terminals AD has been established, the voices set to the terminals B and C are not transmitted to the terminals A and D for the time being. Therefore, as shown in the data D2b, the transfer from the terminal B to the terminal D is not carried out. The transmission from terminal C to terminal D is also temporarily set to "x". However, when the volume is adjusted to listen to the sound from the terminal C during the operation of the terminal D, the transmission data is set to "◯" as shown in the data D2c.

In this state, a request for direct dialogue is sent from terminal B to terminal A. In this way, the server generates the request data R3 from B to A. Moreover, since the direct session has already been established between the terminals AD, a request from the terminal B is also added to the terminal D which is the target thereof.

If the request is approved by the terminal A, the server sets the request from the terminal B to the terminal A as "approved" as shown in the request data R4. However, in a state where the approval is not obtained from the terminal D, the approval of all members is not obtained (refer to step S31 of FIG. 4 ), and the transmission data D2 is not updated.

In this state, the direct dialogue between terminals AD is canceled. In this way, the request from the terminal B to the terminal A can be approved, so the server changes the transmission data D2 to the transmission data D3 accordingly. That is, a direct dialogue is established between the terminals AB. Also, when the direct dialogue between the terminals AD is established, the voices of the terminals C and D are not transmitted to the terminals A and B either, so as a result, the terminal CD also becomes in the same state as the direct dialogue. In this state, if the direct dialogue between the terminals AB is cancelled, the direct dialogue will not be established between all the terminals, so the transmission data D3 is changed to the state of the transmission data D1, and the transmission between all terminals is carried out.

FIG. 6 is an explanatory diagram illustrating an example of a screen of a direct conversation in the online conference system. Fig. 6(a) shows the state before the establishment of the direct dialogue, and Fig. 6(b) shows the picture in the direct dialogue.

As shown in FIG. 6( a ), in this example, the image of oneself is displayed on the upper left, and the images of participants A to C are displayed in other areas. A button 51 for designating a direct chat request in units of groups is displayed on the self image. By clicking either button, a request can be sent to group 1 or group 2. In addition, buttons 52 each displayed as "direct chat" are displayed on the images of the respective participants A to C. FIG. Simply click this button when requesting a direct chat with individual participants selected. In the example of FIG. 6( a ), since the direct chat request has been sent to the participant A, only the button 52 of the participant A is displayed in reverse.

The volume 53 for adjusting the volume is displayed on the screens of the respective participants A to C. In the online conference, the volume of each participant can be individually adjusted by sliding the volume 53 . In the illustration, participant A's volume is high, participant C's volume is low, and participant B is muted. In the present embodiment, the terminal 10 receives video data each including a participant's voice, and reproduces the voice by playing. The video data of the participants is transmitted as individual data, so the volume can also be adjusted individually. That is, in the terminal 10, the processing of playing a plurality of sounds is performed simultaneously. The online conference system 100 of the embodiment is configured as follows: the playback function of the terminal 10 is instructed to indicate the volume corresponding to the operation of the volume 53 displayed on the screen, thereby realizing individual volume adjustment. The volume adjustment method is not limited to the method according to the function of the terminal 10, and it can also be implemented on the server side. In this case, what is necessary is just to make a server receive the adjustment state of each participant's volume 53 from a terminal, and to send to a terminal in the state which synthesize|combines each participant's voice at the corresponding volume.

When the direct dialogue is established, as shown in FIG. 6( b ), a connection line 56 connecting the participants of the direct dialogue is displayed. In the illustration, it means that you are having a direct conversation with participant A. This connection 56 is also displayed on the screen of participant A's terminal. The screens displayed on the terminals of participant B and participant C are not displayed. In this way, participants B and C can avoid recognizing that they are having a direct conversation with participant A. As a variation, the connection 56 of the direct dialogue may be displayed on the terminals of the unrelated participants B and C, and it is also possible to select whether or not to display it.

A release button 55 is displayed at one end of the line 56 . By clicking this button, the established direct chat can be canceled. When you establish multiple direct conversations in such a way that you can have direct conversations with participants A and C, a link 56 and a release button 55 will be displayed with participants A and C respectively, so you can use the release button 55 Dismiss direct dialogue individually. Also, a button 54 for "clear" is displayed on the own screen. When the button 54 is operated, all direct chats related to oneself can be canceled at once.

When sending a request for a direct chat with oneself, as shown in the screen of the participant B in the figure, the button 57 of "Request" is displayed blinking. By clicking button 57, the request can be "approved". In the embodiment, the operation for rejecting the request is not set, but it can also be set to be rejected.

Through the processing and screen display described above, the online meeting system 100 of this embodiment can realize direct dialogue among some of the participants. The processing and screen display described above are merely examples, and various modifications may be made.

C. Conference facilitation function: Next, the conference promotion function will be described. In this embodiment, there is a meeting promotion function to conduct online meetings efficiently and effectively. The following functions are prepared as meeting promotion functions: a function to urge participants to speak by displaying the degree of each participant's speech; a function to promote speech or provide an opportunity to reconsider thoughts by displaying reactions from other participants during speeches functions etc. These functions are described below.

FIG. 7 is a flowchart of participant screen creation processing. Mainly, the processing performed by the screen generator 130 , the image input unit 140 , the texture processing unit 150 , and the analysis processing unit 160 (see FIG. 1 ) is performed by the server constituting the online conference system 100 on hardware. This process is repeatedly executed while the online conference is in progress.

When the processing is started, the server inputs video data including audio from each terminal (step S50). Then, it is judged from which terminal the video data containing the sound that can be called a speech is sent, thereby specifying the speaker in the online meeting (step S51).

Also, input sticker information from other participants to the speaker (step S52). As sticker information, you can use signs expressing opinions and impressions such as approval, surprise, and disapproval for the speech. In the online conference system 100, these stickers are prepared in advance, and by selecting a part of them, participants can simply send their opinions and impressions to the speech without hindering the speech. Text information from each participant can also be included in the sticker information. In this way, specific opinions, feelings, etc. that cannot be expressed with stickers can be conveyed. Sticker information is not limited to speakers, and can be set to be sent to anyone. However, if stickers can be sent in this state, the purpose of each sticker becomes unclear, and complicated information is exchanged in the online meeting, which may hinder the progress of the meeting. Therefore, in this embodiment, the speaker is specified in step S51, and only the sticker information is sent to the speaker effectively.

Next, the server performs emotion analysis processing of participants other than the speaker (step S53). In this embodiment, emotion analysis is performed based on the expression of each participant. This processing can be performed using the participant images received from each terminal, for example, by applying artificial intelligence. Conventional techniques for sentiment analysis already exist, so they can also be exploited.

As sentiment analysis, for example, it is possible to evaluate whether the user's line of sight leaves the screen, and judge the favorability of the user's speech according to the number or frequency. In addition, it is also possible to determine whether the user is dozing off based on the extent to which the user's eyes are opened or the shaking of the head. In this case, it can also be regarded as an indication that the speech is boring and fed back to the speaker, and a warning can also be issued to wake up the user. Furthermore, since such emotional analysis can be performed on all online meeting participants, for example, when the proportion of participants who are judged to be lacking in concentration such as dozing off increases, the moderator or all participants can be urged to take a break. display etc.

Also, the server calculates the speech time and silence time of each participant (step S54). Speech time refers to the length of the speech. The silent time is the time when no speech is made, that is, the elapsed time since the last speech was finished. The figure schematically shows the calculation method of speaking time and silent time. The hatched part in the figure indicates that a speech is in progress. After participant A makes speech a1, participant B makes speech b1, and after that, speaker A makes speech a2 again. In step S54, since the current situation is indicated, the length of the speech a2 currently being made by the participant A becomes the speech time. The speaking time is gradually increased as long as the speaking continues. If the speaking time is displayed, the participant A can recognize that he has been speaking for too long in the online meeting, and can pay attention to compacting the speech before speaking to improve the efficiency of the meeting. As for the participant B, the elapsed time after the speech b1 is finished becomes the silence time. If no speech is made, the silent time also gradually increases. By displaying the silent time, participant B can be urged to speak.

The server also calculates the speaking rate of each participant (step S55). The speaking rate indicates the ratio of each participant speaking in the online meeting. In this embodiment, the speaking rate is calculated by "cumulative speaking time of each participant/time of online meeting". The time of the online meeting also includes a time of silence for all members. Instead of this, it may be set as "cumulative time of speaking by each participant/total speaking time of all participants". In this way, the ratio of each participant can be grasped only for the speaking time. Also, simple replies are included in utterances, so the utterance rate may be calculated only for utterances that exceed a predetermined length and are judged to describe coherent opinions.

The server generates image data representing the above processing results and sends them to each terminal (step S56). In addition, in this example, a part of each processing performed in steps S52 to S55 may also be omitted.

FIG. 8 is an explanatory diagram showing an example of a screen in the online meeting system. The image of oneself is displayed on the upper left, and the images of participants A to C are displayed in the other areas. Indicators 61 for speaking time and silent time are displayed on the screen of each participant including himself. While speaking by yourself, the speaking time gradually increases as shown in the figure. Conversely, when speech is not being made, the silent time gradually increases as shown by the dotted line. The legend shows the status of one's own speech, so participants A to C show the silent time respectively.

A graph 62 showing the speaking rate of each participant is displayed on the lower right of the own screen. The hatched part in graph 62 represents the speaking rate of oneself. The speaking rate is displayed common to all participants, so it is not displayed on the screens of other participants.

Display stickers 60 on the left side of the own screen. The heart-shaped logo is a sticker that expresses approval for the speech,! A texture that expresses surprise, and a texture that expresses disapproval with NG. These are the stickers that participants A to C passed on to themselves during their speeches. In this way, by displaying the sticker, there is an advantage of being able to know the reaction to one's own statement. The types of textures are not limited to these. In addition, characters sent from other participants may be displayed in any area of the screen. Conversely, you can also send stickers to other participants. Various types of interfaces can be considered for posting stickers. For example, each sticker may be displayed on the screen of each participant, and the sticker may be sent by clicking on the sticker. In addition, it is also possible to display a sticker outside the screen in advance, drag it to the screen of each participant, and send it.

The results of the sentiment analysis are displayed on the screen of each participant. In this embodiment, it is judged in the sentiment analysis that it is not an opinion such as agreeing to, opposing any of the speeches, or not knowing, and the result is indicated by an expression mark 63 . A mark 63 indicating approval of the speech is displayed on the screen of the participant A. The logo of participant C indicates that it is neither of them. The logo can be displayed on any part of the screen, and can also be superimposed on participant B's face as shown in logo 65 .

According to the online conference system of the embodiment, by displaying the display shown in FIG. 8 , each person's speech can be promoted. Also, in a real meeting, it is difficult for the speaker to express his opinion or impression without hindering the speech. However, in this embodiment, this can be achieved by effectively utilizing the sticker. Therefore, discussions can be made more active than actual meetings. Furthermore, there is an advantage of being able to infer the intention of each participant by using sentiment analysis. The screen display in FIG. 8 is only an example, and a part thereof may be omitted, and various displays may be performed.

D. Meeting review function: Next, the meeting review function will be described. The meeting review function is a function that analyzes and displays the speech status of each participant in the meeting after the online meeting ends. By making full use of this function, each participant can comment on his negativity in the meeting, his habit of speaking too long, etc. This evaluation can be used to improve the quality of conducting online meetings. Hereinafter, the processing, display screen, etc. of the meeting review function will be described.

FIG. 9 is a flow chart of meeting review processing. It is used to analyze various information processing after the online meeting is over. This processing is mainly performed by the screen generation unit 130 and the analysis processing unit 160 (refer to FIG. 1 ), and is performed by a server that constructs the online meeting system 100 on hardware. Either user starts the process by activating it.

If the process is started, the server first inputs the user ID of the user who started the process (step S60). Then, a list of conferences attended by the user is displayed, and selection of a conference to be analyzed is accepted (step S61). As illustrated in FIG. 1 , the user database 170 records the history of participating in the online meeting, so it is only necessary to refer to the history corresponding to the user ID input in step S60. An image of the meeting list is shown in the figure. Displays information such as time, date, title, and participants of the meeting that is helpful for selecting a meeting as a review object. The user only needs to click on any one and so on to choose from this list.

If the target conference is selected, the server reads the conference data (step S62). The conference materials are recorded in the conference database 171 . Then, keyword extraction processing is performed (step S63). The content of this processing is described below, and the speech of each participant is recognized, and the keyword contained in it is extracted. Keywords are extracted corresponding to utterances.

The server then executes analysis data generation processing (step S64). In this embodiment, the following four types of data are generated. First, count the distribution of speech time and frequency as speech performance. This can be done for all participants or only for those who are performing this process. Through this statistics, the participants can know how long most of the speeches are made, and can know whether they have habits such as long speeches. If all the participants are counted, by comparing with other participants, you can know your speaking time tendencies more objectively. Second, as a reaction analysis, calculate "number of stickers/number of speeches" for each sticker type. In this way, it is possible to know the tendencies of other participants' opinions and impressions such as approval, surprise, and disapproval of one's statement. Reaction Analysis In addition to this, you can also perform various analyzes that can evaluate your own speech, such as the number of times you get the most "approve" or "disagree" stickers, etc. The third is the speaking ratio of each participant. This is the same process as that described above in step S55 of FIG. 7 , and the final results are obtained after the meeting is over. By calculating the speaking ratio, you can evaluate the degree to which you actively participate in online meetings. The fourth is comprehensive evaluation. Comprehensive evaluation can be obtained by various methods. For example, the first to third analysis results may be converted into points based on each standard, and the points may be obtained by summing them up. Alternatively, the comprehensive evaluation may be divided into elements such as positivity and speech length, and the first to third analysis results may be divided into points for each element. In addition, it can be obtained by various methods. A comprehensive evaluation does not necessarily have to be a theoretically "reasonable" evaluation. As long as an objective comprehensive evaluation defined by a predetermined benchmark is displayed for each online meeting, the user can be motivated to improve the quality of the meeting.

The server generates image data representing the above analysis results, and outputs to the terminal (step S65). In this embodiment, two types of screens, a Gantt chart and an analysis report, are prepared. Of course, the output screen is not limited to this. The user can switch these screens as long as he instructs display switching (step S66). The above is the content of the review and processing of the meeting.

FIG. 10 is a flowchart of keyword extraction processing. This corresponds to the processing of step S63 of the conference review processing (FIG. 9). If the processing starts, the server inputs the audio data (step S70). In the case of using video data containing audio, only the audio part may be extracted therefrom. Then, a voice recognition process is performed based on the voice data (step S71). There are known technologies for voice recognition, so it is only necessary to use the known technologies. Next, perform morphological analysis on the text data obtained from the voice recognition, and extract words (step S72). The morpheme analysis can also use known techniques. Candidate words as keywords are extracted in this way.

Next, the server excludes non-candidate words (step S73). Non-candidate terms are terms that are unlikely to be keywords even if they occur frequently in online meetings. For example: pronouns, demonstrative pronouns, and words indicating time and date such as today, yesterday, and tomorrow. Non-candidate words only need to pre-register these words as a database. In addition, the operator can also update the database of non-candidate words by confirming the results of keyword extraction.

The server sorts the extracted words (step S74). In this embodiment, sorting is performed based on frequency of use, proper nouns, number of searches on the Internet, keyword database, and the like. Frequency of use refers to the number of times used in online meetings. It is considered that use by many people is more important as a keyword than use by one person, so it can be evaluated not only by the number of times of use, but also by the number of users. Proper nouns mostly include proper nouns that are the subject of online meetings, such as project names and activity names. In addition, proper nouns are likely to be clues to recall or specify the content of the conversation, so they are ranked high. On the other hand, proper nouns may be unsuitable as keywords, such as personal names or place names, and therefore, for example, sorting may be performed only when a common proper noun is used by many people. The number of searches on the Internet refers to keywords that frequently appear in various search engines on the Internet. In this regard, it can be obtained by accessing the website of a search engine, etc. Also, the keyword database is a database of keywords registered in advance by the user. In this way, even if it is a special word, it can be extracted as a keyword without omission. Keyword extraction may use criteria other than the above. In addition, it is also possible to set the priority for a plurality of criteria and apply it.

If words are sorted in this way, the server specifies and outputs its upper predetermined number as a keyword (step S75). This keyword can be utilized for the screen display of the meeting review process (FIG. 9), etc. In addition, the extracted keywords can also be recorded in the conference database 171 in correspondence with the speeches.

Next, an example of a display screen in the meeting review process will be described. In this embodiment, two types of screens, a Gantt chart and an analysis report, are prepared. The following will explain in order.

FIG. 11 is an explanatory diagram showing a display example of a Gantt chart. In the Gantt chart, the horizontal axis is set to time, and the speeches of each participant in the online meeting are displayed in a strip chart. In the illustration, the speeches of four participants A to D are shown. The square speech frame 70 and the like each indicate that a speech is being made. Each speech frame 70 can also be linked with audio data. For example, the speech can also be played when the speech box 70 is clicked, or the sound of the corresponding time can be played according to the position of the speech box 70 clicked. Video data can also be played together with audio data.

During the meeting, and possibly also while others are speaking, another participant speaks. In the illustration, while participant A shown in the speech frame 70 is speaking, participant B starts speaking. In this case, in the present embodiment, when there is a utterance started later, an emphasis display 72 is performed on the portion overlapping with another utterance, as indicated by the hatched portion in the figure. Other hatched parts also indicate overlapping speeches. It cannot be said that it is a better attitude in a meeting to make a speech that overlaps with another's speech, so emphasizing it in this way has the effect of reminding the participant to control such a speech next time.

The sticker 71 sent in the speech is also displayed in each speech. In this way, it is possible to visually grasp how other participants react to each statement, and through the online meeting, it is possible to easily specify a statement considered important.

Also, keywords extracted corresponding to each utterance are displayed on the lower side of the area where the utterance frame of each participant is displayed. In this way, it is easy to recall what topic each speech is about.

Fig. 12 is an explanatory diagram showing a display example of an analysis report. The upper left shows the user's name and the time, date, subject, and participants of the meeting as the analysis target. The bottom left shows the performance of the speech. In this embodiment, utterance lengths are obtained on the horizontal axis, and the number of utterances corresponding to each utterance length is shown in a bar graph. Speech time may be quantized and counted in units of one minute or the like. The average length of one's own speeches is also displayed in the speech performance. It is also possible to display the overall average including other participants. In this way, one can know the tendency of one's speech being too long or too short each time, and can improve the quality of speech in the future. In addition, the actual speech performance also shows the effective speech rate. There are also speeches that cannot be classified as opinions, such as simply replying or echoing. Then, among all the utterances of the self, utterances longer than a predetermined time are regarded as effective utterances, and the ratio thereof is regarded as an effective utterance rate. The time to be used as the basis for effective speaking can be determined arbitrarily.

The reaction analysis results are displayed in the lower center. Responses of other participants to one's own statement expressed in proportion. The legend shows that the frequency of getting the heart logo, that is, agreeing to the sticker, is nearly half. Reaction analysis can be obtained by various methods. For example, if for every speech you make, 2 of the 4 participants agree, 1 disagrees, and 1 has no response, then 50% agree, 25% disagree, and 25% no response. After performing such an analysis for each utterance, an average may be calculated for all utterances. In addition, as another method, it is also possible to obtain the ratio of each opinion by setting the whole map of approval and disapproval as 100%. In this case, the proportion of "no response" is not displayed in the analysis results.

The bottom right displays the speaking ratio of each participant in the online meeting. The part marked with hatching in the figure is the proportion of his own speech. The calculation method is as described in step S55 of the participant screen creation process (FIG. 7).

The comprehensive evaluation of the online meeting is displayed on the upper right. First, it displays a radar chart divided into items such as enthusiasm, speech length, effectiveness, persuasiveness, and coordination. For example, the method of multiplying the number of times of utterances, the effective utterance rate, the proportion of utterances, and the like by a predetermined coefficient and converting them into points can be adopted for positivity. For other projects, it is enough to define the calculation formula corresponding to the project and convert the analysis results of the online meeting into scores. If the scores obtained in this way are plotted, a radar chart can be drawn. Also, a score as a comprehensive evaluation is displayed on the upper right. This score can also be calculated by various methods. For example, the area of the radar chart can also be set as 100 points when each item is a full score, and the score can be determined according to the area of the radar chart actually obtained.

The analysis report is not only about the results of the user's specific online meeting, but also can show the comparison with the results of the user's past online meetings or the comparison with other participants. In addition, it is also possible to count the speeches of all staff participating in the online meeting, etc., and display and summarize the analysis organized by all staff.

In this embodiment, after the online meeting is implemented through the above-mentioned meeting review function, it is possible to review each participant's speech content, participation attitude, speech habit, and the like. Attending the next online meeting based on this review can improve the efficiency of the meeting. In addition, in this embodiment, as shown in FIG. 11 , the responses or keywords of other participants are displayed in correspondence with each other, so that it is easy to recall the contents of the speeches in the online meeting, or to easily find important speeches. In the embodiment, the meeting review process is described as being implemented after the online meeting ends, but this process may be performed at any time during the meeting. In this way, the quality of the conference can be improved while it is in progress.

E. Effect: The foregoing is a description of the embodiments of the present invention. In the online conference system of the embodiment, the usefulness and convenience of the online conference can be further enhanced by functions such as the direct dialogue function, the conference promotion function, and the conference review function. In addition, functions that cannot be realized in actual conferences can be provided, and the quality of the conference itself can be improved. The various features described in the embodiments do not necessarily have to be fully provided, and some of them can also be appropriately omitted or combined. In addition, the present invention can also be constructed in various modified examples other than the embodiments.

[industrial availability] The present invention can be applied to an online conference system, which is connected to multiple terminals through a network to realize online conferences among the aforementioned terminals.

10: terminal 11: Sending and receiving department 12: Sound input part 13: Video Photography Department 14: Browser 15: Volume adjustment department 16: Command input unit 17: Display control part 18: Sound output control unit 51: button 52: button 53: volume 54: button 55:Release button 56: Connection 57: button 60: Textures 61: indicator 62: Chart 63: Identification 65: Identification 70: speech box 71: Textures 72: Emphasis display 100:Online conference system 110: Sending and receiving department 120: Transmission control department 130:Screen generation department 140: Image input unit 150: Texture processing department 160: Analysis and processing department 161: Sentiment Analysis Department 162: Keyword extraction department 170:User database 171:Conference database 172: Image database

FIG. 1 is an explanatory diagram showing the configuration of an online conference system. Fig. 2 is an explanatory diagram showing the database structure. Fig. 3 is a flowchart of transfer control processing. Fig. 4 is a flow chart of transmission data management processing. Fig. 5 is an explanatory diagram showing the concept of transfer control. FIG. 6 is an explanatory diagram showing an example of a screen of a direct conversation in the online meeting system. FIG. 7 is a flowchart of participant screen creation processing. FIG. 8 is an explanatory diagram showing an example of a screen in the online conference system. FIG. 9 is a flow chart of meeting review processing. FIG. 10 is a flowchart of keyword extraction processing. FIG. 11 is an explanatory diagram showing a display example of a Gantt chart. Fig. 12 is an explanatory diagram showing a display example of an analysis report.

51: button

52: button

53: volume

54: button

55:Release button

56: Connection

57: button

Claims (18)

An online conference system is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, which includes a transmission control unit that transmits audio data and video data between the plurality of aforementioned terminals, wherein the aforementioned transmission When the control unit receives a limited dialogue instruction in the aforementioned online conference, it performs limited transmission to prohibit the transmission of audio data sent and received between the specific terminals in the state of being played by other terminals; the limited dialogue instruction is specified from any of the aforementioned terminals. Part of the above-mentioned multiple terminals including this terminal is used as a specific terminal, and a dialogue is only conducted between the specific terminals; during the aforementioned limited transmission period, audio data from terminals other than the aforementioned specific terminal are also transmitted to the aforementioned specific terminal. at least partly. The online conference system as described in claim 1, which includes a conference database storing group data preliminarily using a part of the terminal as a group, and when the transmission control unit receives a limited dialogue instruction targeting the group , and regard the terminals belonging to the group as the aforementioned specific terminals. The online conference system according to claim 1, wherein, during the limited transmission period, when the transmission control unit designates a part of the specific terminal from a terminal other than the specific terminal and accepts a new limited dialogue instruction, regardless of whether it is specified by the aforementioned As specified in the limited dialogue instruction, all the aforementioned specific terminals are asked whether to approve the limited dialogue instruction. The online conference system according to Claim 1, which includes a screen generating unit that generates a screen in which an indication icon for designating the terminal as the specific terminal is superimposed on the video data from the terminal. The online conference system according to claim 1, which includes a screen generating unit that, when performing the limited transmission, combines images from the specific terminal with each other on the screen displayed on the terminal. Yes, at the same time, a screen showing a cancel icon is generated, and the cancel icon can be used as an instruction to cancel the limited transmission. The online conference system according to any one of Claims 1 to 5, which includes: a transmission control unit that transmits audio data and video data between multiple aforementioned terminals; and a screen generation unit that overlaps images of the aforementioned terminals data, and display the volume icon individually specifying the volume of the terminal. An online conference system is connected to a plurality of terminals through a network to realize an online conference among the aforementioned terminals, which includes: a transmission control unit for mutually transmitting audio data and video data among the plurality of aforementioned terminals; and screen generation a section that receives a response transmitted from any of the aforementioned terminals to any other terminal, and displays the response on the other terminal; and an emotional analysis section that performs emotional analysis of the terminal user based on the image data from the aforementioned terminal; the aforementioned The response is the result of this sentiment analysis. The online meeting system as described in Claim 7, wherein the aforementioned response is information represented by a pre-prepared logo or text. The online conference system according to claim 7, wherein the screen generating unit superimposes and displays responses from other terminals on the video data of the terminal user. The online conference system according to any one of Claims 7 to 9, wherein the screen generating unit superimposes and displays a response from another terminal to the terminal on an image of another terminal of a sender of the response. An online conference system is connected to a plurality of terminals through a network to realize an online conference between the aforementioned terminals, which includes: an analysis and processing unit that counts one of the speaking time and the silent time of the users of each of the aforementioned terminals both; and The screen generation unit displays the statistics result on the terminal. The online conference system according to claim 11, wherein the analysis and processing unit prompts the terminal user for each speaking time. The online conference system according to claim 11, wherein the analyzing and processing unit obtains a ratio of speaking time from each terminal in the online conference. An online conference system is connected to multiple terminals through a network to realize an online conference between the aforementioned terminals, which includes: a voice data input unit, which inputs voice data from the aforementioned terminals; a keyword extraction unit, which extracts the voice data the included keyword; and a screen generating unit that displays the keyword and the audio data on the terminal in correspondence with each other. An online conference system is connected to multiple terminals through a network to realize an online conference between the aforementioned terminals, which includes: a conference database, storing the start time and end time of each speech from each terminal in the aforementioned online conference as the conference data; and the screen generation unit, according to the conference data, displays a Gantt chart drawn by dividing the speech period according to each of the terminals one by one according to the time axis. The online conference system according to claim 15, wherein the screen generating unit, when another terminal starts to speak while one terminal is speaking, emphatically displays the speeches of other terminals that overlap with the speaking time of the first terminal. part. The online conference system according to claim 15 or 16, wherein the responses from other terminals are paired with the utterances and stored in the conference database, and the screen generating unit is displayed by pairing the responses with the utterances. The online conference system according to claim 15 or 16, wherein the screen generating unit displays keywords of each speech specified by the conference database corresponding to the speech.

Images