JP7178015B2 - online meeting system - Google Patents

Description

The present invention relates to an online conference system using communication lines.

2. Description of the Related Art An online conference system is known in which a plurality of remote users establish a conference by connecting terminal devices of the respective users via communication lines. For example, Japanese Laid-Open Patent Publication No. 2002-200001 discloses a video conference system comprising a user terminal device having a video telephone and a remote image transmission circuit. In this videophone conference system, the monitor screen of each terminal device is divided into a plurality of screens, and the image of each user who is a conference attendee is displayed on each divided screen.

In a video conference system such as Patent Document 1, in order to display the faces of attendees on each user terminal device, a photographed image of the attendee's face is transferred to each user terminal device via a communication circuit. . However, since the amount of data of the photographed images is enormous, it takes a long time to input and output the photographed images of each attendee. The load given to the communication circuit is also large. Therefore, as a means of solving the above problem, it is also known to display attendees' avatars (characters serving as alter ego) on the user terminal device instead of the photographed images.

For example, Patent Document 2 discloses a display control unit that displays avatars in which a plurality of users are virtually represented on a display, a behavior detection unit that detects behavior of at least one user among the plurality of users, and a behavior detection unit that detects the behavior of at least one user. A proactiveness determination unit that determines the aggressiveness, which is an index that indicates the user's positive attitude toward the meeting, and the user's behavior detected by the behavior detection unit according to the aggressiveness determined by the aggressiveness determination unit. A conference system is disclosed that includes an action reflecting unit that reflects on an avatar. Patent Document 3 discloses display means for displaying a virtual conference room for holding a virtual conference and displaying animation characters (equivalent to avatars) representing attendees of the conference in the virtual conference room; an instruction means for instructing the action of one's own animation character; an information output means for outputting action information of one's own animation character instructed by the instruction means; A terminal device for a virtual conference system is disclosed, which includes character control means for causing the animation character of the character to act, and for causing the own animation character to act according to the instructed action.

JP-A-4-140990 JP 2014-225801 A JP-A-6-274596

As in Patent Document 2, each user's behavior is detected and each user's behavior is reflected in an avatar. However, when the number of participants in the meeting increases and a large number of avatars are displayed on the screen, the amount of data related to reflecting the behavior of each user on the avatar increases accordingly, and the processing of the user terminal device becomes large. Since the burden and the load on the communication line increase, there is a risk of causing a display defect such as freezing of the avatar's movement, or an audio defect such as the audio of the conference being interrupted. Furthermore, in the configurations of Patent Documents 2 and 3, although it is possible to estimate the degree of participation of each user in the conference by looking at the movement of the avatar on the conference screen, there is no function to evaluate the content of the conference. It was impossible for each user or third party to intuitively know the progress of the meeting, and there was room for improvement in this respect as well.

SUMMARY OF THE INVENTION An object of the present invention is to suppress an increase in the amount of data related to reflecting the actions of each user in an avatar in an online conference system using avatars, even when the number of participants in the conference increases. To prevent the processing load on a user terminal device and the load on a communication line from increasing.
Another object of the present invention is to enable users and third parties participating in the conference to intuitively know the progress of the conference in an online conference system using avatars.

The present invention is an online conference system comprising a conference server 3 and a plurality of terminal devices 4 operated by individual users 5 participating in the conference and connected to the server 3 via communication lines 2. It targets conferencing systems. Then, the terminal device 4 displays a personal avatar 34 (34A) corresponding to the user 5 (5A) who is the operator of the terminal device 4 (4A) and other users 5 (5B to 5F) participating in the conference. a display unit 11 that displays personal avatars 34 (34B to 34F) corresponding to , a motion detection unit 21 that detects the motion of the user 5 via the camera 13, and the motion detection unit 21 of the user 5 detected by the motion detection unit 21 A motion extraction unit 22 for extracting a motion reflected on the personal avatar 34 from motions, and a motion command for transmitting/receiving the motion command indicating the motion extracted by the motion extraction unit 22 between the terminal devices 4 via the communication line 2. A transmitting/receiving section 23 and an action reflecting section 24 for reflecting a movement corresponding to the action command on the personal avatar 34 displayed on the display section 11 based on the action command .

The action extracted by the action extraction unit 22 can be selected by the user 5 who is the operator of the terminal device 4 or by the administrator of the conference server 3 .

A conference avatar 32 indicating the progress of the conference is displayed on the display unit 11 of each terminal device 4 . The terminal device 4 receives an operation related to evaluation of the progress of the conference by the user 5 who is the operator, and has an evaluation unit 28 that transmits an evaluation signal corresponding to the evaluation to the conference server 3, and a conference server. and an evaluation reflecting unit 29 that receives an evaluation command related to the total evaluation result transmitted from 3 and changes the form of the conference avatar 32 displayed on the display unit 11 . The conference server 3 includes an evaluation totaling unit 54 that totalizes the evaluation signals transmitted from each terminal device 4 and transmits an evaluation command to each terminal device 4 .

In the present invention, when the user 5 moves in front of the camera 13 , the motion is detected by the motion detector 21 via the camera 13 . The motion extraction unit 22 extracts motions reflected in the personal avatar 34 from various motions detected by the motion detection unit 21 . When a motion to be reflected is extracted by the motion extraction unit 22, the motion command transmission/reception unit 23 transmits a motion command indicating the motion to the other terminal device 4 via the communication line 2, and the motion reflection unit 24 A motion corresponding to the motion command is displayed on the display unit 11 . As described above, in the online conference system of the present invention, all actions of the user 5 operating each terminal device 4 are not reflected in the personal avatar 34 on the display unit 11, but only limited actions are performed by the personal avatar. 34, it is possible to significantly reduce the amount of data related to action reflection, compared to a configuration in which all actions of the user 5 are reflected in the personal avatar 34. FIG. Therefore, even if the number of participants in the conference increases, the processing load on the terminal device 4 and the load on the communication line 2 are prevented from increasing, and display defects such as freezing of the operation of the personal avatar 34 occur. Also, it is possible to prevent the user 5 from interrupting his/her speech.

If the actions extracted by the action extraction unit 22 are configured to be selectable by the user 5 of the terminal device 4 or the administrator of the server 3, only the optimum action of the user 5 according to the type of the meeting is displayed as the personal avatar. 34 can be reflected. For example, when the conference system of this embodiment is applied to a lecture, etc., and the words of one lecturer are heard by a large number of listeners of several dozen to several hundred, the "applause" It is possible to reflect only the actions of the listeners, who are the users 5, on the personal avatars 34 corresponding to the respective listeners, and not to reflect many other actions on the personal avatars 34. As a result, the processing load on the terminal device 4 and the load on the communication line 2 can be significantly suppressed, and the occurrence of poor display and poor sound can be reliably prevented even though the lecture is attended by a large number of users 5.例文帳に追加can be done. There is also the advantage that the lecture progresses smoothly without the personal avatar 34 making unnecessary movements.

A conference avatar 32 indicating the progress of the conference is displayed on the display unit 11 of each terminal device 4, and the form of the conference avatar 32 changes in response to an operation related to evaluation of the progress of the conference by each user 5. Since the degree of significance of the meeting can be visualized, each user 5 can easily determine whether the current meeting is progressing meaningfully simply by looking at the form of the meeting avatar 32 . can be grasped with certainty.

1 is a schematic configuration diagram of an online conference system according to an embodiment of the present invention; FIG. 1 is a block diagram of a terminal device that constitutes an online conference system; FIG. It is a figure which shows the display content of a terminal device. It is a figure which shows the form change of a meeting avatar. FIG. 4 is a diagram for explaining action recognition and action reflection to an avatar; 1 is a block diagram of a conferencing server; FIG.

1 to 6 show an online conference system according to an embodiment of the invention. FIG. 1 shows a schematic configuration diagram of the conference system. This conference system 1 is composed of a system server (conference server) 3 and a plurality of terminal devices 4 (4A to 4F) which are connected to each other via a communication line 2 such as the Internet. The terminal devices 4 (4A-4F) are operated by users 5 (5A-5F) who are individual conference participants participating in the conference. By using this conference system 1, even when a plurality of users 5 are in different places, these users 5 can hold a conference on the display 11 of the terminal device 4 using the terminal device 4 operated by each of them. can be done. The illustrated example shows an example in which six users 5 participate in a conference using a total of six terminal devices 4 operated by each of them. , but not limited to the above.

FIG. 2 is a block diagram of the terminal device 4. As shown in FIG. The terminal device 4 is a so-called notebook computer, and includes a keyboard 10 operated by a user, a display (display unit) 11 that is a liquid crystal display panel, a microphone 12 for collecting sound, a camera 13 for imaging, and a loudspeaker. A speaker 14 and the like are provided. The control unit 15 of the terminal device 4 comprehensively controls the overall operation of the terminal device 4, and programs stored in a storage unit 16 such as ROM (Read Only Memory) and SSD (Solid State Drive), Alternatively, a program read from the outside via the communication unit 17 is developed in a RAM (Random Access Memory) 18, which is a work area, and the image display unit 19 and the like are controlled according to the program.

FIG. 2 schematically shows the configuration of the control unit 15 when executing a program related to the online conference system (hereinafter simply referred to as "program"). During execution of the program, the control unit 15 It includes a motion detection unit 21, motion extraction unit 22, motion command transmission/reception unit 23, motion reflection unit 24, voice input unit 25, voice output unit 26, operation input unit 27, evaluation unit 28, evaluation reflection unit 29, and the like. Operations of these units will be described later.

FIG. 3 shows a display screen on the display 11 of the online conference system 1 according to the present embodiment. A chat area 31 in which character information typed by is displayed, and a conference avatar area 33 in which a conference avatar 32 is displayed are formed.

As shown in FIG. 3, a personal avatar 34 is displayed in the user area 30 . The personal avatar 34 corresponds to the user 5 who is the operator of the terminal device 4, and is displayed as an alter ego of each user 5 in the virtual space of the online conference. These personal avatars 34 are selected by each user 5 from characters prepared in advance on the program, or created by each user 5 on the program. In FIG. 3, the user area 30 displays six personal avatars 34 (34A-34F) corresponding to six users 5 (5A-5F). In the upper left column of the user area 30, a personal avatar 34 (34A) of the user 5 (5A) who is the operator of the terminal device 4A is displayed.

The conference avatar 32 is displayed as an indicator of the progress of the conference, and the form is changed by receiving the evaluation of the conference by the user who is a conference participant (for example, the three-level evaluation of the progress of the conference, 0 × △). Change. The conference avatar 32 in this embodiment is initially in the form of a character (32A) imitating an eagle pilot wearing a flying cap, and from this form, as shown in FIG. On the other hand, when receiving a low evaluation as meaningless meeting, it changes to a chick-like character (32C). It is set to change.

FIG. 6 shows a block diagram of the system server 3. As shown in FIG. As shown in the figure, the control unit 50 of the system server 3 comprehensively controls the overall operation of the server 3. The control unit 50 includes a ROM in which various data and programs are stored, A storage unit 51 such as an HDD (Hard Disc Drive) is connected to a RAM 52 as a program operating area. The control unit 50 includes a communication unit 53 that transmits and receives various data to and from the terminal device 4 via the communication line 2, and also collects evaluation signals sent from each terminal device 4 and sends an evaluation command to each terminal device. 4 is provided with an evaluation tallying unit 54 that transmits to the 4.

Next, changes in the avatars (personal avatar 34 and conference avatar 32) in the online conference system according to the present embodiment and processing operations of the control unit 15 will be described with reference to FIGS. 1 to 6. FIG. First, the organizer of the online conference selects in advance an action (reflection action) to be reflected on the personal avatar 34 prior to holding the conference. In this embodiment, it is assumed that the organizer of the conference has selected two motions of "nod" and "applause" as reflection motions.

FIG. 3 shows a display mode of the display 11 of the terminal device 4A operated by the user 5A immediately after the conference starts, where the personal avatar 34A of the user 5A and the avatars 34B of the other five users 5B to 5F are shown. 34F, the conference avatar 32 in its initial form 32A is displayed. For example, an utterance of user 5A is picked up by microphone 12 of terminal device 4A, transmitted to other terminal devices 4B to 4F via voice input unit 25 and communication unit 17, and is transmitted to other terminal devices 4B to 4F. It is uttered from the speaker 14 through the output section 26 . Also, in the chat area 31, character information typed by each user 5 operating the keyboard 10 is displayed.

The camera 13 of the terminal device 4A is positioned to capture the upper body of the user 5A, so that the movement of the upper body of the user 5A during the meeting is captured accurately by the camera 13. Digital data captured by the camera 13 is sent to the motion detection unit 21, and through image analysis (motion capture) by the motion detection unit 21, is detected as the motion of the user 5A. Specifically, the motion detection unit 21, for example, “tilts its head”, “horizontally shakes its head”, “vertically shakes its head (nods)”, “waves its hands”, “folds its hands”, A wide variety of actions of the user 5A such as "clapping hands" are detected.

Actions detected by the action detection unit 21 (hereinafter referred to as "detected actions") are sent to the action extraction unit 22 at any time, and the action extraction unit 22 extracts only the actions reflected on the personal avatar 34 from the detected actions. Extract. Specifically, the motion extracting unit 22 compares the detected motion with the reflected motions (“nod” and “applause”) preselected by the previous conference organizer, and if the detected motion matches the reflected motion, , an extraction signal is transmitted to the operation command transmission/reception unit 23 . On the other hand, when the detected motion does not match the reflected motion, no extraction signal is sent from the motion extractor 22 to the motion command transmitter/receiver 23 .

Upon receiving the extraction signal from the motion extracting unit 22 , the motion command transmitting/receiving unit 23 sends a motion command indicating the extracted motion to the motion reflecting unit 24 . Transfer to the terminal device 4 (4B-4F). Upon receiving the action command, the action reflecting section 24 reflects the action on the personal avatar 34A corresponding to the user 5A on the display 11 via the image display section 19. FIG. Specifically, as shown in FIG. 5, when the user 5A performs a clapping motion, the clapping motion is reflected in the personal avatar 34A on the display 11, and when the user 5A performs a nodding motion. , the motion of nodding is reflected in the personal avatar 34A.

This operation is the same in the other terminal devices 4 (4B to 4F), and the operation command sent via the communication line 2 from the operation command transmitting/receiving section 23 of the terminal device 4A operated by the user 5A is When the operation command transmission/reception unit 23 of the terminal device 4 (4B to 4F) receives the operation command, the operation reflection unit 24 of the other terminal device 4 (4B to 4F) displays on the display 11 of each terminal device 4 (4B to 4F) A clapping motion or a nodding motion is reflected in the personal avatar 34A corresponding to the user 5A.

As shown in FIG. 3, at the beginning of the conference, the conference avatar 32 (32A) in the initial form is displayed. As the online conference progresses, each user 5 is asked to evaluate the progress of the conference. As described above, the evaluation is a three-level evaluation of 0 × △, and the user 5 performs the evaluation by placing the cursor on one of the three buttons of 0 × △ displayed on the display 11 and clicking it. be able to. The evaluation unit 28 of the terminal device 4 transmits an evaluation signal corresponding to the evaluation to the system server 3 via the communication unit 17 upon receiving the operation related to the evaluation as described above.

The evaluation aggregating unit 54 of the system server 3 aggregates the evaluation signals sent from the evaluator 28 of each terminal device 4, and makes the conference avatar 32 grow, regress, or maintain the status quo. An evaluation command is sent toward each terminal device 4 . For example, when a majority of the users 5 give a high evaluation (o) as a meaningful conference, the evaluation aggregating unit 54 evaluates to grow the conference avatar 32 displayed on the display 11 of each terminal device 4. Send command. On the other hand, when a majority of the users give a low evaluation (x) as meaningless, the evaluation aggregating unit 54 transmits an evaluation command to cause the conference avatar 32 to regress.

Upon receiving an evaluation command from the evaluation totaling unit 54 of the system server 3, the evaluation reflection unit 29 of each terminal device 4 changes the form of the conference avatar 32 displayed on the display 11 according to the evaluation command. For example, when an evaluation command for growth is received from the form of 32A in FIG. 4, the conference avatar 32 is changed to the form of an officer-like character (32B). On the other hand, when receiving an evaluation command to retreat from the form of 32A in FIG. 4, the conference avatar 32 is changed to the form of a chick-like character (32C).

As described above, in the online conference system 1 according to the present embodiment, all actions of the users 5 (5A to 5F) operating the respective terminal devices 4 (4A to 4F) are performed by the personal avatars 34 (34A) on the display 11. ~ 34F), but only limited actions such as "nod" and "clap" are reflected in the personal avatar 34, so all actions of the user 5 are reflected in the personal avatar 34 The amount of data related to motion reflection can be significantly reduced compared to the configuration in which the operations are reflected. Therefore, even when the number of participants in the conference increases, the processing load on the terminal devices 4 (4A to 4F) and the load on the communication line 2 are prevented from increasing, and the operation of the personal avatar 34 is prevented from freezing. It is possible to prevent the occurrence of a display defect and the occurrence of a voice defect such as the user 5's speech being interrupted.

Since the user 5 of the terminal device 4 or the administrator of the system server 3 can select the motion extracted by the motion extraction unit 22, only the optimal motion of the user 5 according to the type of the meeting can be selected by the personal avatar 34. can be reflected in For example, when the conference system 1 of the present embodiment is applied to a lecture or the like, and the words of one lecturer are heard by a large number of listeners of several tens to several hundred, "applause" Only the actions of the audience members who are the users 5 related to the event are reflected in the personal avatars 34 corresponding to the respective audience members, and many other actions are not reflected in the personal avatars 34. As a result, the processing load on the terminal device 4 and the load on the communication line 2 can be significantly suppressed, and the occurrence of poor display and poor sound can be reliably prevented even though the lecture is attended by a large number of users 5. be able to. There is also the advantage that the lecture progresses smoothly without the personal avatar 34 making unnecessary movements.

A conference avatar 32 indicating the progress of the conference is displayed on the display 11 of each terminal device 4, and the form of the conference avatar 32 changes in response to an operation related to evaluation of the progress of the conference by each user 5. With this configuration, the degree of significance of the meeting can be visualized, so each user 5 can easily and reliably check whether the current meeting is progressing meaningfully simply by looking at the form of the meeting avatar 32. can be grasped.

In the above embodiment, the motions (extracted motions) extracted by the motion extraction unit 22 were "nod" and "clap", but the extracted motions are not limited to these. The display form such as screen division on the display 11 is not limited to that shown in FIG. The forms of the meeting avatar 32 and the personal avatar 34 are also not limited to the illustrated examples.

1 Online conference system 2 Communication line 3 Conference server (system server)
4 terminal device 5 user 11 display unit (display)
13 camera 21 motion detection unit 22 motion extraction unit 23 motion command transmission/reception unit 24 motion reflection unit 28 evaluation unit 29 evaluation reflection unit 32 conference avatar 34 personal avatar 54 evaluation totalization unit

Claims (2)

A conference server (3) and a plurality of terminal devices (4) operated by individual users (5) participating in the conference and connected to the server (3) via communication lines (2). In an online conference system consisting of
A terminal device (4)
A personal avatar (34) corresponding to the user (5) who is the operator of the terminal device (4) and a personal avatar (34) corresponding to another user (5) participating in the conference are displayed. a display unit (11);
a motion detector (21) that detects motion of the user (5) via the camera (13);
a motion extraction unit (22) for extracting a motion reflected in a personal avatar (34) from the motion of the user (5) detected by the motion detection unit (21);
an action command transmitting/receiving section (23) for transmitting/receiving an action command indicating the action extracted by the action extracting section (22) between the terminal devices (4) via the communication line (2);
an action reflection unit (24) for reflecting a movement corresponding to the action command on the personal avatar (34) displayed on the display unit (11) based on the action command;
equipped withand
The action extracted by the action extraction unit (22) is configured to be selectable by the user (5) who is the operator of the terminal device (4) or the administrator of the conference server (3). An online conference system characterized by: A conference avatar (32) indicating the progress of the conference is displayed on the display unit (11) of each terminal device (4),
The terminal device (4)
an evaluation unit (28) that receives an operation related to the evaluation of the progress of the conference by the user (5) who is the operator and transmits an evaluation signal corresponding to the evaluation to the conference server (3);
an evaluation reflection unit (29) that receives an evaluation command related to the total evaluation result transmitted from the conference server (3) and changes the form of the conference avatar (32) displayed on the display unit (11);
with
A conference server (3) has an evaluation totaling section (54) that totalizes evaluation signals transmitted from each terminal device (4) and transmits an evaluation command to each terminal device (4). , the online conference system according to claim 1.

Images