JP2012165170A - Conference device, conference method and conference program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of easily recognizing a participant paying attention and a participant to whom the attention is paid.SOLUTION: A conference device 1 for remote conference through a network comprises: video acquisition means 16 for acquiring video data of a participant who uses the conference device; operation acquisition means 18 for acquiring operation data of the self-participant indicating a participant to whom the attention is paid; reception means 19 for receiving video data and operation data of other participants from another conference device through a network 9; video generation means 20 for generating conference video having video data of the self-participant and other participants laid out in a circular shape; video edition means 21 for setting, to the conference video, an arrow from the video data of the participant paying attention to the participant to whom the attention is paid based on the operation data of the self-participant and the other participants; and display means 14 for displaying the conference video edited by the video edition means 21.

Description

  The present invention relates to a conference apparatus, a conference method, and a conference program for performing a remote conference via a network.

  The demand for remote conferencing is increasing for the purpose of reducing business trip costs in corporate activities. Conventional conference systems include, for example, a telephone (voice) conference system and a Web (video) conference system that operates on a browser. In the Web conference system, if a browser is started, a Web camera is installed, and a headset is attached, a conference can be held while watching each other's participants. Note that Patent Document 1 describes a multipoint communication system capable of acquiring and transferring a right to speak.

JP 2004-248145 A

  When a person performs face-to-face communication, a smooth conversation is realized by exchanging not only linguistic information but also non-linguistic information. In order to exchange non-linguistic information, it is important to recognize who is sending non-linguistic information to who and who has received non-linguistic information. A line of sight is used as one of means for transmitting non-linguistic information. By sending a line of sight, it is possible to draw the attention of the other party without using linguistic information and convey the non-linguistic information to the other party through vision.

  However, in a remote conference via a network, participants cannot exchange gazes. Therefore, in a remote conference, only language information (speech) can be exchanged, and it may be difficult to recognize who is paying attention to who. As a result, although video is used, only language information (speech) is communicated, and there are cases where problems such as poor switching of speakers and inactive conversations occur.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a conference apparatus, a conference method, and a conference program capable of easily recognizing a participant who is paying attention and a participant who is paying attention. Is to provide.

  In order to achieve the above object, the present invention provides a conference device that performs a remote conference via a network, and includes a video acquisition unit that acquires video data of a self-participant who uses the conference device, and a participant of interest. Instructing, operation acquisition means for acquiring operation data of the self-participant, reception means for receiving video data and operation data of other participants from other conference devices via the network, and the self-participants and other participants Video generation means for generating a conference video in which video data are arranged in a circle, and video data of participants who are receiving attention from the video data of the participants who are paying attention based on the operation data of their own participants and other participants Video editing means for setting the arrow to the conference video, and display means for displaying the conference video edited by the video editing means.

  The present invention is a conference method for performing a remote conference via a network, and the conference device instructs a participant to be watched, a video acquisition step of acquiring video data of a self-participant who uses the conference device, An operation acquisition step of acquiring operation data of the self-participant, a reception step of receiving video data and operation data of the other participant from another conference device via the network, and video data of the self-participant and the other participant A video generation step for generating a conference video arranged in a circle, and an arrow from the video data of the participant in focus to the video data of the participant in focus based on the operation data of the participant and other participants A video editing step for setting the video to the conference video, and a display step for displaying the conference video edited in the video editing step.

  The present invention is a conference program for causing a computer to execute the conference method.

  According to the present invention, it is possible to provide a conference device, a conference method, and a conference program capable of easily recognizing a participant who is paying attention and a participant who is paying attention.

1 is an overall configuration diagram of a conference system according to an embodiment of the present invention. It is a figure which shows an example of a meeting image | video. It is a figure which shows an example of the edited meeting image | video. It is a figure which shows an example of the edited meeting image | video. It is a figure which shows an example of the edited meeting image | video. It is a flowchart which shows operation | movement of a video editing part.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is an overall configuration diagram of a conference system according to an embodiment of the present invention. The conference system of the present embodiment is a system for a plurality of conference participants (hereinafter referred to as “participants”) in a remote place to perform a remote conference using a plurality of conference devices 1 connected to a network 9. is there.

  The conference system shown in the figure includes a plurality of conference devices 1 used by each of a plurality of participants, and a conference server 8 connected to these conference devices 1 via a network 9.

  The conference apparatus 1 is, for example, a PC, and includes a video camera 11 that captures images of the self-participant who uses the conference apparatus 1, a microphone 12 that collects the sound spoken by the self-participant, and the self-participant An input device 13 that accepts the above operation, a display device (display) 12 that displays the conference video, and a speaker 15 that outputs the voices of other participants transmitted from the conference server 8.

  The input device 13 is a device that operates a pointer (such as a mouse pointer) displayed on the conference video on the display device 12. For example, a pointing device such as a mouse, a pen, or a touch pad may be used.

  In addition, the conference apparatus 1 receives the video data captured by the video camera 11, the video acquisition unit 16 that captures the audio data collected from the microphone 12, and the operation data input to the input device 13. An operation acquisition unit 18 to be captured, a communication unit 19, a video generation unit 20, and a video editing unit 21 are provided.

  The communication unit 19 transmits the video data captured by the video acquisition unit 16 and the operation data captured by the operation acquisition unit 18 to the conference server 8 via the network 9 and also to the video editing unit 21. . In addition, the video editing unit 21 transmits the audio data captured by the audio acquisition unit 17 to the conference server 8 via the network 9. Further, the communication unit 19 receives audio data of other participants from the conference server 8 via the network 9 and outputs / reproduces them to / from the speaker 15, and also transmits video data and operation data of other participants to the network 9. Is received from the conference server 8 and sent to the video generation unit 20.

  The video generation unit 20 generates a conference video in which the video data of the self-participant captured by the video acquisition unit 16 and the video data of other participants received by the communication unit 19 are arranged in a circle. The video editing unit 21 edits the conference video generated by the video generation unit 20 based on the operation data of the own participant acquired by the operation acquisition unit 18 and the operation data of other participants received by the communication unit 19. Is displayed on the display device 14.

  The conference server 8 receives the participant's video data, audio data, and operation data input from each conference device 1 used by each participant, and transmits them to the other conference devices 1.

  The conference device 1 and the conference server 8 can use, for example, a general-purpose computer system including a CPU, a memory, an external storage device such as an HDD, an input device, and an output device. In this computer system, each function of each device is realized by the CPU executing a predetermined program loaded on the memory. For example, the functions of the conference device 1 and the conference server 8 are as follows. The CPU of the conference device 1 is a program for the conference device 1, and the CPU of the conference server 8 is a program for the conference server 8, respectively. It is realized by executing.

  Further, the program for the conference apparatus 1 and the program for the conference server 8 can be stored in a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, a DVD-ROM, or distributed via a network. You can also

  Next, the conference video of this embodiment will be described.

  FIG. 2 is a diagram illustrating an example of a conference video generated by the video generation unit 20. As shown in the figure, the video generation unit 20 arranges the video data 201 to 204 of each participant in a circle (circumference) at regular intervals. Further, in the illustrated example, the video data 201 of the self-participant A (participant sitting in front of the conference device 1) who uses the conference device 1 is arranged at the bottom of the circle. Then, the video generation unit 20 divides the circle into a sector according to the number of participants so that the area of each participant becomes equal, and sets and displays the dividing line 206. In the example shown in the figure, since four participants are participating in the conference, it is divided into four.

  In the initial state of the conference video, a pointer (such as a mouse pointer) 205 is displayed at the center of the circle. The self-participant uses the input device 13 (such as a mouse) to move the pointer 25 toward the video data of any participant, thereby selecting / instructing the participant of interest.

  For example, as shown in FIG. 3, the self-participant A operates the input device 13 to move the pointer 205 toward the video data 203 of the participant C, and the pointer 205 moves to the area 207 of the participant C. As a result, it is input that the self-participant A is paying attention to the participant C. By this operation, the video editing unit 21 sets the arrow 208 from the video data 201 of the subject participant A who is paying attention to the video data 203 of the participant C who is paying attention to the conference video.

  Note that the maximum number of participants that can be specified by the self-participant is one. That is, it is either not paying attention to anyone (0 people) or paying attention to one person. For example, when the participant D is designated after the participant A designates the participant C as the participant to be noticed, the designation of the participant C is canceled and only the participant D is designated as the participant to be noticed. The It should be noted that the designation of the participant to be noticed is automatically canceled when a predetermined time (first prescribed time) has elapsed since the designation, even if the next participant to be noticed is not designated. To do.

  The pointer 205 moved by the self-participant A returns to the central portion set in the initial state by the operation acquisition unit 18 after a predetermined time (second time) has elapsed (see FIG. 2). Note that the predetermined time (first time) for canceling the designation and the predetermined time (second time) for returning the pointer 205 to the center are independent, and may be the same or different. May be.

  In addition, the video editing unit 21 lowers the resolution of the video data of the participant who is drawing attention and the video data of the participants other than the video data of the participant who is watching (for example, the resolution is ¼, etc.). Lower). In the example shown in FIG. 3, the resolution of the video data 203 of the participant C who is focused and the video data 202 and 204 of the participants B and D other than the video data 201 of the participant A who is focused is Lower than the video data 201 and 203 of the self-participant A and the participant C. It should be noted that the resolution of the video data 201 of the self-participant A is not lowered even when any participant is not paying attention and is not receiving attention.

  In addition, as shown in FIG. 4, when the participant D operates the input device 13 of the conference apparatus 1 used by the participant D and moves the pointer so as to pay attention to the participant A (the pointer is When moved to the area of the participant A), the operation data of the participant D is transmitted to the conference apparatus 1 used by the participant A via the network 9 and the conference server 8. The video editing unit 21 of the conference apparatus 1 of the participant A uses the operation data to move from the video data 204 of the participant D who is paying attention to the video data 201 of the participant A who is paying attention to an arrow 208 ( For example, a dotted arrow) is set for the conference video.

  When the participant C operates the input device 13 of the conference device 1 used by the participant C and moves the pointer so as to pay attention to the participant B, the operation data of the participant C is stored in the network 9. And transmitted to the conference apparatus 1 used by the participant A via the conference server 8. Similarly, when the participant B operates the input device 13 of the conference apparatus 1 used by the participant B and moves the pointer so as to pay attention to the participant C, the operation data of the participant B is stored in the network. 9 and the conference server 8 are transmitted to the conference apparatus 1 used by the participant A. The video editing unit 21 of the conference apparatus 1 of the participant A uses these operation data to place an arrow 209 between the video data 202 of the participant B and the video data 203 of the participant C that are paying attention to each other. (For example, a thick solid arrow in both directions) is set in the conference video.

  In the present embodiment, the video editing unit 21 lowers the resolution of video data of a participant who is not paying attention to the self-participant and is not paying attention to the self-participant. In the example shown in FIG. 4, the resolution of the video data 202 of the participant B and the video data 203 of the self-participant C is set lower than the video data 201 and 204 of the self-participant A and the participant D.

  Further, as shown in FIG. 5, the self-participant A operates the input device 13 to move the pointer 205 toward the video data 201 of the self-participant A (self-participant A), and the pointer 205 is moved to his / her area. By moving, the self-participant A inputs a request for attention of all who requests that all other participants want to pay attention to himself / herself. In response to this all-attention request operation instruction, the video editing unit 21 enlarges and displays the video data 201 of the self-participant A to a predetermined size (for example, 1.2 times).

  Next, the operation of the video editing unit 21 of this embodiment will be described.

  First, the operation acquisition unit 18 acquires operation data operated by the self-participant using the input device 13, and determines a participant that the self-participant pays attention to according to the position of the moved pointer. Then, the operation acquisition unit 18 sends the operation data of the self-participant including the determined participant of interest to the video editing unit 21 and the communication unit 19. The operation data of the self-participant sent to the communication unit 19 is transmitted to all other conference apparatuses 1 via the network 9 and the conference server 8.

  In addition, the communication unit 19 sends operation data of other participants received from the other conference devices 1 to the video editing unit 21 via the network 9 and the conference server 8. Thereby, the operation data of the self-participant and the operation data of the other participants are input to the video editing unit 21. The video editing unit 21 determines, for each participant, which participant the participant is paying attention to based on the input operation data of each participant (when the pointer is in any participant's area). ) Whether any participants are not paying attention (when the pointer is in the center) or whether all the members are requesting attention (when the pointer is in their own area) in a storage unit (not shown) It shall be remembered.

  The video editing unit 21 receives each operation data from the operation acquisition unit 18 or the communication unit 19 and sets the designation of the participant of interest in the storage unit, and then a predetermined time (first predetermined time) has elapsed. Then, the designation of the participant to be noticed is canceled and updated to a state in which no participant is noticed.

  The video editing unit 21 edits the image video generated by the video generation unit 20 with reference to the storage unit in which the operation data of each participant is stored.

  FIG. 6 is a flowchart showing the editing process of the video editing unit 21. The video editing unit 21 repeatedly performs the process shown in FIG. 6 for each participant at a predetermined timing. First, the video editing unit 21 refers to the storage unit and determines whether or not the operation data (participating participant) of the target participant (hereinafter “participant”) is a request for attention (S11). In the case of a request for attention by all (S11: YES), the size of the video data of the participant is enlarged to a predetermined size (S12), and the process proceeds to S15.

  When not all of the attention request (S11: NO), the video editing unit 21 is when the participant is not a self-participant, or when the self-participant is not set in the operation data of the participant in the storage unit. And when the said participant is not set to the operation data of the self-participant of a memory | storage part (S13: NO), the resolution of the video data of the said participant is made low (S14). In other words, if it is not the self-participant, does not pay attention to the self-participant, and does not attract attention from the self-participant, the resolution of the video data of the participant is lowered.

  On the other hand, in the case of a self-participant, either the self-participant is set in the operation data of the participant in the storage unit, or the participant is set in the operation data of the self-participant in the storage unit In the case of (S13: YES), the process proceeds to S15.

  In S15, the video editing unit 21 is a case in which any participant is set in the operation data of the participant in the storage unit (S15: YES), and the attended participant set in S15 is noticed. When the participant is set as a participant who pays attention to the operation data (S16: YES), it is considered that they are paying attention to each other, and both the participant and the participant set in S15 are both A direction arrow is set (S17).

  If the participant is not set in the operation data of the participant set in S15 (S16: NO), the video editing unit 21 sets an arrow from the participant toward the participant set in S15. (S18).

  If no participant is set (S15: NO), and the participant is set as a participant who pays attention to the operation data of any participant (S19: YES), video editing The unit 21 sets an arrow from the participant paying attention to the participant toward the participant (S20).

  The above-described operation of FIG. 6 will be specifically described using the conference video of FIG. 4 as an example. In this case, there is no participant to be noted in the operation data of the participant A (self-participant), and the participant C as the participant to be noted in the operation data of the participant B is stored in the storage unit. It is assumed that participant B is set as the participant to be noted in the operation data, and participant A is set as the participant to be noted in the operation data of participant D.

  Participant A is not a request for attention of all members (S11: NO), is a self-participant (S13: YES), does not pay attention to any participant (S15: NO), and is noted by participant D (S19: YES), the video editing unit 21 sets an arrow 208 from the video data 204 of the participant D to the video data 201 of the participant A (S20).

  Participant B is not a request for attention of all members (S11: NO), and does not pay attention to self-participant A and is not attracting attention from self-participant A (S13: NO). The resolution of 202 is lowered (S14). In addition, since the participant B is paying attention to the participant C (S15: YES) and is also noted by the participant C (S16: YES), the video editing unit 21 displays the video data 202 of the participant B. And a bidirectional arrow 209 are set between the video data 203 of the participant C (S17).

  As for participant C, not everyone's attention request (S11: NO), since self participant A is not noticed and is not attracting attention from self participant A (S13: NO), video data of participant C The resolution of 203 is lowered (S14). Participant C is paying attention to Participant B (S15: YES), and Participant B is also paying attention (S16: YES), so video editing unit 21 uses video data 202 of Participant B. A bidirectional arrow 209 is set between the video data 203 of the participant C and the video data 203 of the participant C (S17). However, since the arrow 209 has already been set in the process of the participant B, the arrow 209 is not set here.

  Participant D is not a request for attention of all members (S11: NO), but is paying attention to participant A (S13: YES, S15: YES), and is not attracting attention from participant A (S16: NO) The video editing unit 21 sets the arrow 208 from the video data 204 of the participant D to the video data 201 of the participant A (S18). Since the arrow 208 has already been set by the processing of the participant A, the video editing unit 21 Then, the arrow 208 is not set.

  In the present embodiment described above, it is possible to easily select a participant of interest even in a conference system using a conference device that does not include a special device for measuring the line of sight, such as a Web conference. Since it is possible to recognize that it is attracting attention, it is possible to realize an environment in which non-linguistic information can be exchanged between participants and a smooth conversation can be realized.

  Specifically, in the present embodiment, video data of all participants is arranged in a circle, and a conference video in which video data of the self-participant using the conference device is arranged at the bottom is generated and displayed on the display device. indicate. Thereby, it is possible to give each participant a sense of reality as if he / she is sitting at a round table and having a meeting.

  Further, in the present embodiment, based on the operation data of each participant, an arrow from the participant's video data of interest to the participant's video data of interest is set in the conference video. In the case of participants who are paying attention to each other, a bidirectional arrow is set between the participants. Thereby, in this embodiment, which participant is paying attention to which participant can be expressed, and the participant can be easily recognized.

  Further, in the present embodiment, the participant can easily select the participant to be noted by an operation of moving the pointer toward the participant to be noted on the conference video. In addition, the pointer moves to the area of any participant and then returns to the center position (initial position) after a predetermined time has elapsed, so the participant simply moves the pointer in the direction of the participant who wants to pay attention to. In this way, participants can be selected freely.

  In the present embodiment, attention is paid to the participant's line of sight by reducing the resolution and blurring the video data of the participant who is paying attention and the participants other than the participant who is paying attention. Non-linguistic information can be transmitted to a participant who has received attention and a participant who has received attention.

  In addition, this invention is not limited to the said embodiment, Many deformation | transformation are possible within the range of the summary. For example, it is possible to switch ON / OFF (use / do not use) of the function of selecting a participant to be noticed by moving the pointer of the conference video as described in the above embodiment with a toggle type switch button. It is done.

  In the above embodiment, for participants who are not paying attention to the self-participant and who are not paying attention to the self-participant, pay attention to a participant other than the self-participant, or a participant other than the self-participant. Even if it is a case where attention has been paid to, the resolution of the video data of the participant has been lowered (FIG. 4: Participant B, Participant C). However, even if the participant does not pay attention to the self-participant and does not receive attention from the self-participant, the participant other than the self-participant pays attention, or the participant other than the self-participant gets attention. The resolution of the video data of the participant may not be lowered.

1: Conference device 11: Video camera 12: Microphone 13: Input device 14: Display device 15: Speaker 16: Video acquisition unit 17: Audio acquisition unit 18: Operation acquisition unit 19: Communication unit 20: Video generation unit 21: Video editing Part 8: Conference server 9: Network

Claims (9)

A conference device for performing a remote conference via a network,
Video acquisition means for acquiring video data of self-participants who use the conference device;
Operation acquisition means for acquiring the operation data of the self-participant, instructing the participant to be noted;
Receiving means for receiving video data and operation data of other participants from other conference devices via a network;
Video generation means for generating a conference video in which video data of the self-participant and other participants are arranged in a circle;
Based on the operation data of the self-participant and other participants, video editing means for setting an arrow from the video data of the participant who is focused to the video data of the participant who is focused on the conference video,
And a display unit for displaying the conference video edited by the video editing unit. The conference device according to claim 1,
The conference apparatus is characterized in that the operation data is input by an operation in which a participant moves a pointer toward any participant in a conference video displayed on the display unit. The conference device according to claim 1 or 2,
The conference apparatus, wherein the video editing unit lowers the resolution of video data of participants other than the participant who is drawing attention and participants who are paying attention to the participant who is drawing attention. The conference device according to claim 1,
In the case of operation data in which a participant moves a pointer toward the participant himself / herself, the video editing unit increases the size of the video data of the participant. A conference method for conducting a remote conference via a network,
The conference equipment
A video acquisition step of acquiring video data of the self-participant who uses the conference device;
An operation acquisition step for acquiring the operation data of the self-participant, instructing the participant of interest,
A receiving step of receiving video data and operation data of other participants from other conference devices via the network;
A video generation step of generating a conference video in which video data of the self-participant and other participants are arranged in a circle;
Based on the operation data of the self-participant and other participants, a video editing step of setting an arrow from the video data of the participant who is paying attention to the video data of the participant who is paying attention to the conference video;
And a display step of displaying the conference video edited in the video editing step. The conference method according to claim 5,
The operation data is input by an operation in which a participant moves a pointer toward any participant in the conference video displayed in the display step. The conference method according to claim 5, wherein:
The video editing step lowers the resolution of video data of participants other than the participant who is drawing attention and participants who are paying attention to the participant who is drawing attention. The conference method according to claim 5,
In the video editing step, in the case of operation data in which a participant moves a pointer toward the participant himself / herself, the size of the video data of the participant is enlarged.   A conference program for causing a computer to execute the conference method according to claim 5.

Images