JP4768578B2 - Video conference system and control method in video conference system - Google Patents

Description

  The present invention relates to a video conference system that realizes a multi-point videophone conference and a control method in the video conference system.

  2. Description of the Related Art Conventionally, in a video conference terminal device used for a multi-point video conference system, by specifying the number of images of the other party to be displayed, the other party participating in the conference is displayed by dividing the screen. With regard to this display method, for example, a technique described in JP-A-9-70031 (Patent Document 1) is known. With this technology, all attendees and only the speakers within them are photographed with separate cameras, and the images of the speakers are displayed in a superimposed manner on all attendee images, so that it is possible to see who is speaking. It has become. A technique for emphasizing and displaying a speaker is also disclosed in Japanese Patent Laid-Open No. 2002-218424 (Patent Document 2).

  Japanese Patent Laid-Open No. 2004-343761 (Patent Document 3) proposes a technique for determining conference media provided to a passive participant based on the activity and selection of a certain active participant. In this technology, feedback from a speaker is provided via a back channel, and content is distributed in real time from a server to participants who are not speaking based on the activity of the speaker.

Japanese Patent Laid-Open No. 9-70031 JP 2002-218424 A Japanese Patent Laid-Open No. 2004-343761

  In the speaker emphasis display in Patent Document 1, the speaker is determined based on the judgment of the presenter or the video conference system operator. In Patent Document 2, the determination of the speaker is automatic determination of the video conference system. However, in any case, the will of the speaker is not reflected.

  In Patent Document 3, the only way to provide information is to speak at a meeting, and the information is transmitted to all attendees. Therefore, information cannot be individually communicated only to specific conference attendees. That is, no means is provided for sending information from a conference participant other than the speaker to another specific conference participant.

  The purpose of the present invention is to solve such a conventional problem, and to reflect the will on the screen when the speaker wants to emphasize the speech, and further, it is possible to transmit information between participants other than the speaker of the video conference Another object of the present invention is to provide a video conference system and a control method for the video conference system.

The present invention provides a video conference system including a plurality of video conference terminal devices connected to a video conference server via a network, wherein the video conference terminal device displays a microphone used when speaking and a video of a conference attendee. A display unit, a first speech mode for speaking to all conference attendees, and a second speech mode for allowing all conference attendees to hear and further highlight the speakers on the screen; The means for switching and selecting the third speech mode that speaks so as to be heard only by a specific conference attendant, and when the second speech mode is selected by the video conference terminal device, Means for creating metadata indicating highlighting and transmitting it to the video conference server .

  In the video conference system of the present invention, when the video conference server receives the metadata, the video conference server creates a video conference video that highlights the speaker who transmitted the metadata, and the video conference terminal device of all conference attendees You may make it deliver to.

  In the video conference system of the present invention, when the video conference terminal device selects the means for managing the addresses of other conference attendees and the third speech mode, the specific conference selected. And a means for transmitting voice data to the address of the attendee.

  Further, in the video conference system according to the present invention, the video conference terminal device reproduces the audio by distinguishing between reception of the audio data in the third speech mode and reception of the audio data in the other speech mode. May be further provided.

  In the video conference system of the present invention, the microphone may include a microphone used in the third speech mode and a microphone used in another speech mode.

  In the video conference system of the present invention, when the video conference terminal device receives audio data in the third speech mode, the video of the conference attendee who transmitted the audio data is distinguished from other conference attendees. You may further provide the means to display as possible.

The present invention relates to a control method in a video conference system including a plurality of video conference terminal devices connected to a video conference server via a network so that all video conference attendees can hear in the video conference terminal device. A first speech mode that speaks, a second speech mode that is audible to all conference participants and highlights the speaker on the screen, and a third speech mode that is audible only to certain conference attendees When the video conference terminal device selects the second speech mode, the video conference terminal device creates metadata indicating highlighting and stores the metadata in the video conference server. It is characterized by transmitting .

  In the control method in the video conference system according to the present invention, when the video conference server receives the metadata, the video conference server creates a video conference video that highlights a speaker who has transmitted the metadata, so that all conference participants' televisions are displayed. You may make it deliver to a conference terminal device.

  Further, in the control method in the video conference system according to the present invention, the video conference terminal device manages addresses of other conference attendees, and when the third speech mode is selected, the selected specific mode is selected. Audio data may be transmitted to the address of the meeting attendee.

  Further, in the control method in the video conference system of the present invention, the video conference terminal apparatus distinguishes between the audio data received when the audio data is received in the third speech mode and the audio data received in the other speech mode. You may make it reproduce.

  Further, in the control method in the video conference system according to the present invention, when the video conference terminal device receives the audio data in the third speech mode, the video of the conference attendee who transmitted the audio data is displayed in another conference attendance. You may make it display so that it may be distinguished from a person.

  According to the present invention, when a speaker wants to emphasize a statement, the highlight on the screen can be performed by his / her own operation, and the intention can be accurately reflected. Furthermore, according to the present invention, since a means for selecting and transmitting a specific person without sending a message to all the meeting attendees is provided, it is possible to transmit information between attendees other than the speaker of the video conference. Become.

  Next, the best mode for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration of a video conference terminal device according to an embodiment of the present invention. FIG. 2 is a functional block diagram showing the entire video conference system according to the embodiment of the present invention.

  In FIG. 1, a video conference terminal device 100 includes a first microphone 101, a second microphone 102, an instruction input unit 103, a camera 104, a display unit 105, a voice reception control unit 106, and voice transmission control. Unit 107, metadata creation unit 108, video control unit 109, attendee address management unit 110, and speaker 111.

  As the video conference system, as shown in FIG. 2, a video conference server 201 and a plurality of video conference terminal devices 100 a to 100 c are connected via a network 202. The video conference terminal devices 100a to 100c have the same configuration as the video conference terminal device 100 of FIG.

  In the video conference terminal device 100 in the present embodiment, there are the following three types of modes when transmitting a message. The first mode is a mode in which all the participants in the video conference call speak so that they can be heard but are not highlighted. Here, this is referred to as “conference speech mode”. The second mode is a mode in which all the attendees of the video conference call can speak and are further highlighted, and this is called “emphasized speech mode”. The third mode is a mode in which a specific conference attendee is selected so that only the attendee can hear the speech, and this is called an “individual speech mode”.

  Returning to FIG. 1, details of the video conference terminal device 100 will be described. The first microphone 101 is a microphone that is used when speaking so as to be heard by all the participants in the video conference call. That is, the microphone 101 is used in the conference speech mode and the emphasized speech mode. On the other hand, the second microphone 102 is a microphone used when speaking so as to be audible only to the attendee designated among the attendees of the videophone conference (individual speaking mode).

  The instruction input unit 103 is a functional block for a speaker to input an instruction. When a speaker speaks using the first microphone 101 in a video conference call and wants to emphasize that he / she can hear all attendees, he / she operates the instruction input unit 103 to input an emphasis instruction. To do. In addition, when the speaker wants to speak only to a specific attendee using the second microphone 102, the instruction input unit 103 is used to input an instruction to that effect and necessary information. The instruction input unit 103 can be realized by an input device such as a keyboard, a mouse, and a dedicated button.

  The camera 104 shoots an image that the attendee wants to see, such as the face of the attendee of the videophone conference, and sends it to the teleconference server 201. The display unit 105 displays an image distributed from the video conference server 201 via the network 202 when participating in the video conference call.

  The audio reception control unit 106 is a functional block that determines and processes audio data received from the network 202 and reproduces it through the speaker 111. The audio data received by the audio reception control unit 106 is distributed to all conference attendees by the video conference server 201 and the audio received from the video conference terminal device of the specific attendee attending the video conference. There is voice. The voice transmission control unit 107 controls the transmission destination of the voice data according to the operation of the instruction input unit 103 and which of the first microphone 101 and the second microphone 102 is used.

  The metadata creation unit 108 is a functional block that creates necessary metadata according to the operation of the instruction input unit 103. Metadata is management data describing information about data, and in the case of the present embodiment, it means emphasis. The created metadata is transmitted to the video conference server 201 in association with the audio data.

  The video control unit 109 is a functional block that processes video displayed on the display unit 105. Display control of attendee video distributed from the video conference server 201 via the network 202 and display control of the attendee list managed by the attendee address management unit 110 are performed.

  The attendee address management unit 110 receives the attendee list from the video conference server 201 at the start of the video conference call, and stores the addresses of other attendees. When speaking using the second microphone 102 to communicate only to a specific meeting attendee, the attendee list is displayed by operating the instruction input unit 103 and the destination, that is, the destination to which the voice data is sent To decide.

  Under the control of the audio reception control unit 106, the speaker 111 distinguishes and reproduces audio transmitted from the video conference server 201 and audio transmitted directly from many attendees.

  Next, the operation of the best mode for carrying out the present invention will be described with reference to the drawings. FIG. 3 is a flowchart showing an operation on the video conference terminal device 100 side when an attendee speaks in an embodiment of the present invention. FIG. 4 is a flowchart showing the operation on the video conference server 201 side in the embodiment of the present invention. FIG. 5 is a flowchart showing an audio receiving operation and a reproducing operation on the video conference terminal 100 side in one embodiment of the present invention.

  First, the video conference terminal device 100 participating in the video phone conference accesses the video conference server 201 via the network 202 and performs a conference participation registration process (step S301 in FIG. 3). When the video conference call is started, the video captured by the camera 104 is transmitted to the video conference server 201 and the video for conference is distributed from the video conference server 201, so that it is received and displayed on the display unit 105. The video conference server 201 also distributes an attendee list including attendee names and addresses, which are stored in the attendee address management unit 110. This attendee list can be displayed on a part of the display unit 105 (step S302). The processing of the received voice will be described later.

  If the conference attendee wants to speak (YES in step S303), the conference attendant wants to speak on the conference (if all attendees are asked) (YES in step S304), and if he wants to emphasize the speech (step S305) YES), it can be highlighted on the screen of the video conference terminal device 100 of all attendees that he / she is speaking. That is, the instruction input unit 103 performs an emphasis instruction input operation, and speaks using the microphone 101 (enhancement speech mode) (step S306).

  The metadata creation unit 108 detects the emphasis instruction from the instruction input unit 103 and creates metadata indicating that the sound is emphasized. The voice transmission control unit 107 transmits this metadata to the video conference server 201 in association with the speech voice.

  If the user speaks on the conference but does not particularly want to highlight (NO in step S305), he / she speaks using the microphone 101 (conference speech mode) without inputting an emphasis instruction (step S307). In this case, the audio transmission control unit 107 transmits the audio data to the video conference server 201 without adding metadata meaning emphasis.

  On the other hand, when it is desired to convey a message only to a specific attendee without speaking at the conference (NO in step S304), the instruction input unit 103 is operated to select a message destination from the attendee list. Then, the user speaks using the microphone 102 (individual speech mode) (step S308). In this case, the voice transmission control unit 107 directly transmits the voice data to the selected destination address. In this manner, in the video conference terminal device 100, the process according to the speech situation is repeated until the conference ends (YES in step S309).

  On the other hand, in the video conference server 201, when the attendees are gathered and the video conference call is started (step S401 in FIG. 4), the attendee list is distributed to the video conference terminal device 100 of the attendee (step S402). When video and audio data are transmitted from the video conference terminal device 100 of each attendee, the data is processed for video conference (displaying a plurality of attendees) (step S403). At this time, if the highlighted metadata is added to the received audio data (YES in step S404), a video that emphasizes the attendee of the video conference terminal that has transmitted the data is created (step S405). ). The created video and audio data is distributed to the attendee's video conference terminal device 100 via the network 202 (step S406). Until the conference ends (YES in step S407), the processing from step S403 to step 406 is repeated.

  Next, regarding the voice reception operation in the video conference terminal device 100, the voice reception control unit 106 refers to the transmission source address of the received voice data and determines whether or not the voice data is delivered from the video conference server 201. Determination is made (step S501 in FIG. 5). If it is audio data received from the video conference server 201, it is reproduced with the main audio and output to the speaker 111 (step S502). On the other hand, if it is not the audio data received from the video conference server 201, it is determined that the speech is from another attendee using the microphone 102, and is reproduced with sub audio and output to the speaker 111 (step S503).

  6A and 6B are diagrams illustrating an example of an image displayed on the display unit 105 of the video conference terminal device 100. FIG. FIG. 6A shows an example of a screen 601 in which the video control unit 109 outputs the video for video conference to the display unit 105 based on the attendee list managed by the attendee address management unit 110. In this screen example, the video of each attendee, the attendee number assigned to the attendee, and attendee names (603a to 603d) are displayed on the screen divided into four. In this screen example, since attendee Jhon gives an instruction for emphasis by the instruction input unit 103 of the video conference terminal device 100 (step S306 in FIG. 3), the emphasis display 602 is made (emphasis). Speak mode). This highlighting is easily noticeable, for example, by displaying a red frame.

  FIG. 6B is an example of a screen when selecting a message destination to perform a message (step S308 in FIG. 3) using the second microphone 102 in the individual message mode. When the instruction input unit 103 is operated to indicate the individual speech mode in which the second microphone 102 speaks, the attendee list 610 managed by the attendee address management unit 110 is displayed. Select the address of the utterance by entering the number. The microphone 102 is activated and collected using the microphone 102, and the voice transmission control unit 107 generates voice data and transmits it to the selected destination.

  Although not shown, a display is added on the screen of the display unit 105 of the video conference terminal device 100 that is the destination of the speech in the individual speech mode so that the attendee who has transmitted the voice data can be identified on the screen. May be. That is, in the video conference terminal device 100 that has received the voice data in the individual speech mode, the voice data sender can be identified from the voice data transmission address, so that the sender can be distinguished from the highlight display 602 on the screen. Just do it. For example, if the emphasis display 602 of the emphasis speech mode is displayed in a red frame, it is possible to know at a glance who has transmitted the individual remark by displaying the sender in the individual speech mode in a blue or yellow frame other than that. .

  Note that the switching of the three speech modes in the above embodiment may be performed by operating the instruction input unit 103 each time a speech is to be made, and the speech mode may be selected. For example, the first microphone 101 may be used as the conference speech mode, and the operation input from the instruction input unit 103 may be performed when the emphasized speech mode and the individual speech mode are used.

  Next, another embodiment of the present invention will be described with reference to the drawings. FIG. 7 is a functional block diagram showing a configuration of a video conference terminal device according to another embodiment of the present invention. In FIG. 7, the video conference terminal device 700 includes a microphone 701, a microphone switching control unit 702, an instruction input unit 703, a camera 704, a display unit 705, a voice reception control unit 706, and a voice transmission control unit 707. , A metadata creation unit 708, a video control unit 709, an attendee address management unit 710, and a speaker 711.

  The difference from the embodiment of FIG. 1 is that the second microphone 102 (FIG. 1) is omitted and a microphone switching control unit 702 is provided instead. That is, one microphone 701 is used in common for a speech on the conference (conference speech mode and emphasis speech mode) and a speech to a specific partner (individual speech mode), and the control operation thereof is the microphone switching control unit 702. Switching with. For example, if there is no particular instruction input at the instruction input unit 703, processing is performed in the conference speech mode (corresponding to the processing in step S307 in FIG. 3), and if there is an emphasis instruction input, processing is performed in the enhanced speech mode (step S306 in FIG. 3). Equivalent to processing). Further, if there is an instruction input in the individual speech mode in the instruction input unit 703, a process of selecting the other party and transmitting voice data is performed (corresponding to step S308 in FIG. 3). Since the configuration and operation of each of the other functional blocks are the same as the corresponding functional blocks in the embodiment of FIG.

  In addition, this invention is not limited only to the above-mentioned embodiment, It can implement in various deformation | transformation in the range which does not deviate from the summary of this invention.

It is a functional block diagram which shows the structure of the video conference terminal device in one Embodiment of this invention. It is a functional block diagram which shows the whole video conference system in one Embodiment of this invention. It is a flowchart which shows the operation | movement by the side of the video conference terminal device in case an attendant speaks in one Embodiment of this invention. It is a flowchart which shows the operation | movement by the side of the video conference server in one Embodiment of this invention. It is a flowchart which shows the audio | voice reception operation | movement and reproduction | regeneration operation | movement by the side of the video conference terminal in one Embodiment of this invention. (A) And (b) is a figure which shows an example of the image | video displayed on the display part of the video conference terminal device in one Embodiment of this invention. It is a functional block diagram which shows the structure of the video conference terminal device in other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Video conference terminal device 101 Microphone 102 Microphone 103 Instruction input part 104 Camera 105 Display part 106 Audio | voice reception control part 107 Audio | voice transmission control part 108 Metadata production part 109 Video control part 110 Attendee address management part 111 Speaker 201 Video conference server 202 Network 700 Video conference terminal device 701 Microphone 702 Microphone switching control unit 703 Instruction input unit 704 Camera 705 Display unit 706 Audio reception control unit 707 Audio transmission control unit 708 Metadata creation unit 709 Video control unit 710 Attendee address management unit 711 Speaker

Claims (11)

In a video conference system including a plurality of video conference terminal devices connected to a video conference server via a network,
The video conference terminal device
A microphone to use when speaking,
A display for displaying the video of the attendees;
A first speech mode that speaks to all conference attendees, a second speech mode that is audible to all conference attendants and highlights the speakers on the screen, and a specific conference attendance Means for switching and selecting a third speech mode in which the user speaks so as to be heard only by the user,
Means for creating metadata indicating highlighting and transmitting to the video conference server when the video conference terminal device selects the second speech mode;
A video conference system characterized by comprising: The video conference server, upon receiving the metadata, creates a video conference video that highlights a speaker who has transmitted the metadata, and distributes the video conference video to video conference terminal devices of all conference participants. 1. The video conference system according to 1 . The video conference terminal device
A means of managing the addresses of other meeting attendees;
Means for transmitting audio data to the address of the selected particular attendee when the third speech mode is selected;
Further videoconferencing system according to any one of claims 1 to 2, characterized in that with a. The video conference terminal device further comprises means for reproducing audio by distinguishing between receiving audio data in the third speaking mode and receiving audio data in another speaking mode. Item 4. The video conference system according to any one of Items 1 to 3 . The video conference according to any one of claims 1 to 4 , wherein the microphone includes a microphone used in the third speech mode and a microphone used in another speech mode. system. The video conference terminal device further includes means for displaying the video of the conference attendee who transmitted the audio data so as to be distinguishable from other conference attendees when receiving the audio data in the third speech mode. The video conference system according to any one of claims 1 to 5 , wherein: A control method in a video conference system including a plurality of video conference terminal devices connected to a video conference server via a network,
In the video conference terminal device,
A first speech mode that speaks to all conference attendees,
A second speech mode that is audible to all attendees and highlights the speaker on the screen;
A third speech mode that speaks only to certain meeting attendees,
, Select by switching the speaker ,
In the video conference system, the video conference terminal device creates metadata indicating highlighting when the second speech mode is selected, and transmits the metadata to the video conference server . Control method. The video conference server, when receiving the metadata, creates a video conference video that highlights a speaker who has transmitted the metadata, and distributes the video conference video to video conference terminal devices of all conference participants. 8. A control method in the video conference system according to 7 . In the video conference terminal device, addresses of other conference attendees are managed, and when the third speech mode is selected, audio data is transmitted to the selected address of the specific conference attendee. control method in the video conference system according to any one of claims 7 to 8, characterized. The teleconference terminal device, and when the audio data received at the third speech mode, at the time of the voice data received at other remarks mode, distinguish of claims 7 to 9, characterized in that the sound reproduced The control method in the video conference system as described in any one. When the video conference terminal device receives audio data in the third speech mode, the video conference terminal device displays the video of the conference attendee who transmitted the audio data so that it can be distinguished from other conference attendees. The video conference system according to any one of claims 7 to 10 .

Images