JPH09307869A - Video conference system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display participants without a sense of incongruity while emphasizing a participant making an utterance at the point of time by deciding the sequence of utterance from voice data received by a server and deciding which participant's image is to be synthesized based on the sequence of utterance and displaying limited participant. SOLUTION: A communication equipment 13 of a server 12 receives communication data from a terminal equipment 1 and a data controller 14 divides the data into moving image and voice data. An utterance sequence decision device 15 decides the sequence of utterance by participants from the voice data, and a data synthesis controller 16 decides which participant's image is to be synthesized based on the sequence data. A data synthesizer 17 selects a participant's image among image data by an instruction from the data synthesis controller 16 and reduces the data and converts the data into synthesis image data. The data controller 14 superimposes again the received data received from the data synthesizer 17 onto the converted data and transfers the resulting data to each terminal equipment via the communication equipment 13 as communication data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to displaying moving image data of conference participants in a system for transferring moving image data and audio data on a network to hold a conference.

[0002]

2. Description of the Related Art A system for connecting a terminal such as a personal computer using a communication line to transfer a moving image, a voice, data, and the like and hold a conference has been conventionally used.

FIG. 7 is a block diagram illustrating a conventional system. The system includes terminal devices 48, 56 to 58, a server 59 and a communication line 55.

Each terminal device, for example terminal device 48,
A camera 49 for capturing video and a microphone 5 for capturing audio
0, moving image data from the camera, voice data from the microphone is converted into data for communication, while the data control device 53 for dividing the data received from the opposite terminal into moving image data, voice data, The communication device 54 includes a communication device 54 that transfers communication data issued by the own terminal device to the server and receives communication data from another terminal device from the server.

The server 59 receives communication data sent from each terminal device and transfers communication data to each terminal, and a communication device 60 that receives the data received from the communication device as moving image data and voice data. Of the data received from the data control device 61 and the data control device 61 that divides the data and sends it to the data synthesis device and converts the data received from the data synthesis device 62 into communication data,
A data synthesizing device 62 is provided for synthesizing image data and audio data, and passing them to the communication device.

The data of the conference participants are input as moving image data from the camera of each terminal device and as voice data from the microphone.

When each of the above data is input,
The data control device of each terminal device superimposes each data as communication data and passes it to the communication device in its own terminal device.

The communication device of each terminal device transfers the communication data to the server 59.

The communication device of the server receives the communication data from the communication device of the terminal device and transfers it to the data control device.

In the data control device, the received data is divided into moving image data and audio data, and each data is passed to the data synthesizing device.

The data synthesizer reduces the received moving image data and converts it into an image in which all the participants are displayed as shown in FIG. The voice data is also converted into a state in which all the participants can receive the same data, and each data is passed to the data control device.

The data control device again superimposes the data received from the data synthesizing device and passes it as communication data to the communication device.

The communication device converts the received communication data into
Transfer to the communication device of each terminal device.

Each terminal passes the communication data received by its own communication device to the data control device, and the data control device divides the communication data into respective data, and then passes the data to the reproduction device suitable for each data, Each data is reproduced.

[0015]

Conventionally, when a video conference is held, most of the participants in the conference are connected to each other at two points, and at most four points are connected. In this case, regarding the display of the moving image of each participant, in the case of the opposite, the other image is displayed, and in the case of four-point connection, the display screen is divided into four, and the image from each point is displayed. .

When the number of participating terminals increases as it is, the display screen is further divided and displayed. However, since the display used in the video conference has a limited resolution, the display area of the moving images of all of the participants becomes smaller as the number of participating terminal devices increases, which may reduce the usefulness of the video conference. It can happen.

An object of the present invention is to provide a method of holding a conference without feeling discomfort in a video conference system connecting multiple points.

[0018]

In order to achieve the above object, the present invention displays, on the screen, a limited number of participants based on the order in which they speak in the video conference system described above. By doing so, it is possible to provide a means for conducting a video conference without feeling uncomfortable without reducing the display image of each participant.

In fact, as a criterion for selecting participants, the present invention uses the order of speech.

Therefore, in the system configuration of the present invention, specifically, the speech order determining device that determines the order of speech from the voice data received by the server, and the image of which participant is selected depending on the determined order. And a data synthesis control device for determining whether to synthesize.

The utterance order determining device determines the order in which the participants utter, and the data synthesizing control device displays the participants whose utterance order is new in the moving image display screen according to the uttering order.
The images of the participants that can be displayed are combined.

When a participant other than the displayed participant speaks and exceeds the displayable number, the participant who speaks first among the displayed participants is removed from the composite image, and Add the speaking participant to the composite image.

By synthesizing the display images of the participants according to the above procedure, it is possible to display the participants who are participating in the current discussion at the conference and the current speaker.

Further, since it is not necessary to display all the participants, it is possible to hold a video conference without reducing the display area per participant even with a display device having a limited display area.

[0025]

FIG. 1 is a block diagram illustrating the present invention.

The system comprises terminal devices 1, 9, 10, 11,
It is composed of a server 12 and a communication line 8.

The terminal device 1 receives the respective data of the camera 2 which captures the moving image data and the microphone 3 which captures the voice data, and uses them as communication data, and also the server 12
The data control device 6 for dividing the communication data received from the device into moving image data and audio data, the display 5, the speaker 4 for reproducing each data, the communication data transferred to the server, and the server for communication. A communication device 7 for receiving data is provided. The terminal devices 9, 10 and 11 have the same configuration.

The server 12 receives the communication data sent from each terminal device and transfers the communication data to each terminal; and the data received from the communication device 13 as voice data and image data. The data is divided into data and sent to the data synthesizing device 17 as well as the data synthesizing device 17
A data control device 14 for converting data received from the device into communication data; voice data is received from the data control device 14, the speech order of the participating terminals is determined, and then the voice data is sent to the data synthesizing device 17 A speech order determination device 15 to be passed to the synthesis control device 16; image data is synthesized into one screen under the control of the data synthesis control device,
A data synthesizer 17 that synthesizes the voice data sent from each terminal as one voice data and passes it to the communication device 13.
And a data synthesis control device 16 for determining which participant is to be synthesized as image data based on the sequence data received from the speech sequence determination device 15 and controlling the data synthesis device 17.

The data of the conference participants of each terminal are input as moving image data from the camera 2 of the terminal device 1 and as voice data from the microphone 3. When each of the above data is input, the data control device 6 superimposes each data as communication data and passes it to the communication device 7. The communication device 7 transfers the communication data to the server 12.

The communication device 13 of the server 12 is the terminal device 1
The communication data is received from the communication device 7 and passed to the data control device 14.

The data control device 14 divides the received data into moving image data and audio data, and delivers the audio data to the utterance order determining device 15 and the moving image data to the data synthesizing device 17.

When the participant utters a voice of a certain level or higher from the received voice data, the utterance order determining device 15 regards the participant as a speaker and adds the utterance to the order data. The sequence data is passed to the data synthesizing device 17 and the data synthesizing control device 16.

The data synthesizing control unit 16 determines which participant's image is to be synthesized from the received sequence data.
Notify the data synthesizer 17.

The data synthesizing device 17 selects the image of the participant according to the instruction from the data synthesizing control device 16 from the received image data, reduces the image, and then transforms it as the synthetic image data 35 as shown in FIG. The voice data is converted so that each participant receives voice data spoken by a participant other than himself, and each data is passed to the data control device 14.

The data controller 14 is the data synthesizer 1
7 is superimposed again and passed to the communication device 13 as communication data.

The communication device 13 transfers the received communication data to the communication device 7 of each terminal device.

The terminal device 1 passes the communication data received by the communication device 7 to the data control device 6, and the data control device 6 divides the communication data into respective data, and then selects a reproduction device suitable for each data. The moving image data is reproduced on the display 5, and the audio data is reproduced on the speaker 4.

Next, the first embodiment of the present invention will be described in detail.

This system, as shown in FIG.
8 to 34, a server 26 and a communication line.

As described with reference to FIG. 1, the terminal device receives the respective data of the camera 2 for capturing the image, the microphone 3 for capturing the sound, and uses them as communication data or data received from the server 12. A data control device 6 for dividing each data, a display 5 for reproducing image data, a speaker 4 for reproducing audio data, a communication device for transferring communication data to a server and receiving communication data from the server Equipped with 7. It is assumed that the display 5 can display up to four people.

The server 12 receives the communication data sent from the terminal device 1 or transfers the communication data to the terminal device 1, and the data received from the communication device 13 as image data. A data control device 14 for dividing the data into voice data and sending the data to the data synthesizing device 17 and converting the data received from the data synthesizing device 17 into communication data, and the voice data from the data control device 14, and the speech order of the participants And the speech synthesizing device 17 for passing the speech data to the data synthesizing device 17 and the speech order to the data synthesizing control device 16, and the data synthesizing device 17 for synthesizing the image data and the audio data and passing them to the communication device. , Which participant is to be combined as image data is determined by the sequence data received from the speech sequence determination device 15. And a data synthesis control apparatus 16 which controls the data synthesizer 17.

The data of the conference participants are input as moving image data from the camera 2 of the terminal device 1 and as voice data from the microphone 3.

When each data is input, the data control device 6 of the terminal device 1 superimposes each data as communication data and passes it to the communication device 7.

The communication device 7 transfers the communication data to the server 12.

The communication device 13 of the server 12 is the terminal device 1
The communication data is received from the communication device 7 and passed to the data control device 14.

The data control device 14 divides the received data into moving image data and audio data, and delivers the audio data to the utterance order determining device 15 and the image data to the data synthesizing device 17.

When the participant utters a voice with a certain level or higher from the received voice data, the speech order determining device 15 regards the participant as a speaker and adds it to the sequence data. The sequence data is passed to the data synthesizing device 17 and the data synthesizing control device 16.

The data synthesizing control device determines which participant's image is to be synthesized from the received sequence data, and informs the data synthesizing device.

The data synthesizing device 17 selects the image of the participant in accordance with the instruction from the data synthesizing control device 16 from the received image data, reduces the image data, and then transforms it as synthetic image data as shown in FIG. The voice data is converted so that each participant receives voice data spoken by a participant other than himself, and each data is passed to the data control device 14.

The data controller 14 is the data synthesizer 1
7 is superimposed again and passed to the communication device 13 as communication data.

The communication device 13 transfers the received communication data to the communication device 7 of the terminal device 1.

The terminal device 1 passes the communication data received by the communication device 7 to the data control device 6, and the data control device 6 divides the communication data into respective data, and then makes a reproduction device suitable for each data. The moving image data is reproduced on the display 5, and the audio data is reproduced on the speaker 4.

The method of determining the display order in this case is performed in the procedure as shown in FIG. In the first embodiment, as shown in FIG.
Steps 22 and 23 are unnecessary.

As shown in FIG. 4, the numbers 1 to 4 are assigned to the respective divided screens, and the participants in front of the terminals A, B, C, D, E,
F, G, H.

In the initial state, no one is displayed on each screen, and when the conference is started (step 19), the numbers 1 to 4 are displayed in the order of utterance. When A, B, C, and D speak in this order, and E, F, G, and H do not speak, the display 1 to 4 is displayed as the screen 36 of FIG. 5 (step 19 and 20). After that, when the non-display person E speaks in the order of A, C, and D (step 21), the participant B who is not speaking out of the displayed participants is replaced. , Participant E is displayed in the position of 2 which B occupied until then. Next, if participant F speaks, the order of speaking is A, C, D, E, F.
Then, the participant F is displayed at the position 1 instead of the participant A (step 24). According to the above procedure, the participant determines whether to display or not to display on the screen according to the statement.

At this time, in order to avoid frequent changes in the display positions of the participants, the display position is not determined by the speaking order, but the participant who has spoken first among the displayed participants is displayed. Use the method of replacing the participants who have not spoken. In this case, the actual screen change is as shown in FIG.

Of course, if the display position can be moved frequently, the utterance order and the display position on the screen may be matched.

Next, the second embodiment will be described.

When it is desired that the chairperson of the conference is always displayed on the screen, it can be realized by setting a fixed display for any participant and switching the display between the other participants. The processing flow in this case is as shown in FIG. 2, and steps 22 and 23, which are unnecessary in the first embodiment, are required.

In the above system, when the participant A is fixed, the participant F is displayed instead of the participant C. The actual flow is as shown in FIG.

[0061]

As described above, the present invention does not display all the participants on the display device, but only the images from the predetermined number of terminals from the conference participants who spoke at the time closest to the present time. Since it is displayed, it is possible to display the participants who are mainly speaking at the time of the conference. As a result, in a video conference system in which the resolution of the display device is not so high and it is difficult to display all the participants, it is possible to display the participants without any discomfort.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is a flowchart illustrating a procedure for determining a display order.

FIG. 3 is a configuration diagram of an embodiment of the present invention.

FIG. 4 is a moving image display screen according to the embodiment of the present invention.

FIG. 5 is a diagram showing a change in screen configuration according to the first embodiment of the present invention.

FIG. 6 is a diagram showing a change in screen configuration according to the second embodiment of the present invention.

FIG. 7 is a block diagram showing a configuration of a conventional video conference system.

[Explanation of symbols]

 1,9,10,11 Terminal device 2 Camera 3 Microphone 4 Speaker 5 Display 6,14 Data control device 7,13 Communication device 15 Speaking order determination device 16 Data synthesis control device 17 Data synthesis device

Claims (4)

[Claims] 1. A video conferencing system in which a plurality of terminals are connected to a center via a communication line, and moving image data and audio data are transferred between the plurality of terminals, wherein the center is connected from the plurality of terminals. Means for collecting moving image data and audio data, and ordering the plurality of terminals based on the audio data from the plurality of terminals, and outputting selection information indicating moving image data from the terminals to be displayed in this order. And a synthesizing means for synthesizing the moving image data from the terminal designated by the selection information into one screen of moving image and transferring the synthesized moving image to the plurality of terminals. . 2. The selecting means determines the speaking order of the conference participants on a terminal-by-terminal basis based on the voice data from the plurality of terminals, and orders each terminal, and a predetermined number of selected terminals are selected based on this ordering. The video conference system according to claim 1, wherein information indicating moving image data from a terminal as a display target is output as the selection information. 3. The selection means outputs, as the selection information, information indicating, as display targets, moving image data from a terminal selected based on the ordering and a specific terminal. Video conferencing system as described. 4. The video according to claim 3, wherein the synthesizing unit synthesizes the moving image data for one screen so that the moving image data from the specific terminal is displayed at a predetermined position on the screen. Conference system.