JPH08321884A - Multi-spot video conference system - Google Patents

Abstract

PURPOSE: To realize a multi-spot video conference system capable of eliminating unnaturalness and inconvenience caused by the connection of a new video conference terminal during a multi-spot video conference or the disconnection of the participating video conference terminal. CONSTITUTION: A video capacity identifying part 6 identifies whether a video conference terminal which should be informed of information concerning the variation of the number of terminals is provided with video capacity or not. Based on the result of the identification result, an informing method selection part selects a video synthesizing part 4 and a speech synthesizing part 5. Namely, when the video conference terminal of the informing destination is provided with video capacity, information concerning the variation of the number of the terminals is given through the use of the video synthesizing part 4. On the other hand, when the video terminal is not provided with video capacity, information concerning the variation of the number of the terminals is given through the use of the speech synthesizing part 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multipoint video conference system for conducting a multipoint video conference by connecting a plurality of video conference terminals to a multipoint communication control device.

[0002]

2. Description of the Related Art In recent years, the development of image compression coding technology and IS
With the spread of high-speed and high-quality digital lines represented by DN, the recommendation for AV (audiovisual) service is ITU-T Recommendation H.264. 320, H.M. 242 and H.H. Two
Various video conferencing systems have been developed in accordance with these recommendations, as defined by 21.

As such a video conference system, 3
A multipoint videoconferencing system capable of conducting a videoconference by connecting multiple points or more has already been devised. In the multipoint video conference system, a multipoint communication control device (hereinafter, referred to as MC) for controlling a plurality of video conference terminals participating in the multipoint video conference is controlled.
A method of forming a multipoint connection via the MCU by connecting to the U.).

As an example of the multipoint connection using the MCU, FIG. 6 shows a connection form when a multipoint video conference is held at four points. In the figure, the video conference terminals 602-
605 is an IS to join a multipoint video conference.
It is connected to the MCU 601 via DN.

According to the conventional multipoint video conference system for connecting multipoints by using the MCU 601, the MCU 601 distributes video to each of the video conference terminals 602 to 605 during multipoint communication. , 1) A method of identifying the current speaker terminal by comparing the audio levels from the respective video conference terminals 602 to 605, and distributing the video sent from the identified speaker terminal, 2) Current connection A method of switching the images sent from the inside video conferencing terminals 602 to 605 at regular intervals and sequentially delivering them, 3) The video from all the video conferencing terminals connected to the multipoint by the MCU 601 is regarded as one image. A method of synthesizing and delivering the synthesized video, or 4) delivering video from a specific video conference terminal according to an instruction from the operation unit of the MCU 601. The method is proposed.

[0006]

However, in the methods of 1), 2) and 4) above, a new video conference terminal is connected in the middle of a multi-point video conference, or a multi-point video conference is involved. If the communication with the existing video conference terminal is disconnected, the information about the connection status of such a video conference terminal is not notified to other video conference terminals of the multipoint video conference, so that the TV that has not been connected until then is disconnected. There is a problem in that the user of the conference terminal suddenly makes a speech, or even if there is a speech to the user of the video conferencing terminal that has been connected until then, there is no response to the speech, which is unnatural or inconvenient. .

Further, even in the case of the method 3) in which the MCU 601 synthesizes and distributes the images from the video conference terminals participating in the multi-point video conference, it is newly connected during the video conference. If the video capability of the video conference terminal is turned off or if the video capability of the video conference terminal to which the composite video is delivered is turned off, the status cannot be known from other video conference terminals. The same problem as above was encountered.

The present invention has been made to solve the above problems, and when a new video conference terminal is connected during a multipoint video conference or communication with a participating video conference terminal is disconnected. It is an object of the present invention to provide a multipoint video conference system that can eliminate the unnaturalness and inconvenience caused by being disturbed.

[0009]

In order to achieve the above-mentioned object, the present invention provides a multipoint video conference for performing a multipoint video conference by connecting a plurality of video conference terminals to a multipoint communication control device. In the system, the multipoint communication control device includes a terminal information storage unit that stores terminal information of a videoconference terminal participating in the multipoint video conference, and a television participating in the multipoint video conference. When the number of terminals monitoring means for monitoring the number of terminals of the conference terminal, and the change in the number of terminals is detected by the number of terminals monitoring means,
Terminal information notifying means for notifying the information about the change in the number of terminals to the video conference terminals participating in the multi-point video conference.

Further, the terminal information notifying means has an image synthesizing means for converting the information regarding the change in the number of terminals into at least one of character data and bit map data and overlaying it on a transmission image.

It is also preferable that the terminal information notifying means has a voice synthesizing means for voice synthesizing the information regarding the change in the number of terminals and adding it to the transmitted voice.

Further, the terminal information notifying means converts the information regarding the change in the number of terminals into at least one of character data and bitmap data and overlays the transmitted image, and the information regarding the change in the number of terminals. The voice synthesizing means for synthesizing voice and adding to the transmitted voice, the video ability identifying means for identifying the video ability of each video conference terminal participating in the multipoint video conference, and the video ability identifying means are identified. It is also preferable to have a selecting means for selecting either the image synthesizing means or the voice synthesizing means based on the video capability.

Further, the information regarding the change in the number of terminals is
It may be composed of the number of terminals after the change and the terminal information of the video conference terminal which is a factor of the change of the number of the terminals stored in the terminal information storage means.

[0014]

According to the multipoint video conference system of claim 1, when the change in the number of terminals is detected by the terminal number monitoring means, the information about the change in the number of terminals participates in the multipoint video conference. The video conference terminal is notified.

Further, according to the multipoint video conference system of claim 2, the information relating to the change in the number of terminals is converted into at least one of character data and bitmap data by the image synthesizing means, and is overlaid on the transmission image. To be done.

According to the multipoint video conference system of claim 3, the information about the change in the number of terminals is voice-synthesized by the voice synthesizing means and added to the transmitted voice.

Further, according to the multi-point video conference system of claim 4, the selecting means selects either the image synthesizing means or the voice synthesizing means based on the video capability identified by the video capability identifying means. To be done.

[0018]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the functional arrangement of a multipoint control unit (MCU) according to this embodiment.

In the figure, the terminal information storage unit 1 stores information (terminal information) of each video conference terminal (hereinafter referred to as a participating video conference terminal) participating in a multipoint video conference, such as a calling address and a user abbreviation. It stores information. The terminal information stored in the terminal information storage unit 1 is directly input from the operation unit (not shown) of the MCU before the multipoint video conference is held, or
When connected to the U, it is assumed that the video conference terminal transmits to the MCU by using LSD (low-speed data) specified by the ITU-T recommended H series and T series.

The terminal information storage unit 1 is connected to the terminal information notification unit 3. The terminal information notifying unit 3 converts the terminal information notified to the participating video conference terminals into at least one of character data and bitmap data and overlays the transmitted image, and the terminal information to notify the participating video conference terminals. A voice synthesizing unit 5 for synthesizing and adding to the transmitted voice, and a notification method selecting unit (hereinafter, referred to as selecting unit) 7 for selecting either the image synthesizing unit 4 or the voice synthesizing unit 5 to notify the terminal information. Have and.

The terminal information notifying section 3 receives a plurality of I's via the terminal number monitoring section 2 for monitoring the number of participating video conference terminals.
Line interface 21 for controlling the SDN communication line
Connected to 1. Further, the image synthesizing unit 4 and the voice synthesizing unit 5 are connected to the line interface 211 via the image coding / decoding unit 207, the voice coding / decoding unit 205, and the demultiplexing unit 210, respectively. The line interface 211 is connected to a plurality of ISDN communication lines and controls each connected line according to the ISDN user network interface.

The selection unit 7 identifies whether or not the video conference terminal to which the terminal information is notified has video capability based on the BAS (bit allocation signal) defined in the ITU-T Recommendation H series. It is connected to the unit 6 and selects either the image synthesizing unit 4 or the voice synthesizing unit 5 based on the identification result of the video capability identifying unit 6. For example, if the videoconference terminal to which the terminal information is notified has video capability, the image combining unit 4 is used to notify the terminal information, and if the videoconference terminal to which the terminal information is notified does not have video capability, the audio information is output. The combining unit 5 is used to notify the terminal information. Also, if the video conference terminal to which the terminal information is notified has video capability,
It is also possible to notify the terminal information using both the image synthesizing unit 4 and the voice synthesizing unit 5.

In the above configuration, when the terminal number monitoring unit 2 monitors the incoming call from the video conference terminal to the MCU and the disconnection of the communication with the participating video conference terminals, the change in the number of the participating video conference terminals is detected. The terminal information of the video conference terminals that has caused the change in the number of participating video conference terminals is retrieved from the terminal information storage unit 1 by the terminal information notification unit 3. The extracted terminal information is overlaid on the transmission image by the image synthesizing unit 4 or added to the transmission voice by the voice synthesizing unit 5 according to the selection by the selecting unit 7, together with the information regarding the change in the number of other terminals. Image synthesizer 4
The image signal generated by the above or the audio signal generated by the audio synthesizing unit 5 is encoded by the image encoding / decoding unit 207 or the audio encoding / decoding unit 205, and then converted into a transmission frame by the demultiplexing unit 210. It is multiplexed and notified to each participating video conference terminal from the line interface 211.

FIG. 2 shows an M that can realize the above-described functional configuration.
It is a block diagram which shows the structure of CU. The operation of the MCU 200 is controlled by the CPU 201 having the functions of the terminal number monitoring unit 2, the terminal information notification unit 3, the video capability identification unit 6, and the notification method selection unit 7 described above.

The CPU 201 is used as a ROM (Read Only Memory) 202 that stores a control program and a terminal information storage unit 1 that stores data such as terminal information, and is also used as a work area of the CPU 201. RAM (Random Access Memory) 203 and user M
An operation unit 204 having an input device (not shown) such as a keyboard, a tablet, and a mouse used for inputting control information for controlling the CU 200, a voice processing unit 206 for generating a voice signal, and a voice signal A voice encoding / decoding unit 205 for performing encoding and decoding of the image, an image processing unit 208 for generating an image signal, an image encoding / decoding unit 207 for encoding and decoding the image signal, a personal computer, etc. A channel interface 209 for connecting external data terminal devices of the same, a demultiplexing unit 210 for performing a multiplexing process and a demultiplexing process for various data, and a plurality of ISs.
Line interface 21 connected to the DN line and controlling each line according to the ISDN user network interface
1 is connected via a bus 212.

The voice encoding / decoding unit 205 uses the ITU-T
According to the audio signal encoding / decoding algorithm defined in the Recommendation G series, the audio signal transmitted from the participating video conference terminal is decoded and the audio signal transmitted to the participating video conference terminal is encoded.

The voice processing unit 206 has the function of the voice synthesizing unit 5 shown in FIG. 1, voice-synthesizes the terminal information notified to the participating video conference terminals to generate a voice signal, and the voice signal is converted into a voice signal. After addition processing is performed on the audio signal decoded by the encoding / decoding unit 205, it is sent to the audio encoding / decoding unit 205 again.

The image encoding / decoding unit 207 is a ITU-T
Recommendation H. In accordance with the image signal encoding / decoding algorithm defined by H.261, the image signal transmitted from the participating video conference terminal and the image signal transmitted to the participating video conference terminal are encoded.

The image processing unit 208 has the function of the image synthesizing unit 4 shown in FIG. 1, converts the terminal information notified to the participating video conference terminals into at least one of character data and bitmap data, and the converted data. Is overlaid on the image signals from the participating video conference terminals decoded by the image coding / decoding unit 207, or the images from all the participating video conference terminals are combined, and then the image coding / decoding unit 2 is again processed.
It is sent to 07.

The demultiplexing unit 210 complies with ITU-T Recommendation H.264. 221, an audio signal sent from the audio encoding / decoding unit 205, an image signal sent from the image encoding / decoding unit 207, and a channel interface 209.
The data input from the external data terminal device connected to the outside through the control data and the control data generated by the control program executed by the CPU 201 are multiplexed in transmission frame units according to the current communication capability and communication mode. The external data terminal device or the CPU 20 which is connected in the audio coding / decoding unit 205, the image coding / decoding unit 207, and the channel interface 209 after the multiplexing process and the received frame are separated into each medium of the constituent unit.
The separation process for sending to 1 is performed.

FIG. 3 is performed in the CPU 201.
It is a flowchart which shows the monitoring processing procedure of the number of participating video conference terminals. The program for executing this process is RO
The video conference terminal stored in the M202 and participating in the multipoint video conference is activated when the video conference terminal is first connected to the MCU 200.

First, the value of the variable n for counting the current number of participating video conference terminals is initialized to "1" (step S100). Next, it is determined whether there is a new incoming call from the video conference terminal, that is, whether the number of participating video conference terminals has increased (step S101). If it is determined that there is an incoming call, the value of the variable n for counting the number of participating video conference terminals is incremented by 1 (step S
102), the reason why the number of participating video conference terminals has changed (here, a new video conference terminal has been connected) and the terminal information (for example, the calling address) of the newly connected video conference terminal are stored in the RAM 203 (step S10
3), the terminal information notification process described later is performed (step S108). When the terminal information notification process is completed, the processes after step S101 are performed again.

On the other hand, if it is determined in step S101 that there is no incoming call from the new video conference terminal, it is determined whether or not there is a video conference terminal of which communication with the MCU 200 has been disconnected among the participating video conference terminals ( If there is a video conference terminal whose communication has been disconnected (step S104), the value of the variable n for counting the number of participating video conference terminals is decremented by 1 (step S105). Then, as in step S103, the factors that change the number of participating video conference terminals (here, the communication with the video conference terminal is disconnected) and the terminal information (calling address, etc.) of the video conference terminal whose communication has been disconnected are It is stored in the RAM 203 (step S106). Furthermore, it is determined whether or not there is a participating video conference terminal to which information regarding the disconnection of communication with the video conference terminal determined in step S104 should be notified (step S107), and there is a participating video conference terminal to which information should be notified. If it is determined that the terminal information notification processing described later is performed (step S108). When the terminal information notification process is completed, the processes after step S101 are performed again. If it is determined that there is no participating video conference terminal to which information is to be notified, this processing operation ends.

In the above-mentioned step S107, whether or not there is a participating video conference terminal to which information should be notified is determined by a variable n for counting the number of participating video conference terminals.
The value of. That is, when it is determined in step S104 that there is a video conference terminal whose communication has been disconnected, the value of the variable n becomes 0 or 1 or more by the process of step S105. Therefore, if the value of the variable n is 0, information is output. It is determined that there is no participating video conference terminal that should be notified,
When the variable n is 1 or more, it is determined that there is a participating video conference terminal to which information is to be notified.

FIG. 4 is a flowchart showing the terminal information notification processing procedure started in step S108.

First, the RAM 2 is preliminarily based on the information (calling address, etc.) about the video conference terminals that have caused the change in the number of participating video conference terminals stored in the RAM 203 in step S103 or step S106.
More detailed terminal information about the video conference terminal stored in 03, for example, the user abbreviation is retrieved (step S201).

Next, the variable i for counting the number of participating video conference terminals notified of the information regarding the change in the number of participating video conference terminals is initialized to "0" (step S20).
2) By comparing the value of the variable i with the number of participating video conference terminals, it is determined whether or not there is a video conference terminal to which information regarding the change in the number of participating video conference terminals has not been notified (step S203). . If the value of the variable i is smaller than the number of participating video conference terminals in step S203, that is, if there is a video conference terminal for which information regarding the change in the number of participating video conference terminals is not notified, the process proceeds to step S204. If it is determined that there is no video conference terminal to which the information regarding the change in the number of participating video conference terminals has not been notified, the terminal information notification process ends.

As the number of participating video conference terminals compared with the value of the variable i, the number n of participating video conference terminals used in the flowchart shown in FIG. 3 may be used. Since the number n of participating videoconference terminals includes the videoconference terminals that are the factors of the incoming call from the new videoconference terminals detected in step S101, However, information regarding the change in the number of participating video conference terminals is notified.

Next, data indicating the presence or absence of video capability of the "i-th video conference terminal" which is the notified video conference terminal for notifying the information regarding the change in the number of participating video conference terminals is acquired (step S204). The presence or absence of video capability is determined based on the data (step S205). Whether or not the video capability is present is sent to the MCU 200 when the video conference terminal is connected to the MCU 200.
It is identified from the BAS (bit allocation signal) specified in the TU-T recommendation H series.

If it is determined that the i-th video conference terminal has the video capability, the terminal information extracted in step S201 is included in the transmission video to be transmitted to the i-th video conference terminal. It is converted into at least one of character data and bitmap data together with other information regarding the change in the number of conference terminals, and is overlaid on the transmission image and transmitted (step S206). In this way, the information regarding the change in the number of participating video conference terminals is notified.

If it is determined that the i-th video conference terminal does not have the video capability, step S201.
The terminal information extracted in step (4) is voice-synthesized with other information related to the change in the number of participating video conference terminals, and is added to the transmission voice transmitted to the i-th video conference terminal to change the number of participating video conference terminals. Notified (step S2)
07).

When the process of step S206 or step S207 is completed, the value of the variable i is incremented by 1 (step S208), and the process returns to step S203.

In FIG. 5, when the number of participating communication terminals increases due to the arrival of a new video conference terminal, the received image of the video conference terminal when the information about the change in the number of terminals is notified by overlaying on the transmitted video. An example is shown.

In the figure, 501 is an image (speaker image) from a video conference terminal identified as a speaker terminal, and the speaker image 501 is transmitted to each of the present participating video conference terminals. The speaker video conference terminal is MCU20.
It is identified by comparing the audio levels of the audio signals sent from each participating video conference terminal sent to 0.

When the MCU 200 receives an incoming call from a new video conference terminal, the information notifying that the video conference terminal is newly connected is overlapped with the area 502 of the speaker video (transmission video) 501. As shown in FIG.
R is used as information to notify the arrival of a new video conference terminal.
The user abbreviations of the video conference terminals stored in advance in the AM 203 and the current number of participating video conference terminals (including the video conference terminal added by this incoming call) are notified.

As described above, according to this embodiment,
Notifying participating videoconference terminals of information related to changes in the number of participating videoconference terminals due to new connection of videoconference terminals or disconnection of communication with videoconference terminals participating in videoconference. You can Therefore, the user participating in the multipoint video conference can grasp the information regarding the change in the number of participating video conference terminals, and the user of the video conference terminal which has not been connected until then suddenly makes a remark. It is possible to eliminate such unnaturalness and inconvenience.

Further, since the terminal information is converted into at least one of character data and bit map data so as to be overlaid on the transmitted image, the users participating in the multipoint video conference can obtain information regarding the change in the number of terminals. Can be obtained visually.

Even if the video conference terminal used does not have the video capability or the video capability is turned off, the users participating in the multi-point video conference are concerned with the change in the number of terminals. Information can be obtained by voice.

Information regarding the change in the number of terminals is provided to all participating video conference terminals by selecting either audio notification or video notification according to the presence or absence of video capability of the video conference terminals of the notification destinations. Can be notified.

In this embodiment, the ISDN line is adopted as the communication line for connecting each video conference terminal to the MCU. However, PSTN (analog telephone network), CSPDN (line switching network) and LAN (local) are used. Needless to say, it can be applied to various lines such as an area network).

[0052]

As described above, according to the multipoint video conference system of claim 1, a multipoint video conference is performed by connecting a plurality of video conference terminals to the multipoint communication control device. In the point-to-point video conference system, the multi-point communication control device participates in the multi-point video conference, and terminal information storage means for storing terminal information of the video conference terminals participating in the multi-point video conference. Terminal number monitoring means for monitoring the number of terminals of the video conferencing terminal being operated, and when a change in the number of terminals is detected by the terminal number monitoring means, information regarding the change in the number of terminals is provided to the multipoint video conference. Since it is configured to have a terminal information notifying means for notifying the video conference terminals participating in the, the users participating in the multipoint video conference can count the number of participating video conference terminals. It is possible to grasp the information to be,
For example, it is possible to eliminate the unnaturalness and inconvenience that a user of a video conference terminal that has not been connected until then suddenly speaks.

According to the multipoint video conference system of claim 2, the information regarding the change in the number of terminals is converted into at least one of character data and bitmap data,
By providing the image synthesizing unit that overlays the transmitted image, the user participating in the multipoint video conference can obtain the information regarding the change in the number of terminals visually.

Further, according to the multipoint video conference system of claim 3, since the voice synthesizing means for voice synthesizing the information regarding the change in the number of terminals and adding it to the transmitted voice is provided, the video conference terminal to be used. The effect that the user participating in the multipoint video conference can obtain the information regarding the change in the number of terminals by voice even if the user does not have the image ability or the image ability is turned off. Is obtained.

Further, according to the multipoint video conference system of claim 4, the terminal information notifying means converts the information relating to the change in the number of terminals into at least one of character data and bitmap data to form a transmission image. An image synthesizing unit for overlaying, a voice synthesizing unit for synthesizing voice and adding information regarding the change in the number of terminals to a transmission voice, and identifying a video capability of each video conference terminal participating in the multipoint video conference. Of the video conferencing terminal by having the video capability identifying means for selecting and the selecting means for selecting either the image synthesizing means or the voice synthesizing means based on the video capability identified by the video capability identifying means. You can select either audio notification or video notification according to the presence or absence of video capability, so all participating video conference terminals can Effect that it is possible to notify information relating to changes in the youngest number.

[Brief description of drawings]

FIG. 1 is a multipoint communication control device (MC) according to an embodiment.
It is a block diagram showing an example of functional composition of U).

FIG. 2 is a block diagram showing a configuration of an MCU capable of executing the functional configuration shown in FIG.

FIG. 3 is a flowchart showing a procedure for monitoring the number of participating video conference terminals according to the embodiment.

FIG. 4 is a flowchart showing a terminal information notification processing procedure according to the embodiment.

FIG. 5 is a diagram showing an example of an image received by a participating video conference terminal.

FIG. 6 is a diagram showing an example of multipoint connection when an MCU is used in a conventional multipoint video conference system.

[Description of Reference Signs] 1 terminal information storage unit 2 terminal number monitoring unit 3 terminal information notification unit 4 image synthesis unit 5 audio synthesis unit 6 video capability identification unit 7 notification method selection unit 201 CPU (terminal number monitoring unit, video capability identification unit , Identification means, selection means, terminal information notification means) 203 RAM (terminal information storage means) 206 voice processing unit (voice synthesis means) 208 image processing unit (image synthesis means)

Claims (5)

[Claims] 1. A multipoint video conferencing system for performing a multipoint videoconference by connecting a plurality of videoconference terminals to a multipoint multipoint communication control device, wherein the multipoint multipoint communication control device comprises: Terminal information storage means for storing terminal information of video conference terminals participating in the video conference, terminal number monitoring means for monitoring the number of terminals of the video conference terminals participating in the multipoint video conference, and the terminal And a terminal information notifying unit for notifying the video conference terminal participating in the multipoint video conference of the information regarding the change in the number of terminals when the number monitoring unit detects the change in the number of terminals. A featured multipoint video conference system. 2. The terminal information notifying means has an image synthesizing means for converting the information regarding the change in the number of terminals into at least one of character data and bitmap data and overlaying the transmitted image. The multipoint video conference system according to 1. 3. The multipoint-to-multipoint communication according to claim 1, wherein the terminal information notifying means has a voice synthesizing means for synthesizing information about the change in the number of terminals and adding it to a transmission voice. Video conferencing system. 4. The terminal information notifying means converts the information regarding the change in the number of terminals into at least one of character data and bitmap data and overlays the transmitted image, and the information regarding the change in the number of terminals. A voice synthesizing means for synthesizing and adding to the transmitted voice, a video ability identifying means for identifying the video ability of each video conference terminal participating in the multipoint video conference, and the video ability identifying means. The multipoint video conference system according to claim 1, further comprising: a selection unit that selects either the image synthesis unit or the audio synthesis unit based on the video capability. 5. The information regarding the change in the number of terminals includes the number of terminals after the change and the terminal information of the video conference terminal which is a factor of the change in the number of terminals stored in the terminal information storage means. The multipoint video conference system according to any one of claims 1 to 4, characterized in that: