JPH02145097A - Decentralized multi-point communication conference system - Google Patents

Abstract

PURPOSE:To observe a video image and to listen to voice preferred by each communication conference equipment by utilizing a packet exchange network as a communication network, allowing each communication conference equipment to send a video packet and a voice packet to all participant equipments or only a required communication conference equipment and synthesizing the video image and adding voice in its own equipment. CONSTITUTION:Communication conference equipments (terminal equipments) 32 - 36 set a logical path with all terminal equipments of n-poit in participating to a conference, process a video signal and a voice signal into packets and send then to all terminal equipments of the n-point. For example, the terminal equipment 32 at a point A makes 4 copies from one video packet and sends them to a point B through a route Ab and terminal equipments 34, 35, 36 at points C, D, E through routes Ac, Ad, Ae respectively simultaneously. The video packet and the voice packet from the received 4 points are converted into a video signal and a voice signal at the point E, and as to the video signal, e.g., 3 points A, B, C are selected and the signal is displayed on a display device of its own terminal equipment with synthesis. As to the voice, the voice signals from all points are added and outputted. Moreover, this is similar in the point B.

Description

[Detailed description of the invention] (Technical field to which the invention pertains) The present invention utilizes a packet network to packetize video and audio signals from each point between n+1 points and transmit them as video and audio packets. Regarding distributed multipoint communication conference system.

(Prior Art) FIG. 9 is a diagram showing a connection configuration of a conventional distributed multipoint communication conference system. In FIG. 1, 1 is a circuit switching network, 2 is a multipoint connection control bag W (hereinafter referred to as MCU), 3 to 7 are communication conference devices (hereinafter referred to as terminals). When a multipoint conference is held between five points of terminals 3 to 7, the terminals 3 to 7 are connected to the MCU 2 via the circuit switched network 1. Since the MCU needs to communicate with terminals at five locations simultaneously, it is connected to the communication network of the circuit switching network 1 through five sets of bidirectional lines 8. One set of this bidirectional line consists of a video line and an audio line, and each terminal sends the video and audio of its own location to MCU 2, and the MCU reduces and synthesizes the host's video and videos from several locations to complete the entire process. Terminal 3
~Send to 7.

FIG. 11 shows an example of a divided and reduced composite image displayed on each terminal 3 to 7, 21 is the display of each terminal, 22 to 25
is a divided small screen, and the image of the point A where the terminal 3 is located is displayed on the light rain surface 22. Point B below.

Images C and D are displayed on divided reduced screens 23, 24, and 25, respectively. Furthermore, the MCU 2 adds the voices from each point only to the necessary points and sends the sum to all the terminals. The video compositing method in the MCU is performed, for example, based on instructions from a terminal serving as a host.

However, this method has the disadvantage that the image displayed on the display is the same at all points, and it is not possible to display the image of a desired point at each point. Similarly, with regard to audio, there was a drawback that the same audio was heard at each location, and it was not possible to listen to only the audio from a favorite location.

FIG. 10 is a diagram showing another connection configuration of the conventional distributed multipoint communication conference system, in which 11 is a packet switching network, 12 is an MCU for the packet switching network, and 13 to 1
7 is a communication conference device (hereinafter referred to as a terminal), MCU
12 is connected to the packet switching network 11 through one bidirectional packet line 18. Each terminal converts audio and video into packets and transmits them as audio packets and video packets over the same line. Further, each terminal sends the video and audio of its own location to the MCU 12, and the MCU reduces and synthesizes the host's video and videos of several locations and sends it to all terminals as shown in FIG. 11, for example. Additionally, the MCU adds the audio from each location to the necessary locations and sends the sum to all terminals. The video compositing method in the MCU is performed, for example, based on instructions from a terminal serving as a host.

However, similar to the method shown in FIG. 9, this method has the disadvantage that the image displayed on the display is the same at all points, and it is not possible to display the image of a desired point at each point. Similarly, with regard to audio, there was a drawback that the same audio was heard at each location, and it was not possible to listen to only the audio from a favorite location.

Another disadvantage is that multipoint communication conferences cannot be conducted without an MCU, and communication conferences cannot be conducted with only a plurality of terminals.

Therefore, in order to conduct a teleconference, it is necessary to make an initial investment in the MCU as a shared facility, and there is a drawback that it is not possible to easily conduct a teleconference using just a terminal.

(Objective of the Invention) The present invention eliminates the drawbacks of the above-mentioned conventional methods, and enables each teleconferencing device (terminal) to view video and hear audio according to the user's preference, thereby increasing the sense of realism. The purpose is to provide each terminal with the voice synthesis and voice addition functions of the multipoint connection control bag 5g (MCU), and to enable multipoint communication conferences with each terminal alone, even without the MCU.

(Structure of the Invention) (Characteristics of the Invention and Differences from the Prior Art) In order to achieve the above object, the present invention uses a packet switching network as a communication network, and each communication conference device (terminal) is a terminal for all conference participants or Video and audio packets are sent only to the necessary terminals, and each terminal performs video synthesis and audio addition within its own terminal, and the multipoint connection control unit (MCU) has the ability to combine video and audio packets. The most important feature is that each terminal has an addition function, making it possible to conduct multipoint communication conferences using just the terminals without the need for an MCU.

The difference is that instead of sending video and audio to all terminals as in the conventional technology, it is sent only to the necessary terminals to enhance the sense of realism, and it is also possible to conduct multipoint communication conferences even with only the terminals.

(Embodiment) FIG. 1 shows a connection configuration diagram of an embodiment according to claim (1) of the present invention, in which 31 is a packet switching network;
2 to 36 are communication conference devices (terminals). During a conference, each terminal sets logical paths with all terminals at n points participating in the conference, packetizes video signals and audio signals, and packetizes video and audio signals to all n points. Send to the terminal.

For example, when the terminal 32 at point A conducts a so-called 5-point communication conference with n=4 with terminals 33, 34, 35, and 36 at points B, C, D, and H, one video packet is sent to 4 Copy each part and add it to the route Ab't'B point as follows: Route Ac
, Ad, Ae, terminals 34.3 at points C, D, and E, respectively.
Send at the same time on 5.36. On the other hand, at point E.

Each route A e g from each point A, B, C, D
Video packets are sent independently on Be, Ce, and De. The same applies to voice packets.

At point E, the received video packets and audio packets from the four points are converted into video and audio signals respectively, and for the video signals, for example, three points A, B, and C are selected and synthesized on the display of the own terminal. indicate. An example of this is shown in FIG. 2. In other words, 41 is a display capable of displaying a multi-window image of point E, 42 to 44 are multi-window displays, 42 shows an image of point A, and 43 and 44 each display B.

An image of point C is displayed. Regarding audio, audio signals from all points are added and output.

In addition, at point 1, for example, n, the received video packets and audio packets from the other four points are converted into video signals and audio signals respectively, and for the video signal, for example, point A,
Select four points, C, D, and E, and display them as a composite on your terminal's display. An example of this is shown in Fig. 3. In other words, 51 is a display that can display multi-window images of points n, 52 to 55 are multi-windows, 52 shows images of point A, and 53° and 54.55 each display C. Images of points , D, and E are displayed. Regarding audio, audio signals from all points are added and output.

I have explained packet switching networks above, but in recent years CCI
The next generation broadband communication network (B-
It is clear that the present invention is similarly applicable to asynchronous transfer (ATM) networks, which are popular as ISDN. In other words,
In a packet-switched network, information is packetized at layer 3, whereas in ATM, information is packetized (called cells in the case of ATV) at layer 1, but other operations are the same.

The present invention also applies to local area networks (L
Obviously, it is also applicable to loop-type or bus-type networks. In this case, it is not necessary to duplicate the same video and audio packets from each terminal for the number of terminals participating in the conference and send them to all terminals.Due to the network topology, all packets pass through each terminal once, so the packets For example, by writing a specific group address, each terminal belonging to the group will not only relay the packet sequentially, but also read it into its own terminal, resulting in the packet being sent to all terminals within the group. The same action as above will be performed. The subsequent video synthesis/display operations and audio addition operations within the terminal are the same as those described in FIG.

FIG. 4 shows a block diagram of the configuration of a teleconferencing device (terminal) for realizing the distributed multipoint teleconferencing system shown in FIG.
This corresponds to the teleconferencing devices 32 to 36 installed at points A-E in the figure. Reference numeral 61 denotes a packet line (constituting the packet switching network 31 in FIG. 1), which is a bidirectional line capable of transmitting and receiving packets. Reference numeral 62 denotes a line control unit which has an interface circuit for the packet line and also sets and cancels a logical path with a communication partner with the packet switching network.

When conducting a communication conference between n+1 points, n other than the own terminal
Set up n logical paths between the terminal and the destination.

This terminal is composed of transmitting systems 63 to 68 and receiving systems 69 to 77. The transmitting system includes a camera 63, a video packet encoding section 64, and a video packet transmitting section 65, which constitute the video system. The audio system includes a microphone 66, an audio packet encoder 67, and an audio packet transmitter 68.

On the other hand, the receiving system includes a video packet receiving section 69, a video packet decoding section 70, a video signal selection section 71, a multipoint video display control section 72, and a display 73. Further, 74 is an audio packet receiving section.

The audio system includes an audio packet decoder 75, an audio adder 76, and a speaker 77.

Next, to explain the operation of the transmission system, the video signal input from the camera 63 is compressed in the amount of information by high-efficiency encoding such as an interframe encoding method in the video packet encoder 64, and the compressed information is For example, it is assembled into a byte-based packet.

The video packet transmitter 65 copies the packet n times for the number of destinations, adds necessary packet headers such as a logical path number assigned to each communication partner to each packet, and sends these packets to the line controller 62. It is converted into a signal format that can be transmitted on the packet line 61 and sent to the packet switching network 31 (
1) to the other party's terminal at point n.

Similarly, the audio signal from the microphone 66 is compressed by an audio packet encoder 67 using a high-efficiency encoding method such as DPCM, and the compressed information is assembled into a byte-based packet, for example.

The voice packet transmitting unit 68 copies the packet n times for the number of destinations, adds necessary packet headers, performs necessary signal conversion in the line control unit 62, and then transfers the packet to the packet switching network 31.
It is sent to the other party's terminal via .

Next, the operation of the receiving system will be explained. Packet line 61
The video packet sent via the above logical bus is received by the line control section 62, converted into a signal format necessary for internal processing of the terminal 60, and sent to the video packet reception section 69.

During an n+1 point conference, the line control unit 62 is connected to terminals at n destinations other than its own terminal by n logical paths. Video packets from each point are sent to all video packet receivers 69. The video packet receiving unit determines which destination the video packet is from based on the logical path number in the packet header, and after removing the packet header, for example.

It is decoded into a video signal by a video packet decoder installed at each destination.

In this decoding method, if video signals from n points are simultaneously received, n independent decoders are required. In this manner, the video packet decoding unit 70 decomposes and decodes the video packets from each of the n points, and obtains a video signal from the n points.

The video signal selection unit 71 selects m( of the video signals at n points).
1≦m≦n) The video signals of the points are selected, for example, according to instructions from the operator of the terminal 60, and sent to the multi-point video display control section 72. In this video display control section, one display 7
3 performs processing for displaying m multi-windows, and displays the results on the display 73. The above-mentioned FIG. 2 or FIG. 3 is an example of the display.

Next, the voice packet sent via the logical bus on the packet line 61 is received by the circuit control section 62, and after signal conversion, is sent to the voice packet reception section 74. All audio packets from n points during the conference are sent to the audio packet receiving section 74. In this voice packet receiving section,
After determining which destination the audio packet is from based on the logical path number in the packet header, etc., and removing the packet header.

For example, it is decoded into an audio signal by an audio packet decoder provided at each destination.

In this decoding method, if audio signals from n points are simultaneously received, n independent decoders are required. The audio packet decoding unit 75 thus decomposes and decodes the audio packets from each of the n points, and obtains audio signals from the n points. The audio addition unit 76 adds the audio at n points and outputs the result from the speaker 77.

When adding the sounds from all points in this way, the sound signal selection unit (not shown) is unnecessary, but j(
1≦j≦n−1) If only the voices at the points are to be added, an audio signal selection unit is required between the audio packet decoding unit 75 and the audio adding unit 76. The voice selection method is to select the top 4 people (local) in order from the person who is speaking most actively (local).
There are conventional methods such as adding .

The case of a packet-switched network has been described above, but currently CC
It is clear that the present invention can be applied to the ATM network, which is being studied at ITT, with the same configuration and operation as shown in FIG. In the case of an ATM network, a logical path between terminals via the ATM network is set up in layer 1, and fixed-length cells of several tens of bytes in layer 1 transmit video information and audio information instead of packets. As in the case of packets, a cell header is added to each cell, and logical path numbers and the like are written therein.

It is also clear that the present invention can be applied to LANs.

However, a LAN has a loop or bus topology, and one packet is sent to all terminals connected to the LAN due to the nature of that topology. Therefore, the n+1 terminals that are conference participants have the same group address number, and each terminal only sends and receives packets with that group address number, so that the same It is no longer necessary to duplicate the packet n times for the number of stations.

In other words, the video packet transmitting unit 65 sends out to the packet line 61 one packet corresponding to the video packet passed from the video packet encoding unit 64 with a group address number added to it, and as a result, all One line control unit 62 that can send the packet to n terminals in the same group passes only the packet having the group address number among the packets passing through n points to the video packet receiving unit 69. The operation of transmitting a packet to point n and the operation of receiving a packet from point n are the same for voice packets.

Next, a connection configuration diagram of an embodiment according to claim (2) of the present invention is shown in FIG. In the figure, 81 is a packet switching network having the same function as the packet switching network 31 in FIG.
~86 is a communication conference device! (terminal).

During a conference, each terminal can set logical paths with all the terminals at n points participating in the conference, packetize video and audio signals, and send them to all the terminals at the necessary points.

Currently, each of the terminals 82 to 86 is A, B, and C.

Assume that there are locations D and n, and a communication conference is being held at these five locations. The terminal 86 at point n is at point A (for example, the explainer)
during a meeting while displaying only the video. n in that case
An example of the point display is shown in Figure 6, where 91 is n.
Although it is possible to display images in multiple windows on the point display, only the image of point A is currently displayed in one of the 92 image windows.

Further, it is assumed that the terminal 83 at point n is in a meeting while displaying images of points A and C. An example of the display of point n in that case is shown in Fig. 7.The display 101 can display the video of point n in a multi-window, but currently, the video of point A and the video of point A are displayed in two video multi-windows 102 and 103. An image of point C is displayed. At this time, the terminal 86 at point n issues a video transmission request only to the terminal 82 at point A, and the terminal 83 at point n issues a video transmission request only to the terminal 82 at point A and the terminal 84 at point C. There is.

Conversely, the terminal 82 at point A receives video transmission requests from two terminals, the terminal 83 at point n and the terminal 86 at point n, packetizes the video signal at point A, and generates the video packet to be transmitted. is duplicated into two, and each logical path Ae and A
It is sent to the terminal 86 at point n and the terminal 83 at point n via two logical paths b.

Also, since the terminal 84 at point C receives the video transmission request from the terminal 83 at point n,
The video signal is packetized and the video packet to be transmitted is sent to the terminal 83 at point n without particularly duplicating it. n
The terminal 86 at the point converts the video packet sent via the logical path Ae into the original video signal, and displays only the video signal (k=1) from the point A that issued the transmission request on its own display 91. indicate. The terminal 83 at point n receives the video packet sent via two logical paths Ab and cb, converts it into the original video signal, and sends the transmission request to points A and C. 2 (k=2) points are displayed on its own display 101.

Regarding audio, as shown in FIG. 1, all terminals transmit audio packets to all other terminals participating in the conference, and each terminal adds and outputs audio signals from n points. Furthermore, if necessary, a method of selectively adding only the audio at point j (1≦j≦n) can also be applied.

FIG. 8 is a block diagram showing the configuration of a communication conference device (terminal) for realizing the distributed multipoint communication conference system shown in FIG. Yes, there are other numbers, 110 is a communication conference device (terminal), 112 is a video signal transmission request receiving unit, and the video signal sent to the own terminal from any of the n terminals currently participating in the conference. The signal transmission request is sent to the line control unit 6.
2, and create a list of video signal transmission destinations.

In the example of FIG. 7, points A and C are written in the video signal transmission destination list of the terminal at point n.

113 is a video packet transmitting section, 114 is a video packet receiving section, and 115 is a video signal transmission request processing section, which transmits a video signal transmission request to each destination and holds a video signal reception destination list. Reference numeral 116 denotes an audio signal selection unit that selects audio signals at j points from the highest audio level per unit time.

Next, the operation of the transmission system will be described. The video signal input from the camera 63 is compressed by high-efficiency encoding in the video packet encoder 64, and the compressed information is assembled into packets. The video packet transmitting unit 113 refers to the video signal transmission destination list in the video signal transmission request receiving unit 112, copies the packet for the number of destinations, and adds necessary packet headers such as the logical bus number assigned to each communication partner. It is added to each packet, and these packets are added to the line control unit 6.
At step 2, the signal is converted into a signal format that can be transmitted over the packet line 61, and sent via the packet switching network 81 to the destination terminal entered in the video signal transmission destination list.

Similarly, the audio signal from the microphone 66 is compressed in the amount of information by a high-efficiency encoding method in the audio packet encoder 67, and the compressed information is assembled into packets.

The voice packet transmitting unit 68 copies the packet n times for the number of destinations, adds the necessary packet headers, performs necessary signal conversion in the line control unit 62, and transmits the packet to the n points via the packet switching network. Sent to the other party's terminal.

Next, the operation of the receiving system will be explained. In order to receive a video packet, the video signal transmission request processing unit 115 transmits a video signal transmission request to each k point (1≦k≦n) that requires a video signal in advance, and The video signal transmission request processing unit 115 stores a list of video signal reception destinations for the corresponding points. This list may be created at the start of the meeting, or participants may be added or deleted during the meeting.

Video packets sent via the logical bus on the packet line 61 are received by the single line control unit 62 and sent to the terminal 110.
The single line control unit 62 converts the signal into a signal format necessary for internal processing and sends it to the video packet receiving unit 114. During an n+1 point conference, the single line control unit 62 connects terminals at n destinations other than its own terminal using n logical buses. It is connected.

Video packets from each location are sent to an all-°C video packet receiving section 114. This video packet receiving unit uses the logical path number etc. in the packet header.

Determine which destination the video packet is from, discard packets sent from destinations other than those listed in the video signal reception destination list, and discard the packet header for packets from any point in the list. After removing the signal, the signal is decoded into a video signal by, for example, a decoder provided for each destination.

In this decoding method, if video signals from n points are simultaneously received, n independent decoders are required. In this manner, the video packet decoding unit 70 decomposes and decodes the video packets from each of the n points, and obtains a video signal from the n points.

This video signal is sent to a multi-point video display control section 72, which performs processing to display m multi-windows on one display, and displays the result on a display 73.

FIG. 6 or 7 is an example of the display.

Next, the voice packet sent via the logical bus on the packet line 61 is received by the single line control section 62, and after signal conversion, is sent to the voice packet reception section 74. All audio packets from n points during the conference are sent to the audio packet receiving section 74. This audio packet receiving unit determines which destination the audio packet is from based on the logical path number in the packet header, removes the packet header, and then decodes it into an audio signal using a decoder provided for each destination, for example. .

In this decoding method, if audio signals from n points are simultaneously received, n independent decoders are required. The audio packet decoding unit 75 thus decomposes and decodes the audio packets from each of the n points, and obtains audio signals from the n points. For example, the audio signal selection unit 116 constantly checks the audio level for each unit time on the audio signals at n points that are output from the audio packet decoding unit 75, and selects, for example, the audio signals at only 4 (j=4) points from the highest level. is selected and passed to the audio adder 76. This audio adder adds the audio at the j point and outputs the result from the speaker 77. Note that when the audio signals from all n points are added, the audio signal selection unit 116 is unnecessary.This embodiment has been described as an example of application to a packet network.

As mentioned earlier, it is obvious that this method can also be applied to ATM networks and LANs.

(Effects of the Invention) As described above, according to the present invention, a packet switching network is used as a communication network, and each communication conference device transmits video packets and audio packets only to all conference participating devices or to necessary communication conference devices. In addition, each teleconferencing device performs video synthesis and audio addition within its own device.
This has the advantage that each teleconferencing device can view images and hear audio according to its preference, thereby increasing the sense of realism.

In addition, since each teleconferencing device has the video synthesis and audio addition functions of the multipoint connection control unit i (MCU),
There is an advantage that a multipoint communication conference can be easily carried out using only a communication conference device without an MCU.

[Brief explanation of the drawing]

FIG. 1 is a connection configuration diagram of an embodiment according to claim (1) of the present invention, and FIGS. 2 and 3 are diagrams showing display examples of the distributed multipoint communication conference devices 32 to 36 of FIG. 1. , FIG. 4 is a configuration block diagram of the distributed multipoint communication conference device shown in FIG. 1, FIG. 5 is a connection configuration diagram of the embodiment according to claim (2) of the present invention, and FIGS. 6 and 7 are FIG. 5 is a diagram showing an example of display display in the distributed multipoint communication conference apparatuses 82 to 86 of FIG. 5, FIG. 8 is a block diagram of the configuration of the distributed multipoint communication conference apparatus of FIG. 5, and FIGS. Diagram showing each side of the connection configuration of the distributed multipoint communication conference system, Part 1
FIG. 1 is a diagram showing an example of the display according to FIGS. 9 and 10. 31, 81...Packet switching network, 32-36.60°82-86.110...Communication conference device, 41,51°73,91.101...Display, 42-44°52-55. ...Multi-window, 61...Packet line, 62...Line control unit, 63...Camera, 6
4... Video packet encoder, 65, 113... Video packet transmitter, 66... Microphone, 67...
- Audio packet encoder, 68...Audio packet transmitter, 69°114...Video packet receiver, 70...
-Video packet decoding unit, 71...Video signal selection unit, 7
2...Multi-point video display control unit, 74...Audio packet receiving unit, 75...Audio packet decoding unit, 76...
Audio addition section, 77... Speaker, 112... Video signal transmission request receiving section, 115... Video signal transmission request processing section, 116... Audio signal selection section. Patent Applicant: Nippon Telegraph and Telephone Corporation Figures 42-44... Marque Weedow 52-55 Maruka Window Figures 42-44

Claims (2)

[Claims] (1) When conducting a distributed multi-point communication conference using a packet network, video and audio signals from each point are packetized and transmitted as video and audio packets between n+1 points. Send packets and audio packets to n different destinations, receive video packets and audio packets from n points sent to your own location, and convert the video and audio packets into the original video and audio signals. However, regarding the video signal,
A maximum of m (1≦m≦n) points are selected among n points, and video signals from the selected m points are displayed on an image display device.As for audio signals, a maximum of m (1≦m≦n) points are selected among the n points. 1
≦j≦n) A distributed multipoint communication conference system is characterized in that points are selected, and audio signals from the selected j points are added and output. (2) When conducting a distributed multi-point communication conference using a packet network, video and audio signals from each point are packetized and transmitted as video and audio packets between n+1 points. Sends packets only to the point for which a transmission request has been made, and sends all audio packets to n points, receives video packets from point n sent to its own point, and issues a transmission request for video signals. A video signal at point k (1≦k≦n) is received, the video packet and audio packet are converted into the original video signal and audio signal, the video signal at point k is displayed on an image display device, and the audio signal is A distributed multipoint communication conference system is characterized in that a maximum of j (1≦j≦n) points among the n points are selected, and audio signals from the selected j points are added and output. .