JP2527202B2 - Multipoint image transmission system - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a multipoint image transmission system in which multiple points are simultaneously connected in a moving image transmission system.

[Conventional technology]

As for this kind of multipoint video conference system, for example, “Business Communication '84, Vol. 21, No. 10, pp.
28-40 "Overview of the video conference system and its future", and FIG. 7 is an explanatory diagram of such a multipoint video conference system. The conventional multipoint video conference system shown in the figure is a conventional encoding device, a conventional multi-point conference control unit (hereinafter referred to as M
Abbreviated as CU). FIG. 8 shows the correspondence between the coded data and the screen in the conventional coding apparatus.
The figure shows that the speed control for each line is performed individually in the conventional MCU. 6 to 9,
(1) is a terminal station, (2) is a line allocated to each terminal station,
(3) is an MCU, (4) is a speed controller for each line, and (5) is a control channel allocated to each terminal station.

Next, the operation of such a conventional example will be described. FIG. 7 (a) shows a request method, in which all the moving pictures transmitted at a plurality of points are sent to the MCU, and the MCU supplies the moving picture of a desired station in response to a request signal from each point on the receiving side. Further, FIG. 7 (b) shows a video multiplex transmission system, which is a system in which all transmission moving images at a plurality of points are sent to the MCU, multiplexed or combined into one video signal and sent to each point. It is considered that the line capacity for sending from the MCU to each terminal is large, or that the coded video is decoded in the MCU and then the video is synthesized and recoded. FIG. 7 (c) shows a broadcast distribution method in which one terminal station (station A in the figure) is the sender of the moving image and the other terminals are all the receivers of the MCU. In a conference, the speaker becomes the sender, and the sender is replaced with the replacement of the speaker.

The reason for adopting these methods lies in the encoder and the MCU at each terminal station, and this point will be described below.

As shown in FIG. 8, which is an explanatory view showing the correspondence between the coded data and the screen in the conventional transcoder, the coded data for one screen is sent as a packet with a header attached for each fixed length frame. To do. Therefore, in principle, the packet or packet group transmitted from the encoding device does not correspond to the screen division. Corresponding is the packet group and the data for one screen. For this reason, encoded data can be handled only on a screen-by-screen basis. Further, as shown in FIG. 9, each terminal station (1) is connected to the MCU (3) via the line (2).
The transmission speed (bidirectional) of each line is controlled by the individual speed control unit (4) via the control channel (5). Therefore, the terminal stations participating in the multipoint conference basically perform speed control regardless of the connection status of the entire conference. Therefore, as described with reference to FIG. 7, a multipoint conference is held by a switching operation in units of terminals and screens. However, in a multipoint conference, the fact that the video that can be displayed by one's own station is only one of a plurality of games is lost, which is one of the merits of the video conference. To display multiple stations simultaneously, as explained in the video multiplex transmission method in Fig. 7 (b), the technique of increasing the speed of the line sent from the MCU to the terminal station and multiplexing and transmitting the video of multiple stations However, the receiving station must have equipment to separate this and to decode it individually. There is also a method of combining the videos of multiple stations with an MCU and transmitting them as if they were one station, but the MCU must have decoding and re-encoding equipment. If multiple terminal stations are in the vicinity of the MCU, the MCU is transmitted without encoding and is synthesized by the MCU.
There is also a method of encoding, but the applicable cases are limited.

[Problems to be solved by the invention]

Since the conventional multipoint video conference has the above-mentioned system, if only one of a plurality of games can be viewed at the same time, or if the intention is to display a plurality of games at the same time, the line speed is increased. There was a problem in that it would be necessary or necessary for additional equipment.

The present invention has been made to solve the above problems, the line speed between each terminal station and the MCU is the same as before, and the encoded data input from each terminal is decoded in the MCU. It is an object of the present invention to obtain a multipoint video conference system that can be combined without being converted and can send the combined coded data to each terminal.

[Means for Solving the Problems] The multipoint image transmission system according to the present invention transmits each of the transmission stations according to the priority at the time of screen division transmitted from each of the transmission stations among the plurality of stations and the number of transmission stations. Image allocation means for deciding allocation of transmission image data amount to stations and outputting to each transmission station as transmission image data amount allocation information, and transmission based on the transmission image data amount allocation information provided in each of the transmission stations Encode the image to generate encoded image data,
A coding device that adds and outputs the transmission image data amount allocation information and coded image data input from each of the transmission stations are multiplexed, and each receiving station adds decoding processing information necessary for image decoding. And image decoding means for outputting to each receiving station, and decoding means provided in the receiving station for decoding the multiplexed coded image data from the image controlling means based on the decoding processing information. It is equipped with and.

[Operation] According to the present invention, the allocation of the transmission image data amount to each transmitting station is determined by the priority and the number of transmitting stations at the time of screen division transmitted from each transmitting station, and each transmitting station is determined based on this determination. Performs the encoding, and the image control means multiplexes the encoded data from each transmitting station and outputs the multiplexed data to each receiving station. Therefore, the amount of encoded image data for each transmitting station is set according to the processing capability of the receiving station. Each receiving station can always receive and display a combined image obtained by combining the images from the transmitting stations even if the number of transmitting stations is changed.

[Embodiment] An embodiment of the present invention will be described below with reference to FIGS. It should be noted that FIG. 1 conceptually illustrates an embodiment of a multipoint video conference according to the present invention.
FIG. 3 illustrates the functions that the encoding device should have in the embodiment of the multipoint video conference system according to the present invention. FIG. 3 shows that the MCU in the embodiment of the multipoint video conference system according to the present invention converts the encoded video data into moving image data. FIG. 4 is a diagram for explaining a process to be performed for each terminal station, and FIG. 4 is a diagram for explaining that the MCU in the embodiment of the multipoint video conference system according to the present invention allocates the transmission rate to each terminal station. 1 to 5, (1) is a terminal station,
(2) is a line allocated to each terminal station, (3) is an MCU as image control means, (4) is a speed control unit for each line, (5) is a control channel allocated to each terminal station, ( 6) is an image allocating means for allocating the transmission speed and the resolution (the number of pixels) as the transmission image data amount to each terminal station according to the multipoint connection status of the conference and outputting it to each terminal station as the transmission image data amount allocation information The conference control unit (7) is a conference control information channel.

Next, the operation of the embodiment will be described. In Fig. 1, five terminal stations are connected to the MCU. If these terminal stations are connected at the same time and a multipoint video conference is performed, one terminal station can transmit the transmitted videos of the remaining four terminal stations. It is desirable to be seen at the same time. In the system according to this embodiment, the moving image received by each terminal station is an image obtained by combining the transmitted moving images of all the terminal stations except the own station. Therefore, the MCU allocates the transmission speed and the resolution (number of pixels) as the transmission image data amount to each terminal station, and each terminal station transmits according to the instruction of the MCU. Allocation of the amount of transmitted image data is determined according to the multipoint connection status and priority. When the number of points is very large and the transmission rate assigned to one terminal station becomes very small, the conventional multipoint connection method described with reference to FIG. 7 can be used together. The MCU synthesizes (multiplexes) the moving image sent from each terminal station for each receiving station as it is with the encoded data and sends it. In the embodiment shown in FIG. 1, since station A has three speakers, the screen is halved. Since the number of stations to be received is 4, the remaining 3 stations are 1/2 ÷ 3 = 1/6 and
/ 6 is assigned. Station A itself does not need to receive the video transmitted by itself, but since other stations are operating in 1/6 mode, you will see a composite display of 4 1/6 size images for 4 stations. . That is, the transmission rate and the number of pixels are determined according to the station having the smallest receivable capacity (both the transmission rate and the number of pixels) among the terminal stations connected at the same time. First
In the illustrated embodiment, the station A is prioritized, so that when a receiving station other than the station A tries to receive the video of the station A at the same time as the video of the other station, the receiving capacity is the smallest. Become. The receivable capacity is the decoding processing capacity.

As shown in FIG. 2, the encoder of each terminal operates at the number of pixels and the transmission speed, which is the transmission image data amount allocation information instructed by the MCU, and the transmission station and The receiving station, priority, etc. are transmitted as a header. The priority here means that the screen is divided by multipoint connection among the modes 1 to 8 shown in FIG. 3 (the larger the mode number, the finer the divided screen becomes). It corresponds to the tolerance of time. Also, the encoding device has resolution (number of pixels)
It is desirable that the transmission speed and the transmission rate be possible, and that these variable elements be switched under the control of the outside so that the transmission and reception are operated even under the same conditions.

As shown in FIG. 4, the MCU performs video composition by combining (multiplexing) a packet group of coded data sent from each station, and adds a new header corresponding to the receiving station. As shown in FIG. 5, each terminal (1) is connected to the MCU (3) by a line (2). Each line has a corresponding speed control unit (4), and exchanges information such as the number of pixels and the transmission speed via the control channel (5) as described above. The MCU has a conference control unit (6), the speed control unit (4) receives information (7) obtained by separating the control channels of the line, and the transmission rate for each line is assigned the number of pixels according to them. Is determined and notified to each speed control unit (4). The conference controller (6) is composed of a processor, and the above processing is performed by software. The outline is shown in FIG. First, the number of stations connected simultaneously is known from the information (7) obtained from each speed control unit (4). (N stations.) At the same time, the priority sent from each station is stored for each station. (The priority is, for example, described in the modes 1 to 8 in FIG. 3 as described above.) Next, from the connected station 1 to the station N, the receiving stations are sequentially connected. Sometimes we decide how to combine other stations (games). This is performed as follows. That is, if there is no claim of priority to the games (stations 2 to N) to be received by the station 1, the allocation to each game becomes equal. In this case, the allocation of each game is calculated as 1 / (N-1). Then, the speed in FIG. 3 is searched, and the speed ratio for mode 1 is 1 / (N-
1) Search for a mode that is less than or equal to 1 and is closest to 1 / (N-1). For example, when N = 3, the mode is 2. A case where there is a difference in priority will be described with reference to FIG. 1 as an example. now,
Station 1 = A station, station 2 = B station, ... N = 5. When the station 2 (station B) decides the allocation of the game, the station 1 (station A) asserts the priority. It is 2 in the mode of FIG. Therefore, first, station 1
Was assigned 1/2. Then the rest is 1-1 / 2 = 1/2, 1/2 / (N-1) -1 = 1 / (2N-4) where N = 5, so 1 / 2N-4 = 1 / 6.

All stations other than station 1 with priority 2 are assigned 1/6. By deciding the allocation in this way, N-1
Time allocation will be decided. At each station, the allocation with the smallest value is determined as the final allocation. After that, the allocation is sent to the speed control unit corresponding to each station. At this time, the allocation information is represented by the mode number. The speed control unit (4) puts the mode number on the control channel and sends it to the coding device (1) of the terminal station so that the transmission is performed at the transmission speed according to the designated allocation.

When synthesizing moving images from multiple stations, the frame rate (the number of frames per unit time) may differ depending on the station. In such a case, it is conceivable to immediately send the encoded data to the receiving station at the stage when the encoded data is given to the MCU until the encoded data of all the games are gathered. Therefore, the header information of the packet includes, as the decoding processing information, the above-mentioned mode number and address information indicating at which position on the received decoded image the corresponding split screen should be fitted. is necessary.

Although the example of performing the combination of encoded data in units of packets has been described, the same applies to the case in which allocation of fixed time slots is performed as a unit.

In the above embodiment, the case where all the terminal stations have the variable resolution and variable transmission rate functions has been described. For example, when it is desired to connect an encoding device having a fixed resolution transmission rate, the opposite station has a variable function. Then, the game can be adapted to the fixed-action encoder via the MCU.

In addition, although the video conference system has been described in the above embodiment, it is also an effective system as a multipoint simultaneous monitoring system, and one of its features is that it is flexible with respect to an increase in the number of points.

Further, in the above embodiment, the case where all the terminal stations are connected to one MCU has been described, but some terminal stations are different from each other.
It may be connected to the MCU. In such a case, one MCU acts as the center, another MCU sends the information of each terminal station to the center, and the center sends the other MCUs.
The number of pixels and the transmission speed are instructed to each terminal via the.

[Effects of the Invention] As described above, according to the present invention, the allocation of the transmission image data amount to each transmitting station is determined by the priority at the time of screen division transmitted from each transmitting station and the number of the transmitting stations. Then, the transmitted image data amount allocation information is output to each transmitting station. At each transmitting station, the transmitted image is encoded based on the transmitted image data amount allocation information to generate encoded image data, and the transmitted image data amount allocation is performed. Add information and output,
The coded image data input from each transmitting station is multiplexed, and each receiving station adds decoding processing information necessary for image decoding and outputs it to each receiving station. The encoded image data sent according to the data amount allocation information is multiplexed while being associated with the receiving station, so that it can be decoded in the MCU without changing the line speed between each terminal station and the MCU. There is an effect that a composite image can be output to the receiving station without arranging the function of.

[Brief description of drawings]

FIG. 1 is a diagram conceptually showing an operation example of a multipoint video conference system according to an embodiment of the present invention, and FIG. 2 is a function of an encoding device in the multipoint video conference system according to the embodiment of the present invention. FIG. 3 is an explanatory diagram of an operation mode of an encoding device in a multipoint video conference system according to an embodiment of the present invention, and FIG. 4 is a moving image of an MCU in the multipoint video conference system according to the embodiment of the present invention. FIG. 5 is an explanatory diagram of processing to be performed on encoded data, FIG. 5 is an explanatory diagram relating to allocation of a transmission rate to each terminal station by the MCU in the multipoint video conference system according to the embodiment of the present invention, and FIG. FIG. 7 shows an operation of a conference control unit of an MCU in a multipoint video conference system according to an embodiment of the invention, and FIG. 7 shows a conventional multipoint video conference system. FIG. 8 is an explanatory view showing the correspondence between the encoded data and the screen in the conventional encoding device, and FIG. 9 is an explanatory diagram showing that the speed control for each line is individually performed in the conventional MCU. . In the figure, (1) is a terminal station, (2) is a line allocated for each terminal station, (3) is an MCU, (4) is a speed control unit for each line, (5)
Is a control channel assigned to each terminal, (6) is a conference control unit, and (7) is a conference control information channel. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (4)

(57) [Claims] 1. A multipoint image transmission system for transmitting coded image data between a plurality of stations connected at a multipoint, the priority at the time of screen division transmitted from each of the transmitting stations of the plurality of stations. And an image allocating means for determining allocation of transmission image data amount to each transmitting station according to the number of transmitting stations, and outputting to each transmitting station as transmission image data amount allocation information, and each of the transmitting stations. , An encoding device that encodes a transmission image based on the transmission image data amount allocation information to generate encoded image data, and adds and outputs the transmission image data amount allocation information, and input from each of the transmission stations. Image control means for multiplexing encoded image data, adding decoding processing information necessary for image decoding to each receiving station, and outputting to each receiving station; and A multipoint image transmission system, comprising: a decoding unit that decodes the multiplexed encoded image data based on the decoding processing information. 2. The multipoint image transmission system according to claim 1, wherein the image control means multiplexes coded image signals according to allocation of fixed time slots. 3. The multipoint image transmission system according to claim 1, wherein the image allocating means outputs split screen allocation information to the encoding means in accordance with allocation of fixed time slots. 4. The image control means multiplies the coded image signals, which are coded image signals bucketed into a fixed length, as a unit. Point image transmission system.