JPH08205115A - Screen compositing and encoding device - Google Patents

Abstract

PURPOSE: To obtain a multipoint video conference system which enables conference states at respective points to be grasped on one screen without deteriorating the video quality. CONSTITUTION: This is a multipoint conference controller 3 which controls the terminal videos of video conferene terminals at N (integer larger than 3) points according to video signals which are generated and encoded by the respective video conference terminals 1. Respective decoded signals are stored in a frame memory 17. The respective decoded signals are read out of the frame memory in the same frame phase by a reducing circuit 18 and reduced to 1/N to generate reduced signals. The respective reduced signals are put together by a compositing circuit 19 to generate a composite signal. This composite signal is encoded by an encoding circuit 20 to generate an encoded signal. The encoded signal is supplied to the respective video conference terminals to generate the terminal videos.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multipoint video conference system using video conference terminals installed at three or more points.
In particular, the present invention relates to a multipoint video control device that is connected to these video conference terminals and controls the terminal video in the video conference terminals.

[0002]

2. Description of the Related Art Conventionally, there has been known a technique of taking a difference between video signals and stopping a coding process of an image having no change point to interpolate a frame drop. Further, techniques relating to a multipoint video conference system are disclosed in, for example, Japanese Patent Laid-Open Nos. 62-105588, 62-108691, and 1-147.
No. 946, etc.

Among them, for example, Japanese Patent Laid-Open No. 62-10558.
In JP-A-8, a variable rate coding / decoding apparatus is used for a video conference terminal and a conference control apparatus, and a coding rate is selected corresponding to the number of points as a transmission path between them, and demultiplexing is performed by variable time slot allocation. It is possible to transmit video signals at multiple points.

FIG. 6 shows a concrete structure of this Japanese Patent Laid-Open No. 62-105588. In FIG. 6, the transmission lines between the video conference terminal 27 and the multipoint conference control device 32 are 1.
It is set to 5Mbps. According to this, by reducing the time slot of the transmission line allocated to each point, it is possible to simultaneously watch the images of a plurality of points on a plurality of video conference terminals.
For example, when a conference is held at three points, in order to watch the images of the second and third points at the video conference terminal 27 at the first point at the same time, 768 kbps (1.5 Mbps / 1.5 Mbps /
2) is assigned to the two video conference terminals 27a and 27a.
'And two variable speed encoding / decoding devices 28a, 28
Images of the second point and the third point are displayed by using a ′ and the demultiplexer 29a. At the second location, the video conference terminal 27b and the variable-speed encoding / decoding device 28b are connected to each one.
It has a stand and is connected to the multipoint conference control device 32 at 1.5 Mbps so that the images at the first point and the third point can be switched and displayed.

[0005]

However, in the case of Japanese Patent Laid-Open No. 62-105588, the conference controller is equipped with a variable speed encoding / decoding device according to the maximum number of video conference terminals connected. Is required. In addition, the video signal at a certain point is decoded by the multipoint conference control device after being encoded at a certain bit rate by the variable speed encoding device,
Since it is re-encoded at another bit rate for transmission to another point, deterioration of video quality (resolution, quality) occurs.

Further, when a 64 kbps line is used for the transmission line between the video conference terminal and the multipoint conference controller, 32 kbps is allocated to each point. In this way, when a low bit rate transmission line is used and the number of connecting points increases, the number of time slots assigned to each point decreases, and the video quality (resolution, movement) is maintained even in a video conference with relatively few movements. There is a problem of deterioration. As described above, when a line with a low bit rate is used, the line capacity assigned to one point, that is, the number of time slots is reduced, and there is a possibility that the image quality is deteriorated and the motion is awkward.

[0007] Therefore, an object of the present invention is to increase the utility of a multipoint conference by making it possible to display the images of a plurality of points on one screen and to grasp the conference situation of each ground on one screen.

Another object of the present invention is to provide a multipoint video conference system capable of grasping the conference status of each ground on one screen without degrading the video quality.

Still another object of the present invention is to provide a screen synthesizing coding device, a multipoint video control device, and a multipoint video control method for realizing the multipoint video conference system.

[0010]

According to the present invention, N (N
Is an integer greater than or equal to 3) and a decoding circuit that decodes N video signals to generate N decoded signals, a frame memory that stores the N decoded signals, and N frames from the frame memory. Read the decoded signal in the same frame phase and set 1
/ N to reduce to N reduced signals to generate N reduced signals, a synthesizing unit to synthesize the N reduced signals to generate a synthesized signal, and a code to encode the synthesized signal to generate a coded signal. And a screen synthesis coding device.

Further, according to the present invention, terminal images at the video conference terminals at N points are generated according to N video signals generated and encoded at the video conference terminals at N points (N is an integer of 3 or more). In the multipoint video control device for controlling, the N
A decoding circuit for decoding N video signals to generate N decoded signals, a frame memory for storing the N decoded signals, and the N decoded signals from the frame memory for the same frame A reduction circuit for reading in phase and reducing to 1 / N to generate N reduced signals, a synthesizing unit for synthesizing the N reduced signals to generate a synthetic signal, and encoding and encoding the synthetic signal. A multipoint video control apparatus comprising: a coding circuit for generating a signal; and means for supplying the coded signal to each of the N video conference terminals to generate the terminal video. To be

According to the present invention, the N video signals generated and encoded by the video conference terminals at N points (N is an integer of 3 or more) are decoded to generate N decoded signals, The N decoded signals are read in the same frame phase to obtain 1 / N
To generate N composite signals, combine the N composite signals to generate a composite screen, treat the composite screen as one screen, encode it, and supply it to each of the N video conference terminals. Thus, a multipoint video conference system is obtained which is characterized by generating a terminal video.

Further, according to the present invention, terminal images at the video conference terminals at N points are generated according to N video signals generated and encoded at the video conference terminals at N points (N is an integer of 3 or more). In the multipoint video control method for controlling, the N
Decoding N video signals to generate N decoded signals, storing the N decoded signals, and reading the stored N decoded signals in the same frame phase. Reducing to 1 / N to generate N reduced signals, combining the N reduced signals to generate a combined signal, and encoding the combined signal to generate an encoded signal. And supplying the coded signal to each of the video conference terminals at the N points to generate the terminal video, thereby providing a multipoint video control method.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a case where a video conference is held at four points as a multipoint video conference system according to an embodiment of the present invention. In FIG. 1, video conference terminals 1a to 1d at four points.
64 kbps or 1 (represented by reference numeral 1 below) and the multipoint conference control device 3 that functions as a multipoint video control device.
Connection is made by lines 2a to 2d (hereinafter represented by reference numeral 2) having a low bit rate such as 28 kbps. Although a transmission device, a switching device, etc. are actually interposed between each video conference terminal 1 and the multipoint conference control device 3, they are omitted in this figure.

From each video conference terminal 1 through each line 2,
The multiplexed signal of the encoded video signal, the audio signal, and the control signal is sent to the multipoint conference control device 3. The multipoint conference control device 3 has a built-in screen synthesizing encoding device, reduces the video signals from the video conference terminal 1 at each point to 1/4, synthesizes them into one screen, and sends them to each point.

Since the multipoint conference control device 3 will be described in detail with reference to FIG. 2 and thereafter, its operation will be briefly described here. In the video conference terminal 1, in order to prevent the buffer memory in the internal encoding device from overflowing and being unable to transmit, the quantizer control is performed according to the generated information amount of the target image and the bit rate of the line, It controls the number of encoded frames. The number of NTSC signal frames is 2.
Although it is 9.97 frames / second, the number of encoded frames is 5 to 10 when a low bit rate line such as 64 kbps is used.
It is about frame / second. The screen synthesis encoding device decodes the data of 5 to 10 frames / second sent from each video conference terminal. Naturally, the decoded frames are 5 to 10 frames / sec. Since the frame phase of the decoded signal is asynchronous, the decoded signal is once written in the frame memory and read in the same frame phase. Convert the number of pixels and the number of lines of the signal read from the frame memory (each thinned out by 1/2),
The image is reduced to 1/4 and the images at 4 points are combined into one screen. The composite video signal is encoded and then multiplexed with the audio signal at each point and sent to each video conference terminal.

By thus reducing the video signal at each point to 1/4, the amount of information generated when re-encoding is reduced to 1
Since it can be reduced to / 4, even if you use a low bit rate line such as 64 kbps or 128 kbps, it is possible to prevent image quality deterioration due to awkward movement of each point, and increase the number of video conferencing terminals according to the number of connected points No need.

FIG. 2 shows the configuration of the multipoint conference control device 3. Line input terminals 4a to 4d (hereinafter represented by reference numeral 4)
The multiplexed signals of the video conference terminals at the respective points input from are separated into video signals, audio signals, and control signals by separation circuits 5a to 5d (hereinafter represented by reference numeral 5). The video signal 6 at each point is decoded by the screen synthesizing coding device 9 as described later, reduced to 1/4, and then the video at each point is synthesized and coded.

Further, the levels of the audio signals 7a to 7d (represented by reference numeral 7 below) at the respective points are compared by the audio comparison circuit 11 and added by the addition / loss insertion circuit 12,
A loss is inserted to prevent howling.

Video signals 6a to 6d at each point (hereinafter referred to as reference numeral 6)
Or a composite video signal is switched by the control signals 8a to 8d (represented by reference numeral 8 below) at each point or the output signal from the audio comparison circuit 11 to broadcast the video signal at any point, The broadcast of the composite video signal and the video of the specific point are selected, and the audio signal output from the addition / loss insertion circuit 12 of the audio signal is transferred to the video / audio multiplexing circuit 13a to.
13d (represented by the reference numeral 13 below) is multiplexed and output to the line output terminals 14a to 14d (represented by the reference numeral 14 below).

FIG. 3 is a block diagram showing the configuration of the screen synthesis coding apparatus 9, FIG. 4 is a timing chart for explaining the operation of the screen synthesis coding apparatus 9, and FIG.
It is an example of a composite screen of the image of the point. The video signals of the video conference terminals at the respective points input from the video input terminal 15 are decoded by the decoding circuits 16a to 16d (hereinafter represented by reference numeral 16). Although the decoding circuit 16 operates at the same clock, the frame phase of the decoded video signal at each point is the decoded video signal (that is, the decoded signal) 2 of FIG.
Not synchronized like 2a to 22d.

The decoded signal is temporarily stored in the frame memory 17a.
17d (hereinafter represented by reference numeral 17), the video signals at respective points are read out in the same frame phase as the frame memory output video signals 23a to 23d in FIG. In addition,
Every frame is read from the frame memory 17.

In the reduction circuits 18a to 18d (hereinafter represented by reference numeral 18), after filtering for removing aliasing noise, the reduction circuit output video signal of FIG. 4 (immediately,
(Reduction signal) 24a to 24d, pixels are thinned to 1/2 in the horizontal direction and the vertical direction, respectively, and reduced to 1/4.

The synthesizing circuit 19 writes the reduced video signal at each point in the internal memory, and synthesizes the video signal 25 shown in FIG.
As described above, the phase of 1/2 frame is read out, and time-division multiplexing is performed to synthesize a video signal of one screen (that is, a synthesis signal). The composite video signal 26 of FIG. 4 is the composite video signal 25.
FIG. The signal of the n frame indicates a signal obtained by decoding and reducing the video signal of the video signal input terminal 15a of FIG. 3, and is the time division multiplexing position of the video signal at the position of FIG. 5A. Similarly, the signal of the n + 1 frame is a signal obtained by decoding and reducing the video signals of the video signal input terminals 15b and 15c of FIG. 3, and is the time division multiplexing position of the video signals at the positions of b and c of FIG.

The coding circuit 20 codes the composite video signal and outputs it to the video output terminal 21 as a coded signal. As a result, the encoded signal is supplied to the video conference terminals at the respective points, and each video conference terminal 1 generates a terminal image as a composite screen of the images at the four points as shown in FIG.

Thus, by reducing the video signal at each point to 1/4, the amount of generated information can be reduced to 1/4. Therefore, the generated subtraction amount of the composite video signal at the four points is also decoded as a whole video signal. It is about 1/4 of that. Therefore, it is possible to re-encode the encoded frame at the video conferencing terminal at each point without dropping the frame, so 64 kbps or 128 kb
Even in a multipoint conference using a low bit rate line such as ps, it is possible to prevent deterioration of video quality due to clutter.

In the present embodiment, the case of synthesizing four screens has been described. However, the number of decoding circuits of the screen synthesizing coding device is increased according to the number of connected points of three or more, and the reduction circuit is configured. It can be changed to compose a composite screen corresponding to the number of points.

[0028]

As described above, according to the present invention,
The amount of generated information can be reduced to 1 / N by reducing the video signal at each point to 1 / N, and even if the composite video signal is re-encoded, the frames encoded at each point are not dropped. It can be re-encoded and the deterioration of video quality (movement clutter) can be prevented.

[Brief description of drawings]

FIG. 1 is a block diagram for explaining a multipoint video conference system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing the configuration of a multipoint conference control device included in the multipoint video conference system of FIG.

FIG. 3 is a block diagram showing a configuration of a screen synthesis encoding device included in the multipoint conference control device of FIG.

FIG. 4 is a timing chart for explaining the operation of the screen synthesizing encoding device in FIG.

FIG. 5 is an explanatory diagram showing an example of a composite screen of images at four points.

FIG. 6 is a block diagram showing an example of a conventional multipoint video conference system.

[Explanation of symbols]

 1a to 1d Video conferencing terminal 3 Multipoint conference control device 16a to 16d Decoding circuit 17a to 17d Frame memory 18a to 18d Reduction circuit 19 Combining circuit 20 Encoding circuit

Claims (4)

[Claims] 1. A decoding circuit for decoding N (N is an integer of 3 or more) video signals to generate N decoded signals,
A frame memory for storing the N decoded signals; and a reduction circuit for reading the N decoded signals from the frame memory in the same frame phase and reducing them to 1 / N to generate N reduced signals. A screen synthesizing coding apparatus comprising: synthesizing means for synthesizing the N reduced signals to generate a synthetic signal, and a coding circuit for coding the synthetic signal to generate a coded signal. 2. Multipoint control for controlling terminal images at video conference terminals at N points according to N video signals generated and coded at video conference terminals at N points (N is an integer of 3 or more). In the video control device, a decoding circuit that decodes the N video signals to generate N decoded signals, a frame memory that stores the N decoded signals, and N frames from the frame memory. A decoding circuit for reading the decoded signal of 1 in the same frame phase and reducing it to 1 / N to generate N reduced signals; a combining means for combining the N reduced signals to generate a combined signal; A coding circuit for coding a signal to generate a coded signal, and means for supplying the coded signal to each of the video conference terminals at the N points to generate the terminal video. Point-to-point video control device. 3. The N video signals generated and encoded by the video conference terminals at N points (N is an integer of 3 or more) are decoded to generate N decoded signals, and the N decoded signals are generated. The encoded signal is read out in the same frame phase and reduced to 1 / N to generate N reduced signals, the N reduced signals are combined to generate a combined screen, and the combined screen is treated as one screen and encoded. A multi-point video conferencing system characterized in that the video is supplied to each of the N video conference terminals to generate a terminal image. 4. Multipoint control for controlling terminal images in N video conference terminals in accordance with N video signals generated and encoded in N (N is an integer of 3 or more) video conference terminals. In the video control method, the N video signals are decoded to generate N decoded signals, the N decoded signals are stored, and the stored N decoded signals are stored. Are read in the same frame phase and reduced to 1 / N to generate N reduced signals, the N reduced signals are combined to generate a combined signal, and the combined signal is encoded and encoded. A multipoint video control method comprising: generating an encoded signal; and supplying the encoded signal to each of the N video conference terminals to generate the terminal video.