JPH0463084A - Television conference system among scattered sites - Google Patents

Abstract

PURPOSE:To give presence and to smoothly expedite a conference by flexibly synthesizing and displaying pictures at plural sites and generating an audio signal from plural speakers correspondingly to a display picture. CONSTITUTION:A video multiplex switching and synthesizing part 29 selects video signals from respective lines in time division based on the indication from a control part 27 and performs segmentation or reduction synthesis and sends the results to video line processing parts 12a to 12d and converts them to codes adapted to transmission lines. Meanwhile, audio signals are decoded by audio line processing parts 24a to 24d and have the audio levels detected, and addition processing of two channels corresponding to respective display pictures is performed by an audio selective addition and distribution part 26. Thereafter, they are encoded by parts 24a to 24d again and are multiplexed with video signals and are sent to lines 5a to 5d. This multiplexed signal is separated to video and audio signals, and they are decoded and are outputted to monitors and speakers.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention enables a plurality of video conference terminals installed in remote locations to communicate with each other via a communication network such as B-ISDN (Broadband Service Integrated Digital Network). The present invention relates to a multipoint video conference in which the terminals are connected and a communication conference is held between these terminals.

[Conventional technology]

A conventional multipoint video conference system using a control node such as a conference brig was configured as shown in FIG. In the figure, 1a to 1d are video conference terminals located at multiple points, and 2 is a multipoint video conference control device. Set lines 4a to 4d to terminals 1a to 1.
Mutual communication was carried out between d. Here, the television conference terminal 1 includes a video system such as a camera and a monitor, an audio system such as a microphone and a speaker, a network control device for connecting to the communication network 3, and the like.

Next, the operation of the system shown in FIG. 6 will be explained. First, a request is made from the video conference terminal 1a to the exchange in the communication network 3 to set up a line, and four lines B4a with the required transmission capacity are connected between the multipoint video conference control device 2 and the video conference terminals 1a to 1d.
Set ~4d. This allows 4 video conference terminals 1
Individual lines are set up in a star shape between a, lb, lc, and ld and the multipoint video conference control device 2, and communication between these devices is performed. Here, in the lines 4a to 4d, the video signal, the audio signal, and the control signal specifying the display screen are multiplexed to perform full-duplex two-way communication, and the video conference terminal 1b and the IC shown in FIG. As shown on the monitor, a reduced composite screen of images from cameras at other points is displayed, and as shown on the monitors of the video conference terminals 1a and 1d, the screen of a specific point (for example, the speaker) is displayed, and the audio mixes (adds) the audio from all destinations and conducts a multipoint video conference while listening to this. The multipoint video conference control device 2 for realizing this will be described below.

FIG. 7 shows an example of the configuration of the multipoint video conference explanation device 2. As shown in FIG. In the figure, lla to lld are line interfaces, 12a to 12d are video line processing units, and 1
Reference numerals 3a to 13d are video reduction units, 142 to 14d are audio line processing units, 15 is a video switching/synthesizing unit, 16 is an audio addition/distribution unit, and 17 is a control unit.

Next, the operation of the system shown in FIG. 7 will be explained. Signals from the video conference terminals 1a to 1d are transmitted through the line interface ll.
A to lid are separated into a video signal, an audio signal, and a control signal. communicated. First, the video signal from each line is decoded by the video line processing units 12a to 12d, and video synchronization (horizontal synchronization, vertical synchronization, and video frame synchronization) is established by the video memory, and then the video signal is processed by the video reduction units 13a to 13d. Processed by a spatial filter, thinning out every other pixel in the vertical and horizontal directions, the video switching/synthesizing unit 15 synchronizes each line, synthesizes or switches each video signal, and then returns to the video line processing unit. 12a-12d
and sent to the line interfaces lla-lid. Here, the selection between the reduced screen and the screen at a specific point is made based on control signals from the video conference terminals 1a to 1d, and this control signal is transmitted through the line interface lla.
After being separated by . On the other hand, the audio signals are separated by the line interfaces lla to lid, decoded by the audio line processing units 14a to 14d, and added by the audio addition/distribution unit 16 excluding the audio at the own point (N-1). Addition processing is performed, and a loss corresponding to the number of points is inserted to prevent howling, and the signals are sent to the respective audio line processing units 14a to 14d. Here N
is the number of points, which in this case is 4.

Next, FIG. 8 shows the configuration of the audio addition/distribution section 16.

As shown in the figure, after first adding up the audio signals of all points, the audio of the own point is removed and a loss is inserted based on the number of points information from the control unit 17, and the audio signals to be sent to each point are create.

In addition, in response to the change in the number of points in FIG. 8, the input audio signal switch is opened and closed by instructions from the control section 17. The audio signal added here is again processed by the audio line processing unit 14a.
In ~14d, a loss is added and converted into a code suitable for the transmission path, and is multiplexed with video signals etc. in line interfaces lla~lid, and sent to video conference terminals 13~1d.
will be sent to. Here, it is troublesome to switch the screen every time the speaker changes on the video conference terminal side, so in order to improve the man-machine interface, the speaker's screen is automatically displayed by detecting the level of the audio signal. There is a way,
In this case, the speaker is identified from the level detection results of the audio line processing units 14a to 14d, this is transmitted to the control unit 17, and this signal is transmitted to the video switching/synthesizing unit 15 to automatically switch the screen.

[Problem to be solved by the invention]

As described above, the audio signal processing method of the multipoint video conference control device 2 is simple addition (N
-1 addition), so the video conference terminal 1a~
If you hold a meeting while looking at the reduced composite screen or cropped composite screen on LD, the audio from all points will be heard from the same speaker, and in the case of a reduced screen, the screens at each site are small, so it will be difficult to hear who is speaking. This has the drawback of making it difficult to identify the dolphins and lacking a sense of realism.

The present invention has been made in view of these points, and its purpose is to provide a multi-point television that can easily infer which video conference terminal is being used to make a statement and that can create a sense of realism. The goal is to obtain a conference system.

[Means to solve the problem]

In order to achieve such an object, the present invention provides a transmission means for transmitting cut-out position information and composite position information of the screen of each point from the video conference terminal to the conference control node as a control signal, and A cropping and compositing means that performs screen cropping based on cropping position information and screen compositing based on compositing position information, and a position on the screen at which the center of the position where images from each point are to be composited is located during screen compositing. and an addition means that performs a plurality of audio addition processes based on the determination result of the determination means, and the addition means performs a plurality of audio addition processes based on a control signal when displaying on a monitor of a video conference terminal. The added audio signal is output from a speaker.

[Effect]

In the multipoint video conferencing system according to the present invention, in order to simultaneously display images from multiple points, the screen at each point is reduced or partially cut out, and these are combined to create a single screen. Correspondingly, multiple audio signals are selectively added and sent to the terminal side using multiple lines. On the terminal side, a plurality of speakers are provided to receive screen synthesis information and output audio signals to the speakers corresponding to the screen.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a multipoint video conference system according to the present invention. In the same figure, 5a to 5
Reference numeral d represents a line as a transmission means having a capacity for transmitting video and audio, 6a to 6d represent video conference terminals, and 7 represents a multipoint video conference control device. In the multipoint video conference control device 7, 122 to 12d are video line processing units, 133 to 13d are video reduction units, 24a to 24d are audio line processing units, and 26 is an audio selection addition/distribution unit as an addition means. , 27 is a control unit as a determination means, 28a to 2
8d is a line interface, and 29 is a video multiplexing switching/synthesizing unit as a cutting/synthesizing means.

The operation of the system shown in FIG. 1 will now be described. In the figure, it is assumed that there is a moderator at the terminal 6a and a speaker is speaking at the terminal 6b. The following designations are made using screen control signals from the video conference terminals 6a to 6d, respectively. That is, in the terminal 6a, the point Pb and the eye point P of the speaker of the terminal 6b are
A composite screen of point a and other points Pc and Pd, since the speaker is speaking on terminal 6b, a composite screen of point Pa of the moderator of terminal 6a and other points Pc and Pd, terminal 6c is a composite screen of the speaker of terminal 6b automatic switching screen, speaker's point pb on terminal 6d
and other points Pa, and specify the composite screen of the PC,
The location information for cutting out a portion of the screen at the relevant point and the location information for compositing are also specified.

On the other hand, in the multipoint video conference control device 7, the video signals from the cameras of the video conference terminals 6a to 6d, the audio signals from the microphones, and the above screen control signals are transmitted to the transmission paths 5a to 5d.
The data is converted into a code suitable for , multiplexed and transmitted, and received by the line interfaces 28a to 28d. The line interfaces 28a to 28d separate the video signal, audio signal, and control signal, respectively, and the video line processing units 12a to 12d,
The information is transmitted to the audio line processing sections 24a to 24d and the control section 27.

First, the video signal is decoded by the video line processing units 12a to 12d, synchronized by the video memory (horizontal, vertical, and video frame synchronization), and filtered by the video reduction units 13a to 13d. The pixels in the vertical and horizontal directions are thinned out one by one. Furthermore, the video multiplexing/synthesizing unit 29 selects video signals from each line in a time-sharing manner (by on/off control of the matrix switch described below) based on instructions from the control unit 27 (screen position information described later). By doing so, the signals are cut out or reduced and combined, and sent to the video line processing units 12a to 12d for encoding suitable for the transmission path.

On the other hand, after the audio signal is decoded by the audio line processing units 24a to 24d, the audio level is detected, and the audio selection addition/distribution unit 26 selects two signals corresponding to each display screen based on instructions from the control unit 27. Performs addition processing for channels (left and right audio). For example, times! For 5a, the voice of line 5b is the voice of line 5C, and the voice of 5d is voice d.
processing is performed with audio c+d. After that, the audio line processing units 24a to 24d encode the signal again, and the line interfaces 28a to 28d multiplex it with the video signal, and send it to the line 5.
Sent from a to 5d. This multiplexed signal is received by the video conference terminals 6a to 6d, separated into video and audio signals, decoded, and output to a monitor and a speaker, respectively.

FIG. 2 shows a configuration diagram of the video multiplexing/synthesizing section 29, which synchronizes and controls the video input signals of the respective lines of the video line processing sections 12a to 12d or the video reduction sections 132 to 13d. The matrix switch S
Turns W on and off. 30a to 30d are superimpose circuits that contain information indicating which point (for example, Pa
This is for superimposing characters (such as Pb, Pc, Pd, etc.) on a video conference terminal. This makes it easy to identify which terminal it is on a composite screen or the like.

Further, in the matrix switch SW, the pixels of each line in the horizontal direction are controlled by the video line processing unit 12a.
- Extracting the video from 12d or video reduction unit 13
The reduced video signals from 2 to 13d are cut out and synthesized. Here, the timing of cutting out is performed based on the position information of the screen control signal from the terminal, and the combining is also performed based on the position information from the video conference terminals 6a to 6d.

In addition, when automatically switching the speaker screen as in the case of the terminal 6c, the control unit 27 controls the audio line processing units 24a to 24d.
The speaker is detected by finding the maximum value from the voice level detection results, and this information is sent to the video multiplexing/synthesizing section 29 to control the matrix switch SW so that the screen of the speaker is always displayed.

FIGS. 3(a) to 3(e) are explanatory diagrams of a method for synthesizing video signals in the matrix switch SW, and for the video from the line 5a, cutout position information is provided based on the screen control signal from the video conference terminal. (XalYal) and (X
a2. A rectangular screen shown in the diagonal relationship of Ya2) is cut out (see Fig. 3 fa)), and position information (Xbl, Yb
1) The square screen shown by the diagonal relationship of (Xb2.Yb2) is cut out (see Fig. 3(b)), and the screen from the line 5a is (Xgl, Ygl) and (Xg2.Yg2), respectively. The screen from line 5b is (Xg 3.Yg 3) and (X
g4. It is synthesized at the bottom left position of the square screen of Yg 4). In addition, the screens from times 11i5c and 5d are also (Xg5.Yg5) and (Xg6.Yg5), respectively.
g6), (Xg7.Yg7) and (
Xg8. Similar compositing processing is performed at the lower center and lower right positions corresponding to the diagonal positions of the screen of Yg8) (3rd
Figure (C1, (see dl)).When cutting out the reduced screen, in order to simplify the process, a spatial filter is applied to remove aliasing noise, and pixels in the horizontal and vertical directions are each reduced by 1/2. They are thinned out one by one (every other one) and combined into a square shape.

FIG. 4 shows the timing of compositing in the video multiplexing switching and compositing section 29, and the line indicated by the dashed-dotted line (
When combining the video information of line 5b (see FIG. 4(a)), from the line synchronization signal of one line (see FIG. 4(C)), the video information of line 5b (see FIG. d
l), video information of line 5c (4th line) between Xg5 and Xg6
On/off control of the timing of the matrix switch SW is performed so that only the video information of the line 5d (FIG. 4(f)) is taken out between Xg7 and Xg8. Similar timing control of matrix switches is performed for other lines, and this is processed for each field and frame (see Figure 4 (bl)).However, automatic switching to the speaker's screen is performed without compositing. In this case, the control unit 27 identifies the speaker by comparing the information of the audio detection circuits of the audio line processing units 24a to 24d, instructs this information to the video multiplexing/synthesizing unit 29, and (see Figure 2) is controlled to always switch to the screen of the speaker's location.

Next, audio processing in this embodiment will be explained using FIG. First, the audio signals separated by the line interfaces 28a to 28d are decoded by the audio line processing units 24a to 24d, and the audio level of each line is detected.At the same time, the audio signal is separated by the audio selection addition/distribution unit 26 to be displayed on each screen. Addition of audio is performed in response to. For example, as shown in the example in Fig. 1, in the case of the terminal 6a, the left channel is selected to have audio S, and the right channel is selected to add audio C and audio d, resulting in audio c+d. Add to the table.

Also, for terminal 6b, the left channel has audio a.
, C, and d are added, and voices a and d are added to the right channel. Similarly, two channels of audio are added based on the screen position information displayed on the terminals 6c and 6d.

FIG. 5 shows the configuration of this audio selection addition/distribution section. As shown in the figure, after adding the voice information of all points -1, subtracting the voice of the eye point and adding (N-1),
Based on instructions from the control unit 27, loss insertion control corresponding to the number of points and audio addition for two channels are automatically performed. This is realized by the circuits shown in the solid line and the broken line, and the former circuit is 201og (M
-1) Insert the loss of dB. M is the number of points to be added.

The latter circuit is composed of a switch circuit and a subtraction circuit, and its connection relationship is adaptively changed according to instructions from the control section 27. That is, the control unit 27 determines whether the center of gravity of the composite screen is on the left or right side of the screen as the left or right channel of the two channels. In this way, based on the screen control signals from the video conference terminals 6a to 6d, two channels are added corresponding to the screen to be displayed, encoded by the audio line processing units 24a to 24d, and sent to the line interfaces 28a to 28d. It is multiplexed with a video signal at 28d and transmitted to each video conference terminal via lines 5a to 5d. On the terminal side, the audio of these two channels is
By reproducing the sound with two speakers, it becomes possible to hear the sound corresponding to the screen. That is, for example, the terminal 6a
Now, audio S is output from the left speaker, and audio C and d are output from the right speaker. In addition, when performing automatic switching display like the video conference terminal 6c, the voice detection circuit of the audio line processing units 24a to 24d detects the speaker, and the screen of the terminal 6b, which is the speaker's screen, is automatically switched and displayed. vinegar·
At the same time, (N-1) audio signals from each ground are added, and this audio signal is output from the speakers on both sides.

〔Effect of the invention〕

As explained above, according to the present invention, screens at multiple points are flexibly combined and displayed, and audio signals are generated from multiple (for example, left and right) speakers corresponding to the display screen, thereby creating a sense of realism. This has the advantage of allowing the meeting to proceed smoothly.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of a multipoint video conference system according to the present invention, FIG. An explanatory diagram of the video signal synthesis method in the system shown in the figure, Fig. 4 is a time chart showing the timing of the synthesis of video signals in the system shown in Fig. 1, and Fig. 5 is an audio selection addition that constitutes the system shown in Fig. 1.・A configuration diagram of a distribution unit; FIG. 6 is a configuration diagram showing a conventional multipoint video conference system; FIG. 7 is a configuration diagram showing a multipoint video conference control device that constitutes the system in FIG. 6; The figure is a block diagram showing a conventional audio addition/distribution section. 3... Communication network, 5a-5d... Line, 6a-6d.
...TV conference terminal, 7...Multipoint video conference control device, 12a-12d...Video line processing unit, 13a-
13d...Video reduction unit, 24a-24d...Audio line processing unit, 26...Audio selection addition/distribution unit, 27...
- Control unit, 28a to 28d... line interface,
29...Video multiplex switching/synthesizing section.

Claims (1)

[Claims] A centralized conference control node can display a composite screen of each point's screen reduced on the monitor screen of the video conference terminal, or can be set up at three or more points by switching the screen of each point and displaying it. In a multipoint video conference system in which the video conference terminals connected to each other are connected to each other to conduct a video conference between multiple points, the video conference terminal transmits to the conference control node cut-out position information and synthesis of the screen of each point. a transmission means for transmitting position information as a control signal; a cut-out and synthesis means for cutting out a screen from a screen at each point based on the cut-out position information; and a cut-out and synthesis means for carrying out screen synthesis based on the synthesis position information; comprising a determining means for determining at which position on the screen the center of the position where the images of each point are to be synthesized is located, and an adding means for performing a plurality of audio addition processes based on the determination result of the determining means; A multipoint video conference system, characterized in that the adding means causes a plurality of speakers to output an added audio signal based on the control signal when displayed on a monitor of the video conference terminal.