JPH0399592A - Screen updating system for video inter-multipoint conference system - Google Patents

Abstract

PURPOSE:To obtain an inter-multipoint video conference system eliminating frozen display screen by providing an information selection means, a talker detection means, a screen updating instructing means and a selection instructing means. CONSTITUTION:Only one of picture data sent from plural 1st terminal equipments 111 being talkers and participants is selected by a selection means 131 and sent to a 2nd terminal equipment 121. When a talker is changed in the terminal equipments 111, it is detected by a talker detection means 141 and an in-frame coding processing is instructed to the terminal equipment 11 being a talker from a screen updating instructing means 151. When a picture processed by the terminal equipment 111 being the talker according to the instruction is sent, a selection instructing means 161 switches the selecting means 131 and the picture data of the terminal equipment being the talker is sent to the 2nd terminal equipment. Thus, the frozen state that the display screen is held at the switching of the talkers is eliminated.

Description

[Detailed Description of the Invention] [Summary] An object of the present invention is to eliminate freezing of the display screen regarding a screen update method of a multi-point video conference system that controls switching of image data between video conference stations installed at multiple points. Then, image data subjected to interframe encoding processing or intraframe encoding processing sent from a plurality of first terminals serving as speakers and participants is selectively selected and output to a second terminal. a selection means, a speaker detection means for detecting a first terminal as a speaker, and a screen update instruction for instructing the first terminal as a speaker detected by the speaker detection means to perform intra-frame encoding processing. and when intra-frame encoded image data is sent from a first terminal that is a speaker detected by the speaker detection means, an instruction to select the first terminal is sent to the selection means. and a selection instruction means.

[Industrial application field]

The present invention relates to a screen update method for a multipoint video conference system that controls switching of image data between video conference stations installed at multiple locations.

[Conventional technology]

There is a video conference system that connects two conference halls located at remote locations through a communication line and exchanges image signals and audio signals with each other. This video conference system has the advantage of saving travel time for both conference participants and ensuring a good atmosphere in the conference hall.

FIG. 4 shows a schematic configuration of the point-to-point video conference system described above. In the figure, the video conference station 41
1 includes a video codec 413 that encodes and decodes video images, and a camera 415 that captures images to be transmitted.
It also includes a monitor device 417 that displays the received video. Similarly, the video conference station 421 uses a video codec 423 . Camera 425. A monitor device 427 is provided.

The video conference stations are connected, for example, via transmission paths each having a line capacity of 384 Kbps in both directions, and image data encoded with high efficiency by moving image codecs 413 and 423 is sent and received. Inter-frame/intra-frame encoding is commonly used as this high-efficiency encoding method, and inter-frame encoding is normally performed, and intra-frame encoding is performed only when there is a large difference between consecutive frames, that is, when there is intense movement. I do.

The configuration of the multipoint videoconferencing system when such a point-to-point videoconferencing system is extended to multiple points is described in the fifth section.
As shown in the figure.

The multipoint video conference system shown in the figure is configured to communicate between a plurality of, for example, four video conference stations (each video conference station is A, B, 0, and D) having the same configuration as the video conference station 411 described above. Each video conference station is connected via a multipoint connection device.

The multipoint connection device in such a multipoint video conference system detects a speaker using, for example, an audio signal for each video conference station, and then receives image data sent from the video conference station that is the speaker. You are transferring to another video conference station where you will be a participant. The video conference station that receives this image data displays the speaker's image on a monitor device and synthesizes the audio signals sent from multiple video conference stations that are the speaker and other participants. Speaker (not shown)
Output from.

For example, when the speaker moves from station A to station B, focus on station 0. If station A is the speaker, the multipoint connection device selects the image data of station A and transmits it to station 0. Normally, station A performs high-efficiency coding using interframe coding and sends the encoded image data to station 0.

Next, when the level of the audio signal of station B increases, the multipoint connection device detects station B as a speaker, selects the image data supplied from station B, and transmits it to station 0.

Station 0, which receives the image data described above from the multipoint connection device, first receives the image data of station A and displays it on the monitor device, and then receives the image data from station B and displays it. conduct.

In addition, as a publicly known example of a point-to-point video conference system or a multi-point video conference system, there is
-43954 publication, JP-A-60-79886 publication,
JP-A-63-280558, JP-A-63-302
There are publications such as No. 685.

[Problem to be solved by the invention]

By the way, the above-mentioned multipoint video conference system has a problem in that a so-called freeze state occurs in which the display screen is held when the speaker is switched.

Each video conference station typically sends interframe-encoded image data to a multipoint connection device, and this encoding is maintained even if the speaker changes. Therefore, when switching from station A to station B, interframe encoded image data from station A is followed by interframe encoded image data from station B, which is then sent to station 0. Then, station 0 detects an error in the image data.After that, station 0 sends an instruction for screen update, that is, intraframe encoding, to B.
The display screen freezes until the intraframe-encoded image data is sent to the station and returned, resulting in an unnatural-looking display.

The present invention was created in view of these points, and an object of the present invention is to provide a screen update method for a multipoint video conference system that eliminates freezing of the display screen.

[Means to solve the problem]

FIG. 1 is a principle block diagram of the screen updating method of the multipoint video conference system of the present invention.

In the figure, a selection means 131 selectively selects image data that has been subjected to inter-frame encoding processing or intra-frame encoding processing, which is sent from a plurality of first terminal devices 11 serving as speakers and participants. and outputs it to the second terminal 121.

The speaker detection means 141 detects the first terminal 111 that becomes the speaker.

The screen update instructing means 151 instructs the first terminal 111, which is the speaker detected by the speaker detecting means 141, to perform intra-frame encoding processing.

The selection instruction means 161 selects the first terminal 111 when intra-frame encoded image data is sent from the first terminal 111 as the speaker detected by the speaker detection means 141. The instruction is sent to the selection means 131.

Therefore, when the speaker changes, the image data is switched after image data corresponding to an instruction for intra-frame encoding processing is sent.

[For production]

Image data sent from a plurality of first terminals 111 serving as speakers and participants is input to the selection means 131, only one of the first terminals 111 is selected, and the image data is transferred to the second terminal. 121.

When there is a change in the speaker in a plurality of terminals 111, the speaker after the change is detected by the speaker detection means 141, and the screen update instruction means 151 performs an intra-frame encoding process on the terminal 111 that has become the speaker. give instructions.

When the intra-frame encoded image data is sent from the terminal 111 that has become the speaker in response to this instruction, the selection instruction means 161 switches the selection means 131 to select the terminal 111 that has become the speaker. The image data is sent to the second terminal 121
sent to.

In the present invention, when the speaker changes, the second terminal 121 changes the image data by switching the image data after receiving the image data according to the instruction for intra-frame encoding processing. At the same time as switching, image data subjected to intra-frame encoding processing can be received and received. Therefore, image data subjected to interframe encoding processing is no longer continuously received from different first terminals 111, and the display screen does not freeze.

〔Example〕

Hereinafter, embodiments of the present invention will be described in detail based on the drawings.

FIG. 2 shows the configuration of a multipoint connection device in an embodiment to which the screen update method of the multipoint video conference system of the present invention is applied.

Here, the correspondence between the embodiment of the present invention and FIG. 1 will be shown.

The first terminal 111 and the second terminal 121 correspond to each video conference station (station A, station B, station 0, station D).

The selection means 131 includes video selectors 211, 213.2.
It corresponds to 15.217.

The speaker detection means 141 corresponds to the speaker detection section 231.

The screen update instruction means 151 includes update request units 241 and 243.
, 245, 247. Multiplexing part 251, 253° 255.2
It corresponds to 57.

The selection instruction means 161 includes the intraframe encoding detection unit 221
corresponds to

Examples of the present invention will be described below assuming that the correspondence relationship as described above exists.

For example, four video conference stations (station A, 8 stations, 20 stations, and station D) are connected to the multipoint connection device shown in FIG. 2, and the overall configuration of the multipoint video conference system is as follows. Consider what is shown in Figure 5.

In FIG. 2, 211, 213, 215, 217 are video selectors, 221 is an intraframe encoding detection unit,
231 is the speaker detection unit, 241°243.245,24
7 indicates an update request section, and 251, 253, 255, and 257 indicate multiplex sections, respectively.

The video selector 211 is for sending image data to the A station, and selects any of the image data input from the video conference stations other than the A station, that is, the B station, the 0 station, and the D station. Output.

Similarly, the video selector 213 is for sending image data to station B, the video selector 215 is for sending image data to station 0, and the video selector 217 is for sending image data to station D. Select one of them and output.

The intraframe encoding detection unit 221 detects, based on the image data sent from each video conference station, that the encoding method of this image data is intraframe encoding. For example, a video conference station performed intraframe encoding processing. When a flag indicating intra-frame encoding processing is set in the frame header section, the intra-frame encoding detection unit 221 detects intra-frame encoding by extracting this flag from the image data.

The intraframe encoding detection unit 221 also controls the selection state of image data in each of the video selectors 211 to 217. An instruction to switch the selection state is sent to each video selector based on the detection result of the intraframe encoding described above and the speaker detection result by the speaker detection unit 231, which will be described later.

The speaker detection unit 231 performs speaker detection based on audio data sent from each video conference station. A video conference station that corresponds to audio data with a high audio level that continues for a predetermined period of time or more is determined to be a speaker,
An instruction to send the image data to the determined video conference station is sent to the intraframe encoding detection unit 221.

The intra-frame detection unit 221 receives the detection results from the speaker detection unit 231, and then detects the intra-frame encoded image data sent from the video conference station determined to be the speaker. Sends a switching instruction to the video selector.

For example, the speaker detection unit 231 detects that station A has become the speaker, and then the intraframe encoding detection unit 221
When the intra-frame encoded image data from the A station is detected, the intra-frame encode detector 221 sends a switching instruction to the video selectors 213 to 217 other than the video selector 211 corresponding to the A station. The video selectors 213 to 217 switch their selection states in response to this switching instruction, and thereafter select and output the image data sent from station A.

Further, the speaker detection unit 231 inputs the speaker detection result (speaker determination result) to the update request units 241 to 247.

The update request unit 241 sends a screen update request, that is, an intraframe coding instruction, to the A station when it is determined that the A station is the speaker. This intraframe coding instruction for station A is as follows:
The image data output from the video selector 211 is multiplexed by the multiplexer 251 and sent to station A.

Similarly, each of the update request units 243 to 247 is for sending an intra-frame encoding instruction to each station when it is determined that each of station B, station 0, and station D is a speaker. The intraframe encoding instruction output from the update request section is multiplexed with image data by multiplexing sections 253 to 257, and sent to each corresponding station.

Each video conference station performs intra-frame encoding of the image data after receiving this intra-frame encoding instruction, so that the intra-frame encoded image data is then detected by the intra-frame encoding detection unit 221. , each video selector is switched.

FIG. 3 shows the operation timing of the embodiment of the present invention described above. Focusing on station 0, the operation timing is shown when the speaker shifts from station A to station B.

In the figure, "speaker detection" indicates the speaker detection result for station B by the speaker detection unit 231, and changes to a high state when it is determined that station B is the speaker. The "update request" indicates an intraframe encoding instruction output from the update request unit 243 corresponding to the B station. The rB station image data indicates image data input from the B station to the multipoint connection device, and is usually subjected to interframe encoding. “Intraframe coding detection” indicates the intraframe coding detection state detected by the intraframe coding detection unit 221 corresponding to station B, and a high state corresponds to the detection state. "Video selector output" indicates the content of image data output from the video selector 213 corresponding to the B station.

As shown in Figure 3, when the speaker changes from station A to station B,
The speaker detection section 231 detects the speaker of the B station, and an update request from the update request section 243 is sent to the B station. Station B outputs intra-frame encoded image data in response to this update request, and after the intra-frame encoded image data is detected by the intra-frame encoding detection unit 221, the video selector 213 switches to display the intra-frame encoded image data. The encoded image data of station B is selected and output.

In this way, when the speaker detection unit 231 in the multipoint connection device detects a speaker, an instruction for intraframe encoding is sent to the video conference station determined to be the speaker. In this state, the video selector will not be switched.
The screen at each videoconference station therefore displays the previous speaker.

Next, the video conference station that received the intra-frame encoding instruction outputs intra-frame encoded image data, and when the intra-frame encoding detection unit 221 in the multipoint connection device detects this image data, Switch each video selector.

Therefore, since the display is not switched until intra-frame encoded image data is sent, the display screen will not freeze. Furthermore, since an instruction for intra-frame encoding is sent immediately when the multipoint connection device detects a speaker, it is possible to switch the screen to the speaker in the shortest possible time.

In the embodiment of the present invention, speaker detection is performed based on voice data, but any method of speaker detection may be used. For example, a signal may be sent from a video conference station serving as a speaker, and this signal may be detected by the speaker detection unit 231.

[Effects of the Invention] As described above, according to the present invention, when the speaker changes, the image data is switched after the image data corresponding to the instruction for intra-frame encoding processing is sent. , the second terminal can receive and receive image data subjected to intra-frame encoding processing at the same time as the image data is switched. Therefore, it is possible to eliminate freezing of the display screen of the second terminal, which is extremely useful in practice.

[Brief explanation of drawings]

Fig. 1 is a principle block diagram of the screen update method of the multipoint video conference system of the present invention, Fig. 2 is a block diagram of the multipoint connection device of the embodiment, Fig. 3 is an operation timing diagram of the embodiment, and Fig. 4 The figure is a block diagram of a two-point video conference system, and FIG. 5 is a block diagram of a multi-point video conference system. In the figure, 111 is a first terminal, 121 is a second terminal, 131 is a selection means, 141 is a speaker detection means, 151 is a screen update instruction means, 161 is a selection instruction means, 211, 213, 215, 217 are Video selector, 221 is an intra-frame encoding detection unit, 231 is a speaker detection unit, 241, 243, 245, 247 is an update request unit, 251
．． 253, 255, and 257 are multiplex parts. Figure 2.

Claims (1)

[Claims] (1) A plurality of first terminals (111
) for selectively selecting image data that has been subjected to inter-frame encoding processing or intra-frame encoding processing and outputting the selected image data to a second terminal (121); speaker detection means (141) for detecting the first terminal (111); and a first speaker detected by the speaker detection means (141).
screen update instruction means (151) for instructing the terminal (111) to perform intra-frame encoding processing; and a first speaker detected by the speaker detection means (141).
When image data subjected to intra-frame encoding processing is sent from the first terminal (111), this first terminal (111)
) selection instruction means (161) for sending an instruction to select the selection means (131);