JPH04177993A - Picture coding system and inter-multi-point video conference system using picture coding system - Google Patents

Abstract

PURPOSE:To generate a divided screen, a talker screen and a subscriber designation screen by coding a subframe sequentially to generate a picture code, obtaining N-sets of subframes through the sequential decoding of the picture code, and synthesizing the N-sets of subframes to reproduce one-frame picture. CONSTITUTION:One frame of an original picture A is made up of N(e.g. 16) picture elements and subframes A1-A4 of the original picture A are formed respectively by thinning one kind of picture elements in total 4 kinds of picture elements, from the original picture. Each of the subframes A1-A4 is coded in the unit of subframes sequentially by a picture coder 15 and an outputted picture code 16 is outputted in a format of a serial signal resulting in sequential coding signal A1'-A4' from the subframes A1-A4. In this case, the picture code 20 consists of the four subframes A1'-A4'. When the picture code 20 is decoded by a picture decoder 21, the subframes A1-A4 as the 1/4 reduction picture from the original picture is obtained. When the picture of the subframes A1-A4 is synthesized, the reproduced picture A of the original picture is obtained. Thus, a split pattern, a talker pattern and a subscriber designation pattern are simply generated.

Description

Detailed Description of the Invention (Industrial Application Field) The present invention relates to an image encoding method for a multipoint video conference conducted between a plurality of terminals, and a multipoint video conference method using this image encoding method. It is related to.

(Prior Art) Conventionally, as a technology in this field, for example, the technology shown in FIGS. 5 and 6 has been considered.

The configuration will be explained below.

In FIG. 5, terminals 101 to 105 are connected to a multipoint video conference device 100, and one person per terminal, a total of five
Participants A-E are participating in a video conference. There are the following three types of screen display modes that the multipoint video conference device 100 provides to each terminal.

■ Split screen display: multiple images of participants in a multipoint video conference are reduced and displayed on one screen. In FIG. 5, terminals 101 and 105 are in this mode.

゛■ Target designation screen display: Displays the image of the partner designated by the multipoint video conference participant. In Figure 5, terminal 1
03 is in this mode.

■Speaker screen display: This multipoint video conference device detects the participant who is currently speaking, and displays the image of that speaker. In FIG. 5, terminals 102 and 104 are in this mode.

FIG. 6 shows the multipoint video conference device 10 in FIG.
FIG. 2 is a block diagram showing the configuration of 0.

In FIG. 6, 200 r to 200. are signal lines from each terminal, into which image signals, audio signals, and control signals are input, and are connected to the separation circuit 201.

A separation circuit 201 converts signals from each terminal into an image signal 20.
2, ~202. ．． , audio signals 2071-207, ,
The control signals are separated into 2131 to 213°.

Image decoders 2031 to 203 receive image signals 202 . 202 is decoded to reproduce the original image, and the frame memory 204. ~204. write to.

The screen editor 205 creates a split screen ((2) above) from the frame memory 204, ~204° in which images from each terminal are stored.

The image encoders 206□ to 206ゎ encode the divided screens created by the screen editor 205, respectively, and the encoded signals are connected to the multiplex circuit 214.

Audio signal 2 from each terminal output from the separation circuit 201
07, ~207. are the audio decoders 2081-2, respectively.
08, it is converted into a linear audio signal.

The audio mixer 209 is connected to the audio decoder 208 . ~208. The linear audio signals output from the audio encoders 2101 to 210 . Output to.

The audio maximum value detection circuit 211 is connected to the audio decoders 208I to 208I.
The terminal (speaker terminal) having the highest audio level is detected from among the linear audio signals output from 08, and the identification number 212 of that terminal is output to the multiplex circuit 214.

Control signal 2 from each terminal output from the separation circuit 201
13. ~2133 are input to multiplex circuit 214.

The control signals 213□ to 213 represent the screen display mode (the above-mentioned split screen display, destination designation screen display, and speaker screen display) requested by each terminal.

The multiplexing circuit 214 includes an image decoder 206 . ~206. The split screen image signal output from the separation circuit 201, the image signal 202 from each terminal output from the separation circuit 201. ~202°,
Control signal 2 from each terminal output from the separation circuit 201
13. ~213, , the maximum value speaker identification number 212 output from the voice maximum value detection circuit 211, and the mixed voice signal output from the voice encoders 2101 to 210 are input, and these image signals and voice signals are input. are multiplexed, and the signal lines 215, to 215 . It is sent to each terminal via . Here, the image signals sent by the multiplexing circuit 214 to each terminal are control signals 2131 to 213 . selected according to That is, when a terminal requests split screen display mode, the image encoder 206. ~206. When one of the outputs is selected and the target area designation screen display mode is requested, the image signal 202. -2029 is selected and the speaker screen display mode is requested, the audio maximum value detection circuit 211
Identification number 2 of the terminal with the highest audio level detected in
12, the image signals 2021-202. One of them is selected.

(Problem to be Solved by the Invention) However, in the multipoint video conference device having the above configuration, in order to create a split screen, an image decoder, an image encoder,
Since a frame memory and a frame memory are required, there are problems in that the cost increases and the hardware scale becomes very large. Furthermore, since image signals sent from conference participants are decoded, edited, and re-encoded, there are problems in that image quality deteriorates (S/N ratio deteriorates) and image signals are delayed.

The present invention solves the above problems and provides an image encoding method that can reduce the cost and hardware scale of a multipoint video conference device, and that does not cause deterioration of image quality or delay of image signals, and The purpose of this invention is to provide a multipoint video conference system using this image encoding system.

(Means for Solving the Problem) In order to achieve the above object, the image encoding method of the present invention has the following features: N subframes corresponding to 1/N reduced images of one frame are constructed by thinning out each frame, the subframes are sequentially encoded to create image codes, and the image codes are sequentially transmitted, and on the receiving side,
It is configured to sequentially receive the transmitted image codes, sequentially decode the image codes to obtain N subframes, and synthesize the N subframes to reproduce one frame image. be.

In addition, in order to achieve the above object, in the multipoint video conference system of the present invention, each terminal transmits one frame of image by N pixels (
However, after configuring N subframes corresponding to 1/N reduced images of one frame by thinning out one pixel per pixel (N is an integer of 2 or more), the image code obtained by sequentially encoding each subframe is One frame image is created by combining N subframes obtained by sequentially decoding image codes sent to a multipoint video conference device or switched and transmitted by the multipoint video conference device in subframe units. The multipoint video conferencing apparatus is configured to switch the image code sent from each terminal in subframe units and send it to each terminal.

(Function) According to the present invention, since the image encoding method and the multipoint video conference system are configured as described above, the multipoint video conference device can use the image encoding method of the present invention sent from each terminal. By switching the encoded image code in subframe units and sending it to each terminal, it is possible to create a split screen, a speaker screen, and a destination designation screen.

(Example) Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows an embodiment of the image signal encoding method used in the multipoint video conference system of the present invention. Here, original image 1
It is assumed that the number of pixels per frame is 4×4=16 pixels, and that a 4-split screen is used.

One frame of the original image A is composed of 4×4=16 pixels, and each of the subframes A1 to A4 of the original image A is obtained by thinning out one pixel out of every four pixels from the original image. Therefore, each of the subframes A1 to A4 is also a reduced screen of the original image A.

Note that this thinning process is performed using a frame memory.

The subframes A1 to A4 are sequentially encoded in units of subframes by an image encoder 15. Note that the image encoding algorithm used by the image encoder 15 is not related to the gist of the present invention, so it will not be specified here.

The image code 16 output from the image encoder 15 is AI', A2'', which is obtained by sequentially encoding subframes A1 to A4.
It is output in the A3' and A4' serial signal format.

FIG. 2 shows an embodiment of the image signal decoding system used in the multipoint video conference system of the present invention.

The image code 20 is the code of four subframes AI', A2
°, A3', and A4'. By decoding this image code 20 with an image decoder 21, subframes A1 to A4 are obtained as 1/4 reduced images of the original image. By combining the images of these subframes A1 to A4, a reproduced image A of the original image is obtained.

Here, for convenience, the number of pixels is 4x4=16 and the number of subframes is 4, but these numbers are arbitrary.

By the way, the image encoding method of the present invention can be applied to search services and priority control by performing hierarchical encoding.

In other words, when encoding image signals and audio signals, it is possible to layer the encoded data into several layers according to their importance, and to transmit only the highly important data during communication without transmitting all the encoded data. An encoding method that allows the reproduction of images and audio on the receiving side.For example, in the search service mentioned above, if the user searches in detail from the beginning, the search time and cost will increase. (For example, by using the portion corresponding to the center of the screen among N subframes, or the portion equivalent to 4 bits of the image code obtained by quantizing the image signal with 8 bits, etc.), the search time can be reduced. and cost savings.

In addition, ATM (Asyn
A chronous transfer mode (Asynchronous Transfer Mode) network is about to be adopted, but one of the drawbacks of this ATM network is cell discard. In particular, in the case of image communication, the amount of information to be transmitted is extremely large, so quality deterioration due to cell discard is a serious problem. Traffic can be alleviated by not transmitting (data of low importance), that is, by performing priority control.

FIG. 3 is a block diagram showing an embodiment of the multipoint video conference device of the present invention.

In FIG. 8, signal lines 30, .about.30 DEG from each terminal are connected to a separation circuit 31. Image signals, audio signals, and control signals are input to the signal lines 30□ to 30°.

The separation circuit 31 separates the signals transmitted from each terminal into image signals 32.-320, audio signals 34.-349, and control signals 33.-33.

The audio decoders 381-38° convert the audio signals 341-34 output from the separation circuit 31 into linear audio signals.

The audio mixer 39 is connected to the audio decoder 38 . ~3B, are mixed (for example, N-1 addition) and output as a mixed audio signal.

Speech encoder 40. ~40. encodes the mixed audio signal output from the audio mixer 39.

The audio maximum value detection circuit 37 includes an audio decoder 38. The terminal (speaker terminal) having the highest audio level is detected from the linear audio signal output from the angle of ~38°, and the identification number 36 of that terminal is output to the switching circuit 35.

The switching circuit 35 receives image signals 321 to 32s outputted from the separation circuit 31 and control signals 33. 339, and the terminal identification number 36 output from the audio maximum value detection circuit 3a are connected. This control signal 33. - 33 are composed of information corresponding to the image display mode (split screen display, destination designation screen display, speaker screen display) requested by each terminal.

The multiplexing circuit 43 receives the image codes 41 . ~41° and audio encoder 40, audio codes 421-42. output from ~40°. is multiplexed and sent to each terminal.

FIG. 4 is an explanatory diagram of the operation of the switching circuit.

In Figure 4, the participants in the multipoint video conference are A-H.
5 people, and the split screen is 4 split screens, and participants A and E
Assume that participant B and D request split screen display, participant C requests display on the speaker screen, and participant C requests display on the target screen. Furthermore, each terminal employs the image encoding method described above. In addition,
In FIG. 4, the same components as in FIG. 3 are given the same numbers.

Image codes 32□ to 32. from each terminal are input to the switching circuit 35. is the same as in Figure 1 (, 1 frame is divided into 4
There are four codes (
Participant A's terminal consists of AI' to A4').

The switching circuit 35 converts these image codes 321 to 3
2. The control signals 33□ to 336 and the speaker terminal identification number 36
Switch and output according to the following. For example, participant A
Since participants A and E request split-screen display, the switching circuit 35 edits the image codes of terminals other than participant A or E to 41 or 41g. Since participants B and D have requested speaker screen display, the speaker terminal identification number 36
The image code Al'~A4' of the terminal specified by 41□
Or it is output from the switching circuit 35 like 414. Participant C requests display of the ground designation screen (
Here, the participant's terminal is specified), and the image code of the specified terminal is 41. The output from the switching circuit 35 is as follows.

Here, for convenience, the number of participants in the multipoint video conference is set to five, and the split screen is set to four split screens, but these are arbitrary.

Furthermore, the present invention is not limited to the above embodiments,
Various modifications are possible based on the spirit of the present invention, and these are not excluded from the scope of the present invention.

(Effects of the Invention) As described above in detail, according to the present invention, the image encoding method is performed on the transmitting side by converting the image of one frame into one pixel per N pixels (N is an integer of 2 or more). N subframes corresponding to 1/N reduced images of one frame are constructed by thinning out each frame, the subframes are sequentially encoded to create an image code, and the image code is sequentially transmitted. The system is configured to sequentially receive transmitted image codes, sequentially decode the image codes to obtain N subframes, and synthesize the N subframes to reproduce one frame of image. In the multipoint-to-point video conference system, each terminal thins out one frame of image by one pixel per N pixels (N is an integer of 2 or more) using the above image encoding method. After configuring N subframes corresponding to 1/N reduced images, each subframe is sequentially encoded and the obtained image code is sent to the multipoint video conference device, or the multipoint video conference device The image code sent out by switching in subframe units is
The N subframes obtained by sequential decoding are combined to reproduce one frame image, and the multipoint video conference device selects the image code sent from each terminal according to the control signal and transmits it to each terminal. It was configured to send to. Therefore, even if an image decoder, an image encoder, and a frame memory are not prepared in the multipoint video conference device (even if the image encoding method of the present invention is sent from the multipoint video conference participant terminals), Split screens, speaker screens, and destination designation screens can be created by simply selecting encoded image signals using the switching circuit installed in the multipoint videoconferencing device. Scale and cost are greatly reduced.

In addition, decoding of image signals within multipoint video conferencing equipment,
Since editing and encoding are not performed, deterioration in image quality and delay in image signals do not occur.

[BRIEF DESCRIPTION OF THE DRAWINGS] FIG. 1 is an explanatory diagram of an embodiment of an image encoding method used in the multipoint video conference system of the present invention, and FIG. 2 is an illustration of an image decoding method used in the multipoint video conference system of the present invention. FIG. 3 is a block diagram showing an embodiment of the multipoint video conference system of the present invention, FIG. 4 is an explanatory diagram of the operation of the switching circuit in FIG. 3, and FIG. 5 is a conventional FIG. 6 is a block diagram showing the configuration of a conventional multipoint video conference device. 15... Image decoder, 21... Image encoder, 31.
...Separation circuit, 35...Switching circuit, 37...
- Audio maximum value detection circuit, 38. ~38. ...Audio decoder, 39...Audio mixer, 40, ~40. . . . Audio encoder, 43 . . . Multiplex circuit, 100 . Patent applicant: Oki Electric Industry Co., Ltd. Representative: Patent attorney: Takeshi Sugiyama (3 others);
q Yuuki!

Claims (2)

[Claims] (1) On the transmitting side, (a) one frame of image is thinned out by one pixel per N pixels (N is an integer of 2 or more) to form N subframes, (b) the subframes are sequentially (c) Sequentially transmit the image codes; (d) Sequentially receive the transmitted image codes; (e) Sequentially decode the image codes to generate N (f) combining the N subframes to reproduce one frame image; (2) In a multipoint video conference system in which a conference is held between multiple terminals via a multipoint video conference device, each terminal transmits one frame of image by N pixels (however, N
is an integer greater than or equal to 2), one pixel at a time is thinned out to construct N subframes corresponding to 1/N reduced images of one frame, and then the image code obtained by sequentially encoding each subframe is transmitted between the multipoints. Image codes sent to a video conference device or switched and sent by the multipoint video conference device in subframe units are sequentially decoded and N subframes obtained are synthesized to reproduce one frame image. , a multipoint video conference system, wherein the multipoint video conference device switches the image code sent from each of the terminals in subframe units and sends the video code to each of the terminals.