JPH08307844A - Image data transmitter, terminal equipment and multi-point controller used for the transmitter - Google Patents

Abstract

PURPOSE: To provide the image data transmitter in which an opposite image to be displayed can be selected freely by each terminal equipment and deterioration in image quality can be suppressed attended with processing of decoding and re-coding. CONSTITUTION: Output image data of terminal equipments 1a-1e at each point are sent to a multi-point controller 2. In this case, the image data are processed into a packet and a header including sender information such as an identification signal ID of its own terminal equipment is added to the data. The controller 2 applies high speed time division multiplex to output image data from the terminal equipments 1a-1e to obtain multiplexed data, which are sent respectively to the terminal equipments 1a-1e. The terminal equipments 1a-1e can use output image data from the terminal equipments 1a-1e according to the multiplexed data separately and selectively and a displayed opposite image can be selected optionally. The controller 2 conducts only multiplexing processing of output image data from the terminal equipments 1a-1e, no decoding and re-cording processing is conducted and the deterioration in the image quality can be suppressed attended with the processing.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data transmission device suitable for application to, for example, a video conference system, a terminal device used for the same, and a multipoint control device. Specifically, by transmitting the multiplexed data of the output image data of each terminal device to each terminal device from the multipoint control device, each terminal device can freely select the partner image to be displayed, and the decoding process and re-encoding can be performed. The present invention relates to an image data transmission device capable of suppressing image quality deterioration due to conversion processing, a terminal device used for the same, and a multipoint control device.

[0002]

2. Description of the Related Art FIG. 7 shows a configuration example of a conventional video conference system. The video conference system of the example of FIG.
It is composed of terminal devices 51a to 51e at five points A to E and a multipoint control unit (MCU: Multipoint Control Unit) 52. The multipoint control device 52 is the terminal device 5
1a to 51e, a data switching unit 53 for selectively extracting one image data from the encoded image data VDa to VDe transmitted from 1a to 51e, and an image extracted by the data switching unit 53 A data distribution unit 54 for distributing data and transmitting the data to the terminal devices 51a to 51e.
And have.

In this video conference system, image data VDa-V output from the terminal devices 51a-51e.
De is transmitted to the multipoint control unit 52 and the data switching unit 5
3 is supplied. Then, the image data output from the terminal device at one point, which is selectively taken out by the data switching unit 53 of the multipoint control unit 52, is distributed by the data distribution unit 54 and transmitted to the terminal devices 51a to 51e, respectively. As a result, in each of the terminal devices 51a to 51e, only the image of one point is displayed by the image data extracted by the data switching unit 53 of the multipoint control device 52.

FIG. 8 shows another example of the configuration of a conventional video conference system. The video conference system of the example of FIG. 8 includes terminal devices 61a to 61e at five points A to E,
It is composed of a multipoint control device 62. The multipoint control device 62 decodes the image data VDa to VDe that are output from the terminal devices 61a to 61e and that have been encoded, such as compressed, and decode the decoding units 63a to 63e, respectively.
And to combine the image data decoded by these decoding units 63a to 63e to form image data to be transmitted to each of the terminal devices 61a to 61e, and to re-encode and transmit to the terminal devices 61a to 61e. And a data synthesizing / distributing section 64.

In this video conference system, image data VDa-V output from the terminal devices 61a-61e.
De is transmitted to the multipoint control unit 62, and is decrypted by the decoding unit 63a.
Decoding processing is performed in each of ~ 63e. Then, the image data output from the decoding units 63a to 63e is supplied to the data combining / distributing unit 64, and the image data combining process is performed to form the image data to be transmitted to each of the terminal devices 61a to 61e. For example, the image data to be transmitted to the terminal device 61a is formed so that the images of the points of the terminal devices 61b to 61e are simultaneously displayed on the same screen. Image data to be transmitted to the other terminal devices 61b to 61e are also formed in the same manner. The image data to be transmitted to each of the terminal devices 61a to 61e formed by the data combining / distributing unit 64 is re-encoded and transmitted to the terminal devices 61a to 61e.
Thereby, in each of the terminal devices 61a to 61e, the images of the other four points are simultaneously displayed on the same screen.

[0006]

In the video conference system of the example of FIG. 7, the terminal devices 51a to 51e at the respective points are
There is a problem that only one image can be displayed at each point. Further, in the video conference system of the example of FIG. 8, since the decoding and re-encoding processing is performed in the multipoint control device 62, the image quality is deteriorated, and each terminal device 61a.
Since the image data combined by the multipoint control device 62 is supplied to the devices 61e to 61e, the partner image to be displayed cannot be arbitrarily selected.

Therefore, it is an object of the present invention to allow each terminal device to freely select a partner image to be displayed, and to suppress image quality deterioration due to decoding and re-encoding processing.

[0008]

An image data transmission apparatus according to the present invention comprises a plurality of terminal devices and a multipoint control device, and transmits output image data of the plurality of terminal devices to the multipoint control device. The multipoint control device transmits output multiplexed data obtained by multiplexing output image data of a plurality of terminal devices to a plurality of terminal devices.

The terminal device according to the present invention comprises packetizing means for packetizing the image data, and header adding means for forming the output image data by adding a header containing at least sender information to the packetized image data. An image data separation means for separating output image data of a predetermined transmission source based on the header from time-division multiplexed data of a plurality of packetized output image data to which a header including at least transmission source information is added. And memory means for storing the output image data of each transmission source separated by this image data separation means, and image display for selectively displaying an image based on the output image data of each transmission source stored in this memory means And means.

The multipoint control device according to the present invention includes a memory means for storing a plurality of packetized output image data to which a header containing at least transmission source information is added, and a memory means for storing the memory means. Data multiplexing means for time-divisionally multiplexing a plurality of image data to form output multiplexed data.

[0011]

The image data output from the plurality of terminal devices is transmitted to the multipoint control device for multiplexing, and the multiplexed data output from the multipoint control device is transmitted to the plurality of terminal devices. In the terminal device, for example, after image data is packetized, a header including transmission source information is added and output image data to be transmitted to the multipoint control device is formed. Further, in the terminal device, for example, output image data of a predetermined transmission source is separated based on the header from the multiplexed data transmitted from the multipoint control device, and the image based on the separated output image data is selectively displayed. To be done. In the multipoint control device, for example, a plurality of packetized output image data to which headers including transmission source information are added are time-division multiplexed to form output multiplexed data.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. This example is an example applied to a multipoint video conference system. FIG. 1 shows the multipoint video conference apparatus of this example as a whole. The multipoint video conference apparatus of this example includes terminal devices 1a to 1e at five points A to E and a multipoint control device 2. Terminal device 1a-
The output data MDa to MDe of 1e are respectively transmitted to the multipoint control unit 2 via the line. Further, the output data MDm of the multipoint control device 2 is transmitted via the line to the terminal devices 1a ...
1e, respectively.

FIG. 2 shows a transmitter 11 of the terminal devices 1a-1e.
Is shown. The transmitter 11 is a video camera 11
A / D converter 112 for converting the video signal output from the digital signal 1 into a digital signal, and this A / D converter 112
An encoding circuit 113 for performing encoding such as compression on the image data output by the encoder, and a packetizing circuit 114 for packetizing the encoded image data output by the encoding circuit 113. , A header generation circuit 11 for generating a header including an identification signal ID of its own terminal device as transmission source information, a signal indicating image data, and the like.
5 and a header adding circuit 116 for adding the header created by the header creating circuit 115 to the packetized image data output from the packetizing circuit 114.

The transmitting unit 11 also converts an audio signal from a microphone (not shown) into a digital signal and then performs an encoding process such as compression to form audio data, and an audio encoding unit 117 and image data and audio. User data section 118 for outputting user data other than data, packetized image data VD output by header adding circuit 116, audio data AD and user data output by audio encoding section 117 And a multiplexing circuit 119 for time-division-multiplexing the user data UD output from the unit 118 to form multiplexed data MD for transmission to the multipoint control device 2. The voice data AD output from the voice encoding unit 117 and the user data UD output from the user data unit 118 are also
It is in the form of being packetized and having a header including an identification signal ID of its own terminal device and a signal for identifying the type of data added.

With the above arrangement, the video camera 111
The output video signal is converted into a digital signal by the A / D converter 112, then encoded by the encoding circuit 113, and then packetized by the packetizing circuit 114. A header is added to the image data packetized by the adding circuit 116 to obtain the image data VD.
Is formed. Then, the image data VD, the audio data AD from the audio encoding unit 117, and the user data unit 11
The user data from 8 is time-division multiplexed by the multiplexing circuit 119 to form multiplexed data MD (MDa to MDe), and this multiplexed data MD is transmitted to the multipoint control unit 2 via the line.

FIG. 4 shows the structure of the multipoint control unit 2. The multipoint control device 2 includes terminal devices 1a to 1e.
The image data VDa to VDe, the audio data ADa to ADe, and the user data UDa to UDe are respectively transmitted from the multiplexed data MDa to MDe transmitted based on the header.
Separation circuits 201a to 201e for separating the image data V and the image data V separated by the separation circuits 201a to 201e.
Packet capture circuits 202a to 202e for temporarily capturing Da to VDe in the memory, and separation circuits 201a to
It has decoding circuits 203a to 203e for decoding the audio data ADa to ADe separated by 201e.

Further, the multipoint control device 2 is decoded by a multiplexing circuit 204 for time-division-multiplexing the image data VDa-VDe captured in the packet capturing circuits 202a-202e and decoding circuits 203a-203e. A synthesizing circuit 205 for synthesizing the voice data ADa to ADe, and an encoding circuit 206 for re-encoding the voice data synthesized by the synthesizing circuit 205;
The data switching circuit 207 is provided for selectively extracting any one of the user data UDa to UDe separated by the separation circuits 201a to 201e. Here, the multiplexing circuit 204 performs time-axis compression and time-division multiplexing processing. That is, the packet capture circuits 202a to 202e
The image data VDa to VDe captured respectively in
For example, the data is sequentially read out at a speed five times as high as the writing speed, supplied to the multiplexing circuit 204, and time-division multiplexed.

Further, the multipoint control unit 2 includes a multiplex circuit 20.
4, the multiplexed data VDm output from the encoder 4 and the encoding circuit 20.
6 output audio data AD _MIX and user data UD selectively extracted by the data switching circuit 207
_{The SEL} is time-division multiplexed and the multiplexed data MDm
Has a multiplexing / distributing circuit 208 for transmitting each of these to the terminal devices 1a to 1e.

In the above configuration, the terminal devices 1a to 1e
Multiplexed data MDa to MDe transmitted from
(Shown in A to E) are separation circuits 201a to 201e, respectively.
Of the image data VDa to VDe, the audio data ADa to
ADe and user data UDa-UDe are separated. Then, the image data VDa to VDe separated by the separation circuits 201a to 201e are respectively received by the packet capture circuit 202.
It is supplied to the multiplexing circuit 204 via a to 202e, time-axis compressed, and time-division multiplexing processing is performed to obtain multiplexed data VDm.

Also, the audio data ADa to ADe separated by the separation circuits 201a to 201e are supplied to the decoding circuits 203a to 203e, respectively, and are decoded and then synthesized by the synthesis circuit 205, and the synthesized speech data is encoded. It is supplied to the circuit 206. In the encoding circuit 206, after the re-encoding process is performed, the data is packetized and a header including a signal for identifying the audio data is added to the audio data AD _MIX . Also, the user data U separated by the separation circuits 201a to 201e.
Each of Da to UDe is supplied to the data switching circuit 207 and one of them is selectively taken out.

Then, the multiplexed data VDm output from the multiplexing circuit 204, the audio data AD _MIX output from the encoding circuit 206, and the user data UD _SEL extracted by the data switching circuit 207 are sent to the multiplex distribution circuit 208. The supplied data is time-division multiplexed and multiplexed data MDm (see FIG. 5).
(Shown in F) is formed, and this multiplexed data MDm is transmitted to each of the terminal devices 1a to 1e.

FIG. 3 shows the receiver 12 of the terminal devices 1a to 1e.
Is shown. The receiving unit 12 is a multipoint control device 2
A separation circuit 121 for separating the image data VDm, the audio data AD _MIX , and the user data UD _SEL based on the header from the transmitted multiplexed data MDm, and a header further than the image data VDm separated by the separation circuit 121. A packet separation circuit 122 for separately obtaining the image data VDa to VDe based on the above, an audio decoding unit 123 for performing decoding processing of the audio data AD _MIX separated by the separation circuit 121, and a separation. Circuit 121
And a user data section 124 for processing the user data UD _SEL separated by.

A buffer memory 125a for accumulating the image data VDa to VDe (excluding the header portion) which are sequentially separated by the packet separation circuit 122.
-125e and these buffer memories 125a-125
Decoding circuits 126a to 126e for decoding the image data VDa to VDa stored in e, and a video RAM (VRAM) to which the image data output from the decoding circuits 126a to 126e is supplied and written. 1
27 and a video monitor 129 for displaying an image based on the image data read from the video RAM 127.
And have.

In this case, the memory control means is not shown,
The video RAM 127 has decoding circuits 126a to 126e according to an image to be displayed on the video monitor 129.
Selection of image data to be output, thinning control, and write address control of the video RAM 127 are performed. This allows the video monitor 129 to selectively display images of five points including the user. For example, the display screen of the video monitor 129 is divided into four areas, and the images of four points excluding the self are divided and displayed in each area.

In the above structure, the multiplexed data VDm transmitted from the multipoint control unit 2 is separated by the separation circuit 121.
Image data VDm, audio data AD _MIX
And user data UD _SEL . The audio data AD _MIX is supplied to the audio unit 123 and subjected to decoding processing and the like. As a result, in each of the terminal devices 1a to 1e, it is possible to output synthetic voice by the voice data AD _MIX . The user data UD _SEL is supplied to the user data unit 124 for processing.

The image data VDm separated by the separation circuit 121 is supplied to the packet separation circuit 122, separated into image data VDa-VDe based on the header, and sequentially stored in the buffer memories 125a-125e. Decoding processing is performed in the decoding circuits 126a to 126e. Then, according to the image to be displayed on the video monitor 129, the decoding circuits 126a to 126e.
Image data output from the video RAM 12 selectively
This video R is written to a predetermined address of 7
After the image data read from the AM 127 is converted into an analog signal by the D / A converter 128, the video monitor 1
29.

Thus, on the video monitor 129, an arbitrary image based on the image data VDa to VDe, for example, an image at four points other than itself is divided and displayed. In addition, FIG.
Each of the terminal devices 1a to 1e at the points E to E schematically shows a state in which the images of the four points other than the self are divided and displayed.

As described above, according to this example, the multi-point control unit 2 time-division-multiplexes the multiplexed data VDm from the multipoint control unit 2 into the terminal units 1a to 1e at the respective points.
Since each of the terminal devices 1a to 1e separates and selectively uses the image data VDa to VDe from the multiplexed data VDm, the partner image to be displayed on the video monitor 129 can be arbitrarily selected. The screen layout can be freely set. For example, video monitor 1
The screen of 29 is divided into four, and the images of four points other than the self can be simultaneously divided and displayed in each area. Further, according to this example, the multipoint control device 2 time-division-multiplexes the image data VDa to VDe transmitted from the terminal devices 1a to 1e, and transmits the multiplexed data VDm to the terminal devices 1a to 1e. Works like. Therefore, the multipoint control device 2 does not perform the process of decoding and re-encoding the image data, and there is an advantage that the deterioration of the image quality due to the process can be suppressed.

Although the image data and the like are packetized by the terminal devices 1a to 1e in the above embodiment, they may be packetized by the multipoint control device 2. Further, in the above-mentioned embodiment, A to
Although an example of the multipoint video conference system having the terminal devices 1a to 1e at five points E is shown, it goes without saying that the number of terminal devices is not limited to this.

[0030]

According to the present invention, since the multiplexed data of the output image data of each terminal device is transmitted from the multipoint control device to each terminal device, the output image data of each terminal device is separated from the multiplexed data. By using each terminal device, the partner image to be displayed can be arbitrarily selected and the screen layout can be freely set. Further, in the multipoint control device, since the output image data of each terminal device is multiplexed and the decoding process and the re-encoding process are not performed, the image quality deterioration due to the decoding process and the re-encoding process may occur. Can be suppressed.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a multipoint video conference apparatus according to an embodiment of the present invention.

FIG. 2 is a system diagram showing a configuration of a transmission unit of a terminal device according to an embodiment.

FIG. 3 is a system diagram showing a configuration of a receiving unit of the terminal device in the example.

FIG. 4 is a system diagram showing a configuration of a multipoint control device according to an embodiment.

FIG. 5 is a diagram showing input / output data of a multipoint control device.

FIG. 6 is a diagram showing an image display example of each terminal point according to the embodiment.

FIG. 7 is a diagram showing a configuration example of a conventional multipoint video conference system.

FIG. 8 is a diagram showing another configuration example of a conventional multipoint video conference system.

[Explanation of symbols]

 1a to 1e Terminal device 2 Multipoint control device 11 Transmitter of terminal device 12 Receiver of terminal device 111 Video camera 113 Encoding circuit 114 Packetizing circuit 116 Header adding circuit 119,204 Multiplexing circuit 121,201a to 201e Separation circuit 122 Packet separation circuit 125a to 125e Buffer memory 126a to 126e Decoding circuit 127 Video RAM 129 Video monitor 202a to 202e Packet capture circuit 208 Multiple distribution circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H04N 7/24 H04N 7/13 Z

Claims (4)

[Claims] 1. A plurality of terminal devices and a multipoint control device are provided, wherein output image data of the plurality of terminal devices are transmitted to the multipoint control device, and the multipoint control device controls the plurality of terminal devices. An image data transmission device, comprising: transmitting output multiplexed data obtained by multiplexing output image data of a plurality of the terminal devices to the above. 2. The plurality of terminal devices packetize image data, respectively, and add a header containing at least sender information to the packetized image data to form the output image data. The image data transmission device according to claim 1, wherein the point control device time-division-multiplexes the output image data of the plurality of terminal devices to form the output multiplexed data. 3. Packetizing means for packetizing image data, header adding means for adding output data to the packetized image data by adding a header containing at least sender information, and at least sender information Image data separation means for separating output image data of a predetermined transmission source based on the header from time division multiplexed data of a plurality of packetized output image data to which respective headers including Memory means for respectively storing the output image data of each transmission source separated by the data separation means, and image display means for selectively displaying an image based on the output image data of each transmission source stored in the memory means. A terminal device, comprising: 4. Memory means for storing a plurality of packetized output image data to which a header including at least transmission source information is added, and a plurality of output image data stored in the memory means. A multipoint control device, comprising: a data multiplexing means for multiplexing and forming output multiplexed data.