JPH11331795A - Video conference image data control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video conference image data control system capable of continuing a conference, without showing the disturbance of a dynamic image displayed on a terminal display to a user, even when an error occurs in dynamic image data. SOLUTION: User's voice data inputted from a microphone 2 in one terminal 1 and dynamic image data picked up by a camera 3 are compressed by a compressing device 6, the compressed data are multiplexed by a multiplexer 8 and the multiplexed data are stored in a data buffer 11 and transmitted from a 1st communication equipment 13 to a 2nd communication equipment 16 in a counter terminal 15 through a communication channel 14. In the terminal 15, a data buffer 18 stores the received sound data and dynamic image data, a separator 21 separates the sound data from the dynamic image data and an extending device 23 extendes respective separated data, and at the detecting of error occurrence in the dynamic image data, the device 23 allows an image buffer 27 to display the data of a block on the same position as a frame immediately prior to the block generation of the error.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a TV conference system using a communication line, which is used when a terminal detects that an error has occurred in moving image data being received.
The present invention relates to a TV conference image data control system that enables a terminal user of a conference system to continue a TV conference without seeing disturbed images.

[0002]

2. Description of the Related Art A conventional TV conference image data control system is described in Japanese Patent Application Laid-Open No. 02-126783. As shown in FIG. 6, this conventional TV conference moving image data control system obtains a moving image of a user who uses one terminal 101 with a camera 102 installed on one terminal 101. The moving image data is subtracted from the moving image data of one frame before in the transmission unit 104 of one terminal 101, stored as an inter-frame difference signal in the inter-frame difference memory 105, and further transmitted to the compression device 111 via an adder. Is entered. The difference data of the moving image of the inter-frame difference signal compressed by the compression device 111 is transmitted from the communication device 113 to the communication device 116 of the opposite terminal 115 of the TV conference moving image data control system via the communication line 114.

[0003] The communication device 116 receives the difference data and outputs it to the decompression device 112.
At the same time, is added to the video data of one frame before from the frame memory 123, and is added to the display 12
7 is output. The display 127 is one terminal 10
This is displayed as a moving image of the first user. The camera 128 of the opposing terminal 115 and the part of the inter-frame difference memory 120 of the transmission unit 119 have the same configuration as the one terminal 101 and perform the same operations. Also, one terminal 1
The decompression device 112 on the 01 side, the frame memory 109 of the receiving unit 106, the adder 110, and the display 103 are also configured and operate in the same manner as the counter terminal 115 side.

In a conventional TV conference system having such a configuration, for example, the communication device 116 of the opposite terminal 115
When the error detecting unit 107 of the receiving unit 106 of one terminal 101 detects that there is an error in the differential data transmitted from the transmitting unit 119, the transmitting unit 1 of the opposite terminal 115
The adder 110 adds the combined difference data transmitted from 19 through the communication device 116 to the moving image data two frames before stored in the frame memory 109 of the receiving unit 106, and the moving image is added. , The moving image is immediately restored. When an error occurs, an error signal is output from the error detector 107 of the receiving unit 106 of one terminal 101, and a digital video signal is supplied from the sub-frame memory 108 to the frame memory 109 to correct the frame memory. Is performed.

[0005] The transmitting section 119 of the opposite terminal 115 transmits the difference data obtained by combining the newly obtained inter-frame difference and the data of the inter-frame difference memory 120 to the receiving section 106 of one terminal 101. Transmit to The receiving unit 106 combines the moving image of the difference data transmitted with the moving image of the previous frame stored in the frame memory 109 and displays the combined image. In this way, only when the moving image at the time of occurrence of the error becomes the same as the moving image one frame before, a normal moving image can be displayed on the display 103 when the next combined difference data is supplied. ,
The time required for restoring a moving image is reduced.

[0006]

However, this prior art has the following problems. That is, the transmitting unit 10
At 4,119, the data of the previous frame must be saved. The reason is that only the difference data is transmitted, and the intra frame cannot be used. As an approximation technique, Japanese Patent Application Laid-Open No. 62-200994
In the publication, when an error occurs as a moving image communication system, a frame error signal is input to the graphic display device, where the error-free moving image data one frame before is displayed, and the frame error signal is With reference to the data input to the input device, where no error has occurred,
It is disclosed that encoding and transmission enable error control that is rich in real time. However, even in the case of this publication, the above problem has not been solved.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned conventional problems. Even when an error has occurred in moving image data, the user does not see disturbance of the moving image displayed on the display of the terminal. It is an object of the present invention to provide a TV conference image data control system capable of continuing a conference.

[0008]

In order to achieve the above object, a TV conference image data control system according to the present invention comprises:
First communication data generating means for compressing and multiplexing moving image data and voice data of one terminal user of the V conference system and generating communication data for transfer to an opposite terminal of the TV conference system; A second communication data generation unit for compressing and multiplexing the moving image data and the audio data of the opposite terminal user to generate communication data for transfer to the one terminal; and the first communication data generation unit. A first communication device for transferring the generated communication data from the one terminal to the opposite terminal, and receiving the communication data generated by the second communication data generating unit; Transferring the communication data generated by the communication data generation means to the one terminal;
A second communication device that receives the communication data generated by the communication data generation unit, and separates and expands the communication data received by the first communication device to reproduce moving image data and audio. And a first reproducing unit for displaying the moving image data of the immediately preceding frame when an error occurs in the moving image data, and separating and expanding the communication data received by the second communication device. And a second reproducing unit that reproduces the moving image data and the sound and that displays the moving image data of the immediately preceding frame when an error occurs in the moving image data.

[0009] The moving image data and the voice data of the user of one terminal of the TV conference system are compressed and multiplexed by the first communication data generating means of the one terminal, and the communication data is transmitted to the opposite terminal by the first communication means. Forward. When the second communication means of the opposite terminal receives the communication data generated by the first communication data generation means, it outputs the communication data to the second reproduction means,
The second reproducing means separates and decompresses the communication data to reproduce the moving image and voice of one terminal user, and when an error occurs in the moving image data, displays the moving image data of the immediately preceding frame. Act like so.

Similarly, the moving image data and voice data of the user of the opposite terminal of the TV conference system are compressed and multiplexed by the second communication data generating means of the opposite terminal, and the communication data is transmitted to the second communication means by the second communication means. Transfer to terminal. When the first communication means of one terminal receives the communication data generated by the second communication data generation means, it outputs it to the second reproduction means, and the second reproduction means separates and decompresses the communication data. When the moving image and the voice of the opposite terminal user are reproduced and an error occurs in the moving image data, the moving image data of the immediately preceding frame is displayed.
Therefore, according to the present invention, even when an error occurs in the moving image data, the conference can be continued without showing the user any disorder of the moving image displayed on any terminal.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a TV conference image data control system according to the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the first embodiment of the present invention. In FIG. 1, for convenience of explanation, the terminal 1 is one terminal used by the user of the TV conference system, and the terminal 15 will be described as an opposite terminal facing the one terminal 1. One terminal 1
A microphone 2 (hereinafter, referred to as a microphone) that collects a voice of a user of the one terminal 1 and converts the voice into an electric signal.
And a camera 3 for capturing a moving image.

The audio signal output from the microphone 2 and the moving image data output from the camera 3 are sent to a compression device 6. The compression device 6 compresses these data under the control of the data control device 10 and sends the data to the multiplexing device 8. The multiplexing device 8 multiplexes the compressed data output from the compression device 6 and outputs the multiplexed data to the data buffer 7, where it is temporarily stored. The microphone 2, the camera 3, the compression device 6, the multiplexing device 8, and the data buffer 7 constitute a first communication data generation unit.

The output of the data buffer 7 is transmitted to the communication device 1
3 and is transferred from the first communication device 13 to the second communication device 16 of the opposite terminal 15 through the communication line 14. The first communication device 13 and the second communication device 1
6 both have a transmission and a reception function. The first communication device 13 transmits the communication data held in the data buffer 7 to the opposite terminal 15 and the second communication device of the opposite terminal 15. The communication data transmitted from the communication device 16 can be received. Similarly, the second communication device 1
Numeral 6 is capable of receiving communication data transmitted from the first communication device 13 and transmitting communication data generated by the opposite terminal 15 to one terminal 1.

In the opposite terminal 15, the second communication device 16
Is transmitted to the data control device 19 and the data buffer 18 from the one terminal 1. The data buffer 19 temporarily stores communication data from the second communication device 16 and outputs the data to the separation device 21. The separation device 21 separates voice data and moving image data of the user of the one terminal 1 included in the communication data, and sends the data to the decompression device 23.

The decompression device 23 decompresses audio data and moving image data, respectively, and
In addition, the moving image data is output to the moving image buffer 27 after constructing the reproduction moving image data according to the type of the frame. The data buffer 19, the separation device 21, the expansion device 23, the speaker 27, and the moving image buffer 27 constitute a second playback unit. When detecting that an error has occurred in the moving image data of the communication data transmitted from one terminal 1, the decompression device 23 notifies the data control device 19 of an error detection message and, at the same time, notifies the data control device 19 of the error. Is notified so that the data of the previous frame is displayed for the block in which the error occurred.

The data control device 19 which has received the message from the decompression device 23 transmits the data of the block in which the error was found to the second communication device 16 to one terminal 1.
In the next frame, instead of an inter (difference) block,
Notify to transmit an intra (intra-frame coded) block. One terminal 1 transmits the moving image data of the block in which the error has occurred to the opposite terminal 15 in response to the message as an intra block. In addition, the opposite terminal 15 that has received the moving image data from the first communication device 13 of the one terminal 1 transmits the data of the next frame to the block in which the error occurred in the previous frame, as in the normal operation. It is designed to be written. In this way, even when an error occurs in the received moving image data, the moving image can be displayed without being disturbed.

Also, at the opposite terminal 15, a microphone 24 which collects the voice of the user of the opposite terminal 15, converts the voice into an electric signal and outputs the electrical signal to the compression device 22, and captures the moving image data of the user 25 that outputs the data to the compression device 22
Is installed. The compression device 22 receives the audio signal from the microphone 24 and the moving image data of the camera 25, compresses them, and outputs the compressed data to the multiplexing device 20. The multiplexing device 20 multiplexes both audio and video data as communication data and outputs the multiplexed data to the data buffer 11.

The microphone 25, camera 26, compression device 22, multiplexing device 20, and data buffer 11 constitute a second communication data generating means. The communication data stored in the data buffer 11 is transmitted from the second communication device 16 to the first communication device 13 of the one terminal 1 via the communication line 14. First communication device 13
Receives the communication data, the data control device 10
Is output to the data buffer 12. The data control device 10 controls the data of the compression device 6 according to the state of the input communication data.

The data buffer 12 temporarily stores communication data from the opposite terminal received by the first communication device 13,
The data is output to the separation device 9. The separating device 9 separates the audio signal of the communication data from the moving image data and outputs the separated data to the decompressing device 7. The decompression device 7 decompresses the audio signal and the moving image data compressed on the transmission side to construct reproduction data, outputs the audio signal to the speaker 4, and outputs the moving image data to the moving image buffer 5. , There to hold. The data buffer 12, the separation device 9, the expansion device 7, the speaker 4, and the moving image buffer 5 constitute a first reproducing unit.

When there is an error in the moving image data among the communication data transmitted from the opposite terminal 15, the decompression device 7 notifies the data control device 10 of an error detection message. ing. At the same time, the decompression device 7 displays the data of the immediately preceding frame with respect to the block in which the error has occurred in the moving image buffer 5. The data control device 10 having received the message from the decompression device 7 transmits the data of the block in which the error has been found in the opposite terminal 15 to the first communication device 13,
In the next frame, notification is made to transmit an intra block instead of an inter block. In response to the message, the opposite terminal 15 transmits the moving image data of the block in which the error has occurred to the one terminal 1 as an intra block.

The opposite terminal 15 that has received the moving image data of the communication data from one terminal 1 writes the data of the next frame to the block in which the error occurred in the previous frame, as in the normal operation. Like that.
In this way, even if an error occurs in the received moving image data, the moving image can be displayed without being disturbed.

Next, the operation of the first embodiment configured as described above will be described with reference to the flowchart of FIG. The flowchart of FIG. 2 shows the flow of the operation of the first embodiment and a second embodiment described later.
In describing the operation of the first embodiment, an operation in the case where communication data is transferred from one terminal 1 to the opposite terminal 15 will be described. First, when the TV conference starts (step A1), the data of the user of the TV conference system includes moving image data from the camera 3 of one terminal 1,
The audio data is input to the compression device 6 from the microphone 2. When each of the audio data and the moving image data is input to the compression device 6, the compression device 6 compresses the data and passes it to the multiplexing device 8.

The multiplexing device 8 multiplexes audio data and moving image data as communication data and stores the multiplexed data in the data buffer 11. The first communication device 13 extracts communication data from the data buffer 11 and transfers it to the opposite terminal 15 through the communication line 14 (step A2). Opposite terminal 15
The communication device 16 receives communication data from the communication device 13 of the terminal 1 and stores the data in the data buffer 18.

The separation device 21 separates the communication data stored in the data buffer 18 into moving image data and audio data, and passes the data to the decompression device 23. The decompression device 23 decompresses the received moving image data and audio data, constructs the moving image data for reproduction according to the type of the frame, and transfers the data to the moving image buffer 27.
FIG. 3 and FIG. 4 are explanatory diagrams of the compression and decompression methods of the moving image data. The moving image data is divided by time as shown in FIG.

Each frame 31 is shown as a frame 41 in FIG. 4. As shown in FIG. 4, the frame 41 is divided into blocks 42 each having a predetermined number of pixels, for example, 8 pixels vertically and 8 pixels horizontally, The compression and decompression of image data are performed in block units. Intra (intra-frame coded) blocks that compress and decompress video data using only the data in that frame do not use other frame data, so it is not possible to reconstruct video using only that frame data. It becomes possible.

An inter (differential encoding) block for compressing and decompressing moving image data using data of adjacent frames requires data of another frame when reconstructing moving image data. However, a moving image cannot be reproduced with data of only that frame. However, since only the difference data needs to be transmitted / received, the data amount can be extremely reduced.

In this way, the communication data compressed and transmitted from one terminal 1 is expanded by the expansion device 23, and when the expansion device 23 confirms that no error is detected in the moving image data, (Step A3), the audio data is passed from the expansion device 23 to the speaker 26, and the speaker 26 is driven to generate audio from the speaker 26. That is, it is reproduced. At the same time, the moving image data is transferred to the image buffer 27, and the image buffer 2
7 and can be displayed as a moving image on a display (not shown). That is, it is possible to reproduce a moving image.

In the expansion device 23, the compression device 2
When detecting that an error has occurred in the moving image data included in the communication data of one terminal 1 input from the terminal 1 (step A3), the decompression device 23 sets the image buffer 27
Is notified to display the data of the frame immediately before the erroneous block 42 (step A4). The image buffer 27 that has received this message does not display new data for the block on the display, but displays the data of the block at the same position in the previous frame on the display (step A).
5).

As described above, in the first embodiment, when an error is found in the moving image data, the block disturbed by the error is compared with the moving image data of the block at the same position one frame before. By performing the display, the user of the opposite terminal 15 does not need to show the disturbance of the moving image displayed on the display. Therefore, the TV conference can be continued as it is. In the operation of the first embodiment, the communication data is transferred from one terminal 1 to the opposite terminal 15. The same operation is performed when the communication data is transferred toward 1. If an error has occurred in the moving image data of the user of the opposite terminal 15 included in the communication data in that case, the decompression device 7 detects the error and performs the same processing as described above.

Next, a second embodiment of the present invention will be described. In the second embodiment, the configuration of the TV conference image data control system as a system is the same as that of the first embodiment. In the second embodiment, the opposite terminal 1
5, the decompression device 23 receives the data unit 51 as communication data received from one terminal 1 as shown in FIG.
In the case where the moving image data of the data unit 51 is an error 52 as shown in FIG.
When the decompression device 23 detects that there is an error, the decompression device 23 notifies the moving image buffer 27 of a message so as to display the data of the block of the immediately preceding frame with respect to the erroneous block 54 of the frame 53, and Then, a message of the discovery of the error 52 is notified to the data control device 19 (step A6).

The data control device 19 which has received the message from the decompression device 23 transmits the data of the block in which the error was found to the communication device 16 in the next frame, not the inter block but the intra block. Is transmitted (step A7). One terminal 1 responds to the above message by using the moving image data of the block in which the error has occurred as an intra block as the first block.
The data is transmitted from the communication device 13 to the opposite terminal 15 via the communication line 14 (step A8).

The opposite terminal 15 which has received the communication data including the moving image data from one of the terminals 1 transmits the next frame to the block in which the error has occurred in the previous frame in the image buffer 27 in the same manner as in the normal operation. Is written and displayed on the display (step A9), and a series of processing is ended (step A10). In this way, even when an error occurs in the received moving image data, the moving image can be displayed without being disturbed.

[0033]

As described above, according to the present invention, communication data including audio data and moving image data is transmitted between terminals facing each other, and an error occurs in the moving image data on the receiving terminal side. When the video data is found, the video data of the immediately preceding frame is displayed. Therefore, even if an error occurs in the video data, the disturbance of the video displayed on the display of the terminal is not affected by the TV. The conference can be continued without showing the conference system user.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of a TV conference image data control system according to the present invention.

FIG. 2 is a flowchart showing the operation flow of the first embodiment and the second embodiment of the TV conference image data control system according to the present invention.

FIG. 3 is an explanatory diagram showing a state in which moving image data applied to the first embodiment of the TV conference image data control system according to the present invention is divided by time into frames.

FIG. 4 is an explanatory diagram in a case where the frame shown in FIG. 3 is divided into blocks of 8 pixels in width and 8 pixels in height, and moving image data is compressed and expanded in units of blocks.

FIG. 5 is a block diagram showing a moving image data of a frame immediately before an erroneous block when an error occurs in moving image data applied to a second embodiment of the TV conference image data control system according to the present invention; FIG.

FIG. 6 is a block diagram showing a configuration of a conventional TV conference image data control system.

[Explanation of symbols]

1 ... One terminal, 2,24 ... Microphone, 3,25 ...
Camera, 4, 26 ... Speaker, 5, 27 ... Image buffer, 6, 22 ... Compression device, 7, 23 ... Decompression device,
8, 20 multiplexing device, 9, 21 separating device, 1
0, 19 ... Data control device, 11, 12, 17, 18
... data buffer, 13 ... first communication device, 16 ...
Second communication device, 14 ... communication line, 31, 41, 53 ...
... frame, 42, 54 blocks.

Claims (13)

[Claims] 1. First communication data for compressing and multiplexing moving image data and voice data of one terminal user of a TV conference system and generating communication data for transfer to an opposite terminal of the TV conference system. Generating means; second communication data generating means for generating communication data for compressing and multiplexing the moving image data and audio data of the opposite terminal user and transferring the data to the one terminal; and the first communication A first communication device for transferring the communication data generated by the communication data generation means from the one terminal to the opposite terminal, and receiving the communication data generated by the second communication data generation means; Transferring the communication data generated by the second communication data generation means to the one terminal, and transferring the communication data generated by the first communication data generation means A second communication device for receiving, separating and expanding the communication data received by the first communication device to reproduce moving image data and audio,
And a first reproducing means for displaying moving image data of a previous frame when an error occurs in moving image data, and separating and expanding the communication data received by the second communication device to create a moving image. Plays back image data and audio,
And a second reproducing means for displaying the moving image data of the immediately preceding frame when an error occurs in the moving image data, and a TV conference image data control system. 2. The method according to claim 1, wherein the first communication data generating unit divides the frame of the moving image data of the one terminal user into blocks of predetermined vertical and horizontal pixels and compresses the blocks. Item 3. The video conference image data control system according to Item 1. 3. The second communication data generating means divides a frame of moving image data of the opposite terminal user into blocks of predetermined vertical and horizontal pixels and compresses the blocks. 2. The video conference image data control system according to 1. 4. The first reproducing means divides a frame of the moving image data included in the communication data received by the first communication device into blocks of predetermined vertical and horizontal pixels and decompresses the blocks in units of blocks. The video conference image data control system according to claim 1, wherein 5. The second reproducing unit divides a frame of the moving image data included in the communication data received by the second communication device into blocks of predetermined vertical and horizontal pixels and expands the blocks in units of blocks. The video conference image data control system according to claim 1, wherein 6. The first communication data generating means captures a voice of a user of the one terminal and outputs voice data, and captures an image of the user of the one terminal. A first camera that outputs moving image data, a first compression device that inputs and compresses the audio data and the moving image data, and the audio data and the moving image that are compressed by the first compression device. A first multiplexing device that multiplexes the image data to generate the communication data; and a first data buffer that stores the communication data generated by the first multiplexing device. The TV according to claim 1,
Conference image data control system. 7. The second communication data generating means collects a voice of a user of the opposite terminal and outputs voice data, and a moving image by capturing an image of the user of the opposite terminal. A second camera that outputs data, a second compression device that inputs and compresses the audio signal and the moving image data, the audio data and the moving image data compressed by the second compression device, A second multiplexer for multiplexing the communication data to generate the communication data, and a second data buffer for storing the communication data generated by the second multiplexer. The TV conference image data control system according to claim 1. 8. The third data buffer for holding communication data generated by the second communication data generation means received by the first communication device, the third data buffer comprising: A first separation device that separates image data and audio data extracted from a data buffer, a first expansion device that expands moving image data and audio data separated by the first separation device, respectively, A first speaker that is driven by the audio data expanded by the first expansion device to reproduce audio and a first speaker that reproduces the moving image data expanded by the first expansion device and displays the video data on a display; 2. The TV conference image data control system according to claim 1, further comprising: an image buffer. 9. When the first decompression device detects that an error has occurred in the moving image data received by the first communication device, the first decompression device transmits the same image of the previous frame to the first image buffer. 9. The video conference image data control system according to claim 8, wherein moving image data of a block at a position is displayed. 10. When the first decompression device detects that an error has occurred in the moving image data received by the first communication device, the first decompression device stores the same image of the previous frame in the first image buffer. At the same time as notifying the message for displaying the moving image data of the block at the position, the moving image data of the block in which the error has occurred is sent to the first data control device provided in the one terminal in the next frame. The TV conference image data control system according to claim 8, wherein the first communication device is notified to perform intra-block transmission from the second communication device to the first communication device. 11. A fourth data buffer for holding communication data generated by the first communication data generation means received by the second communication device, the second reproduction means comprising:
A second separating device that separates the image data and the audio data extracted from the fourth data buffer, and a second decompressor that decompresses the moving image data and the audio data separated by the second separating device, respectively. A device, a second speaker that is driven by the audio data expanded by the second expansion device to reproduce audio, and a display that reproduces the moving image data expanded by the second expansion device and reproduces the moving image data. The TV conference image data control system according to claim 1, further comprising a second image buffer to be displayed. 12. The second decompression device, upon detecting that an error has occurred in the moving image data transmitted from the one terminal and received by the second communication device,
12. The moving image data of a block at the same position one frame before a block disturbed by an error is reproduced in the second image buffer.
The video conference image data control system according to the above. 13. When the second decompression device detects that an error has occurred in the moving image data received by the second communication device, the second decompression device stores the same image of the previous frame in the second image buffer. At the same time as notifying a message for displaying moving image data of the block at the position, the moving image data of the block in which the error has occurred is sent to the second data control device provided in the opposite terminal in the next frame. The TV conference image data control system according to claim 11, wherein the second communication device is notified to perform intra-block transmission from one communication device to the second communication device.