JPH06334997A - Visual telephone equipment - Google Patents

Abstract

PURPOSE:To display encoded and decoded pictures of user's own picture with a simple constitution by using the encoded and decoded user's own picture signals stored in a picture memory having a variable delay function for use of motion compensation in a user's own picture encoding circuit and displaying the encoded and decoded user's own pictures on the display part of a user's own terminal simultaneously with a reception picture during speaking. CONSTITUTION:A quantized signal is branched into two, and one is sent to the other party of communication through a multiplexing/demultiplexing circuit 105, and the other is inversely quantized by an inverse quantizer 115 and is inversely converted by an inverse converter 116 and is stored in picture memories 118a and 118b having the two variable delay functions for use of motion compensation. One of these memories is used for comparison picture to generate a next difference picture. The other is used for read of the memory of a PinP circuit 108. The picture written in the memory of the PinP circuit 108 by the write priority of the reception picture has the format converted by an inverse format conversion circuit 109 and is subjected to D/A conversion by a D/A conversion circuit 107 and is displayed on a monitor 106.

Description

Detailed Description of the Invention

[0001]

INDUSTRIAL APPLICABILITY The present invention is capable of displaying the same image as the image received at the receiving side on the self-display unit by displaying the self-image on the self-display unit after encoding and decoding the self-portrait. The present invention relates to a videophone device that can be used.

[0002]

2. Description of the Related Art FIG. 5 is a block diagram showing a schematic configuration of a first example of a conventional videophone device. In the figure, 101 is a camera, 102 is an A / D conversion circuit, 103 is a format conversion circuit, 104 is a self-picture coding circuit, and 105 is a multiplexing / multiplexing circuit.
Separation circuit, 106 is a monitor, 107 is a D / A conversion circuit,
Reference numeral 108 is a PinP circuit, 109 is an inverse format conversion circuit, and 110 is a partner image decoding circuit. In this conventional example, an image captured by the camera 101 is converted into an A / D conversion circuit 10
After A / D conversion in 2, the image is branched and the PinP circuit 1
In the method of 08, the image is added to the other image after the inverse format conversion from the inverse format conversion circuit 109, D / A converted by the D / A conversion circuit 107, and then displayed on the monitor 106.

The PinP circuit 108 is a circuit for dropping one of the two image input signals to the level of a digital signal, reducing it, adding it to the other image signal, and returning it to an analog signal for display. Then, it is used as a module for displaying as a parent-child screen.

FIG. 6 is a block diagram showing a schematic configuration of a second example of a conventional videophone device. The same components as those in FIG. 5 are designated by the same reference numerals and the description thereof will be omitted. This conventional second
An example is an A / D conversion circuit 1 for an image captured by the camera 101.
After A / D conversion in 02, format conversion is performed in the format conversion circuit 103, the image is branched, and the PinP circuit 1
At 08, the other image is added to the other image after being encoded by the other image decoding circuit 110, inverse format conversion is performed by the inverse format conversion circuit 109, and D is performed by the D / A conversion circuit 107.
This is a method of displaying on the monitor 106 after A / A conversion.

In the first and second examples, the self-portrait displayed on the monitor 106 is a self-portrait that is neither encoded nor decoded.

[0006]

[Problems when displaying a self-portrait without encoding / decoding] When communicating with a videophone device, the self-portrait may not be projected on the partner screen or may be out of focus. Self-portrait during communication for reasons such as checking how the self-portrait is displayed, such as confirming where the self-portrait is pointing to the other party to explain to the communication partner Must be displayed on the self-display section.

In the above-mentioned first and second conventional examples, the image which is not subjected to the image encoding / decoding process is displayed together with the received image in the folding PinP circuit 108.

When image information is transmitted using a communication line, the amount of information transmission is limited by the communication line speed.
Images with a large amount of information cannot be transmitted as they are. Therefore, an image obtained by performing information compression and frame omission on the original image is transmitted.

FIG. 7 shows a situation in which three types of images, an original image, a folded image by a conventional method, and an encoded / decoded image for which information compression / frame omission has been performed, deteriorate with the passage of time. 1 to 6 are original images, 1 ′ to 6 ′ are folded images by the conventional method, 1 ″, 4 ″, and 6 ″ are encoded / decoded images. In the folded images by the conventional method, A / D is performed for each frame. Conversion / D / A conversion or A / D conversion
Format conversion, reverse format conversion, and D / A conversion are performed. The deterioration of the image due to this processing is not so large that it can be seen by human eyes. However, when image encoding / decoding processing including image compression / frame omission is performed, the amount of information for each frame decreases due to image compression,
Since the frames are skipped due to omission of the frames, the image becomes awkward in motion. Furthermore, since encoding / decoding processing takes time, encoding / decoding delay also occurs.

The image received by the receiving side is an image that has been subjected to these encoding / decoding processing, and therefore an image different from the original image seen by the transmitting side during transmission is received.
Therefore, even if the sender tries to talk to the receiver while pointing to the self-portrait, the image cannot be determined on the receiving side due to image compression, the image itself does not reach due to frame omission, or the sound due to image delay may occur. There is a problem that communication cannot be established due to misalignment. [Problem in Circuit Configuration When Encoding / Decoding Self-Portrait] FIG. 8 shows a configuration in which a self-portrait decoding device is provided in the self-device for displaying the encoded / decoded image on the self-display unit. It is a figure. The same components used in the above figures are designated by the same reference numerals, and the description thereof will be omitted. 110-a
Is a self-picture decoding circuit. The operation is such that, after the self-picture encoding circuit 104 performs self-picture encoding processing, the signal is branched and a dedicated self-picture decoding circuit 110-a is provided separately from the partner picture decoding circuit 110 from the communication partner for decoding. And the image decoded by the partner image decoding circuit 110 and Pi
The nP circuit 108 is used for addition, and the monitor 10 of the self-display unit
Display in 6.

In this method, two decoding circuits must be possessed, which has a drawback that the circuit scale becomes very large.

In consideration of the above points, the present invention is capable of displaying self-portrait encoded / decoded images with a simple structure without providing a self-portrait decoding circuit in order to solve the problems. It is an object of the present invention to provide a videophone device that can be used.

[0013]

A videophone device according to the present invention uses an encoded and decoded self-picture signal stored in a picture memory having a variable delay function for motion compensation in a self-picture coding circuit, It has a display control means for displaying the encoded and decoded self image on the display unit of the self terminal at the same time as the received image during communication.

[0014]

According to the present invention, the image transmission side is subjected to the conversion processing and the quantization processing on the self-picture encoding circuit, and further, the accumulated image subjected to the dequantization and the inverse conversion processing is extracted to extract the accumulated image on the receiving side. It is possible for the transmitting side to extract the same image as the image after the encoding and decoding processing displayed on the screen.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Before explaining the embodiments, the operating principle of the present invention will be explained. FIG. 1 shows a principle configuration diagram of the present invention.
In the figure, the same components as those used in the above figures are designated by the same reference numerals, and the description thereof will be omitted.
Reference numeral 118 is an image memory having a variable delay function for motion compensation. H. 261 (JP-H261 p × 64 kbit /
In the image coding of the video coding method for audiovisual services), the first input image of the first frame is converted, quantized and branched, and one of them is multiplexed / demultiplexed 105
To multiplex and send to the circuit. The other is an image memory 118 having a variable delay function for motion compensation after being inversely quantized and inversely transformed.
Accumulated in. When the image of the next second frame is input, the image of the first frame stored in the image memory 118 having the variable delay function for motion compensation is extracted and compared with the image of the second frame to create a difference image. This is converted into image information of the second frame, quantized and then branched, one is sent to the multiplexing / demultiplexing circuit 105, the other is dequantized and inversely converted, and the second frame is added to the first frame image. It is stored in the image memory 118 having a variable delay function for motion compensation as an eye encoded / decoded image. In the present invention, the accumulated image that has been encoded and decoded is used and displayed on the monitor 106 of the self-display unit. As a display control unit for this purpose, each unit is controlled by a CPU (not shown) or the like. [Embodiment 1] FIG. 2 shows a first embodiment of the present invention based on the above principle.
3 is a block diagram showing the configuration of the embodiment of FIG. In the figure, the same components as those used in the above figures are designated by the same reference numerals, and the description thereof will be omitted. In FIG. 2, 111 is an INTRA / INTER image changeover switch, 112
Is an INTRA / stored image switch, 113 is a converter, 114 is a quantizer, 115 is an inverse quantizer, 116
Is an inverse converter, 117 is a motion compensation circuit, 118-a, 11
8-b is an image memory having a variable delay function for motion compensation in the self-picture encoding circuit 104, and 119 is a frame memory. In this embodiment, there are two image memories 118-a and 118-b each having a variable delay function for motion compensation for accumulating a decoded image created in the self-picture encoding circuit 104.
One is provided as a differential image creation memory for encoding, and the other is an access dedicated memory accessed by the memory of the PinP circuit 108.

The INTRA image is an image frame 1
The image (all of one image) is referred to, and the INTER image is the difference image between the first and second image frames.

When image transmission is performed, it is not possible to transmit all image frames due to the information transmission capability on the line side. Therefore, the first one sends the full screen (INTRA) and 2
After the first sheet, the difference (difference) from the first sheet is checked, and only the different portion is transmitted to reduce the amount of information.
The difference image is the INTER image.

Next, the operation will be described. INTRA
The image of the first frame is input to the INTER image changeover switch 111, converted by the converter 113, and the quantizer 1
Quantization is performed at 14. This signal is branched and one is multiplexed
It is sent to the separation circuit 105 and sent to the communication partner through the line. The other quantizer is inversely quantized by the inverse quantizer 115, the motion compensation information of the motion compensation circuit 117 is used, and the inverse transformer 11
Inverse conversion is performed at 6, and the signal is divided into two and stored in two image memories 118-a and 118-b having a variable delay function for motion compensation. One of them is for a comparison image to create a next difference image. When the next frame is input, the stored image is taken out from the image memory 118-a having the variable delay function for motion compensation, compared, and the difference image is processed in the same manner as the first frame. The other one is for reading the memory of the PinP circuit 108, and is read at a rate of about once every 1/30 seconds and written in the memory of the PinP circuit 108. At this time, since the received decoded image is also written in the memory of the PinP circuit 108 in the same manner, there is a possibility of memory conflict, but in this case, writing of the received image is prioritized. The image written in the memory of the PinP circuit 108 is format-converted by the inverse format conversion circuit 109, D / A converted by the D / A conversion circuit 107, and then the monitor 10 is displayed.
6 is displayed. [Embodiment 2] FIG. 3 is a block diagram showing the configuration of a second embodiment of the present invention. In the figure, the same components as those used in the above figures are designated by the same reference numerals, and the description thereof will be omitted. Reference numeral 118-c is an image memory having a variable delay function for motion compensation, and 120 is a control device for controlling access to the image memory 118-a having a variable delay function for motion compensation.

Next, the operation will be described. INTRA
The image of the first frame is input to the INTER image changeover switch 111, which is converted by the converter 113 and quantized by the quantizer 114. This signal is branched, one of them is sent to the demultiplexing / demultiplexing circuit 105, the other is dequantized by the dequantizer 115, the motion compensation information of the motion compensating circuit 117 is used, and the inverse transformer 116 reverses. A converted image is created by performing conversion, and the encoded / decoded image is stored in the image memory 118-a having a variable delay function for motion compensation. INTRA / INTER image selector switch 11
When the image of the next 2nd frame is input to 1, the stored image of the 1st frame stored in the image memory 118-a having the variable delay function for motion compensation is read out.
A difference image from the image of the frame is created, the difference image is similarly converted by the converter 113, quantized by the quantizer 114, and inversely quantized by the dequantizer 115, and the motion compensation circuit 11
The image memory 118 having the variable delay function for motion compensation is performed by the inverse converter 116 using the motion compensation information of No.
The image is stored in the image memory 118-a having a variable delay function for motion compensation for the next difference image creation. Thereafter, the encoding is repeated similarly.

In parallel with encoding, the image stored in the image memory 118-a having a variable delay function for motion compensation is read out, superposed by the PinP circuit 108 on the decoded image of the received image, and by the inverse format conversion circuit 109. The format is converted, the D / A conversion circuit 107 performs D / A conversion, and the result is displayed on the monitor 106.

The control device 120 controls access to the image memory 118-a having a variable delay function for motion compensation. The control method provides superiority or inferiority of access right to the image memory 118-a having the variable delay function for motion compensation, and when a high access right issues an access request, a low access right waits. Regarding the specific order of access rights, <1> read access for creating a difference image is given the highest priority, and <2>
Decryption image write access is second, <3> Pi
The read access for writing to the memory of the nP circuit 108 is the third.

FIG. 4 shows a flowchart of the access control method in the second embodiment of FIG. In this figure (S1)
(S6) indicates each step. The reading for creating the difference image and the writing of the decoded image occur randomly depending on the information amount of the image. About 1/30 second if early
Reading or writing is performed at a rate of once, or once every few seconds if it is slow (S1). Therefore, as the access control method, the image memory 1 having a variable delay function for motion compensation for writing to the memory of the PinP circuit 108 in <3>
18-a for reading image information and PinP circuit 108
Is written once every 1/30 seconds (S
2), (S3). When the read access for creating the differential image of <1> and the write access of the decoded image of <2> occur, the operation of <3> is stopped (S4), and <1>
As soon as the work of <2> is completed, the operation of <3> is restarted.
When the memory of the PinP circuit 108 is not updated because the memory of the image memory 118-a having the variable delay function for motion compensation is occupied by the work of <1> and <2> (S5) and (S6), , The previous image stored in the memory of the PinP circuit 108 is retransmitted. As for encoded and decoded images, the time period during which access is not possible is in units of several milliseconds, so even if there is a wait for access, there is almost no effect on the image seen by the human eye.

The partner image decoding circuit 110 decodes the received image, writes it in the frame memory 119, and writes the contents of the frame memory 119 in the memory of the PinP circuit 108 at a rate of about 1/30 second. PinP circuit 10
The inverse format conversion circuit 10 for the images added in 8
Inverse format conversion at 9 and D at D / A conversion circuit 107
After A / A conversion, it is displayed on the monitor 106.

In the present embodiment, the image written in the image memory 118-a having the variable delay function for motion compensation is shared as the encoded image and the encoded self-image, and the first example is adopted.
The number of memories can be reduced as compared with. However, an image slightly different from the image received by the communication partner will be received.

[0025]

As described above in detail, the present invention uses the self-picture signal after coding and decoding stored in the picture memory having the variable delay function for motion compensation in the self-picture coding circuit during communication. Since it has a means for displaying the encoded and decoded self-image on the display unit of the self-terminal at the same time as the received image, the image transmission side undergoes conversion processing and quantization processing on the self-image encoding circuit, and then the inverse quantum. The image encoding received by the communication partner is extracted by extracting the image that has undergone encoding and inverse conversion processing.
It is possible for the transmitting side to extract the same image as the image after the decoding process as the self image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a principle configuration of the present invention.

FIG. 2 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 4 is a diagram showing an access control flowchart of the embodiment of FIG.

FIG. 5 is a block diagram of a first example showing a configuration of a conventional videophone device.

FIG. 6 is a block diagram of a second example showing the configuration of a conventional videophone device.

FIG. 7 is a diagram for explaining a difference between an original image, a cut-back image, and an encoding / decoding processed image.

FIG. 8 is a block diagram showing a configuration example of a videophone having two conventional decoding circuits.

[Explanation of symbols]

 101 camera 102 A / D conversion circuit 103 format conversion circuit 104 self-picture coding circuit 105 multiplexing / separation circuit 106 monitor 107 D / A conversion circuit 108 PinP circuit 109 reverse format conversion circuit 110 partner image decoding circuit 111 INTRA / INTER image switching Switch 112 INTRA / stored image switching switch 113 Converter 114 Quantizer 115 Inverse quantizer 116 Inverse converter 117 Motion compensation circuit 118 Image memory with variable delay function for motion compensation 119 Frame memory 120 Control device

Claims (1)

[Claims] 1. An image memory having a variable delay function for motion compensation, which stores a self-picture signal after encoding and decoding, comprising: a self-picture signal input using a self-picture coding circuit is coded and then sent to a line. , And a videophone device capable of decoding a signal received from a line into an image signal by a partner image decoding circuit and displaying the image signal on its own display unit, in an image memory having a variable delay function for motion compensation in the own image encoding circuit. A television characterized by having display control means for displaying the encoded and decoded self-portrait on the display unit of the self-terminal at the same time as the received image during communication using the stored self-portrait signal after encoding and decoding. Telephone device.