JPH06165148A - Dynamic picture communicate equipment - Google Patents

Abstract

PURPOSE:To provide a dynamic picture communicating equipment capable of executing dynamic picture communication having picture quality and motion appropriate for a coding control condition corresponding to a called terminal. CONSTITUTION:When an operator asks the size of a display screen in the called terminal and inputs the known size of the display screen in the called terminal from an external switch 13, a called terminal image size input part 14 supplies the display screen sizes Sj to a quantizing step value setting part 15. Thereby a quantizing step value Sf meet for the display screen size of the called terminal is selected and supplied to a quantizing part 6. The quantizing part 6 executes quantization based upon the selected step value Sf. Consequently coded data Sh having alphapicture qualitybeta and alphamotionbeta meet for the display screen size of the called terminal are transmitted to the called terminal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture communication apparatus for carrying out real time communication of moving pictures through a communication line.

[0002]

2. Description of the Related Art A moving image communication apparatus used for a videophone, a video conference, etc. is known. Generally, in this type of device, the transmission rate of the communication line is lower than the amount of information generated on the transmission side, and therefore it is necessary to compress the image signal by image encoding. As a moving image coding method in this case, the H.264 standard recommended by CCITT (International Telegraph and Telephone Consultative Committee) is recommended. The H.261 system is generally used.

This H.264 In the H.261 system, the input image is converted into a common intermediate format (Common Intermediate Format or Q
uarter CIF: hereinafter abbreviated as CIF / QCIF) It is necessary to perform format conversion into a signal. In addition, H. 261
The methods include an intra-frame coding mode and an inter-frame coding mode. Intra-frame coding is to perform coding by closing within one frame. On the other hand, the inter-frame coding is to predict a frame to be coded based on the previously coded frame and code the prediction difference. In general, interframe coding is more efficient and more frames can be transmitted than intraframe coding. In this inter-frame coding, a moving image with better motion is transmitted.
A prediction screen is created using motion compensation (hereinafter abbreviated as MC), and a difference between frames is obtained.

Here, MC is a method used for more efficiently performing interframe coding, and the block in the frame to be coded is the most similar to the block in the frame coded one frame before. The block is searched for in the same frame as the block in the frame to be coded, and is replaced with the block in the frame coded one frame before which is most similar.

If the inter-frame difference with the predicted image obtained as a result of this replacement is obtained, the value of the inter-frame difference signal becomes small, and the coding efficiency increases accordingly. In addition, the inter-frame difference signal is discriminated by valid / invalid block for each small block, and then the discrete block cosine transform (Discrete CosineTra) which is one of orthogonal transforms is performed on the valid block.
nsform: hereinafter abbreviated as DCT), quantization and variable length coding are performed.

In the determination of the valid / invalid block, the difference signal in the small block is compared with a predetermined threshold value (valid block determination threshold value). If the difference signal is larger than the threshold value, the valid block is determined. This is done by making it an invalid block. However, the magnitude of the difference signal at this time is obtained by the sum of absolute values of the difference values, the sum of squares of the difference values, or the like. As a result, a minute change on the screen is determined as an invalid block and is not transmitted, and the coding efficiency is increased.
That is, if the number of invalid blocks increases, the amount of transmission information per frame decreases, the number of frames that can be transmitted per unit time increases, and a moving image with better motion can be transmitted.

However, if the valid block determination threshold value is too large, even if the change on the screen is obvious, it becomes an invalid block and is not transmitted. Therefore, only a part of the previous frame stays on the decoded image, which causes this. This causes distortion, which significantly deteriorates subjective image quality. From this, the effective block determination threshold value is generally set to a small value, but in this case, conversely, the reduction of the generated code amount becomes insufficient. In addition, since the transmission speed of the communication line is constant, the number of transmission frames per unit time decreases as the generated code amount per screen increases, so a frame sufficient to realize a natural motion is obtained. It is not possible to send the number, and a moving image with awkward motion is sent.

In the quantization, the transform coefficient value after DCT is divided by a predetermined integer value (quantization step value),
The generated code amount can be significantly reduced by setting the transform coefficient value as a discrete value for the quantization step, but on the other hand, when the quantization step value is large and the quantization is rough, the error from the original image signal becomes too large. Therefore, distortion occurs in the decoded image, and the subjective image quality of the decoded image deteriorates. Therefore, generally, the quantization step value is set to a small value such that no distortion occurs in the decoded image. However, in this case as well, the amount of generated code is not sufficiently reduced, so that it is not possible to transmit a sufficient number of frames to realize a natural motion, and only a moving image with awkward motion can be transmitted.

Further, even when the effective block determination threshold value or the quantization step value is large and the decoded image is distorted, the communication partner terminal (hereinafter referred to as the partner terminal) is externally connected. The distortion may not be noticeable depending on the appropriate encoding control condition. For example, even if an image is distorted to the same extent by the same quantization step value, it is very conspicuous on a large screen with a display screen of 20 inches or more.
It may not be noticeable on a small screen of about inch.
In such a case, if the screen of the other terminal is a large screen, the quantization step value should be made small to improve the image quality even if the number of transmission frames is reduced, but if it is a small screen, it should be left as it is. The number of quantization steps is sufficient, and it is not necessary to reduce the number of transmission frames.

[0010]

However, at the start of communication, it is not possible to know how large the display screen of the other terminal is, so for safety reasons, distortion is visible even on a large screen. There is no choice but to use a quantization step value that is not significant, that is, a small quantization step value. For this reason, the amount of generated codes per screen increases, the number of transmission frames decreases, and the motion becomes poor. Further, if the partner terminal has a small screen in such a communication state, the quantization step value is increased to reduce the amount of generated information, and the number of transmission frames is increased accordingly.
Although it is possible to implement moving image communication with smoother movement, such processing cannot be performed because the display screen size of the partner terminal is unknown. In the end, the conventional moving image communication device has a problem that only moving image communication in which motion does not always occur can be performed.

The present invention has been made under such a background, and provides a moving image communication apparatus capable of performing moving image communication of image quality and motion suitable for the coding control condition corresponding to the partner terminal. The purpose is to do.

[0012]

In order to solve the above-mentioned problems, the invention according to claim 1 divides a digital image signal into small blocks composed of a plurality of pixels, and performs an encoding process for each small block. Means for determining whether or not to perform using an effective block determination threshold, encoding means for orthogonally transforming, quantizing, and variable-length encoding the digital image signal of the small block, and a quantization step value at the time of the quantization. And a transmission means for multiplexing the coded data coded by the coding means and sending the coded data to the communication line, while the multiplexed coded data supplied via the communication line. Receiving means for receiving and separating the encoded data separated by the receiving means and variable length decoding,
A code corresponding to a partner terminal in a moving image communication apparatus having a decoding means for performing inverse quantization and an inverse orthogonal transform, and a display means for displaying an image based on the decoded digital image signal decoded by the decoding means. It is characterized by comprising means for inputting a coding control condition to the terminal itself and means for changing the valid block determination threshold value or the quantization step value according to the inputted coding control condition.

According to a second aspect of the present invention, the digital image signal is divided into small blocks composed of a plurality of pixels, and it is determined for each of the small blocks whether or not encoding processing is to be performed, using a valid block determination threshold value. Means, an encoding means for orthogonally transforming, quantizing, and variable-length encoding the small block digital image signal, a means for changing the quantization step value at the time of the quantization, and an encoding by the encoding means. Receiving means for receiving the multiplexed coded data supplied through the communication line and separating the multiplexed coded data, and transmitting the multiplexed coded data to the communication line. Decoding means for performing variable length decoding, dequantization and inverse orthogonal transformation on the encoded data separated by the means, and based on the decoded digital image signal decoded by the decoding means In a moving picture communication device having a display means for displaying an image, means for transmitting and receiving the coding control conditions corresponding to the partner terminal and the own terminal to each other using in-channel unique codes, and the received partner terminal. And means for changing the effective block determination threshold value or the quantization step value in accordance with the encoding control condition corresponding to.

[0014]

According to the first aspect of the present invention, when the operator inputs the coding control condition corresponding to the partner terminal to the own terminal, the valid block determination threshold value or the quantum value is determined according to the input coding control condition. Change the activation step value. As a result, the effective block determination threshold value or the quantization step value at the time of encoding is controlled to a value that matches the encoding control condition corresponding to the partner terminal.

According to the second aspect of the present invention, at the time of communication, the other terminal which has received the other terminal transmits and receives the encoding control conditions corresponding to each of the other terminal and the own terminal by using the unique code of the in-channel. The valid block determination threshold value or the quantization step value is changed according to the corresponding coding control condition. With this, in addition to the operation according to the first aspect of the present invention, the coding control condition can be automatically recognized without the operator having to input the coding control condition corresponding to the partner terminal.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. First, before describing the configuration of the embodiment, a method of recognizing a coding control condition corresponding to a partner terminal and a subjective evaluation experiment result which is the basis of this embodiment will be described. As a method of knowing the coding control condition corresponding to the partner terminal during communication, there is a method of asking the communication partner by voice for the coding control condition corresponding to the partner terminal. When the operator inputs the coding control condition corresponding to the partner terminal, which is found by this, to his / her terminal (hereinafter referred to as own terminal), the effective block determination threshold value or the quantization step on the transmission side according to the input value. You can control the value. This method corresponds to the first embodiment described later.

As a method of automatically recognizing the coding control condition corresponding to the partner terminal during communication, CCITT
Recommendation H. There is a method of using the non-standard protocol defined in 221. FIG. 221 shows a non-standard protocol in 221. As shown in the figure, the format according to this non-standard protocol is non-standard capability, that is, the presence / absence of "the ability to control the coding parameter of the own terminal according to the coding control condition corresponding to the partner terminal", the command, and the message length. , A country code and a provider code, and a message section.

Here, FIG. 4 shows a procedure in which the terminals X and Y match the coding control conditions with each other according to the non-standard protocol. As shown in the figure, after the terminals X and Y notify each other of the communication start, they establish frame synchronization,
Further, mutual exchange of terminal capabilities is started.

If the partner terminal has non-standard capability, that is, "capability to control the coding parameter of the self-terminal in accordance with the coding control condition corresponding to the partner terminal", a non-standard command is used. The other terminals and notify each other of the "coding control conditions of the own terminal". Upon receiving this non-standard command, both terminals immediately control the effective block determination threshold value or the quantization step value of their own terminal to match the coding control condition corresponding to the partner terminal. Note that such a procedure is basically based on CCITT Recommendation H.264. 242. Further, this method corresponds to the second embodiment described later.

Next, the result of the subjective evaluation experiment, which is the basis of this embodiment, will be described. In this experiment, the coding control condition corresponding to the partner terminal is the display screen size of the partner terminal. First, in FIGS. 5 to 8, the same image is encoded by changing the quantization step value, and these are displayed on a small-screen LCD monitor having a different screen size, and the "image quality" is displayed.
FIG. 5 is a diagram showing a result of a subjective evaluation experiment for “movement” and “movement”. 5 shows the result of evaluating the "image quality" for the CIF image, FIG. 6 shows the result of evaluating the "motion" for the CIF image, FIG. 7 shows the result of evaluating the "image quality" for the QCIF image, and FIG. 8 shows the result for the QCIF image. The results of evaluating "movement" are shown.

Note that this experiment was conducted at a visual distance of 40 cm (fixed) assuming a small-screen desk-top television phone, and FIG. 9 shows the experimental environment at this time. That is, the NTSC signal Vd which the VTR 20 reproduces and outputs the image is encoded and composited in the moving image CODEC 21 while changing the quantization step value Qd by the personal computer 22.

The decoded image signal Vd 'output from the moving image CODEC 21 is distributed by the video distributor 23 and supplied to the liquid crystal monitors 24 to 26 of each screen size. In this way, the image displayed on each liquid crystal monitor is evaluated by the evaluator M. Also, the score in this case is an absolute scale of 5
At the stage (5 ... very good, 4 ... good, 3 ... normal, 2 ... bad, 1 ... very bad), there were 10 image experts, and after the experiment, the average score (Mean Opinion Score:
Hereinafter, referred to as MOS value) was determined.

Now, comparing FIG. 5 and FIG. 6 or FIG. 7 and FIG. 8, in both CIF images and QCIF images, the subjective evaluation characteristics of “image quality” differ depending on the screen size, but regarding “movement” It can be seen that there is no difference in the characteristics depending on the screen size. In other words, the difference in screen size does not affect "motion", but it affects "image quality". Therefore, when controlling the quantization step value by screen size, only the effect on "image quality" is affected. It turns out that it is good to consider.

Therefore, the actually usable level is set to MOS.
If the value is "3" (normal), in the case of a CIF image, the quantization step value is about "14" when the screen size is 5.6 inches and the quantization step value when the screen size is 4 inches, as shown in FIG. It can be seen that the quantization step value may be about "24" when the screen size is 3 inches.

On the other hand, in the case of the QCIF image, as shown in FIG. 7, when the screen size is 5.6 inches, the quantization step value is "6".
It can be seen that the quantization step value may be about “8” when the screen size is 4 inches, and the quantization step value may be about “12” when the screen size is 3 inches. Also,
This experiment was conducted on a small screen monitor, but when the display screen size of the other terminal is larger, the quantization step value smaller than the above value (“8” or more in the case of CIF, (4 or more in the case of QCIF)
It is guessed that you can use.

The actual resolution of the small-screen liquid crystal monitor is 5.6 inches and has 240 horizontal pixels × 120 vertical pixels and 4 pixels.
The size of inch is 160 pixels wide by 120 pixels, and the size of inch is 130 pixels wide by 120 pixels vertically. That is,
Since the resolution of the LCD monitor of 4 inches or less is lower than the resolution of the QCIF image (horizontal 176 × vertical 144 pixels), in the case of an LCD monitor with a display screen of 4 inches or less, the CIF image (resolution is 352 horizontal × 288 vertical pixels). ) And the QCIF image, there is no difference in subjective image quality.

For example, if the display screen is 4 inches, then FIG.
From FIG. 8, the quantization step value of the CIF image at the MOS value “3” is “20” and the quantization step value of the QCIF image is “8”.
The OS value is “2.5” in the CIF image and “2.7” in the QCIF image. Further, when the display screen is 3 inches, the quantization step value of the CIF image at the MOS value “3” is “24” and the quantization step value of the QCIF image is “12”. The MOS value is "2.7" in the CIF image, QC
It is “3.2” in the IF image.

As described above, the "movement" of the QCIF image is better regardless of whether the display screen is 4 inches or 3 inches, but there is a difference between the two in terms of image quality as described above. Absent. Therefore, in the case of a liquid crystal monitor whose display screen of the partner terminal is 4 inches or less, it is extremely effective to control so as to forcibly transmit the QCIF image.

Now, based on the above, the configuration of the embodiment will be described below. FIG. 1 shows the first of the present invention.
It is a block diagram which shows the structure of an Example. In this figure, 1 is a camera unit, which outputs a captured image as an analog image signal Sa. Reference numeral 2 is an A / D (analog / digital) converter, which converts the analog image signal Sa supplied from the camera unit 1 into a digital image signal Sb,
Output this. 3 is a format conversion unit, which is A /
The digital image signal Sb supplied from the D conversion unit 2 is converted into C
It is converted into an IF or QCIF (CIF / QCIF) signal Sc and is output.

Reference numeral 4 is an MC unit for motion compensation. This M
The C unit 4 calculates the inter-frame difference between the CIF / QCIF signal Sc and the predicted image created by motion compensation, and outputs the inter-frame difference signal Sd. Further, the MC unit 4 receives the CIF / QCIF input without taking the inter-frame difference.
The signal Sc may be output as it is.

A DCT unit 5 performs DCT on the interframe difference signal Sd supplied from the MC unit 4, and outputs a DCT conversion coefficient Se. A quantizer 6 quantizes the DCT transform coefficient Se supplied from the DCT unit 5 according to a predetermined quantization step value Sf, and outputs a quantized DCT coefficient value Sg. Reference numeral 7 denotes a variable length coding unit, which codes the DCT coefficient value Sg supplied from the quantization unit 6 and outputs coded data Sh. Reference numeral 8 denotes a data multiplexing unit, which is the encoded data Sh supplied from the variable length encoding unit 7.
Are multiplexed and sent to the communication line 9.

Reference numeral 10 denotes an inverse quantizer, which is a quantized D
The CT coefficient value Sg is inversely quantized and output. Reference numeral 11 denotes an inverse DCT unit, which performs inverse DCT on the output of the inverse quantization unit 10 and outputs this. 12 is a local memory,
The locally decoded image Si supplied from the output of the inverse DCT unit 11 is stored as the locally decoded image Si. The local decoded image Si is also used by the MC unit 4 to obtain an inter-frame difference when the next frame is encoded.

An external switch 13 is used by the operator to input the display screen size of the other terminal. Reference numeral 14 denotes a partner terminal image size input unit, which supplies an input value (display screen size Sj) corresponding to the operation of the external switch 13 to the quantization step value setting unit 15. The quantization step value setting unit 15
The quantization step value Sf corresponding to the display screen size Sj of the partner terminal is selected and supplied to the quantization unit 6.

Next, the operation of the embodiment having the above configuration will be described. When the communication line 9 is connected and communication is started, the analog image signal Sa output from the camera unit 1 is A / D.
The conversion unit 2 converts the digital image signal Sb. This digital image signal Sb is further converted into a CIF / QCIF signal Sc in the format conversion section 3.

The CIF / QCIF signal Sc is M
The inter-frame difference with the prediction image created by using the motion compensation is obtained by the C unit 4, and the inter-frame difference signal Sd is obtained as D
It is supplied to the CT unit 5. Further, the inter-frame difference signal Sd is subjected to DCT in the DCT section 5 to become a DCT transform coefficient Se, and then quantized in the quantizing section 6. The quantization step value Sf at this time is an initial setting value (default value).

Further, the quantized DCT coefficient value Sg
Is converted into coded data Sh in the variable length coding unit 7, multiplexed in the data multiplexing unit 8, and then transmitted to the communication line 9 and transmitted to the partner terminal. On the other hand, the quantized DCT coefficient value Sg is the inverse quantization unit 10 and the inverse D
Is converted into a local decoded image Si via the CT unit 11,
It is stored in the local memory 12.

After that, during communication, when the operator audibly asks for the display screen size of the partner terminal, and inputs the display image size of the partner terminal found by the external switch 13, the partner terminal image size input section 14 displays. The screen size Sj is supplied to the quantization step value setting unit 15. As a result, the quantization step value Sf suitable for the display screen size of the partner terminal is selected and supplied to the quantization unit 6. After that, the quantization unit 6 performs quantization based on the newly set quantization step value Sf. As a result, the encoded data Sh having “image quality” and “motion” suitable for the display image size of the partner terminal is transmitted to the partner terminal.

As described above, according to the present embodiment, during communication, the operator inputs the display screen size of the partner terminal informed by voice, so that the quantization suitable for the display screen size of the partner terminal is performed. The step value is set. As a result, moving image communication with “image quality” and “motion” suitable for the display screen size of the partner terminal is realized.

Next, a second embodiment of the present invention will be described. FIG. 2 is a block diagram showing the configuration of the embodiment.
In this figure, parts common to the parts shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted. Further, the difference of this embodiment from the first embodiment described above is that the recognition of the display image size of the partner terminal is performed according to CCITT Recommendation H.264. 242 /
H. 221, which is automatically performed by the communication control unit 16 conforming to 221.

The operation of this embodiment will be described below. First, when the communication line 9 is connected and communication is started, the communication control unit 16 causes CCITT recommendation H.264. Based on 242, the capability check of the own terminal and the partner terminal is performed. At this time, non-standard capabilities are also checked. As a result, if both terminals have non-standard capabilities, that is, "capability to control the coding parameters of their own terminals according to the coding control conditions corresponding to the partner terminal", set to the voice / image communication mode. Then, the display screen size of the own terminal is transmitted by a non-standard command indicating “encoding control condition of the own terminal”.

Upon receiving the display screen size of the self terminal by the non-standard command, the partner terminal immediately sets the quantization step value suitable for the display screen size of the self terminal. on the other hand,
As soon as the own terminal receives the display screen size Sj of the other terminal by a non-standard command from the other terminal, the display screen size Sj of the other terminal is immediately changed to the other terminal image size input unit 14
Is supplied to the quantization step value setting unit 15 via the, and the quantization step value Sf of the own terminal is set to a value suitable for the display screen size Sj of the other terminal.

As a result, the quantizer 6 quantizes the DCT transform coefficient Se based on the quantization step value Sf suitable for the display screen size Sj of the partner terminal. Other operations are the same as those in the first embodiment described above. In this way, also in this embodiment, the encoded data having the "image quality" and the "motion" suitable for the display image size of the partner terminal is transmitted to the partner terminal.

As described above, in the case of this embodiment, in addition to the effects of the first embodiment, the quantization step value Sf at the start of communication is obtained.
Is automatically set to a value suitable for the display screen size Sj of the partner terminal.
There is no need to enter j. Also, needless to say
The encoded data transmitted from the partner terminal is also data having “picture quality” and “motion” suitable for the display screen size of the own terminal.

[0044]

As described above, according to the present invention, the moving image communication of the image quality and the movement suitable for the coding control condition corresponding to the partner terminal is realized, and particularly, the display screen of the partner terminal is small. When it is a screen, it is possible to realize moving image communication with smoother movement without degrading image quality.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of the present invention.

FIG. 2 is a block diagram showing the configuration of a second embodiment of the present invention.

FIG. 3 CCITT Recommendation H.264. 22 is a diagram showing a non-standard protocol in 221. FIG.

FIG. 4 CCITT Recommendation H. FIG. 242 is a diagram showing a non-standard capacity exchange by 242.

FIG. 5 is a graph showing a subjective evaluation experiment result (in the case of a CIF image) of the influence of the quantization step value on the image quality when the screen size is different.

FIG. 6 is a graph showing a subjective evaluation experiment result (in the case of a CIF image) of the influence of the quantization step value on the motion when the screen size is different.

FIG. 7 is a graph showing a subjective evaluation experiment result (in the case of QCIF image) of the influence of the quantization step value on the image quality when the screen size is different.

FIG. 8 is a graph showing a subjective evaluation experiment result (in the case of a QCIF image) of the influence of the quantization step value on the motion when the screen size is different.

FIG. 9 is a block diagram showing a subjective evaluation experiment environment.

[Explanation of symbols]

 1 camera section 2 A / D conversion section 3 format conversion section 4 MC section 5 DCT section 6 quantization section 7 variable length coding section 8 data multiplexing section 9 communication line 10 inverse quantization section 11 inverse DCT section 12 local memory 13 External switch 14 Remote terminal image size input unit 15 Quantization step value setting unit 16 Communication control unit

Claims (2)

[Claims] 1. A unit for dividing a digital image signal into small blocks composed of a plurality of pixels, and determining for each of the small blocks whether or not to perform coding processing by using an effective block determination threshold; Coding means for orthogonally transforming, quantizing, and variable-length coding a digital image signal, means for changing a quantization step value at the time of quantization, and coded data coded by the coding means are multiplexed. ,
A transmitting unit for transmitting to the communication line, a receiving unit for receiving the multiplexed encoded data supplied via the communication line and separating the same, and an encoded data separated by the receiving unit. A moving image communication apparatus having a decoding means for performing variable length decoding, inverse quantization and inverse orthogonal transformation, and a display means for displaying an image based on the decoded digital image signal decoded by the decoding means, It is provided with means for inputting a coding control condition corresponding to a terminal to the terminal itself, and means for changing the valid block determination threshold value or the quantization step value according to the input coding control condition. Video communication device. 2. A means for dividing a digital image signal into small blocks made up of a plurality of pixels, and for each small block, a means for judging whether or not to perform an encoding process by using an effective block judgment threshold; Coding means for orthogonally transforming, quantizing, and variable-length coding a digital image signal, means for changing a quantization step value at the time of quantization, and coded data coded by the coding means are multiplexed. ,
A transmitting unit for transmitting to the communication line, a receiving unit for receiving the multiplexed encoded data supplied via the communication line and separating the same, and an encoded data separated by the receiving unit. A moving image communication apparatus having a decoding means for performing variable length decoding, inverse quantization and inverse orthogonal transformation, and a display means for displaying an image based on the decoded digital image signal decoded by the decoding means, A means for transmitting and receiving the coding control conditions corresponding to each of the terminal and the own terminal to each other by using the unique code of the in-channel, and according to the received coding control conditions corresponding to the partner terminal, the effective block determination threshold value or And a means for changing the quantization step value.