JP3197893B2 - Wireless image communication terminal - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a wireless image communication terminal device for use in videophone, videoconference, and the like, and particularly for performing image restoration using intra coding.

[0002]

2. Description of the Related Art A conventional digital moving picture signal encoding method is described in ITU-T, which was recommended in March 1993.
Recommendation H. 261. H. H.261 is an encoding system including three element technologies described below.

The first is a motion compensation prediction method. The input picture and the pre-coded picture are compared, and the amount of motion between them is measured (motion detection). An input picture is predicted from the motion amount and the pre-coded picture. By calculating the difference (prediction error signal) between the predicted image (prediction image) and the input image picture, and transmitting the prediction error signal and the above-described motion amount to the receiving side, image information can be transmitted with a small data amount. I do.

[0004] The second is a discrete cosine transform. The aforementioned prediction error signal is transformed into the frequency domain. When a prediction error signal is converted into a frequency domain, power is concentrated in a specific frequency domain (low frequency domain). By utilizing this feature and combining with the third method described below, image information can be transmitted with a smaller data amount.

The third is a variable length coding system. This uses the fact that the power is biased to a specific frequency,
By expressing coefficients of frequencies with high appearance frequency with short code length and coefficients of frequencies with low appearance frequency with long code length,
This is a method for shortening the average code length.

By using this, image information can be transmitted with a small amount of data.

The above three element technologies are not applied to the entire image picture, but are applied to each macroblock obtained by dividing the picture into blocks (macroblocks) of 16 * 16 pixels.

Further, there are the following two elemental techniques as techniques for suppressing image quality deterioration due to transmission errors.

The first is cyclic refresh intra coding. In this method, as shown in FIG. 8, intra coding without using the above-described predictive coding is performed on a plurality of macro blocks in one picture. Intra coding is not a method of coding a prediction error using a previous picture, but a method of performing discrete cosine transform of the pixel value of the current macroblock as it is and performing variable length coding. Therefore, even if a transmission error occurs in the previous picture and image quality deteriorates, the transmission of the transmission error can be prevented by transmitting the intra-coded macroblock. Cyclic refresh intra coding is a technique for preventing image quality deterioration by periodically performing intra coding of a plurality of macro blocks among macro blocks included in one picture.

The second is the group of blocks (GO)
B). This is a method of transmitting encoded data of a plurality of macroblocks as one synchronization unit.
H. 261 will be described as an example. QCIF (Quarter Common Input Format) is used as the format of the input image. In QCIF, a luminance signal is 176 × 144 pixels and a color difference signal is 88 × 72 pixels, and is divided into 11 horizontal × 9 vertical macro blocks.

[0011] The GOB is composed of 11 horizontally continuous macroblocks as one synchronization unit, with a synchronization word and a header added to the beginning (see FIG. 9). By configuring a plurality of macroblocks as one synchronization unit in this way, even when a transmission error occurs, the effect of the transmission error can be suppressed to this GOB, and other GOBs can be used.
Will not spread.

[0012]

SUMMARY OF THE INVENTION Conventional cyclic
In the refresh method, the number of macroblocks encoded in one picture is M, and the number of macroblocks included in one picture is N. It takes N / M pictures until one entire picture is refreshed. Therefore, once the influence of a transmission error appears on the screen, it takes a long time to be refreshed, and there is a problem that a portion where the image quality is deteriorated is displayed on the screen for a long time.

Further, since the prediction coding method is used,
In a moving part, the part where the image quality has deteriorated spatially spreads over time.

In order to recover a deteriorated image portion in a short time, there is a method in which it is determined whether or not a macroblock is included in an important area in a screen, and only the macroblock determined to be important is intra-coded. In such a method, consider a case where an important area moves with time. In FIG. 10, the upper part is a coded picture, and the lower part is a picture decoded on the receiving side. The shaded portions are determined to be important regions and are intra-coded portions. In the first picture shown in the lower part of FIG. 1, a black portion of the intra-coded macroblock is
Indicates that the image quality has deteriorated due to a transmission error. When the important area in the second picture moves to the lower right, the part having deteriorated image quality remains without being refreshed. This portion has a problem that it remains until it is determined as an important region again.

[0015] It is an object of the present invention to provide a wireless image communication terminal apparatus which recovers the image quality deterioration of an important area in a short time and does not retain the image quality deterioration of a part which was an important area in the past. .

[0016]

According to the present invention, there is provided a wireless image communication terminal device, comprising: a raster / macroblock converting means for dividing an input digital image signal from an image input into macroblocks; An inter / intra control means for determining whether or not the current macroblock is to be intra-coded; and a macroblock determined to be intra-coded in units of the divided macroblocks, which is intra-coded together with the audio signal. In a wireless image communication terminal device that transmits and receives a wireless signal, separates the image signal into a video signal and an audio signal, and decodes and displays the separated image signal, the current one of the macro blocks included in one picture Important area determination means for determining whether a macroblock is included in a moving area; An important area memory capable of storing, for each block, the number of times that a macroblock included in a moving area is intra-coded, and storing the number of times more than two; When the intra-control means determines that intra-coding is performed, the number of times of intra-coding of the important area memory can be reduced.

[0017]

Embodiment 1 Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG.

FIG. 1 shows a configuration of an embodiment of the present invention. First, each component will be described. In FIG. 1, 10
Reference numeral 1 denotes a raster / macroblock conversion, reference numeral 102 denotes a motion detection unit, reference numeral 103 denotes a frame memory, which stores a reproduced image of a previous picture. 104 is a memory control unit, 105 is an important area determination unit, 106 is an important area memory, which stores the number of times each macroblock should be intra-coded, and 0 indicates a non-important area. 107 is an INTER / INTRA control unit,
108 is a subtractor, 109 is a discrete cosine transform unit (DCT), 110 is a quantization unit, 111 is a variable length coding unit for DCT coefficients, 112 is inverse quantization, and 113 is This is an inverse discrete cosine transform, 114 is an adder, 115 is a variable length coding unit for motion vectors, and 116 is a multiplexing unit for DCT coefficients and motion vectors.

Reference numeral 120 denotes macroblock data, and 1
21 is a difference sum between the macroblock in the same position as the current macroblock and the previous picture, 122 denotes a motion vector, 123 is a value indicating the number of times of whether critical areas are intra-coded. Next, the operation of each unit will be described in detail.

The input image is divided into 16 * 16 pixel size blocks (luminance signals) and 8 * 8 pixel size blocks (color difference signals) in the raster macroblock conversion unit 101 and output. Next, in the motion detection unit 102, the macro block data 120 and the frame memory 1
The motion vector 122 of the current macroblock is output by comparing the current macroblock with the previous picture in the block 03. Further, a sum 121 of difference values between the current macroblock data 120 and the macroblock at the same position in the previous picture is calculated.

Next, an important area determination result 123 indicating whether the current macroblock is an important area and the number of times of intra-coding is read from the important area memory 106.
Here, if the current macroblock is not included in the important area, the INTER / INTRA determination unit 107 switches the switch to the frame memory side, and inputs the predicted image corrected by the motion vector to the subtractor. If the area is the important area, the switch is switched to the “0” side, a pixel value of zero is input to the subtractor 108, and the number of times the current macroblock in the important area memory 106 is to be intra-coded is reduced by one. If this value becomes 0, this macroblock is regarded as an insignificant area, and if it is other than 0, it is regarded as an important area and intra-coded the number of times indicated by the value.

In the important area determination unit 105, the difference value 121
Is compared with a predetermined threshold value. If the value is larger than this value, the current macroblock is determined to be an important area. If it is determined that the area is the important area, the contents of the current macro block in the important area memory 106 are updated. Assuming that the number of times a macroblock included in the important area should be intra-coded is three, the content of the current macroblock in the important area memory 106 is three.

In the subtracter 108, a difference between the current macro block 120 and the predicted image (zero when it is determined that the intra coding is performed) is calculated. Discrete cosine transform unit 10
9 and is quantized by the quantization unit 110. The quantized coefficients are encoded by the variable-length encoding unit 111 into a variable-length code. Further, the quantized coefficient is inversely quantized by the inverse quantization unit 112 to be a DCT coefficient, and is converted to a difference value by the inverse discrete cosine transform unit 113. The adder 114 reproduces the image of the current picture by adding the difference value to a predicted image (zero when it is determined to be intra-coded), and stores the reproduced image in the frame memory 103. The motion vector 122 is subjected to variable-length coding in the variable-length coding unit 115, and together with the variable-length code of the DCT coefficient,
6, multiplexed and output.

The updating of the important area memory 106 will be described in more detail with reference to FIG. In FIG. 2, the upper diagram shows the contents of the important area memory in each picture. The middle diagram corresponds to each picture in the upper stage, and shows an area determined to be important. Similarly, the lower diagram also corresponds to each picture in the upper stage, and shows the contents of the important area memory updated after the important area determination.

The contents of the important area memory indicate, for each macroblock, whether or not the macroblock is included in the important area. It is indicated by a numerical value, where 0 indicates an insignificant area and anything other than 0 indicates an important area. A non-zero value indicates how many times the macroblock should be intra-coded. As described above, the flag indicating whether the area is important / insignificant and the number of times intra-coding is to be performed are represented by one memory.

A description will be given step by step with reference to FIG. The number of macroblocks intra-coded into one picture is 2
Assume that

Since the contents of the important area memory have not been updated yet (b), there is no macroblock to be intra-coded. In the first picture, it is assumed that the important area is determined as shown in FIG. After that, the important area of the first picture is reflected in the important area memory (c). Assuming that a macroblock included in an important region is intra-coded three times, a value of 3 is written in that region.

In the second picture, two macroblocks are intra-coded (shaded macroblocks) according to the content (d) of the important area memory, and the value of the macroblock in the important area memory is reduced by one. The value in the memory corresponding to the intra-coded macroblock is 2, this macroblock is stored as an important area,
After that, it is intra-coded twice.

Next, it is assumed that the important area in the second picture is determined as shown in FIG. The important area of the second picture is reflected in the important area memory (f). This is continued in the third, fourth, fifth,...

As described above, the contents of the important area memory are updated for each picture, and macro blocks in the area are sequentially intra-coded.

In the above description, the important region is determined using the sum of the difference values between the current macroblock and the macroblock at the same position in the previous picture. However, the present invention limits the method for determining the important region. Not something.

In addition, the order of intra-coding macroblocks in the important region is set to be continuous in the horizontal direction. However, the present invention does not limit the order of intra-coding.

It can be easily analogized that the number of intra-coded pictures and the number of intra-coded macro blocks included in the important area can be arbitrarily set.

Further, the present invention can be realized by software, and the present invention can be realized by reading from a recording medium storing the software. In addition, a decoder that can decode data encoded according to the present invention can provide an easy-to-view image in which recovery of image quality in an important area is quick even when image quality is deteriorated due to an error that occurs during transmission. is there.

Further, if the codec having both the encoder and the decoder according to the present invention is used on both the transmitting side and the receiving side, real-time bidirectional image communication can be performed.

Further, the present invention is effective in wireless transmission in which an error is likely to occur during transmission, and is particularly effective in a wireless image communication terminal device as shown in FIG. 12, reference numeral 1201 denotes a receiving unit; 1202, a camera unit; 1203, a monitor unit; 1204, a speaker;
Reference numeral 205 denotes a control unit, and 1206 denotes a mouthpiece. Camera unit 1
The image captured from 202 is encoded by the internal image encoding device. The voice of the sender is captured from the mouthpiece 1206, encoded, multiplexed with the video signal, and transmitted.
In addition, a receiving unit 1201 receives a radio signal from a game,
Video and audio signals are separated. The video signal is decoded by the internal image decoding device and displayed on the monitor unit 1203. The audio signal from the game is similarly decoded and output from the speaker 1204.

Although the present embodiment is a wireless portable terminal, it can also be realized in a form in which a wireless transmitting / receiving device is connected to a computer.

Embodiment 2 Hereinafter, an embodiment of the present invention will be described with reference to FIGS. FIG. 3 shows the configuration of the embodiment of the present invention. First, each component will be described. 3, the same components as those shown in FIG. 1 are denoted by the same reference numerals and description thereof will be omitted.

In FIG. 3, reference numeral 301 denotes INTER / IN
The TRA control unit 302 is a synchronization word generation unit. 310
Is the macroblock number of the current macroblock, and 3
11 is the number of macroblocks to be intra-coded into one picture (this is M), and 312 is the number of macroblocks contained in one picture (this is N).

Next, the operation of each section will be described in detail.

The raster / macroblock converter 101 divides the input image into macroblocks and outputs the current macroblock number 310. The macroblock number is 1 in the upper left macroblock and 2 in the horizontal direction.
・ It increases to 3 ・ 4 ... After reaching the right end, counting is continued from the left end one step below (FIG. 4).

The INTER / INTRA determination section 301 determines whether or not the current macroblock is to be INTRA-encoded, based on the macroblock number 310 and the values of N and M. Suppose M is 5 and N is 99.

The five intra-coded macroblock numbers (M1, M2, M3, M4, M5) are determined.
First, the initial values of M1 to M5 are determined. Since GOB uses eleven horizontally continuous macro blocks as one synchronization unit, five intra-coded macro blocks are determined in a different order.

In order that macroblocks to be intra-coded are included in different synchronization units, macroblocks at positions spaced as long as possible in the horizontal direction are intra-coded. That is, the initial values of the numbers to be intra-coded are as follows.

M1 = 1 M2 = N / M * 1 = 99/5 * 1 = 19 M3 = N / M * 2 = 99/5 * 2 = 39 M4 = N / M * 3 = 99/5 * 3 = 59 M5 = N / M * 4 = 99/5 * 4 = 79 This value is set as an initial value, incremented by 1 for each picture, and set to 1 when the value of N is exceeded. The order of macroblocks to be intra-coded is as shown in FIG.

When the input macroblock number 310 is M
If it is equal to any one of 1 to M5, intra coding is performed, and the switch is switched to the “0” side.
If they are not equal to each other, normal INTER coding is performed, and the switch is switched to the predicted image output side.

The synchronizing word generator 302 outputs the synchronizing word and header information about the corresponding synchronizing unit for every 11 macroblocks, and the multiplexing unit 116 multiplexes the information together with DCT coefficients and motion vector information which have been subjected to variable length coding. Output.

In this method, the macroblocks to be intra-coded in the order shown in Expression 1 are determined. However, even if the macroblocks are determined by the method shown in FIG. 7 using the order shown in FIG. Macroblocks to be intra-coded can be arranged in different synchronization units.

The present invention can also be realized by software, and the present invention can be realized by reading from a recording medium containing the software. In addition, a decoder that can decode data encoded according to the present invention can reduce the number of missing intra-coded macroblocks even when image quality is deteriorated due to an error that occurs during transmission, It is possible to provide an easy-to-view image.

Further, if the codec having both the encoder and the decoder according to the present invention is used on both the transmitting side and the receiving side, real-time two-way image communication can be performed.

The present invention is effective in wireless transmission in which an error is likely to occur during transmission, and is particularly effective in a wireless image communication terminal device as shown in FIG. Further, the present embodiment is a wireless portable terminal, but it can also be realized in a form in which a wireless transmitting / receiving device is connected to a computer.

[0052]

As described above, according to the present invention, it is possible to recover image quality deterioration of an important portion in a picture in a short time, and to maintain image quality deterioration of a portion which was an important region in the past. Is also gone. Therefore, even if image transmission is performed using a transmission path in which a transmission error occurs, it is possible to provide an image that is easy to see.

According to the present invention, intra-coded macroblocks are transmitted in different synchronization units. Therefore, even if the transmitted synchronization unit is lost due to a transmission error, not all intra macroblocks are lost, so that a portion where the image quality is deteriorated is quickly recovered, and an easy-to-view image can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram showing the configuration of an embodiment of the first invention.

FIG. 2 is a diagram showing a state in which an important area memory is updated.

FIG. 3 is a diagram showing a configuration of an embodiment of the second invention.

FIG. 4 is a diagram showing the order of macroblock numbers.

FIG. 5 is a diagram showing an example of an arrangement of macro blocks to be intra-coded according to the present invention;

FIG. 6 shows the order of macroblock numbers for determining a macroblock to be intra-coded.

FIG. 7 is a diagram showing another example of the arrangement of macro blocks to be intra-coded in the present invention.

FIG. 8 is a layout diagram of an intra macro block in a conventional cyclic refresh method.

FIG. 9 is a configuration diagram of a GOB.

FIG. 10 is a diagram showing a problem when a conventional important region is intra-coded.

FIG. 11 is a diagram showing positions of intra-coded macroblocks included in GOB.

FIG. 12 illustrates an example of a wireless image communication terminal.

[Explanation of symbols]

Reference Signs List 101 raster / macroblock conversion unit 102 motion detection unit 103 frame memory 104 memory control unit 105 important area determination unit 106 important area memory 107 INTER / INTRA determination unit 108 subtractor 109 discrete cosine transform unit 110 quantization unit 111 variable length coding Unit 112 inverse quantization unit 113 inverse discrete cosine transform unit 114 adder 115 variable length coding unit 116 multiplexing unit 120 macroblock data 121 difference value sum 122 motion vector 123 important area determination result 301 INTER / INTRA determination unit 302 synchronization word Generation unit 310 Macroblock number 311 Number of macroblocks to be intra-coded in one picture 312 Number of macroblocks in one picture 1201 Receiving unit 1202 Camera unit 1203 Monitor unit 1204 Over mosquitoes 1205 control unit 1206 mouthpiece

Claims (1)

(57) [Claims] 1. A raster / macroblock converter for dividing an input digital image signal from an image input into macroblocks, and determining whether or not a current macroblock among the macroblocks included in one picture is intra-coded. Inter / intra control means for performing the above-mentioned divided macroblocks as a unit, and macroblocks determined to be intra-coded intra-encoded and transmitted together with the audio signal; In a wireless video communication terminal device which separates a video signal into an audio signal and decodes and displays the separated video signal, it is determined whether a current macroblock is included in a moving area among macroblocks included in one picture. Important area determination means for determining, and macroblocks included in the moving area An important area memory capable of storing the number of times of encoding for each block and storing the number of times more than two times, and it is determined that the intra / intra control means performs intra encoding based on the contents of the important area memory. Wherein the number of times of intra-coding of the important area memory is reduced in the event of the occurrence.