JPH0818969A - Image processing device - Google Patents

Abstract

PURPOSE:To prevent the picture quality of a reproducing image signal from being deteriorated by controlling a decoder means corresponding to the correction result of an error correction means for the image signal of a block represented by the motion vector of the block desired to decode by a decoder means. CONSTITUTION:An inputted analog image signal is converted to a digital signal by an A/D conversion circuit 101, and it is divided into the blocks consisting of eight picture elements in the horizontal direction and eight lines in the vertical direction by a blocking circuit 102. A blocked signal is encoded by a compression/encoder circuit 103. and after the information quantity of the signal is compressed, it is outputted to a formatter 104. The formatter 104 converts the image signal to a signal of format suitable for transmission by adding a synchronizing signal for the synchronization of a transmission line, etc. At this time, an error correction/encoder circuit 105 performs error correction encoding by adding parity data on image data and motion vector data. The image signal converted in such a way is outputted to a decoder side via the transmission line 106.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus, and more particularly to an apparatus for obtaining a reproduced image by encoding and transmitting an image signal and decoding it at the receiving side.

[0002]

2. Description of the Related Art In the field of digital transmission of moving images, research on image coding technology has been actively conducted, and image signals using motion compensation predictive coding capable of good image transmission even at a low data rate have been developed. An apparatus for obtaining a reproduced image by decoding is realized.

Image data handled by this type of device is
It is encoded using intra-field encoding that encodes image data of one screen only with data in the same screen, predictive encoding that encodes inter-field and inter-frame difference data between screens that are temporally different. There is.

When the image data and the encoding mode data encoded in this way are transmitted through the transmission line and decoded by the receiving side, the error occurring in the data in the transmission line is corrected and the decoding is performed. ing. However, such an error correction may cause an error that cannot be corrected.
If the error cannot be corrected, the decoding device cannot correctly decode the image data regardless of whether the uncorrectable error is the encoded image data or the encoding mode. Therefore, in such a case, the image is corrected by replacing the image of the portion corresponding to the uncorrectable image signal with the correctly decoded image data to obtain a reproduced image.

[0005]

However, in such a device, when the image data used for the correction has already been interpolated by other image data, it is already different from the original image at that time. . In this type of encoding method, the modified image becomes the reference image for decoding in the subsequent screens, so it often differs from the actual reference image, and the error from the actual image data is decoded. It appears for the image that was displayed, which led to deterioration of the reproduced image.

In view of the above problems, the present invention provides an image processing apparatus capable of obtaining an image with little deterioration in image quality even when an uncorrectable error occurs in encoded data. The purpose is to

[0007]

SUMMARY OF THE INVENTION In order to solve the conventional problems and achieve the above object, the present invention provides an input means for inputting a block-coded image signal and motion vector information for each block. , An error correction means for correcting an error in the image signal, a decoding means for decoding the image signal using the motion vector of each block of the image signal via the error correction means, and a decoding means for decoding the image signal And a control means for controlling the decoding means according to the correction result of the error correction means for the image signal of the block indicated by the motion vector of the block.

[0008]

Since the present invention is configured as described above, it is possible to obtain a reproduced image with little image quality deterioration even when an uncorrectable error occurs on the transmission path.

[0009]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an encoding device and a decoding device as an embodiment of the present invention. First,
The operation on the encoding side in FIG. 1 will be described.

The input analog image signal is converted into a digital image signal by the A / D conversion circuit 101, and further, by the blocking circuit 102, 8 pixels in the horizontal direction and 8 pixels in the vertical direction.
It is divided into blocks of lines. This block-shaped digital image signal is encoded by the compression / encoding circuit 103 as described later, and the information amount thereof is compressed and then output to the formatter 104. Formatter 10
Reference numeral 4 converts the image signal into a signal of a format suitable for transmission by adding a synchronization signal or the like for synchronizing the transmission path. At this time, the error correction coding circuit 105 performs error correction coding by adding parity data to the image data and the motion vector data described later. The image signal converted in this way is output to the decoding device side via the transmission path 106.

Next, the operation of the compression / encoding circuit 103 in this embodiment will be described.

In the present embodiment, the image coding circuit 10
3 and the operation of the image decoding circuit 113 are MPEG (Moving
Encoding and decoding adapted to the Picture coding Expert Group).

FIG. 2 is a block diagram showing the configuration of the compression / encoding circuit 103. As shown in FIG. 4, the compression encoding circuit 103 uses a discrete cosine transform (Discrete Cosine Tran).
sformation (DCT) circuit 202, quantization (Quantizati
on: Q) circuit 203, variable length coding (Variable Length)
Coding (VLC) circuit 204, rate control circuit 206,
The local decoding circuit 207, the motion compensation circuit 211, the motion vector detection circuit 214, the output buffer 205 and the like are roughly configured.

In FIG. 4, the image data input from the blocking circuit 102 in FIG. 1 is transmitted to the DCT circuit 202 via the switch 201.

The switch 201 is an intra-frame image (I frame) obtained in a coding mode in which input image data is exclusively coded within a frame, or forward prediction for coding a difference value from a temporally preceding frame. Bidirectional prediction that encodes a coded image (P frame), a difference value with a temporally preceding frame or a subsequent frame, or a difference value with an interpolated frame from both frames, which has the smallest data amount. It can be switched depending on whether it is an encoded image frame (B frame),
In the case of the I frame, it is connected to the a contact, and in other cases, it is connected to the b contact.

In the case of an I frame, the DCT circuit 202 performs DCT to transform the data in the spatial domain into the data in the frequency domain, and the DCT coefficient obtained by this is quantized in the quantization circuit 203. Then, after variable length coding is performed by the variable length coding circuit 204, the buffer 2
It is stored in 05.

On the other hand, in the case other than the I frame, the switch 201 is connected to the contact b to perform the motion compensation described above. That is, reference numerals 208 and 209 denote an inverse quantization circuit and an inverse DCT circuit which form the local decoder 207, and the data quantized by the quantization circuit 203 is the local decoding circuit 2
At 07, the original image is restored.

Further, 210 is an adder, 215 is a switch that is closed only in cases other than I-frames, 216 is a subtractor, and the locally decoded image data is the motion compensation circuit 2
11 is input to the memory 212. Then, the motion compensation circuit 213 refers to the motion vector detected by the motion vector detection circuit 214, and then the memory control circuit 213 causes the memory 212 to read a predetermined frame (preceding frame, subsequent frame or their interpolated frame) or field. The image data of the corresponding block in is output.

The output of the motion compensation circuit 211 is the subtractor 2
A difference value is obtained by subtraction processing from the input image data at 16, and the difference value is encoded by the DCT circuit 202, the quantization circuit 203, and the variable length encoding circuit 204, and stored in the buffer 205. .

The motion vector detection circuit 214 obtains a motion vector by comparing the frame data to be encoded with predetermined reference frame data. The detection output of the motion detection circuit 214 is a motion compensation circuit. The memory 21 is supplied to the memory control circuit 213 of the memory 211.
2 designates image data to be output. Further, the rate control circuit 206 controls the code amount by switching the quantization step in the quantization circuit 203 based on the occupied amount of the encoded data in the buffer 205.

Next, the operation on the receiving side will be described. The data transmitted through the transmission path 106 is synchronized by the synchronization detection circuit 107 and temporarily stored in the memory 108. And
In the error correction circuit 109 that accesses the memory 108, the error correction process using the above-mentioned parity data is separately performed on the encoded image data and the motion vector data. For uncorrectable data, error information (hereafter error flag) in block units for encoded image data
Is output to the decompression / decoding circuit 110. Also, for the motion vector data, an error flag is output for each motion vector data of each block. From the memory 108, encoded image data and encoding mode information are input to the decompression / decoding circuit 110. The decompression / decoding circuit 110 decodes the coded image data as described later, and
Image data that cannot be decoded is subjected to interpolation processing and output to the line / scan conversion circuit 111. The line / scan conversion circuit 111 converts the image signals decoded by the decompression / decoding circuit 110 in the order of input on the encoding side, and then outputs them to the D / A conversion circuit 112, and the D / A conversion circuit 112.
Converts a digital image signal into an analog image signal and outputs it.

Next, the decompression / decoding circuit 1 according to the present embodiment.
The operation of 10 will be described. FIG. 3 shows the decompression / decoding circuit 1
It is a block diagram which shows the structure of 10.

Hereinafter, description will be made with reference to the flowchart of FIG. The encoded data output from the memory 108 is input to the data separation circuit 301, and is separated into encoded image data and motion vector data. The encoded image data is output to the variable length code decoding circuit 302, and the motion vector data is output to the motion vector memory 307. The motion vector memory 307 stores one frame of motion vector data.

The error flag for the motion vector is input to the interpolation judgment circuit 309, and the interpolation judgment circuit 30
In step S101, it is determined whether or not the error flag for the motion vector is "1". When the error flag is “0”, the input motion vector data from the motion vector memory 307 is directly stored in the memory control circuit 3
11 (step S102), and if "1",
The motion vector of the block at the same position on the screen one frame before stored in the motion vector memory 307 is read and output to the memory control circuit 311 (step S1).
03).

The error flag for the image data output from the error correction circuit 109 is input to the flag memory 308 and the interpolation determination circuit 309. The flag memory 308 can store an error flag of image data of a block corresponding to one frame. The interpolation determination circuit 309 reads out the error flag of the block indicated by the read motion vector from the flag memory 308 based on the motion vector read out from the motion vector memory 307 as described above (step S104). It is determined whether the error flag is "1" (step S105). This means that the block indicated by the motion vector, that is, the reference image data when decoding the image of the P or B frame has already been interpolated. In this embodiment, in such a case, Does not use its image signal as a reference screen.

In step S105, the error flag is "0".
If so, it is further determined in step S106 whether the error flag for the input image data is "1".
When the error flag is “0”, it means that the motion vector is reproduced correctly (error correction is possible) and the image data is also reproduced correctly. Therefore, first, the data separation circuit 310 outputs the data. The variable-length code decoding circuit 302 decodes the variable-length code for the image data thus obtained. Then, the inverse quantization circuit 303 performs inverse quantization with the quantization coefficient according to the encoding, and the inverse DCT circuit 304
After converting the data in the frequency domain into the data in the spatial domain, the data is output to the a terminal of the adder 305 and the switch 306 (step S107).

The switch control circuit 312 also receives mode information (not shown) and a control signal from the interpolation determination circuit 309. The switch control circuit 312 is the interpolation determination circuit 3
When the control signal from 09 outputs the signal indicating the operation up to step S106 described above, it is determined whether the input image data is the I frame image or the P or B frame image based on the input mode information. (Step S10
8). If the input image data is an I-frame image, the switch control circuit 312 connects the switch 306 to the terminal a (step S113), and outputs the image signal decoded as described above to the line / scan conversion circuit 111 and the memory. The data is output to 310 (step S114).

If the input image data is P or B frame image data, the switch control circuit 31 is used.
2 connects the switch 306 to the b terminal. Further, the memory control circuit 311 reads the decoded image signal stored in the memory 310 based on the motion vector output from the interpolation control circuit 309, and outputs it to the adder 305 (step S110). Then, the input image signal and the image signal read from the memory are added by the adder 305 (step S111) and output to the line / scan conversion circuit 111 and the memory 310 (step S112). Here, the memory 310 can store decoded image signals for at least four frames.

When the error flag of the input image data is "1" in step S106, that is, when the image of the block indicated by the motion vector is not interpolated and the input image data cannot be error-corrected. , Interpolation determination circuit 3
Reference numeral 09 designates a control signal indicating that effect to the switch control circuit 312.
Output to. When this signal is input, the switch control circuit 312 receives the signal from the switch 30 regardless of the mode information.
6 is connected to the c terminal (step S115). And
The memory control circuit 311 reads the decoded image signal in the previous frame of the input image data from the memory 310 based on the motion vector output from the interpolation determination circuit 309, and the line scan conversion circuit 111 and the line scan conversion circuit 111 via the c terminal of the switch 306. Output to the memory 310. Thus, when the encoded image data cannot be error-corrected,
The decoded image signal of the reference screen stored in the memory 310 is output as it is.

If the error flag of the image data of the block indicated by the motion vector is "1" in step S105, that is, if the image of the block indicated by the motion vector is an interpolated image signal, it is used in step S104. The changed motion vector is changed so as to indicate a block around the block indicated by this motion vector, and is output to the memory control circuit 311 (step S117). Then, the process proceeds to step S106. The operation after step S106 is as described above.

The operation of this embodiment is shown in FIGS.
Will be explained.

FIG. 5 shows a state of images of two consecutive frames, and it is assumed that the triangular object 401 has moved to the position of 402 in the figure. Now, the input image data is a block 403, and this block 403 is a motion vector 40.
The difference from the block 404 is coded based on 5.

When the data of the block 403 is reproduced correctly, the data of the block 403 is decoded and output to the adder 305. Then, the image signal of the block 404 that has already been decoded and stored in the memory 310 is read based on the motion vector 405, added by the adder 305, and output.

In the case where the block indicated by the motion vector 405, that is, the image signal of the block 404 has already been interpolated, the interpolation judgment circuit 309 selects, for example, the surrounding blocks A to H of the block 405 shown in FIG. The motion vector is output to the memory control circuit 311 so as to be read. Here, block A above block 405
When using, the motion vector 405 ′ is output to the memory control circuit 311.

As described above, according to this embodiment, when the block indicated by the motion vector has already been interpolated, the decoded image signal of the surrounding block which is considered to have a high correlation with the block indicated by the motion vector is used as the reference image signal. It is outputting. Therefore, the difference between the interpolated image and the surrounding image is reduced, and the error between the interpolated image and the actual image signal is less likely to affect the subsequent decoded image signal. It will be possible.

Next, a second embodiment of the present invention will be described. FIG. 7 is a block diagram showing the configurations of an encoding device and a decoding device as a second embodiment of the present invention.

The configuration of FIG. 7 is almost the same as that of the device shown in FIG. 1, but in this embodiment, the line scan conversion circuit 1 is used.
An interpolation circuit 113 is provided in the subsequent stage of 11, and interpolation is performed by the decoded image signals converted in the input order by the line / scan conversion circuit 111.

The decoding operation of this embodiment will be described below with reference to FIGS. 8 and 9.

FIG. 8 is a decompression / decoding circuit 1 in this embodiment.
It is a block diagram which shows the structure of 10. In FIG. 8, the coded data whose code error has been corrected by the error correction circuit 109 is input to the data separation circuit 301, where it is separated into image data and motion vector data. The motion vector data is stored in the motion vector memory 307 and the switch 31.
It outputs to a terminal of 3.

The error flag for the motion vector output from the error correction circuit 109 is input to the interpolation determination circuit 309, and the interpolation determination circuit 309 controls the switch 313 based on this error flag to ensure that the input motion vector data is correct. When it is reproduced, the switch 313 is connected to the a side to output the input motion vector data as it is to the memory control circuit 311, and when the error cannot be corrected, the switch 313 is connected to the b side and the movement is performed. The motion vector of the block at the same position on the screen one frame before is read from the vector memory and output to the memory control circuit 311.

The interpolation judgment circuit 309 also sets the error flag of the block indicated by the motion vector on the basis of the error flag for the input image data and the motion vector corresponding to the input image data read from the motion vector memory. It is read from, and it is determined whether the error flag is "1", and a control signal is output to the interpolation circuit 113 described later.

The following operation of the decompression / decoding circuit 110 is similar to that of the above-described embodiment. That is, the input image data is decoded as described above, and when the input image data is the I frame, the switch 314 is connected to the side a based on the mode information to output the decoded image signal as it is, or the P or B frame. If it is, the switch 314 is connected to the b side and the addition signal with the decoded image signal read from the memory 310 is output.

The image signal thus composited by the decompression / decoding circuit 110 is converted into the line scan conversion circuit 11
The data is converted in the input order at 1 and output to the interpolation circuit 113.

FIG. 9 is a block diagram showing the structure of the interpolation circuit 113.

In FIG. 9, the image signal output from the line scan conversion circuit 111 is output to the delay circuit 501 and the memory 502. Also, the decompression / decoding circuit 1
The control signal output from the 10 interpolation determination circuit 309 is input to the control circuit 505.

Based on this control signal, the control circuit 505 correctly reproduces the block indicated by the motion vector used at the time of decoding, and when it is not interpolated, connects the switch 504 to the terminal a and outputs it from the delay circuit 501. Output an image signal.

When the block indicated by the motion vector is interpolated, the switch 504 is connected to the terminal b, and the memory 502 and the intra-field interpolation circuit 50 are connected.
3 is controlled to generate an interpolated image signal using the image signal in the same field as the input image signal, and output through the switch 504.

Here, the operation of the intra-field interpolation circuit 503 will be described with reference to FIG. Now, assume that the block X in FIG. 10 is an error block. Also,
Assuming that the intra-field interpolation circuit 503 performs processing on a field-by-field basis, the odd field lines x1, x3, x5, x7 of the error block X are the top lines x of the odd fields among the lines of the correctly reproduced block.
Block A, which is the line of the same field closest to 1.
A7 and the line b1 of the block B, which is the line of the same field closest to the bottom line x7, are interpolated in the field.

After the processing of the odd field is completed, the processing of the even field is performed. The even field lines x2, x4, x6, x8 of the error block X are also line a of the block A as in the case of the odd field lines described above.
8 and the line b2 of the block B are interpolated linearly in the field.

That is, in the odd field, x1 =
(4 × a7 + b1) / 5, x3 = (3 × a7 + 2 × b
1) / 5, x5 = (2 × a7 + 3 × b1) / 5, x7 =
(A7 + 4 × b1) / 5, and in the even field, x2 = (4 × a8 + b2) / 5, x4 = (3 × a
8 + 2 × b2) / 5, x6 = (2 × a8 + 3 × b2) /
5, x8 = (a8 + 4 × b2) / 5. The intra-field interpolated image signal thus generated is switched by the switch 504.
Is output via.

As described above, in this embodiment, the image signal after being converted in the input order is more correlated with the image signal in which the block indicated by the motion vector used at the time of decoding is the interpolated image. Since the high intra-field interpolated image signal is generated and output, the difference between the correctly reproduced image and the interpolated image is reduced.

In the first embodiment described above, when the block indicated by the motion vector has already been interpolated, the motion vector is changed to indicate the blocks around the block, but the invention is not limited to this. For example, in this case, the decoded image signal at the same position on the immediately previous screen may be uniformly output via the c terminal of the switch 306 instead of the adder 305. In this case, since the interpolation determination circuit 309 does not need to change the motion vector, the load on the interpolation determination circuit 309 is reduced.

[0054]

As is apparent from the above description, in the present invention, the decoding means is controlled in accordance with the error correction result of the block indicated by the motion vector of the image signal to be decoded, so that the motion vector indicates It becomes possible to combine the image signals by using the decoded image signals of the blocks that are considered to have a high correlation with the blocks. Therefore, the difference between the interpolated image and the surrounding image is reduced, and the error from the actual image signal is less likely to affect the subsequent decoded image signal, and the deterioration of the image quality of the reproduced image signal is prevented. Will be possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an encoding device and a decoding device as a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a compression / encoding circuit in FIG.

3 is a block diagram showing a configuration of a decompression / encoding circuit in FIG.

4 is a flow chart for explaining the operation of the apparatus of FIG.

5 is a diagram for explaining the operation of the apparatus of FIG.

FIG. 6 is a diagram for explaining the operation of the apparatus of FIG.

FIG. 7 is a block diagram showing configurations of an encoding device and a decoding device as a second embodiment of the present invention.

8 is a block diagram showing a configuration of a decompression / decoding circuit in FIG.

9 is a block diagram showing a configuration of an interpolation circuit in FIG.

FIG. 10 is a diagram for explaining the operation of the interpolation circuit.

[Explanation of symbols]

 103 compression / decompression circuit 109 error correction circuit 110 decompression / decoding circuit 309 interpolation determination circuit

Claims (6)

[Claims] 1. An input unit for inputting a block-coded image signal and motion vector information for each block, an error correction unit for correcting an error in the image signal, and an image signal for the image signal via the error correction unit. Decoding means for decoding the image signal by using the motion vector of each block, and the decoding means according to the correction result of the error correction means for the image signal of the block indicated by the motion vector of the block to be decoded by the decoding means. An image processing apparatus comprising: a control unit that controls a decoding unit. 2. The control means, in response to the fact that the correction result of the error correction means for the image signal of the block indicated by the motion vector is uncorrectable, the image of the block around the block indicated by the motion vector. The image processing apparatus according to claim 1, wherein the decoding means is controlled so as to decode the image signal using a signal. 3. The screen other than the screen including the block to be decoded according to the fact that the correction result of the error correction unit for the image signal of the block indicated by the motion vector is uncorrectable. 2. The image processing apparatus according to claim 1, wherein the decoding means is controlled so as to decode the image signal by using the image signal of the block at the same position in. 4. The control means uses the image of the block indicated by the motion vector in response to the fact that the correction result of the error correction means for the image signal of the block indicated by the motion vector is not uncorrectable. The image processing apparatus according to claim 2, wherein the decoding means is controlled so as to decode the image signal. 5. The image signal of a screen including the block to be decoded, when the control means cannot correct the correction result of the error correction means for the image signal of the block indicated by the motion vector. The image processing apparatus according to claim 1, wherein the decoding means is controlled so as to decode the image signal using. 6. The image processing apparatus according to claim 1, wherein the image signal is predictively coded.