JP3449370B2 - Image data decoding method and apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data decoding method and an image data decoding apparatus for decoding image data highly efficient coded by orthogonal transformation.

[0002]

2. Description of the Related Art As a method for highly efficient encoding of image signals, for example, so-called MPEG (Moving Picture Experts G) is used.
In the standardization proposal by roup), a high-efficiency coding method of image signals for so-called digital storage media is specified. The principle of the high-efficiency encoding system of the image signal by MPEG is as follows.

That is, in this high-efficiency coding system, the redundancy in the time axis direction is first reduced by taking the difference between the images, and then the so-called discrete cosine transform (DCT: Discrete Transform).
te Cosine Transform) processing and variable length coding (VLC:
Variable Length Coding) processing is used to reduce the redundancy in the spatial axis direction.

First, the redundancy along the time axis will be described below. Generally, in a continuous moving image, temporally preceding and succeeding images are very similar to an image of interest (that is, an image at a certain time). For this reason, for example, as shown in FIG. 1, if the difference between the image to be encoded now and the image preceding in time is calculated and the difference is transmitted, the redundancy in the time axis direction can be obtained. It is possible to reduce the amount of information to be transmitted by reducing it. The image coded in this way is called a forward predictive coded image (Predictive-coded picture, P picture or P frame) described later.

Similarly, the difference between the image to be encoded from now on and the interpolated image formed temporally forward or backward or forward and backward is calculated, and the difference of the smaller value is transmitted. By doing so, it becomes possible to reduce the redundancy in the time axis direction and reduce the amount of information to be transmitted. The image encoded in this way is a bidirectionally predictive encoded image (Bidirectionally Predic
tive-coded picture, B picture or B frame). In FIG. 1, an image indicated by I is an intra-coded image (intra-coded image: In
tra-coded picture, I picture or I frame), the image indicated by P in the figure indicates the P picture, and the image indicated by B in the figure indicates the B picture.

Further, so-called motion compensation is performed in order to create each predicted image. That is, according to this motion compensation,
For example, a block of 16 × 16 pixels (hereinafter referred to as a macroblock), which is composed of unit blocks of 8 × 8 pixels, is created, and a position where the difference is smallest in the vicinity of the position of the macroblock in the previous image. , And the difference with this searched macroblock can be taken to reduce the amount of data that must be sent. Actually, for example, in the P picture (forward predictive coded image), the difference between the predicted image after motion compensation and the difference between the predicted image after motion compensation and the predicted image after motion compensation is 16 × 16 for the smaller amount
It is selected and encoded in units of macroblocks of pixels.

However, in the above-mentioned case, a lot of data has to be sent with respect to, for example, a portion (image) coming out from behind after the object moves. Therefore, for example, in the B picture (bidirectional predictive coded image), the already decoded temporally forward or backward image after motion compensation, and an interpolated image created by adding both of them will be encoded from now on. The difference between the image and the image to be encoded and the one that does not take the difference, that is, the one with the smallest data amount out of the four images to be encoded from now on are encoded.

Next, the redundancy in the spatial axis direction will be described below. The difference of the image data is not transmitted as it is, but discrete cosine transform (DCT) is applied to each unit block of 8 × 8 pixels. The DCT expresses an image not by the pixel level but by how many frequency components of a cosine function are included. For example, by a two-dimensional DCT, data of a unit block of 8 × 8 pixels is 8 × 8. Is converted into the coefficient block of the component of the cosine function of. For example, an image signal of a natural image taken by a television camera is often a smooth signal. In this case, the data amount can be efficiently reduced by performing the DCT process on the image signal.

FIG. 2 shows the structure of data handled by the above-mentioned coding system. That is, the data structure shown in FIG. 2 has a block layer, a macroblock layer, a slice layer, a picture layer, and a GOP (Group) in order from the bottom.
of Picture) layer and a video sequence layer. The layers will be described in order from the bottom in FIG.

First, in the block layer, as shown in FIG. 2F, the block of the block layer is composed of 8 × 8 pixels (8 lines × 8 pixels) adjacent to each other in luminance or color difference. The DCT (discrete cosine transform) described above is applied to each unit block.

In the above macroblock layer, the macroblocks of the macroblock layer are four luminance blocks (luminance unit blocks) Y0, Y1, Y2 and Y3 that are adjacent to each other in the left-right and up-down directions, as shown in FIG. 2E. In the image, the color difference block (color difference unit block) Cr, Cb corresponding to the same position as the luminance block is composed of a total of six blocks. The order of transmission of these blocks is Y
The order is 0, Y1, Y2, Y3, Cr, Cb. Here, in the encoding method, what is used for the predicted image (reference image for taking the difference), whether the difference does not have to be sent, or the like is determined for each macro block.

The slice layer is, as shown in FIG. 2D,
It is composed of one or a plurality of macroblocks that are continuous in the scanning order of the image. At the beginning of this slice (header), the difference between the motion vector and the DC (direct current) component in the image is reset, and the first macroblock has data indicating its position in the image, and thus the error is It is designed to be able to return even if it happens.
Therefore, the length and starting position of the slice are arbitrary and can be changed depending on the error state of the transmission path.

In the picture layer, each picture, that is, each picture is composed of at least one or a plurality of slices as shown in FIG. 2G. And
According to the encoding method, the intra-coded image (I picture or I frame), the forward predictive coded image (P picture or P frame), and the bidirectional predictive coded image (B picture or B frame) as described above are used. ), DC intra-coded image (DC coded (D) pict
ure) are classified into four types of images.

Here, in the intra-coded image (I picture), only information closed in one image is used when being coded. Therefore, in other words, when decoding, an image can be reconstructed using only the information of the I picture itself. Actually, the DCT processing is performed as it is and the encoding is performed without taking the difference. This encoding method is generally inefficient, but if this encoding method is put everywhere, random access and high-speed reproduction are possible.

In the forward predictive coded image (P picture), an I picture or P picture which is located earlier in time at the input and already decoded is used as a predictive picture (a picture serving as a reference for taking a difference). To do. Actually, whichever is more efficient is selected for encoding the difference from the motion-compensated predicted image or for encoding as it is without taking the difference (intra-code).

The bidirectional predictive coded image (B picture)
In the above, three types of I-pictures or P-pictures, which are temporally positioned in advance and already decoded, and interpolated pictures made from both of them, are used as prediction pictures. This allows
It is possible to select the most efficient one of the above three types of differential encoding after motion compensation and the intra code in macroblock units.

The above DC intra-coded image is an intra-coded image composed of only DC coefficients in DCT, and cannot be present in the same sequence as the other three types of images.

As shown in FIG. 2B, the GOP layer has 1
Alternatively, it is composed of a plurality of I pictures and 0 or a plurality of non-I pictures. The I-picture interval (for example, 9) and the I-picture or B-picture interval (for example, 3) are free. Further, the interval between the I picture and the B picture may change inside the GOP layer.

As shown in FIG. 2A, the video sequence layer is composed of one or a plurality of GOP layers having the same image size and image rate.

As described above, in the case of transmitting a moving image standardized by the high-efficiency encoding method based on MPEG, first, an image obtained by compressing one image within a picture is sent, and then this image is transmitted. The difference from the motion-compensated image is transmitted.

However, it has been found that the following problems occur when the image to be encoded is an interlaced image.

When the interlaced image is encoded as a picture on a field-by-field basis, the vertical positions of the fields alternate. Therefore, when transmitting the still part of a moving image,
Since the difference information is generated every time the field is changed and must be transmitted, the coding efficiency is reduced in the still portion of the moving image.

Further, when the encoding process is performed in the field unit, the block is constructed in the field unit, so that the interval between pixels is widened and the correlation is lowered as compared with the case where the block is constructed in the frame unit. Efficiency is reduced.

On the other hand, if the interlaced image is coded as a picture on a frame-by-frame basis, it is necessary to process a so-called comb-shaped image with respect to the moving part in the frame. For example, as shown in FIG. 3, when there is a moving body CA such as an automobile in front of a stationary moving body, there is movement between fields when the I frame is seen, and such a portion is comb-shaped as shown in FIG. It becomes an image of KS. Therefore, a high frequency component that does not exist in the original image is transmitted, and the coding efficiency is reduced.

Further, in the frame-by-frame encoding processing,
Since two consecutive fields that make up one frame are collectively coded, predictive coding cannot be used between the two fields. Therefore, the minimum distance of the predictive coding is one frame (two fields), so that the motion is faster or the motion is more complicated than the field-based coding process in which the minimum distance of the predictive coding is one field. For images, the frame-by-frame encoding process is disadvantageous.

As described above, the coding efficiency may decrease in each of the field-based coding processing and the frame-based coding processing, and conversely, they are cases where the other coding efficiency is high.

[0027]

SUMMARY OF THE INVENTION Therefore, the present invention has been proposed in view of the above-mentioned circumstances, and for interlaced images, an image with little motion, an image with many motion, and an image in which both of them are mixed. It is an object of the present invention to provide an image data decoding method and an image data decoding device for decoding image data obtained by encoding.

[0028]

In order to solve the above-mentioned problems, the image data decoding method according to the present invention subdivides an image of one frame consisting of two fields having an interlaced structure into blocks. In an image data decoding method for decoding image data that has been encoded such as discrete cosine transform in a field unit or a frame unit in a block, in a frame in which the encoding process is performed in a frame unit, each block is field predicted or Frame prediction and frame decoding are performed to form a frame.
In the frame in which the encoding process is performed in field units, each block is subjected to field prediction and field decoding processes to form a frame. Between the frame coded in the frame unit and the frame coded in the field unit, the field coded in the field unit is two fields of the frame processed frame or the field processed field. To perform a prediction process and a field decoding process to form a field.

Further, the image data decoding apparatus according to the present invention subdivides an image of one frame consisting of two fields having an interlaced structure into blocks, and in each block, discrete cosine transform in field units or frame units, etc. In the image data decoding apparatus that decodes the encoded image data of, the frame in which the encoding process is performed on a frame-by-frame basis performs field prediction or frame prediction for each block, and the frame decoding process is performed to determine the frame. A unit for forming a frame, and a frame for which the encoding process is performed on a field-by-field basis, each block is field-predicted, and a field decoding process is performed to form a frame,
Between the frame coded in the frame unit and the frame coded in the field unit, the field coded in the field unit is two fields of the frame processed frame or the field processed field. , And means for forming a field by performing a field decoding process.

[0030]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Specific embodiments to which the present invention is applied will be described below with reference to the drawings.

1. Image Data Encoding Method / Image Data Decoding Method Field / frame-based encoding / decoding processing to which the present invention is applied will be described.

When the picture shown in FIG. 1 described above is in field units, it is shown in FIG. 5, for example, when the field structure is taken into consideration. In FIG. 5, the upper part is the first
A field (for example, an odd field) is shown, and the lower row shows a second field (for example, an even field). 1/6
Two fields that are temporally adjacent to each other at an interval of 0 second form a frame. Then, in the field-based coding / decoding processing, each picture is coded / decoded in field units.

Further, the I picture (intra-picture coded picture) / P picture (forward predictive coded picture) / B picture (bidirectional predictive coded picture) in the GOP layer shown in FIG. 2B described above.
FIG. 6 shows a specific example in which the configuration pattern is changed. 5 and 6 are the same in that encoding / decoding processing is performed on a field-by-field basis, only the configuration patterns of pictures in the GOP layer are different. By the way, as shown in FIG. 6, when the first field and the second field have the same coding type, when the first field and the second field are collectively coded / decoded, the code of the frame unit is The decoding / decoding process is shown in FIG.

Although various variations can be considered for the motion prediction / compensation in the encoding / decoding processing shown in FIGS. 5, 6 and 7, simple concrete examples thereof are shown in FIGS.
Shown in 10. In these figures, for example, pictures I2 to P5 shown in FIG. 8 and thick dashed arrows such as pictures I3 to P6 shown in FIG. 9 represent motion prediction to P pictures, for example, shown in FIGS. The thin dashed arrows from the picture I2 to the picture B0, etc. represent motion prediction from B picture. Furthermore, in FIGS. 10A and 10B,
A solid arrow represents motion prediction of a macro block having a frame structure described later, and a broken arrow represents motion prediction of a macro block having a field structure.

In the field-unit coding / decoding process, a P picture, for example, the picture P5 shown in FIG.
Uses a picture I2 which is located earlier in time and has already been decoded as a prediction image (a reference image for taking a difference), and a picture P8 is located earlier in time as a prediction image and has already been decoded. Picture P5 is used. Also, for example, the picture P6 shown in FIG. 9 uses the picture I3 as a predicted image, and the picture P7 uses the picture P6 as a predicted image. Next, a B picture, for example, FIG.
The picture B4 shown in (3) uses three types of pictures, i.e., the picture I2 or the picture P5 which has been previously decoded in time and has been interpolated from both, as a prediction picture. Further, for example, the picture B4 illustrated in FIG. 9 uses three types of pictures I3 or P6 as a prediction image and an interpolated image made from both of them.

On the other hand, in the frame-by-frame encoding / decoding process, a P picture (frame), for example, a frame composed of the pictures P6 and P7 shown in FIG. 10A is a frame composed of the pictures I2 and I3 as predicted images. The frame composed of the pictures P10 and P11 uses the frame composed of the pictures P6 and P7 as the prediction image. Next, a B picture (frame), for example, a frame composed of the pictures B4 and B5 shown in FIG. 10B is a frame composed of the pictures I2 and I3 that have been previously decoded as the predicted image and are already decoded. Three types of frames, that is, a frame composed of the pictures P6 and P7 and an interpolated image made from both of them, are used.

As described above, the encoding / decoding processing in the field unit and the frame unit has the same encoding / decoding processing procedure, but as described below, there is a difference in the block configuration and the motion prediction / compensation. There is.

(1) Block Structure In the frame-based coding / decoding processing, the first field and the second field are collectively coded / decoded, so that a block in which the first field and the second field are grouped is configured. Yes, but in field-based processing, a block consists of only one of the fields.

(2) In the coding / decoding processing in units of motion prediction / compensation frames, since the first field and the second field are coded / decoded together, the first field and the second field belonging to the same frame Motion prediction for two fields is not used, but motion prediction from the first field to the second field is used in field-based processing.

Details of the above (1) block configuration and (2) motion prediction / compensation will now be described.

(1) Block Configuration FIG. 11 is a diagram showing the configuration of blocks inside a macroblock in the field-unit encoding / decoding processing. As shown in FIG. 11, field-unit encoding / decoding is performed. In the decoding process, the field-structured macroblock is composed of only the first field or the second field.

On the other hand, FIG. 12 is a diagram showing the configuration of blocks inside a macroblock in the encoding / decoding processing on a frame-by-frame basis. In the encoding / decoding processing in units of frames, in addition to the field-structured macroblocks shown in FIGS. 12A and 12B, frame-structured macroblocks can be taken as shown in FIG. 12C. That is,
In the field-structured macroblock, as shown in FIG. 12A, the coding / field-based coding shown in FIG. 11 is performed.
As shown in FIG. 12B, in addition to the first and second fields having the same macroblock structure as the decoding process, the macroblock is divided into upper and lower two blocks, and the upper half is the first field only and the lower half is Can also consist of only the second field. As shown in FIG. 12C, the macroblock having a frame structure includes a first field and a second field.

As described above, in the frame-based coding / decoding process, a frame-structured macroblock can be used in addition to the field-structured macroblock in the field-based coding / decoding process.

By the way, the switching between the field-structured macroblock and the frame-structured macroblock is performed, for example, on a field-by-field basis determined by a coding method determination circuit 21 (see FIG. 23) which constitutes an image data coding apparatus described later. This is realized by controlling the read address when reading a block from the buffer memories 7 and 8 by the identification information (hereinafter referred to as the encoding method information) for identifying whether the encoding process is the encoding process of the frame or the encoding process of the frame unit. be able to. Further, in the image data decoding device to be described later, it is written in the coded bit stream (Bit stream) received from the image data coding device or the like in the inverse variable length coding circuit 31 (see FIG. 27) ( (Overlapping) flag is detected, and it is determined based on this flag whether the encoding system is in field unit or frame unit, and the information is supplied to the motion compensation circuits 42, 43 and the like to store the information in the buffer memories 37, 38. It can be realized by controlling the read address.

That is, the above-mentioned buffer memories 7, 8,
37, 38 and, for example, as shown in FIGS.
It is configured with a memory having a storage capacity of × 720 pixels. In the frame configuration, image data is stored in the frame configuration as shown in FIG. 13, and in the field configuration, the image data is stored in the field configuration. . Note that FIG.
In 4, the two fields do not have to be consecutive in time. Further, in this example, the size of the buffer memory is set to the size of the frame, but the size is not limited, and may be larger, or may be configured to store a plurality of frames. Further, in the block diagrams of FIGS. 23 and 27, the buffer memory is divided into two in order to facilitate the correspondence with the encoding / decoding processing, but in the actual configuration, it is not necessary to divide it. They may be combined in one buffer memory.

(2) Motion prediction / compensation In the field unit coding / decoding processing, when motion prediction / compensation is performed, for example, prediction from picture I2 to picture B3 shown in FIG. 8 or picture shown in FIG. The motion prediction from the first field to the second field belonging to the same frame is used like the prediction from P6 to the picture P7.

However, in the frame-by-frame encoding / decoding processing, since two fields are encoded / decoded collectively as shown in FIG. 7, from the first field to the second field belonging to the same frame. Do not use motion estimation to.

As described above, field-based coding /
Since the motion prediction of the decoding process uses the motion prediction from the first field to the second field belonging to the same frame, the minimum interval between pictures for motion prediction is short, and the motion prediction of the encoding / decoding process in frame units is performed. Includes. In addition, in the above-described specific example, the motion prediction of a macroblock having a frame structure is specifically shown. However, since the same motion prediction is performed only for two fields in the macroblock, the motion prediction of a macroblock having a field structure is performed. Two motion predictions can be used instead. Furthermore, the motion prediction in the frame unit may not be the motion prediction indispensable for the encoding / decoding processing in the frame unit, but may be only the motion prediction in the field unit.

As described above, in the interlaced image coding / image data decoding method according to the present invention, both the block structure and the motion prediction control method are used for the field unit and frame unit coding / decoding processing. By corresponding to the encoding / decoding processing of, the encoding / decoding in both methods is possible.

In this case, it is necessary to inform the image data decoding apparatus of what range and which encoding processing has been performed by the image data encoding apparatus, that is, whether it is in field unit or frame unit.

In order to realize this, according to the present invention, a flag indicating whether a certain range of the coded image is processed in a field unit or a frame unit in a part of the coded bit stream (coded image data). To provide. In addition,
The certain range is, for example, a sequence, GOP, or picture. Specifically, in the image data encoding device, identification information for identifying whether a certain range of an image has been encoded in units of field units or frame units is used as a flag at a predetermined position of the encoded bitstream. Set (superimpose). In the image data decoding device, by decoding the predetermined position of the coded bit stream by the inverse variable length coding circuit 31 as described above, the unit of decoding processing is determined, and decoding is performed based on the unit. , Images can be played.

The motion prediction in the field unit / frame unit encoding / decoding process is not limited to the specific examples shown in FIGS.
For example, the encoding / decoding processing in units of fields using motion prediction shown in FIGS. 15 and 16 can be switched between the encoding / decoding processing in units of frames and the layer in which an image is present.

Specifically, FIG. 15A shows the GO shown in FIG.
FIG. 9 is a diagram showing a specific example different from the P picture prediction method of the field-unit coding / decoding processing shown in FIG. 8 in the configuration pattern of I / P / B pictures in the P layer.
FIG. 5B is a diagram showing a specific example different from the B picture prediction method of the encoding / decoding processing in units of fields shown in FIG. 8. That is, for example, as the predicted image of the picture P8, while the picture P5 is used in the specific example shown in FIG. 8, as shown in FIG.
You may make it use 2 etc. Further, as the prediction image of the picture B4, for example, in the specific example shown in FIG. 8, three types of the picture I2 or the picture P5 and the interpolated image made from both of them are used, while as shown in FIG. 15B. , Picture I2, picture P5, picture P
8 or the picture P11 and the interpolated image synthesized from them may be used.

FIG. 16A shows the I / O in the GOP layer shown in FIG.
16B is a diagram showing a specific example different from the P picture prediction method of the field-unit encoding / decoding processing shown in FIG. 9 in the P / B picture configuration pattern, and FIG.
FIG. 10 is a diagram showing a specific example different from the B-picture prediction method of the field-unit encoding / decoding process shown in FIG. That is, for example, as the predicted image of the picture P6, while the picture I3 is used in the specific example shown in FIG. 9, the picture I2 or the like may be used as shown in FIG. 16A. Further, for example, as the predicted image of the picture B4, in the concrete example shown in FIG. 9, three types of the picture I3 or the picture P6 and the interpolated image made from both of them are used, while as shown in FIG. 16B. Alternatively, the picture I2, the picture I3, the picture P6 or the picture P7, and the interpolated image synthesized from them may be used.

Furthermore, for example, the field-unit encoding / decoding process and the frame-unit encoding / decoding process may be combined.

For example, FIGS. 17 and 20 show specific examples in the case where a plurality of pictures are coded / decoded in frame units, and subsequent pictures are coded / decoded in field units. FIG. 17 is a diagram showing a combination of the frame-based encoding / decoding processing example shown in FIG. 7 and the field-unit encoding / decoding processing example shown in FIG. 5, and FIG. 20 is shown in FIG. It is a figure which shows the combination of the example of a frame encoding / decoding process and the example of a field encoding / decoding process shown in FIG.

First, in FIG. 17, for example, the picture P
As shown in FIG. 18, a frame composed of pictures 6 and P7 uses a frame composed of pictures I2 and I3 as a predicted image, and a picture P10 uses a picture P6 or the like as a predicted image. Good. On the other hand, for example, the frame composed of the pictures B4 and B5 is, as shown in FIG. 19, a frame composed of the pictures I2 and I3 or a frame composed of the pictures P6 and P7 and an interpolated image synthesized from them. For example, as shown in FIG. 19, the picture B8 may use the picture P6, the picture P7, or the picture P10, and the interpolated image synthesized from them.

Next, in FIG. 20, for example, the frame composed of the pictures P6 and P7 uses the frame composed of the pictures I2 and I3 as the prediction image, as shown in FIG. For example, the picture P6 or the like may be used as the image. On the other hand, for example, the frame composed of the pictures B4 and B5 is, as shown in FIG. 22, a frame composed of the pictures I2 and I3 or a frame composed of the pictures P6 and P7 and an interpolated image synthesized from them. For example, as shown in FIG. 22, the picture B8 may use the picture P6, the picture P7, the picture P10 or the picture P11, and the interpolated image synthesized from them.

Thus, as shown in FIGS.
There is no problem even if the encoding / decoding processing in frame units and the encoding / decoding processing in field units are combined. In other words, in the image data encoding method and the image data decoding method according to the present invention, the motion of the interlaced image is switched by switching the encoding / decoding processing in frame units and the encoding / decoding processing in field units. It is possible to efficiently encode a small number of images, a large amount of moving images, and an image in which both of them are mixed.

2. Image Data Encoding Device FIG. 23 is a block diagram showing a specific circuit configuration of the image data encoding device to which the present invention is applied.

This image data encoding device (encoder)
As shown in FIG. 23, has a local decoding circuit including an inverse quantization circuit 2 to a gate 17 having the same circuit configuration as the image data decoding device described later.

First, when image data of a picture (field or frame) is input via the terminal 1, these image data are temporarily stored in the buffer memory 18. Specifically, for example, as shown in FIG. 24A, image data is input in the order of pictures I0, B1, B2, P3 ... And rearranged in the encoder processing order as shown in FIG. 24B. The motion prediction as shown in FIG. 1 is performed between the rearranged pictures. Note that the input pictures I0, B1, B2, P3 ...
Correspond to fields in FIGS. 5 and 6, and correspond to frames in FIG.

The rearranged pictures are used for motion vector detection in the motion vector detection circuit 19. The motion vector detection circuit 19 detects a motion vector required to generate a picture for prediction based on the already encoded picture. That is, the forward and backward pictures are held in the buffer memory 18, and the motion vector is detected between the current picture and the current reference picture. Here, in the detection of the motion vector, for example, the motion vector is the one in which the sum of the absolute values of the differences between pictures in macroblock units is the smallest.

The sum of absolute values of the motion vector and the difference between pictures in units of macroblocks is calculated by the coding method determination circuit 2
Sent to 1. The coding system determination circuit 21 uses a later-described algorithm to code a picture of a certain layer.
That is, it is determined whether the field-based coding processing or the frame-based coding processing is performed. The information of the encoding method (field unit or frame unit) and the motion vector are sent to the motion compensating circuits 12 and 13, the variable length encoding circuit 25, etc., and used for management of the buffer memories 7 and 8 and will be described later. Is transmitted to the image data decoding device. Further, the information on the encoding method is sent to the control circuit 16. From the control circuit 16, gates 5, 17,
A control signal for controlling the encoding method is output to the change-over switches 6, 9, 10, 15.

Further, the sum of absolute values of the motion vector and the inter-picture difference in macroblock units is sent to the intra-picture / forward / backward / bidirectional prediction judgment (hereinafter simply referred to as prediction judgment) circuit 20. The prediction determination circuit 20 determines the prediction mode of the reference macroblock based on this value.

This prediction mode is used in the control circuit 16
Sent to. Based on this prediction mode, the control circuit 16 outputs a control signal to the gates 5 and 17 so as to switch the intra-picture / forward / backward / bidirectional prediction in macroblock units. Further, the control circuit 16 sends the motion vector corresponding to the selected prediction mode to the buffer memories 7 and 8 to generate a predicted image. Specifically, the input image itself is generated in the intra-picture coding mode, and each predicted image is generated in the forward / backward / bidirectional prediction mode. The specific operations of the gates 5 and 17 and the changeover switches 6, 9, 10 and 15 will be described later with reference to the gate 3 of the image data decoding device.
5, 47 and the changeover switches 36, 39, 40, 45 are the same, and therefore the description thereof is omitted here.

Predicted images from the buffer memories 7 and 8 are
The differencer 22 is supplied to the differencer 22 and generates difference data between the predicted image and the current image. The difference data is a discrete cosine transform (DCT).
m) circuit 23.

The DCT circuit 23 uses the two-dimensional correlation of the image signal to perform the discrete cosine transform of the input image data or the difference data in block units, and the resulting D
The CT conversion data is supplied to the quantization circuit 24.

The quantization circuit 24 quantizes the DCT transform data with a quantization step size (scale) determined for each macroblock and slice, and performs so-called zigzag scanning on the result. Then, the obtained quantized data is subjected to variable length coding (hereinafter referred to as VL
C: Variable Length Code) circuit 25 and inverse quantization circuit 2.

The quantization step size used for quantization is
By feeding back the buffer remaining amount of the transmission buffer memory 26, a value that does not cause the transmission buffer memory 26 to fail is determined. This quantization step size is also V
It is supplied to the LC circuit 25 and the inverse quantization circuit 2 together with the quantized data.

Here, the VLC circuit 25 uses the quantized data,
Quantization step size, prediction mode, motion vector, coding method information, etc. are subjected to variable length coding processing, and coding method information (field unit or frame unit) is added to the header of a predetermined layer and transmitted. The data is supplied to the transmission buffer memory 26 as data.

The transmission buffer memory 26 temporarily stores the transmission data in the memory and then outputs it as a bit stream at a predetermined timing. In addition, the transmission buffer memory 26 outputs a quantization control signal in macroblock units according to the amount of residual data remaining in the memory. Is fed back to the quantization circuit 24 to control the quantization step size. As a result, the transmission buffer memory 26 adjusts the amount of data generated as a bit stream, and maintains an appropriate amount of data (the amount of data that does not cause overflow or underflow) in the memory. .

For example, when the remaining amount of data in the transmission buffer memory 26 is increased to the allowable upper limit, the transmission buffer memory 2
6 reduces the data amount of the quantized data by increasing the quantization step size of the quantization circuit 24 according to the quantization control signal.

On the contrary, the transmission buffer memory 26
When the remaining amount of data in the quantizer circuit 24 is reduced to the allowable lower limit value,
The data amount of the quantized data is increased by reducing the quantization step size of.

As described above, the bit stream from the transmission buffer memory 26 whose data amount has been adjusted is multiplexed with the encoded audio signal, sync signal, etc.,
Further, a code for error correction is added, and after a predetermined modulation is applied, it is recorded as so-called pits of unevenness on a so-called master disk through, for example, a laser beam. A so-called stamper is formed using the master disk, and a large number of duplicate disks (for example, an image recording medium such as an optical disk) is formed by the stamper. Also,
For example, it is transmitted to an image data decoding device described later via a transmission path. The image recording medium is not limited to an optical disc or the like, and may be a magnetic tape or the like.

On the other hand, the output data of the inverse DCT circuit 3 is added to the predicted image by the adder circuit 4 to perform local decoding. The operation of this local decoding is the same as that of the image data decoding device to be described later, and therefore its explanation is omitted here.

The picture composition pattern is not limited to the pattern shown in FIG. 3. For example, if the encoder processing order is different, the picture composition pattern is also different. Specifically, although there are various variations in the processing order of encoding the P picture and the B picture temporally sandwiched between them, the buffer memories 7 and 8 are only different in the processing order. It is possible to respond by changing the control.

Here, a specific algorithm of the encoding system determination circuit 21 will be described with reference to the flowchart shown in FIG.

The encoding system determination circuit 21 is arranged to detect the first field (eg, odd field) of the current frame to be encoded.
To a second field (for example, an even field) is used to select an encoding method.

Specifically, in step ST1, the coding method determination circuit 21 obtains this motion vector for all macroblocks in the frame to be coded, and proceeds to step ST2.

In step ST2, the coding method determination circuit 21 determines the horizontal (x) component and the vertical (y) component of the motion vector.
The median of the component is obtained, and the process proceeds to step ST3. Specifically, the median of the horizontal component is calculated as follows. First, the horizontal components of the motion vector are arranged in order of descending power. Then, the value of the data at the center is set as the median Mv _{x of the} horizontal component. Similarly, the median Mv _{y of the} vertical component is obtained.

In step ST3, the coding system determination circuit 21 determines the vector MV (M obtained in step ST2.
Since v _x , Mv _y ) is a parameter indicating the movement of the entire screen, the magnitude R of this vector MV is obtained as a parameter indicating the magnitude of the movement of the entire screen, and step S
Proceed to T4. This R is calculated by the following equation 1.

R = | MV | = sqrt (Mv _x ² + Mv _y ² ) ... Equation 1 In step ST4, the coding method determination circuit 21
The encoding system is switched according to R obtained in step ST3. Since a field-based coding method (coding processing) is advantageous for a fast-moving image and a frame-based coding method (coding processing) for a low-moving image, the coding method determination circuit 21 sets R to a predetermined value. When the darkness value is less than or equal to TH, the process proceeds to step ST5, and the coding method for each frame is selected. Otherwise, the process proceeds to step ST6, and the field-based coding method is selected.

Thus, in the image data coding apparatus to which the present invention is applied, the motion vector between the first field and the second field of the same frame, or the median of the motion vector of the motion vector, is calculated according to the motion of the image. By determining whether the encoding process is performed in field units or in frame units based on the size, it is possible to efficiently perform an image with little motion, an image with a lot of motion, and an image in which both of them are mixed. Encoding can be done.

3. Image Data Format Next, a specific example of the bit stream output from the image data encoding device, that is, the image data format will be described.

FIG. 26A shows an example in which identification information (encoding system information) for identifying that the encoding process is performed in frame units or field units is added to the header of the video sequence layer as a flag. FIG. In the video sequence layer, as shown in FIG. 26A, a start code, a horizontal size, a vertical size, etc. are written in predetermined bits (indicated by numerals in the figure). In this example, after the frame frequency, identification information for identifying whether the encoding process is performed in field units or frame units is added as a 1-bit flag. The position where the flag is added may be another place.

Further, as shown in FIG. 26B, the above flag may be added to the header of the GOP layer. That is, in this example, the identification information is added as a 1-bit flag after the time code. The position to which this flag is added may be another place as well.

Further, FIG. 26 described above is an example in which a flag is added to the MPEG standard, but the flag may be written by utilizing the extension area of each layer which is predetermined by MPEG. For example, in the case of the video sequence layer, the extension area (Se
quence Extension Byte) or user data. Specifically, for example, FIG.
As shown in 6C, a flag may be added to the extension area of the picture layer of the MPEG standard. At this time,
A 2-bit flag is used to represent the encoding method. This 2-bit flag indicates the following information.

00: picture coded in frame units 01: first field of picture coded in field units 10: second field of picture coded in field units 11: spare

4. Image Data Decoding Device FIG. 27 is a block diagram showing a specific circuit configuration of an image data decoding device (decoder) to which the present invention is applied.

Inverse variable length coding (hereinafter, inverse VLC: Invers
A circuit 31 called "e-Variable Length Coding" reverse bit-lengths the bit stream supplied from the image data encoding device (encoder) shown in FIG. 23 and the bit stream obtained by reproducing an image recording medium such as an optical disk. The data is encoded and output to the inverse quantization circuit 32. At the same time, the inverse VLC circuit 31 decodes the data such as the motion vector, the quantization width (quantization step size), the encoding information, etc. which are written (superposed) together with the image data at the time of encoding.

In particular, the inverse VLC circuit 31 decodes the flag added to the header of the video sequence layer, the GOP layer, the picture layer, etc., and obtains information on whether the decoding process is in frame units or field units. . This information is supplied to the control circuit 46, which generates a control signal for frame processing or field processing. Specifically, the control circuit 46 generates various control signals in response to the output of the inverse VLC circuit 31, and in addition to the gate 35, the gate 47, the changeover switches 36, 39, 40, 45, etc. are connected to predetermined contacts. Switch to the direction.

That is, in the case of frame processing, if one picture is, for example, 720 (pixels) × 480 (lines), and one macroblock is 16 × 16 pixels, 1350 macroblocks complete one picture processing. Generates a control signal for processing the next picture. On the other hand, in the case of field processing, similarly, 675 macroblocks are regarded as the completion of one picture processing, and a control signal for processing the next picture is generated. By managing the scheduling of the buffer memories 37 and 38, decoding is performed in frame units or field units.

Further, in the inverse VLC circuit 31, a control signal indicating whether the decoding process is performed in frame units or field units is also sent to the motion compensation circuits 42, 43, and the motion compensation circuits 42, 43 Buffer memory 3
By controlling the addresses of 7 and 38, decoding is performed in frame units or field units as described below.

The inverse quantization circuit 32 inversely scans and inversely quantizes the data subjected to the inverse VLC processing to obtain the inverse DCT.
Output to the circuit 33. The inverse DCT circuit 33 performs an inverse DCT (Inverse Discrete Cosine Transform) process on the input data and outputs it to the adding circuit 34. The predictive image data switched and selected by the selector switch 45 is input to the adding circuit 34 via the gate 47, and the predictive image data is added to the output data of the inverse DCT circuit 33 to obtain the decoded image data. Is generated.

When the output of the adder circuit 34 is an I picture or a P picture, the gate 35 is opened and the decoded image data is transferred via the change-over switch 36 to the buffer memory 3
7 or buffer memory 38 and stored.

Specifically, when the output of the adder circuit 34 is an I picture or a P picture, the changeover switches 39, 4
0 is switched to the contact a side. Further, the changeover switch 36 is alternately switched between the contact point a and the contact point b, and the picture (I picture or P picture) output from the adding circuit 34 is alternately stored in the pair of buffer memories 37 and 38. .

For example, as shown in FIG. 24A described above, pictures I0, B1, B2, P3, B4, B5, P6, B
In the image data encoding device (encoder), the image data arranged in the order of 7, B8, and P9 are pictures I0, P3, B1, B2, P6, and B as shown in FIG. 24B.
When processed in the order of 4, B5, P9, B7, B8, the reverse V
The data is also input to the LC circuit 31 in this order as shown in FIG. 24D.

As a result, as shown in FIG. 24E, for example,
If the decoded data of the picture I0 is stored in the buffer memory 37, the decoded data of the picture P3 is stored in the buffer memory 38 as shown in FIG. 24F, and further, the buffer memory 37 as shown in FIGS. 24E and F. The data of the picture I0 is updated to the data of the picture P6, and the data of the picture P3 in the buffer memory 38 is updated to the data of the picture P9.

Following pictures I0 and P3, picture B
1 or the data of the picture B2 is input to the adding circuit 34 from the inverse DCT circuit 33, the data of the picture I0 stored in the buffer memory 37 is the motion compensation circuit 42.
In, after being motion compensated corresponding to the motion vector,
It is supplied to the interpolation circuit 44. The data of the picture P3 stored in the buffer memory 38 is motion-compensated in the motion compensation circuit 43 in accordance with the motion vector, and then supplied to the interpolation circuit 44. Interpolation circuit 44 is inverse VL
Corresponding to the data input from the C circuit 31, the respective inputs from the motion compensation circuits 42 and 43 are combined at a predetermined ratio. This combined data is selected by the changeover switch 45 and supplied to the adder circuit 34 via its contact point b and the gate 47. The adder circuit 34 adds the data from the inverse DCT circuit 33 and the data selected by the changeover switch 45,
The picture B1 or the picture B2 is decoded.

When the pictures B1 and B2 are decoded from only the previous picture I0, the changeover switch 45 is changed over to the contact a side, and when the pictures B1 and B2 are decoded only from the subsequent picture P3, the changeover switch 45 is changed over to the contact c side. The data of the picture I0 or the picture P3 is added by the addition circuit 34, respectively.
Is supplied to.

The changeover switch 39 can be switched to the side opposite to the changeover switch 36. That is, when the changeover switch 36 is changed over to the contact a side (b side), the changeover switch 39 is changed over to the contact b side (a side). Therefore, after the picture I0 is stored in the buffer memory 37, the changeover switch 36 is changed over to the contact b side, and when the picture P3 is stored in the buffer memory 38, the changeover switch 39 is changed over to the contact a side.
At this time, the changeover switch 40 is changed over to the contact a side, so that the picture I0 is read from the buffer memory 37 and supplied to the display 41 via the changeover switches 39 and 40 as shown in FIG. 24G. The playback image is displayed. When the pictures B1 and B2 are output from the adder circuit 34, the changeover switch 4
0 is switched to the contact b side, and the pictures B1 and B2 are supplied to the display 41 as shown in FIG. 24G. Next, the changeover switch 39 is changed over to the contact b side, and the changeover switch 40 is changed over to the contact a side, so that FIG.
As shown in F, the picture P3 already stored in the buffer memory 38 is read out and supplied to the display 41.

Here, a specific circuit configuration of the inverse VLC circuit 31 will be described.

As shown in FIG. 28, the inverse VLC circuit 31 is provided with a barrel shifter 31a and a code decoding unit 31b, and the input code is set to 1 by the barrel shifter 31a.
It is sent to the code decoding unit 31b in units of 6 or 32 bits. The code decoding unit 31b includes a code table and a matching circuit (not shown), and performs matching between the input code and the code in the code table.
If there is a match, the data (da
ta) and its code length (CL) are output.

The data (data) is supplied to another circuit of the inverse VLC circuit 31 (not shown) and subjected to appropriate processing, and then the above-mentioned control circuit 46 and motion compensation circuit 43.
Etc.

On the other hand, the length (CL) of the code is sent to the barrel shifter 31a as the shift amount to be shifted next, and the barrel shifter 31a outputs the next code 16 according to the shift amount.
Alternatively, the data is output to the code decoding unit 31b in 32-bit units.

Therefore, the flag for identifying the frame unit / field unit is also written in the code table as a header together with other codes.

Thus, in the image data decoding apparatus to which the present invention is applied, the image data provided in a part of the header such as the video sequence layer, the GOP layer or the picture layer is encoded in the field unit or the frame unit. The image data can be reproduced by detecting the flag for identifying the fact and performing the decoding process according to the flag.

By the way, the image data decoding apparatus of the above-mentioned embodiment is an apparatus capable of executing both the decoding processing in the field unit and the frame unit, but, for example, only the decoding processing in only one of them is performed. A device that cannot execute may determine whether or not the device can perform decoding based on the flag.

Specifically, as shown in FIG. 27, for example, a judgment circuit 51 for judging whether or not decoding is possible based on the flag supplied from the inverse VLC circuit 1 and the judgment result are displayed. A display unit 52 is provided.

Then, for example, in the image data device which performs the decoding process only in the field unit, for example, the operation according to the flowchart shown in FIG. 29 is performed, and in the image data decoding device which performs the decoding process only in the frame unit, The operation is performed according to the flowchart shown in FIG.

That is, in the image data device which performs the decoding process only in the field unit, the judgment circuit 51 inputs the flag in step ST1 and then, in step ST1,
Go to 2.

In step ST2, the judgment circuit 51
Judges whether the flag is a frame predicate, and if so, proceeds to step ST4, and otherwise proceeds to step ST3.

In step ST3, the judgment circuit 51
Displays that decryption is not possible on the display unit 52, and ends.

In step ST4, the judgment circuit 51
Displays that decryption is possible on the display unit 52 and ends.
Then, in step ST5, a decoding process is performed.

On the other hand, in the image data device which performs the decoding process only on the frame, the judgment circuit 51 inputs the flag in step ST1, and then proceeds to step ST2.

In step ST2, the judgment circuit 51
Judges whether the flag is in units of fields, and if so, proceeds to step ST4, and if not, proceeds to step ST3.

In step ST3, the judgment circuit 51
Displays that decryption is not possible on the display unit 52, and ends.

In step ST4, the judgment circuit 51
Displays that decryption is possible on the display unit 52 and ends.
Then, in step ST5, a decoding process is performed.

As a result, the user can easily know the reason why the image data cannot be reproduced on the device by looking at the display on the display unit 52.

Tables 1 and 2 show specific examples of the identification information for identifying whether the encoding process is performed in the field unit or the frame unit. An example of this is the so-called IS0 /
ICEJTC1 / SC29 / WG11 N November 25, 1992
25-NOV-92 Test Model 3, Draft published as a document
This is the specification of the picture layer in Revision 1.

[0122]

[Table 1]

[0123]

[Table 2]

[0124]

According to the present invention, an image of one frame consisting of two fields having an interlace structure is subdivided into blocks, and each block is subjected to coding processing such as discrete cosine transform in field units or frame units. When decoding data, a frame for which encoding processing has been performed in frame units performs field prediction or frame prediction for each block and frame decoding processing to form a frame, and the encoding processing is performed in field units. The frame thus extracted is subjected to field prediction and field decoding processing for each block to form a frame. The field coded in the field unit between the frame coded in the frame unit and the frame coded in the field unit is the two fields of the frame to be frame processed or the field processed. A field is formed by performing a prediction process and a field decoding process using the field. As a result, it is possible to decode image data obtained by encoding an interlaced image with little motion, an image with many motion, and an image in which both of them are mixed.

[Brief description of drawings]

FIG. 1 is a diagram showing a relationship between pictures in MPEG.

FIG. 2 is a diagram showing a structure of a video frame in the MPEG.

FIG. 3 is a diagram showing a specific example of an image having a moving body.

FIG. 4 is a diagram showing an image blurred in a comb shape.

FIG. 5 is a diagram showing a specific example of encoding / decoding processing in units of fields.

FIG. 6 is a diagram showing another specific example of the encoding / decoding processing in field units.

FIG. 7 is a diagram illustrating a specific example of encoding / decoding processing in frame units.

[Fig. 8] Fig. 8 is a diagram illustrating a specific method of motion prediction in the field-unit encoding / decoding process illustrated in Fig. 5.

9 is a diagram showing a specific method of motion prediction in the field-unit encoding / decoding process shown in FIG.

[Fig. 10] Fig. 10 is a diagram illustrating a specific method of motion prediction in the frame-by-frame encoding / decoding process shown in Fig. 7.

[Fig. 11] Fig. 11 is a diagram illustrating a configuration example of a block in a macroblock in a field-unit encoding / decoding process.

[Fig. 12] Fig. 12 is a diagram illustrating a configuration example of blocks in a macroblock in a frame-based encoding / decoding process.

FIG. 13 is a diagram showing a specific configuration example of a buffer memory of an image data encoding device to which the present invention has been applied.

FIG. 14 is a diagram showing a specific configuration example of a buffer memory of the image data encoding device.

[Fig. 15] Fig. 15 is a diagram illustrating another specific method of motion prediction in the field-unit encoding / decoding process illustrated in Fig. 5.

[Fig. 16] Fig. 16 is a diagram illustrating another specific method of motion prediction in the field-unit encoding / decoding process illustrated in Fig. 6.

FIG. 17 is a diagram illustrating an example of a specific combination of frame-based encoding / decoding processing and field-based encoding / decoding processing.

[Fig. 18] Fig. 18 is a diagram illustrating a specific method for predicting the motion of a P picture in the encoding / decoding process illustrated in Fig. 17.

[Fig. 19] Fig. 19 is a diagram illustrating a specific method of motion prediction of a B picture in the encoding / decoding process illustrated in Fig. 17.

[Fig. 20] Fig. 20 is a diagram illustrating another example of another specific combination of frame-based encoding / decoding processing and field-based encoding / decoding processing.

[Fig. 21] Fig. 21 is a diagram illustrating a specific motion prediction method for P pictures in the encoding / decoding process illustrated in Fig. 20.

22 is a diagram showing a method of predicting the motion of a B picture in the encoding / decoding process shown in FIG.

FIG. 23 is a block diagram showing a specific circuit configuration of an image data encoding device to which the present invention has been applied.

[Fig. 24] Fig. 24 is a diagram illustrating the relationship of pictures input to the image data encoding device.

FIG. 25 is a flowchart showing an algorithm of a coding method determination circuit which constitutes the image data coding device.

FIG. 26 is a diagram showing a specific format of a header of encoded image data.

FIG. 27 is a block diagram showing a specific circuit configuration of an image data decoding device to which the present invention has been applied.

FIG. 28 is an inverse VL constituting the image data decoding device.
It is a block diagram which shows the concrete circuit structure of C circuit.

FIG. 29 is a flowchart for explaining the operation of the image data decoding device to which the present invention has been applied.

FIG. 30 is a flowchart for explaining the operation of the image data decoding device to which the present invention has been applied.

[Explanation of symbols]

31 inverse variable length coding circuit, 32 inverse quantization circuit, 33
Inverse DCT circuit, 34 adder circuit, 35 gate, 36,
39, 40, 45 selector switch, 37, 38 buffer memory, 42, 43 motion compensation circuit, 44 interpolation circuit,
46 control circuit, 51 judgment circuit, 52 display section

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 00-7/68 H04N 5/91-5/95

Claims (2)

(57) [Claims] 1. An image of one frame consisting of two fields having an interlaced structure is subdivided into blocks, and each block decodes image data that has been subjected to coding processing such as discrete cosine transform in field units or frame units. In the image data decoding method, for a frame in which the coding process is performed in frame units, each block is subjected to field prediction or frame prediction and frame decoding process is performed to form a frame, and the coding process is performed in field units. The divided frame is formed by performing field prediction and field decoding processing on each block to form a frame, and is provided between the frame coded in the frame unit and the frame coded in the field unit. The field encoded by the unit is
An image data decoding method, characterized in that a field is formed by performing a prediction process and a field decoding process using two fields of the frame processed frame or the field processed field. 2. An image of one frame consisting of two fields having an interlaced structure is subdivided into blocks, and each block decodes image data that has been subjected to coding processing such as discrete cosine transform in field units or frame units. In the image data decoding device, a frame in which the encoding process is performed in frame units, field prediction or frame prediction is performed for each block, and the frame decoding process is performed to form a frame, and the encoding process is performed in field units. In the frame performed in step 1, each block is field-predicted, field decoding processing is performed to form a frame, the above-mentioned frame-unit coded frame and the above-mentioned field-unit coded frame The fields that are coded in field units between
An image data decoding apparatus, comprising means for forming a field by performing prediction processing using the two fields of the frame subjected to the frame processing or the field subjected to the field processing, and performing field decoding processing.