JP2000041247A - Image data decoding method and image data decoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decode image data obtained by coding an interlace image with less motion, an interlace image with much motion, and an image being mixture of the both. SOLUTION: An inverse VLC circuit 31 applies inverse variable length coding processing to a bit stream and provides an output of the processing to an inverse quantization circuit 32 and decodes data denoting a motion vector and information such as quantization width and coding. Furthermore, the inverse VLC circuit 31 decodes a flag added to a header of a video sequence layer, a GOP layer or a picture layer or the like and gives information denoting decoding processing in the unit of frames or fields to a control circuit 46, and the control circuit 46 generates a control signal for frame processing or field processing to throw gates 35, 47 and changeover switches 36, 39, 40, 45 or the like to a position of prescribed contacts to decode coded image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image data decoding method and an image data decoding apparatus for decoding image data which has been highly efficiently coded by orthogonal transformation.

[0002]

2. Description of the Related Art As a method for efficiently encoding an image signal, for example, a so-called MPEG (Moving Picture Experts G) is used.
In the standardization proposal by Roup), a high-efficiency encoding method of an image signal for a so-called digital storage medium is defined. The principle of the high-efficiency encoding method of an image signal by MPEG is as follows.

That is, in the high-efficiency coding method, first, redundancy in the time axis direction is reduced by taking a difference between images, and then, a so-called discrete cosine transform (DCT: Discrete transform) is performed.
te Cosine Transform processing and variable length coding (VLC: V
ariable Length Coding) processing to reduce the redundancy in the space axis direction.

First, the redundancy in the time axis direction will be described below.

In general, in a continuous moving image, an image before and after in time is very similar to an image of interest (ie, an image at a certain time). Therefore, for example, as shown in FIG. 1, if the difference between the image to be coded and the image ahead in time is calculated and the difference is transmitted, the redundancy in the time axis direction can be reduced. It is possible to reduce the amount of information to be transmitted. The image encoded in this manner is called a forward predictive encoded image (Predictive-coded picture, P picture or P frame) described later.

Similarly, a difference between an image to be coded and a temporally forward or backward or an interpolated image formed from forward and backward is calculated, and a difference between the small values is transmitted. By doing so, it is possible to reduce the amount of information to be transmitted by reducing the redundancy in the time axis direction. An image encoded in this way is a bidirectionally encoded image (Bidirectionally Predictive
-coded picture, B picture or B frame). In FIG. 1, an image indicated by I in the figure is an intra-coded image (intra-coded image: Intra-
coded picture, I picture or I frame), an image indicated by P in the figure indicates a P picture, and an image indicated by B in the figure indicates a B picture.

In order to generate each predicted image, so-called motion compensation is performed. That is, according to this motion compensation,
For example, a block of, for example, 16 × 16 pixels (hereinafter referred to as a macro block) composed of unit blocks of 8 × 8 pixels
(Macroblock)), search for the place with the least difference near the position of the macroblock in the previous image,
By taking the difference from the searched macroblock, the data that must be sent can be reduced. Actually, for example, in the case of a P picture (forward prediction coded image), the difference between the predicted image after motion compensation is calculated and the difference between the predicted image after motion compensation is not calculated. Are encoded in units of macroblocks of 16 × 16 pixels.

However, in the case described above, a large amount of data must be sent, for example, with respect to a portion (image) coming out from behind the moving object. Therefore, for example, in the case of a B picture (bidirectionally coded image), it is going to encode the already decoded temporally forward or backward image after motion compensation and a compensated image formed by adding both of them. Of the four differences between the image and the image that does not take the difference, that is, the image that is about to be encoded,
The one with the smallest data amount is encoded.

Next, the redundancy in the space axis direction will be described below.

The difference between the image data is not transmitted as it is, but is subjected to a discrete cosine transform (DCT) for each unit block of 8 × 8 pixels. The DCT expresses an image not by a pixel level but by how many frequency components of a cosine function are included. For example, a two-dimensional DCT is used.
According to T, the data of the unit block of 8 × 8 pixels is 8 ×
8 is converted into a coefficient block of the component of the cosine function.
For example, an image signal of a natural image captured by a television camera is often a smooth signal. In this case, the data amount can be efficiently reduced by performing DCT processing on the image signal.

Here, FIG. 2 shows the structure of data handled by the above-mentioned encoding method. That is, the data structure shown in FIG. 2 includes a block layer, a macroblock layer, a slice layer, a picture layer, and a GOP (Group
of Picture) layer and a video sequence layer. Hereinafter, description will be made in order from the lower layer in FIG.

First, in the block layer, as shown in FIG. 2F, a block of the block layer is composed of 8 × 8 pixels (8 lines × 8 pixels) having adjacent luminance or color difference. The DCT (discrete cosine transform) described above is
It is applied to each unit block.

In the macroblock layer, as shown in FIG. 2E, the macroblocks in the macroblock layer include four luminance blocks (luminance unit blocks) Y0, Y1, Y2, and Y3 adjacent to each other in the left, right, up, and down directions. On the image, there are a total of six blocks including color difference blocks (color difference unit blocks) Cr and Cb which are located at the same position as the luminance block. The order of transmission of these blocks is Y0, Y1, Y
2, Y3, Cr, and Cb. Here, in the coding method, what is used for a predicted image (a reference image for obtaining a difference), whether or not the difference need not be transmitted, and the like are determined in units of macroblocks.

As shown in FIG. 2D, the slice layer is composed of one or a plurality of macro blocks connected in the scanning order of the image. At the beginning of this slice (header), the difference between the motion vector and the DC (direct current) component in the image is reset, and the first macroblock has data indicating the position in the image, so that an error occurs. It is designed to allow you to return if something happens. Therefore, the length of the slice and the starting position are arbitrary, and can be changed according to the error state of the transmission path.

In the picture layer, the picture, ie, 1
As shown in FIG. 2C, each single image is composed of at least one or a plurality of slices. Then, according to the coding method, each of the above-described intra-coded image (I picture or I frame), forward predicted coded image (P picture or P frame), and bidirectional predicted coded image (B picture or B frame) ) And a DC intra-coded image (DC coded (D) picture).

In the case of an intra-coded image (I picture), only information that is closed in one image is used when the image is coded. Therefore, in other words, when decoding, an image can be reconstructed using only the information of the I picture itself. Actually, encoding is performed by DCT processing without taking a difference. This encoding method is generally inefficient, but if it is included everywhere, random access and high-speed reproduction can be performed.

In the forward prediction coded image (P picture), a prediction image (a reference image for taking a difference) is
Use the previously decoded I-picture or P-picture located earlier in the input. In practice, the more efficient one of coding the difference from the motion-compensated predicted image and coding the difference without taking the difference (intra code) is selected for each macroblock.

In the bidirectional predictive coded image (B picture), three types of predictive images are used: an I-picture or a P-picture which is located earlier in time and has already been decoded, and an interpolated image formed from both of them. I do. This makes it possible to select the most efficient one of the three types of difference coding after motion compensation and the intra code in macroblock units.

A DC intra-coded image is an intra-coded image composed of only DC coefficients in DCT, and cannot exist in the same sequence as the other three types of images.

As shown in FIG. 2B, the GOP layer is composed of one or more I pictures and zero or more non-I pictures. The interval between I pictures (for example, 9) and the interval between I or B pictures (for example, 3) are free. Further, the interval between the I picture and the B picture may be changed inside the GOP layer.

As shown in FIG. 2A, the video sequence layer includes one or a plurality of GOs having the same image size and image rate.
It is composed of a P layer.

As described above, when transmitting a moving picture standardized by the high-efficiency coding method based on MPEG, a picture obtained by compressing one picture in a picture is sent first, and then this picture is moved. The difference from the compensated image is transmitted.

However, it has been found that the following problem occurs when the image to be encoded is an interlaced image.

If an interlaced image is coded as a picture in units of fields, the vertical position will be alternately different between fields. Therefore, when transmitting the still part of the moving image,
Since the difference information is generated every time the field is changed and must be transmitted, the coding efficiency is reduced in a still portion of the moving image.

Further, when the encoding process is performed in units of fields, blocks are formed in units of fields, so that the intervals between pixels are widened and the correlation is reduced as compared with the case where blocks are formed in units of frames. Efficiency decreases.

On the other hand, if an interlaced image is coded as a picture on a frame-by-frame basis, a so-called comb-shaped image must be processed for a moving part in the frame. For example, as shown in FIG. 3, when there is a moving object CA such as a car in front of a stationary moving object, I
Looking at the frame, there is movement between the fields, and such a portion becomes a comb-shaped KS image as shown in FIG. For this reason, a high-frequency component that does not exist in the original image is transmitted, and the coding efficiency is reduced.

Further, in the encoding process for each frame,
Since two consecutive fields constituting one frame are encoded together, predictive encoding cannot be used between the two fields. For this reason, the minimum distance of the predictive coding is one frame (two fields), so that the motion is faster or the motion is more complex than the field-based coding process in which the minimum distance of the predictive coding is one field. For an image, encoding processing on a frame basis is disadvantageous.

As described above, the coding efficiency may be reduced in each of the coding processing in the field unit and the coding processing in the frame unit. On the other hand, in the other case, the coding efficiency is high.

[0029]

SUMMARY OF THE INVENTION Accordingly, the present invention has been proposed in view of the above-mentioned circumstances, and has been proposed for an interlaced image, an image having a small amount of motion, an image having a large amount of motion, and an image in which both are mixed. It is an object of the present invention to provide an image data decoding method and an image data decoding device for decoding image data obtained by encoding.

[0030]

According to the present invention, there is provided an image data decoding method for decoding image data which has been subjected to an encoding process in units of blocks each having a two-dimensional arrangement of a plurality of pixels, and having an interlaced structure. A first flag for identifying that the image data provided in a part of the header of the image data has been processed in units of fields or frames when obtaining an image of one frame including one field; A second flag for identifying that the discrete cosine transform of the block has been performed in a field unit or a frame unit is detected, and the decoding process is performed in a field unit based on the first and second flags. when,
When the inverse discrete cosine transform of a block is performed on a field basis or the decoding process is performed on a frame basis, the inverse cosine transform of the block is adaptively switched on a field basis or a frame basis.

The above-described image data has a hierarchical structure. In the image data decoding method according to the present invention, the first and second flags are detected from a header of a predetermined layer of the image data, and the first and second flags are detected. , The decoding process is adaptively switched on a field-by-field or frame-by-frame basis for each predetermined layer.

Image data according to the present invention for decoding image data which has been encoded in units of a block consisting of a two-dimensional array of a plurality of pixels to obtain an image of one frame consisting of two fields having an interlaced structure The decoding apparatus includes a first flag provided in a part of a header of a predetermined layer of the image data, the first flag indicating whether an image included in the layer is processed in a field unit or a frame unit, and a first flag of each block. Detecting means for detecting a second flag for identifying that the discrete cosine coding has been processed in field units or frame units; and image data based on the first and second flags from the detecting means. And a judging means for judging whether decoding is possible.

Further, according to the present invention, decoding is performed on image data that has been encoded in units of blocks each having a two-dimensional arrangement of a plurality of pixels to obtain one frame image composed of two fields having an interlaced structure. The image data decoding apparatus includes a first flag provided in a part of a header of a predetermined layer of the image data for identifying that the image data included in the layer has been processed in a field unit or a frame unit. Detecting means for detecting that the discrete cosine transform of each block has been processed on a field or frame basis; and a second flag for detecting the first and second flags from the detecting means. Decoding means for selecting one of field processing and frame processing for each predetermined layer for decoding.

[0034]

Embodiments of the present invention will be described below with reference to the drawings.

1. Image Data Encoding Method / Image Data Decoding Method A description will be given of encoding / decoding processing in units of fields and frames to which the present invention is applied.

In the case where the picture shown in FIG. 1 described above is a field unit, the picture is shown in FIG. 5, for example, in consideration of the field structure. In FIG. 5, the upper part is the first
A field (for example, an odd-numbered field) represents a second field (for example, an even-numbered field). 1 /
Two fields temporally adjacent to each other at an interval of 60 seconds constitute a frame. Then, in the encoding / decoding processing in field units, each picture is encoded / decoded in field units.

Also, the I picture (intra-coded picture) / P picture (forward predicted coded picture) / B picture (bidirectional predicted coded picture) in the GOP layer shown in FIG. 2B described above.
FIG. 6 shows a specific example in which the configuration pattern is changed. FIGS. 5 and 6 differ from each other only in the configuration pattern of the pictures in the GOP layer, and are the same in that the encoding / decoding processing is performed in field units. By the way, as shown in FIG. 6, when the encoding type of the first field and the second field is the same, if the first field and the second field are collectively encoded / decoded, the code per frame is The decoding / decoding process is shown in FIG.

Various variations are conceivable for the motion prediction / compensation in the encoding / decoding processing shown in FIGS. 5, 6, and 7. A simple concrete example is shown in FIGS.
It is shown in FIG. In these figures, thick dashed arrows such as picture I2 to picture P5 shown in FIG. 8, for example, picture I3 to picture P6 shown in FIG. 9, indicate motion prediction to the P picture, and for example, pictures shown in FIGS. A thin broken arrow from I2 to picture B0 indicates a motion prediction to a B picture. 10A and 10B,
A solid arrow indicates motion prediction of a macroblock having a frame configuration described later, and a broken arrow indicates motion prediction of a macroblock having a field configuration.

In the encoding / decoding processing on a field basis, a P picture, for example, picture P5 shown in FIG.
Uses the previously decoded picture I2 located earlier in time as a predicted image (a reference image for taking a difference), and the picture P8 is located earlier in time as a predicted image and has already been decoded. Used picture P5. Further, for example, the picture P6 shown in FIG. 9 uses the picture I3 as the prediction image, and the pitcher P7 uses the picture P6 as the prediction image. Next, a B picture, for example, FIG.
The picture B4 shown in (1) uses three types of pictures I2 or P5, which are located earlier in time and have already been decoded, as a predicted picture, and an interpolated picture created from both of them. Also, for example, picture B4 shown in FIG. 9 uses three types of picture I3 or picture P6 and an interpolated picture created from both of them as a predicted picture.

On the other hand, in the encoding / decoding processing in units of frames, a P picture (frame), for example, a frame composed of pictures P6 and P7 shown in FIG. , And a frame composed of pictures P10 and P11 uses a frame composed of pictures P6 and P7 as a predicted image. Next, a B picture (frame), for example, a frame composed of pictures B4 and B5 shown in FIG. 10B is a frame composed of pictures I2 and I3 which are located temporally earlier as a predicted picture and have already been decoded, or Three types of frames, composed of pictures P6 and P7, and interpolated images made from both of them are used.

As described above, the encoding / decoding processing in the field unit and the frame unit has the same encoding / decoding processing procedure. However, as described below, there is a difference between the block configuration and the motion prediction / compensation. There is.

(1) Block Configuration In the encoding / decoding processing for each frame, the first field and the second field are encoded / decoded collectively, so that a block composed of the first field and the second field is composed. Although processing can be performed on a field-by-field basis, a block is composed of only one of the fields.

(2) Motion prediction / compensation In the encoding / decoding processing for each frame, the first field and the second field are collectively encoded / decoded. Although motion prediction for two fields is not used, motion prediction from the first field to the second field is used in field-by-field processing.

Here, the details of the above (1) block configuration and (2) motion prediction / compensation will be described.

(1) Block Configuration FIG. 11 is a diagram showing the configuration of a block inside a macroblock in encoding / decoding processing in field units. As shown in FIG. 11, encoding / decoding in field units is performed. In the decoding process, a macroblock having a field configuration is composed of only the first field or only the second field.

On the other hand, FIG. 12 is a diagram showing the configuration of a block inside a macroblock in encoding / decoding processing in units of frames. In the encoding / decoding processing in units of frames, as shown in FIG. 12C, macroblocks having a frame configuration can be taken in addition to the macroblocks having the field configuration shown in FIGS. 12A and 12B. That is,
In a macroblock having a field configuration, as shown in FIG. 12A, encoding / coding in a field unit shown in FIG.
As shown in FIG. 12B, in addition to the first and second fields having the same macroblock configuration as the decoding process, as shown in FIG. 12B, the macroblock is divided into upper and lower blocks, and the upper half has only the first field and the lower half has Can be composed of only the second field. Further, as shown in FIG. 12C, the macroblock having the frame configuration is composed of a first field and a second field.

As described above, in the encoding / decoding processing in units of frames, macroblocks having a frame configuration can be used in addition to macroblocks having a field configuration in the encoding / decoding processing in units of fields.

The switching between the macroblock having the field configuration and the macroblock having the frame configuration is performed, for example, on a field-by-field basis determined by a coding method determination circuit 21 (see FIG. 23) constituting an image data coding apparatus described later. When reading out blocks from the buffer memories 7 and 8 by using identification information (hereinafter referred to as coding system information) for identifying whether the coding processing is the coding processing or the coding processing in frame units, the read address is controlled. be able to. In an image data decoding device to be described later, the inverse variable length coding circuit 31 (see FIG. 27) writes (superimposes) in a coded bit stream (Bitstream) received from the image data coding device or the like. The flag is detected, and based on the flag, it is determined whether the coding method is a field unit or a frame unit. The information is supplied to the motion compensation circuits 42, 43 and the like to read out the buffer memories 37, 38. It can be realized by controlling the address.

That is, the buffer memories 7, 8,
For example, 480 as shown in FIGS.
In the frame configuration, the image data is stored in a frame configuration, and in the field configuration, the image data is stored in a field configuration, as shown in FIG. . FIG.
At 4, the two fields need not be consecutive in time. In this example, the size of the buffer memory is set to the size of a frame. However, the size is not limited and may be larger, or a configuration in which a plurality of frames are stored may be used. Further, in the block diagrams of FIGS. 23 and 27, the buffer memory is divided into two in order to make it easy to cope with the encoding / decoding processing. However, it is not necessary to divide the buffer memory in an actual configuration. They may be combined into one buffer memory.

(2) Motion Prediction / Compensation In the coding / decoding processing in the field unit, at the time of motion prediction / compensation, for example, prediction from picture I2 shown in FIG. 8 to picture B3 or picture prediction shown in FIG. Like the prediction from P6 to picture P7, motion prediction from the first field to the second field belonging to the same frame is used.

However, in the encoding / decoding processing on a frame basis, as shown in FIG. 7, since two fields are encoded / decoded collectively, the second field belonging to the same frame is shifted from the first field. Does not use motion prediction.

As described above, the encoding /
Since the motion prediction in the decoding process uses the motion prediction from the first field to the second field belonging to the same frame, the minimum interval between the pictures to be motion predicted is short, and the motion prediction in the encoding / decoding process in frame units. Is included. In the specific example described above, the motion prediction of a macroblock having a frame configuration is specially shown. However, the motion prediction is performed only for the same motion prediction for two fields in the macroblock. Two motion predictions can be substituted. Further, the motion prediction on a frame basis is not a motion prediction indispensable for the encoding / decoding processing on a frame basis, but may be only a motion prediction on a field basis.

As described above, in the method of encoding / decoding an interlaced image according to the present invention, the encoding / decoding processing in units of fields and frames is performed by using both a block configuration and a method of controlling motion prediction. Encoding / decoding in both methods is possible by corresponding to the encoding / decoding processing.

In this case, it is necessary to notify the image data decoding apparatus to what extent the image data encoding apparatus has performed which encoding processing, that is, the field unit or the frame unit.

In order to realize this, according to the present invention, a flag indicating whether a certain range of the coded image is processed in units of fields or frames is included in a part of the coded bit stream (coded image data). Is provided. In addition,
The certain range is, for example, a sequence, a GOP, or a picture. Specifically, in the image data encoding device, identification information for identifying whether a certain range of an image is encoded in a field unit or a frame unit is set as a flag at a predetermined position in an encoded bit stream as a flag. Set (superimpose). In the image data decoding device, by decoding a predetermined position of the coded bit stream by the inverse variable length coding circuit 31 as described above, a unit of decoding processing is determined, and decoding is performed based on the unit. , Images can be played.

Incidentally, the motion prediction in the encoding / decoding processing in field units and frame units is not limited to the specific examples shown in FIGS.
For example, encoding / decoding processing in units of fields using motion prediction shown in FIGS. 15 and 16 can be switched between encoding / decoding processing in units of frames and a certain layer of an image.

More specifically, FIG. 15A shows the GO shown in FIG.
FIG. 9 is a diagram illustrating a specific example different from the P picture prediction method of the field unit encoding / decoding processing illustrated in FIG. 8 in the configuration pattern of the I / P / B picture in the P layer;
FIG. 5B is a diagram illustrating a specific example different from the B picture prediction method in the encoding / decoding processing in the field unit illustrated in FIG. 8. That is, for example, picture P5 is used as a predicted image of picture P8 in the specific example shown in FIG.
2 or the like may be used. For example, as the predicted image of the picture B4, in the specific example shown in FIG. 8, three types of the picture I2 or the picture P5 and the interpolated image made from both of them are used, as shown in FIG. 15B. , Picture I2, picture P5, picture P
8 or the picture P11 and an interpolated image synthesized therefrom may be used.

FIG. 16A shows the I / O in the GOP layer shown in FIG.
FIG. 16B is a diagram illustrating a specific example different from the P picture prediction method of the field unit encoding / decoding processing illustrated in FIG. 9 in the configuration pattern of the P / B picture, and FIG.
FIG. 21 is a diagram illustrating a specific example different from the B picture prediction method in the encoding / decoding processing in the field unit illustrated in FIG. That is, for example, the picture I3 is used as the predicted image of the picture P6 in the specific example illustrated in FIG. 9, but the picture I2 or the like may be used as illustrated in FIG. 16A. For example, as a predicted image of the picture B4, in the specific example shown in FIG. 9, three types of a picture I3 or a pitcher P6 and an interpolated image made from both are used, as shown in FIG. 16B. Alternatively, a picture I2, a picture I3, a picture P6 or a picture P7, and an interpolated image synthesized therefrom may be used.

Further, for example, the encoding / decoding processing in units of fields and the encoding / decoding processing in units of frames may be combined.

For example, FIGS. 17 and 20 show specific examples in which a plurality of pictures are encoded / decoded in frame units, and then the subsequent pictures are encoded / decoded in field units. FIG. 17 is a diagram showing a combination of the encoding / decoding processing example of the frame unit shown in FIG. 7 and the encoding / decoding processing example of the field unit shown in FIG. 5, and FIG. 20 is shown in FIG. FIG. 7 is a diagram showing a combination of an example of frame encoding / decoding processing and an example of field encoding / decoding processing shown in FIG. 6.

First, in FIG. 17, for example, picture P
As shown in FIG. 18, the frame composed of P6 and P7 uses a frame composed of pictures I2 and I3 as a predicted image, and for example, picture P10 uses picture P6 and the like as a predicted image. Is also good.

On the other hand, for example, a frame composed of pictures B4 and B5 is, as shown in FIG.
2 or I3 frames or pictures P6, P
For example, picture B8 is composed of picture P6, picture P7 or picture P1 as shown in FIG.
0 and an interpolated image synthesized therefrom may be used.

In FIG. 20, for example, as shown in FIG. 21, a frame composed of pictures P6 and P7 uses a frame composed of pictures I2 and I3 as a predicted image. For example, a picture P6 or the like may be used as an image. On the other hand, for example, as shown in FIG. 22, a frame composed of pictures B4 and B5 is a frame composed of pictures I2 and I3 or a frame composed of pictures P6 and P7 and an interpolated image synthesized therefrom. For example, as shown in FIG. 22, the picture B8 may use a picture P6, a picture P7, a picture P10 or a picture P11, and an interpolated image synthesized therefrom.

Thus, as shown in FIGS.
There is no problem in combining the encoding / decoding processing in frame units and the encoding / decoding processing in field units. In other words, in the image data encoding method and the image data decoding method according to the present invention, the motion of the interlaced image is switched by switching between the encoding / decoding processing in frame units and the encoding / decoding processing in field units. Efficient coding can be performed for a small number of images, a large amount of motion, and an image in which both are mixed.

2. Image Data Encoding Device FIG. 23 is a block diagram showing a specific circuit configuration of an image data encoding device to which the present invention has been applied.

This image data encoding device (encoder)
Has a local decoding circuit including an inverse quantization circuit 2 to a gate 17 having the same circuit configuration as an image data decoding device described later, as shown in FIG.

First, when image data of a picture (field or frame) is input via the terminal 1, these image data are temporarily stored in the buffer memory 18. Specifically, for example, as shown in FIG. 24A, image data is input in the order of pictures I0, B1, B2, P3,..., And rearranged in the encoder processing order as shown in FIG. 24B. Motion prediction as shown in FIG. 1 described above is performed between the rearranged pictures. The input pictures I0, B1, B2, P3,.
Corresponds to a field in FIGS. 5 and 6, and corresponds to a frame in FIG.

The rearranged pictures are used for detecting a motion vector in the motion vector detecting circuit 19. The motion vector detection circuit 19 detects a motion vector necessary for generating a picture for prediction based on the already coded picture. That is, the front picture and the rear picture are stored in the buffer memory 18, and the motion vector is detected between the current reference picture and the current picture. Here, in the detection of the motion vector, for example, a motion vector that minimizes the sum of absolute values of the differences between pictures in macroblock units is used as the motion vector.

The sum of the absolute value of the motion vector and the difference between pictures in macroblock units is calculated by the
Sent to 1. The encoding method determination circuit 21 uses a later-described algorithm to encode a picture of a certain layer,
That is, it is determined whether the encoding process is in field units or in frame units. The information of the coding system (field unit or frame unit) and the motion vector are sent to the motion compensation circuits 12 and 13, the variable length coding circuit 25 and the like, and used for managing the buffer memories 7 and 8,
It is transmitted to an image data decoding device described later. Also, the information on the encoding system is sent to the control circuit 16.
The control circuit 16 outputs a control signal for controlling the encoding method to the gates 5 and 17 and the changeover switches 6, 9, 10 and 15.

Further, the sum of the absolute value of the motion vector and the difference between pictures in the macroblock unit is sent to an intra / forward / backward / bidirectional prediction determination (hereinafter simply referred to as prediction determination) circuit 20. The prediction determination circuit 20 determines the prediction mode of the reference macroblock based on this value.

This prediction mode is controlled by the control circuit 16
Sent to Based on this prediction mode, the control circuit 16 outputs a control signal to the gates 5, 17 and the like so as to switch between intra-picture / forward / backward / bidirectional prediction in macroblock units. Further, the control circuit 16 sends a motion vector corresponding to the selected prediction mode to the buffer memories 7 and 8 to generate a predicted image. Specifically, in the case of the intra-picture encoding mode, the input image itself is generated, and in the case of the forward / backward / bidirectional prediction mode, each predicted image is generated. The specific operations of the gates 5 and 17 and the changeover switches 6, 9, 10, and 15 are described in the gate 3 of the image data decoding apparatus described later.
5, 47 and the changeover switches 36, 39, 40, 45 are the same as those of FIG.

The predicted images from the buffer memories 7 and 8 are
The difference data is supplied to the difference unit 22, and the difference unit 22 generates difference data between the predicted image and the current image. The difference data is obtained by a discrete cosine transform (DCT).
m).

The DCT circuit 23 uses the two-dimensional correlation of the image signal to perform a discrete cosine transform of the human-powered image data or the difference data in block units, and obtains the resulting D
The CT conversion data is supplied to the quantization circuit 24.

The quantization circuit 24 quantizes the DCT transform data with a quantization step size (scale) determined for each macroblock and slice, and performs a so-called zigzag scan on the result. Then, the obtained quantized data is subjected to variable length coding (hereinafter referred to as VL).
C: Variable Length Code) circuit 25 and the inverse quantization circuit 2.

The quantization step size used for quantization is:
By feeding back the remaining buffer amount of the transmission buffer memory 26, the transmission buffer memory 26 is determined to have a value that does not fail. This quantization step size is also V
The data is supplied to the LC circuit 25 and the inverse quantization circuit 2 together with the quantized data.

Here, the VLC circuit 25 generates the quantized data,
Performs variable-length coding on the quantization step size, prediction mode, motion vector, coding system information, etc., and adds the coding system information (field unit or frame unit) to the header of a predetermined layer and transmits it. The data is supplied to the transmission buffer memory 26 as data.

The transmission buffer memory 26 temporarily stores the transmission data in the memory, outputs the bit stream as a bit stream at a predetermined timing, and controls the quantization control signal in units of macro blocks in accordance with the amount of residual data remaining in the memory. Is fed back to the quantization circuit 24 to control the quantization step size. Thus, the transmission buffer memory 26 adjusts the amount of data generated as a bit stream, and maintains an appropriate remaining amount of data (a data amount that does not cause overflow or underflow) in the memory. .

For example, when the remaining amount of data in the transmission buffer memory 26 increases to the allowable upper limit, the transmission buffer memory 2
Numeral 6 reduces the data amount of the quantized data by increasing the quantization step size of the quantization circuit 24 by the quantization control signal.

On the contrary, the transmission buffer memory 26
When the remaining data amount decreases to the permissible lower limit, the transmission buffer memory 26 stores the quantization circuit 24 in response to the quantization control signal.
By reducing the quantization step size, the data amount of the quantized data is increased.

As described above, the bit stream from the transmission buffer memory 26 whose data amount has been adjusted is multiplexed with the encoded audio signal, synchronization signal, and the like.
Further, after a code for error correction is added and a predetermined modulation is applied, it is recorded as so-called uneven pits on a master disk via, for example, a laser beam. Then, a so-called stamper is formed using the master disk,
Further, a large number of duplicate disks (for example, image recording media such as optical disks) are formed by the stamper. Further, the image data is transmitted to, for example, an image data decoding device described later via a transmission path. The image recording medium is not limited to an optical disk or the like, but may be a magnetic tape, for example.

On the other hand, the output data of the inverse DCT circuit 3 is added to the predicted image by the adding circuit 4, and local decoding is performed. The operation of the local decoding is the same as that of the image data decoding device described later, and the description is omitted here.

The picture configuration pattern is not limited to the pattern shown in FIG. 3. For example, if the encoder processing order is different, the picture configuration pattern is also different. More specifically, there are various variations in the processing order of encoding a P picture and a B picture temporally interposed therebetween, but these are merely changes in the processing order. You can respond by changing the control.

Here, a specific algorithm of the encoding method determining circuit 21 will be described with reference to a flowchart shown in FIG.

The encoding system determination circuit 21 is a first field (eg, an odd field) of the current frame to be encoded.
, A coding method is selected using a motion vector to a second field (for example, an even field).

More specifically, in step ST1, the coding method determination circuit 21 obtains this motion vector for all macroblocks in the frame to be coded, and proceeds to step ST2.

In step ST2, the coding method determining circuit 21 determines whether the horizontal (x) component and the vertical (y)
The median of the component is obtained, and the process proceeds to step ST3. Specifically, the median of the horizontal component is calculated as follows. First, the horizontal components of the motion vector are arranged in descending order. Then, the value of the data at the center is set as the median Mv _{x of the} horizontal component. Similarly, the median M of the vertical component
Find v _y .

In step ST3, the encoding method determining circuit 21 determines the vector MV (M
v _x , Mv _y ) is a parameter representing the motion of the entire surface on both sides. Therefore, the magnitude R of the vector MV is obtained as a parameter representing the magnitude of the motion on the entire surface, and the step ST
Proceed to 4. This R is obtained by the following equation 1.

R = | MV | = sqrt (Mv _x ² + Mv _y ² ) Equation 1 In step ST 4, the coding scheme determination circuit 21
The coding system is switched based on R obtained in step ST3. For a fast-moving image, a field-based coding method (coding process) is advantageous, and for a low-movement image, a frame-based coding method (coding process) is advantageous. If it is equal to or less than the darkness value TH, the process proceeds to step ST5, and an encoding method for each frame is selected. Otherwise, the process proceeds to step ST6, where an encoding method for each field is selected.

Thus, in the image data encoding apparatus to which the present invention is applied, specifically, a motion vector between the first field and the second field of the same frame, or a median of the motion vector according to the motion of the image. By deciding whether to perform the encoding process on a field basis or on a frame basis based on the size, it is possible to efficiently encode an image with little motion, an image with much motion, or an image in which both are mixed. Can be performed.

3. Image Data Format Next, a specific example of a bit stream output from the image data encoding device, that is, an image data format will be described. FIG. 26A is a diagram illustrating an example in which identification information (encoding scheme information) for identifying that encoding processing has been performed in frame units or field units is added as a flag to the header of the video sequence layer. is there. In the video sequence layer, as shown in FIG. 26A, a start code, a horizontal size, a vertical size, and the like are written in predetermined bits (indicated by numerals in the figure). In this example, after the frame frequency, identification information for identifying whether the encoding process has been performed in a field unit or a frame unit is added as a 1-bit flag. The position where this flag is added may be another location.

As shown in FIG. 26B, a flag may be added to the header of the GOP layer. That is, in this example, identification information is added as a 1-bit flag after the time code. The location where this flag is added may be another location as well.

FIG. 26 shows an example in which a flag is added to the MPEG standard. However, the flag may be written using an extension area of each layer which is predetermined by MPEG. For example, if it is a video sequence layer, it may be written in an extension area (Sequence Extension Byte)
It may be written in user data. Specifically, for example, as shown in FIG. 26C, a flag may be added to an extension area of a picture layer of the MPEG standard. At this time, a 2-bit flag is used to represent the encoding method. The 2-bit flag indicates the following information.

00: Picture coded on a frame basis 01: First field of a picture coded on a field basis 10: Second field of a picture coded on a field basis 11: Reserved

4. FIG. 27 is a block diagram showing a specific circuit configuration of an image data decoding device (decoder) to which the present invention is applied.

Inverse variable length coding (hereinafter, inverse VLC: Inver
The circuit 31 (se Variable Length Coding) reverses the bit stream supplied from the image data encoding device (encoder) shown in FIG. 23 and the bit stream obtained by reproducing an image recording medium such as an optical disk. The variable-length encoding process is performed and output to the inverse quantization circuit 32. Also,
At the same time, the inverse VLC circuit 31 decodes data such as a motion vector, quantization width (quantization step size), and encoding information that are written (superimposed) together with the image data at the time of encoding.

In particular, the inverse VLC circuit 31 decodes a flag added to the header of the video sequence layer, GOP layer, picture layer, or the like, and obtains information on whether the decoding process is performed on a frame basis or on a field basis. This information is supplied to a control circuit 46, which generates a control signal for frame processing or field processing. Specifically, the control circuit 46 generates various control signals corresponding to the output of the inverse VLC circuit 31, and connects the gate 35, the gates 47, the changeover switches 36, 39, 40, 45 and the like to predetermined contacts. Switch to the direction of.

That is, in the case of frame processing, if one picture is, for example, 720 (pixels) × 480 (lines) and one macroblock is 16 × 16 pixels, 1350 macroblocks are completed in one picture processing. To generate a control signal for processing the next picture. On the other hand, in the case of field processing, a control signal for processing the next picture is generated similarly with the completion of one picture processing for 675 macroblocks. Also, by managing the scheduling of the buffer memories 37 and 38, decoding is performed in frame units or field units.

The inverse VLC circuit 31 also sends a control signal indicating whether the decoding process is performed in frame units or field units to the motion compensation circuits 42 and 43, and the motion compensation circuits 42 and 43 , 38, decoding is performed in frame units or field units as described below.

The inverse quantization circuit 32 inversely scans and inversely quantizes the data subjected to the inverse VLC processing, and performs inverse DCT.
Output to the circuit 33. The inverse DCT circuit 33 performs an inverse DCT (Inverse Discrete Cosine Transform) process on the input data and outputs the processed data to the addition circuit 34. The predicted image data switched and selected by the changeover switch 45 is input to the addition circuit 34 via the gate 47. The predicted image data is added to the output data of the inverse DCT circuit 33, and the decoded image data is output. Generated.

When the output of the adder circuit 34 is an I picture or a P picture, the gate 35 is opened, and the decoded image data is transferred to the buffer memory 3 via the changeover switch 36.
7 or buffer memory 38 and stored.

More specifically, when the output of the adding circuit 34 is an I picture or a P pitcher, the changeover switches 39 and 4
0 is switched to the contact a side. The changeover switch 36 is alternately switched between a contact a and a contact b, and the picture (I picture or P picture) output from the addition circuit 34 is alternately stored in a pair of buffer memories 37 and 38.

For example, as shown in FIG. 24A, pictures I0, B1, B2, P3, B4, B5, P6, B
The image data arranged in the order of 7, B8, and P9 is converted into pictures I0, P3, B1, B2, P6, and B6 in an image data encoding device (encoder) as shown in FIG. 24B.
4, B5, P9, B7, and B8, the inverse V
As shown in FIG. 24D, data is also input to the LC circuit 31 in this order.

As a result, for example, as shown in FIG.
Assuming that the decoded data of the picture I0 is stored in the buffer memory 37, as shown in FIG. 24F, the decoded data of the picture P3 is stored in the buffer memory 38, and further, as shown in FIGS. Is updated to the data of the picture P6, and the data of the picture P3 in the buffer memory 38 is updated to the data of the picture P9.

Following the pictures I0 and P3, the picture B
When the data of picture 1 or picture B2 is input from the inverse DCT circuit 33 to the addition circuit 34, the data of the picture I0 stored in the buffer memory 37 is transferred to the motion compensation circuit 42.
In, after motion compensation corresponding to the motion vector,
The signal is supplied to the interpolation circuit 44. The data of the picture P <b> 3 stored in the buffer memory 38 is subjected to motion compensation corresponding to the motion vector in the motion compensation circuit 43, and then supplied to the interpolation circuit 44. The interpolation circuit 44 has the inverse VL
The respective inputs from the motion compensation circuits 42 and 43 are combined at a predetermined ratio in accordance with the data input from the C circuit 31. The synthesized data is selected by the changeover switch 45 and supplied to the adding circuit 34 through the contact b and the gate 47. The adding circuit 34 adds the data from the inverse DCT path 33 and the data selected by the changeover switch 45 to decode the picture B1 or the picture B2.

When the pictures B1 and B2 are decoded only from the previous picture I0, the changeover switch 45 is switched to the contact a side, and when the pictures B1 and B2 are decoded only from the subsequent picture P3, the changeover switch 45 is set to the contact c side. Is switched to
The data of the picture I0 or the picture P3 is supplied to the adding circuit 34.

The changeover switch 39 can be switched to the opposite side of the changeover switch 36. That is,
When the changeover switch 36 is switched to the contact a side (b side), the changeover switch 39 is switched to the contact b side (a side). Therefore, picture I0 is stored in buffer memory 3
7, the changeover switch 36 is switched to the contact b side, and when the picture P3 is stored in the buffer memory 38, the changeover switch 39 is switched to the contact a side. 24G, the picture I0 is read from the buffer memory 37 as shown in FIG.
0 to the display 41, and the display 41 displays a reproduced image. Adder circuit 34
When the pictures B1 and B2 are output, the changeover switch 40 is switched to the contact b side, and the pictures B1 and B2 are supplied to the display 41 as shown in FIG. Next, the changeover switch 39 is changed over to the contact b side and the changeover switch 40 is changed over to the contact a side, and as shown in FIGS. Supplied to

Here, a specific circuit configuration of the above-described inverse VLC circuit 31 will be described.

As shown in FIG. 28, the inverse VLC circuit 31 includes a barrel shifter 31a and a code decoding unit 31b, and the input code is processed by the barrel shifter 31a.
The data is sent to the code decoding unit 31b in units of 6 or 32 bits. The code decoding unit 31b includes a code table, a matching circuit, and the like (not shown), and performs matching between the input code and the code in the code table.
If there is a match, the code type is used to
ta) and its code length (CL) are output.

The data (data) is supplied to another circuit (not shown) of the inverse VLC circuit 31 and, after being subjected to appropriate processing, the control circuit 46 and the motion compensation circuit 43 described above.
And so on.

On the other hand, the length (CL) of the code is sent to the barrel shifter 31a as a shift amount to be shifted next, and the barrel shifter 31a stores the next code in accordance with the shift amount.
The data is output to the code decoding unit 31b in units of 6 or 32 bits.

Therefore, a flag for identifying a frame unit / field unit is also written in the code table as a header together with other codes.

Thus, in the image data decoding apparatus to which the present invention is applied, for example, image data provided in a part of a header such as a video sequence layer, a GOP layer or a picture layer is subjected to encoding processing in field units or frame units. The image data can be reproduced by detecting a flag for identifying the fact and performing a decoding process according to the flag.

The image data decoding apparatus according to the above-described embodiment is an apparatus that can execute both the field-based and frame-based decoding processing. For example, only one of the decoding processing is performed. In a device that cannot execute the process, it may be determined whether or not the device can perform decoding based on the flag.

More specifically, as shown in FIG. 27, for example, a judgment circuit 51 for judging whether or not decoding is possible based on a flag supplied from the inverse VLC circuit 31, and a result of the judgment are displayed. A display unit 52 is provided.

For example, in an image data decoding apparatus that performs decoding processing only in field units, for example, FIG.
In the image data decoding device performing the operation according to the flowchart shown in FIG.
For example, the operation according to the flowchart shown in FIG. 30 is performed.

That is, in the image data decoding apparatus that performs the decoding process only on a frame basis, the decision circuit 51 inputs a flag in step ST1, and then proceeds to step ST2.

In step ST2, the judgment circuit 51
Determines whether the flag is in frame units, and if so, proceeds to step ST4. If not, the process proceeds to step ST3.

In step ST3, the judgment circuit 51
Displays on the display unit 52 that decoding is not possible, and ends.

In step ST4, the judgment circuit 51
Indicates that decoding can be performed on the display unit 52. And
In step ST5, a decoding process is performed.

On the other hand, in the image data decoding apparatus that performs the decoding process for only the field, the decision circuit 51 inputs the flag in step ST1, and then proceeds to step ST2.
Proceed to.

In step ST2, the judgment circuit 51
Determines whether the flag is in field units, and if so, proceeds to step ST4. If not, the process proceeds to step ST3.

At step ST3, the judgment circuit 51
Displays on the display unit 5D that decoding is not possible, and ends.

In step ST4, the judgment circuit 51
Indicates that decoding can be performed on the display unit 52. And
In step ST5, a decoding process is performed.

As a result, the user can easily know the reason why the image data cannot be reproduced by the image data decoding device by looking at the display on the display section 52.

Tables 1 and 2 show specific examples of identification information for identifying whether the encoding process has been performed on a field basis or on a frame basis. This example is a so-called IS0 /
25-NOV-92 Test Model 3, Draft Re, issued as an N document by ICEJTG / SC29 / WG11 on November 25, 1992
This is the specification of the picture layer in vision 1.

[0126]

[Table 1]

[0127]

[Table 2]

[0128]

According to the present invention, image data that has been coded in units of blocks each having a two-dimensional arrangement of a plurality of pixels is decoded to obtain an image of one frame including two fields having an interlaced structure. At this time, a first flag for identifying that the image data provided in a part of the header of the image data has been processed in a field unit or a frame unit, and the discrete cosine transform of each block is performed in a field unit or a frame unit. And a second flag for identifying that the decoding process has been performed. Based on these first and second flags, when the decoding process is performed on a field basis, the inverse discrete cosine transform of the block is performed in the field. In units or when the decoding process is in frame units,
The inverse cosine transform of the block is adaptively switched on a field or frame basis. As a result, it is possible to decode image data obtained by encoding a low-motion image, a high-motion image, and an image in which both are mixed.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a relationship between pictures in MPEG.

FIG. 2 is a diagram showing a configuration of a video frame in MPEG.

FIG. 3 is a diagram showing a specific example of an image having a moving object.

FIG. 4 is a diagram showing an image blurred in a comb shape.

FIG. 5 is a diagram illustrating a specific example of an encoding / decoding process on a field basis.

FIG. 6 is a diagram illustrating another specific example of the encoding / decoding processing in the unit of a field.

FIG. 7 is a diagram illustrating a specific example of encoding / decoding processing in units of frames.

8 is a diagram illustrating a specific motion prediction method in the encoding / decoding processing in the unit of a field illustrated in FIG. 5;

9 is a diagram illustrating a specific motion prediction method in the encoding / decoding processing in the unit of a field illustrated in FIG. 6;

10 is a diagram illustrating a specific method of motion prediction in the encoding / decoding processing in units of frames illustrated in FIG. 7;

FIG. 11 is a diagram illustrating a configuration example of a block in a macroblock in encoding / decoding processing in field units.

FIG. 12 is a diagram illustrating a configuration example of a block in a macroblock in encoding / decoding processing in frame units.

FIG. 13 is a diagram illustrating a specific configuration example of a buffer memory of an image data encoding device to which the present invention has been applied.

FIG. 14 is a diagram illustrating a specific configuration example of a buffer memory of the image data encoding device.

15 is a diagram illustrating another specific motion prediction method in the field-based encoding / decoding processing illustrated in FIG.

16 is a diagram illustrating another specific motion prediction method in the field-based encoding / decoding processing illustrated in FIG.

FIG. 17 is a diagram illustrating an example of a specific combination of encoding / decoding processing in frame units and encoding / decoding processing in field units.

FIG. 18 shows P in the encoding / decoding processing shown in FIG.
FIG. 4 is a diagram illustrating a specific method of predicting motion of a picture.

19 is a diagram illustrating B in the encoding / decoding processing shown in FIG.
FIG. 4 is a diagram illustrating a specific method of predicting motion of a picture.

FIG. 20 is a diagram illustrating an example of another specific combination of encoding / decoding processing in frame units and encoding / decoding processing in field units.

21 is a diagram illustrating P in the encoding / decoding processing shown in FIG. 20;
It is a figure which shows the concrete method of the motion estimation of a pitcher.

FIG. 22 shows B in the encoding / decoding processing shown in FIG. 20;
FIG. 4 is a diagram illustrating a method of predicting motion of a pitcher.

FIG. 23 is a block diagram illustrating a specific circuit configuration of an image data encoding device to which the present invention has been applied.

FIG. 24 is a diagram illustrating a relationship between pictures input to the image data encoding device.

FIG. 25 is a flowchart illustrating an algorithm of an encoding method determination circuit included in the image data encoding device.

FIG. 26 is a diagram illustrating a specific format of a header of encoded image data.

FIG. 27 is a block diagram illustrating a specific circuit configuration of an image data decoding device to which the present invention has been applied.

FIG. 28 is a block diagram illustrating a specific circuit configuration of an inverse VLC circuit included in the image data decoding device.

FIG. 29 is a flowchart illustrating the operation of the image data decoding device to which the present invention has been applied.

FIG. 30 is a flowchart for explaining the operation of the image data decoding device to which the present invention is applied.

[Explanation of symbols]

31 inverse variable length encoding circuit, 32 inverse quantization circuit, 33
Inverse DCT circuit, 34 addition circuit, 35 gate, 36,
39, 40, 45 changeover switch, 37, 38 buffer memory, 42, 43 motion compensation circuit, 44 interpolation circuit,
46 control circuit, 51 judgment circuit, 52 display unit

Claims (4)

[Claims] 1. An image data decoding device which decodes image data which has been subjected to an encoding process in units of a block having a two-dimensional arrangement of a plurality of pixels, and obtains an image of one frame including two fields having an interlaced structure. A first flag provided in a part of a header of the image data for identifying that the image data is processed in a field unit or a frame unit; and a discrete cosine transform of each block is performed in a field unit. Alternatively, a second flag for identifying that the block has been performed in frame units is detected, and based on the first and second flags, when the decoding process is performed in field units, the inverse discrete cosine of the block is detected. If the conversion is performed on a field basis or the decoding process is performed on a frame basis, the inverse cosine transform of the block is performed on a field basis. Or the image data decoding method of performing switching adaptively in units of frames. 2. The image data has a hierarchical structure. The first and second flags are detected from a header of a predetermined layer of the image data, and the first and second flags are detected based on the first and second flags. 2. The image data decoding method according to claim 1, wherein the decoding process is adaptively switched on a field basis or on a frame basis for each predetermined layer. 3. An image data decoding device that decodes image data that has been subjected to encoding processing in units of a block composed of a two-dimensional array of a plurality of pixels to obtain an image of one frame including two fields having an interlaced structure. In the apparatus, a first flag provided in a part of a header of a predetermined layer of the image data and indicating whether an image included in the layer is processed in a field unit or a frame unit, A second for identifying that the discrete cosine coding was processed on a per-field or per-frame basis.
Detection means for detecting the first and second flags, and based on the first and second flags from the detection means,
A determination unit configured to determine whether the image data can be decoded. 4. An image data decoding device which decodes image data which has been subjected to an encoding process in units of a block having a two-dimensional arrangement of a plurality of pixels, and obtains an image of one frame including two fields having an interlaced structure. In the apparatus, a first layer provided in a part of a header of a predetermined layer of the image data for identifying that the image data included in the layer has been processed in a field unit or a frame unit.
And a second flag for identifying that the discrete cosine transform of each block has been processed in a field unit or a frame unit; and a first and a second flag from the detecting unit. An image data decoding apparatus, comprising: decoding means for selecting one of field processing and frame processing for each of the predetermined hierarchies for decoding in accordance with a flag.