JP3265290B2 - Apparatus and method for block decoding of interlaced scanned digital video signal - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to efficient information encoding of interlaced scanned digital video sequences for digital storage or transmission media.

[0002]

BACKGROUND OF THE INVENTION The results of efforts to standardize digital encoding methods for video signals are actively evolving today. Such standardization efforts include the study of digital encoding of various video signal formats at different bit rates. Depending on the scanning technique that determines how the video sequence is captured by the recording device and refreshed on the display device, the video sequence is classified into two types. That is, a progressive scan video sequence and an interlaced scan video sequence. In a progressive scan video sequence, the frames in the sequence are sequentially captured and refreshed line by line from the top to the bottom of the frame. A frame of the interlaced scanned video sequence consists of two fields, the even fields are formed from the even lines of the frame,
Odd fields are formed from odd lines of the frame. Acquisition and refreshing are performed sequentially from the top to the bottom of the field in the even field, and so on in the odd field. Since many sources are now of interlaced scan format (eg, NTSC, PAL), much research effort has been directed to efficient coding of interlaced scanned video sequences.

[0003] Conventional research results are classified into three types. That is, a frame encoding process, a field encoding process, and an interpolation field encoding process.
In the frame encoding process, for example, Kazuto K
"Consideration of B" by amikura and Taikun Li
it Allocation and Distribution of MovingIm
age Coding for Storage Media ", DAPA Meet
ing Report, Fourth Telematic Symposium, Tok
According to yo, February 1991, video sequences are basically coded frame by frame, in which case the even fields are interlaced (the frames are processed as if they were sequential). ). In the frame encoding process, each frame is partitioned into blocks of pixel data, which are then combined into an MPEG Video
Simulation Model Three, International Organ
ization for Standardization, Coded Present
ation of Picture and Audition Information, 199
0, processing by a block coding method such as discrete cosine transform (DCT) with or without motion compensation (MC), as described in ISO-IEC / JTC1 / SC2 / WG8 MPEG90 / 041 Can be. If a frame is encoded independently of the other frames in the sequence and without performing different processing on the even and odd fields, the process is called an intra-frame encoding process. If the frame is encoded under motion compensation using other frames in the sequence,
That process is called the interframe encoding process. In short, a frame encoding process is a process in which a video sequence is encoded by only an intra-frame encoding process or by a combination of an intra-frame and an inter-frame encoding process.

[0004] In the field encoding process, a video sequence is first divided into two sequences, an even field sequence and an odd field sequence.
Each of the even and odd field sequences undergoes the same encoding process (eg, Ichiro Andoh, Y. Yamada).
An external paper, "Some Considerations and Experimen
ts of Interlaced Moving Image Coding ”, DAP
A Meeting Report, Fourth Telematic Symposi
um, Tokyo, February 1991). The motion compensation or motion interpolation in the field coding process is independent of each of the even and odd field sequences or
It can be performed interdependently on two sequences as shown in a paper by chiro Andoh. Thus, the field encoding process is just an intra-field encoding process or a combination of an intra-field and an inter-field encoding process.

In the case of an interpolated field encoding process,
Only one field sequence is encoded and the other is reproduced by a prediction or interpolation process based on the encoded field sequence. An example of this encoding process is described in a paper by Heng-Ming Wang and Dimitris Anastassiou, entitled "High-Qualit
y Coding of the Even Fields Based on the
Odd Fields of the Enterlaced Video Sequen
ces ", IEEE Transaction on Circuits and
Systems, January 1991, and MPEG Video Si
mulation Model Three, Internationalizational Organiza
tion for Standardization, Coded Representation
on of Picture and Audito Information, 1990, I
It is described in SO-IEC / JTC1 / SC2 / WG8 MPEG90 / 041.

[0006]

When the objects in an interlaced scanned video sequence remain stationary, the correlation between adjacent pixels on each line and on mutually adjacent lines is high, so that a conventional discrete cosine transform code is used. Achieving a high compression ratio in a frame encoding process (using high correlation between lines) compared to a field or interpolated field encoding process by commonly known image compression methods such as encoding methods Can be. In this case, inter-frame motion compensation can also be performed with better efficiency. However, if the objects in the sequence start to move, the correlation between the fields will be reduced, and thus, for example, the efficiency of the DCT method will be better when a field coding process is used. Furthermore, when motion compensation is involved in this situation,
An inter-field coding process that performs motion compensation individually for each field has the advantage of providing better prediction of the motion of objects in the sequence. In general, the processes of frame coding, field coding, and interpolated field coding are only suboptimal in terms of efficiency if the motion of objects in the sequence can be very dynamic. Improving the efficiency of video coding schemes is particularly important in high bit rate reduction coding schemes. Also, in the encoding process,
If the motion compensation has been performed,
Performs inverse processing of the inter-frame motion compensation
Inter-field motion compensation in the
The inter-field in the decryption process
Decoding must be performed by performing inverse processing of the motion compensation processing.
I have to.

[0007]

SUMMARY OF THE INVENTION In order to utilize (intra / inter) frame encoding and (intra / inter) field encoding processes, it is necessary to determine which of the frame and field encoding processes to use. A method for adaptively inspecting each region of the frame can be devised. As such, stationary regions within the frame are encoded by taking advantage of the high spatial correlation within the frame and efficiently using the frame encoding process. Also, regions in the frame corresponding to moving objects in the sequence are coded using a field coding process as better correlation is found in these regions within each field.

The method first partitions each frame of the input interlaced scanned video signal into small blocks of pixel data, determines the difference in pixel values between the two fields in each of the blocks, If the error between the two fields is determined to be small, the block may be subjected to a frame encoding process, otherwise the block may be subjected to a field encoding process. In the decoding process, a video signal coded for each block is decoded, and the decoded video signal is converted into an image using a previous motion vector according to side information added for each block. or the reverse process of field motion compensation process, behind
Inter-field motion compensation using motion vectors
Perform inverse processing or use an interface using both motion vectors.
Either reverse the field motion compensation or
Either reverse the inter-frame motion compensation using the motion vector or use the
The inverse of the motion compensation process, or both motion vectors
The inverse of interframe motion compensation using
Do one .

[0009]

According to the above method, a block having motion between fields, that is, a field having a low correlation can be separated from a block having a high spatial correlation between the fields. A field coding process or a frame coding process may be optimally applied to achieve the best results. In the case of the frame coding process, the blocks are intra-frame coded using normal block coding techniques such as the DCT method, or using block coding techniques with inter-frame motion compensation. It can be inter-frame coded. As in the case of the field coding process, the two fields within the block are divided into sub-blocks and either intra-field coded using normal block coding methods or inter-field coded with inter-field motion compensation. Field encoded. Also, according to the above-described decoding method,
Then, it can be correctly decoded.

[0010]

DETAILED DESCRIPTION OF THE INVENTION Prior art motion compensation (eg, MP
EG Video Simulation Model Three, Internati
onal Organization for Standardization, Coded
Representation of Picture and Audio Infor
mation, 1990, see ISO-IEC / JTC1 / SC2 / WG8MPEG90 / 041)
In the interlaced scanning video sequence,
Frames, which can also be called images, can be classified into three types.
That is, (1) an intra-coded frame (ie, I
Frames)-each frame is coded using information only from itself; (2) predictive coded frames (i.e., P frames)-each frame is motion compensated from past I or P frames (3) bi-directionally predicted coded frames (ie, B-frames)-each frame is past and / or
Or encoded using motion compensated predictions from future I or P frames. FIG. 5 shows an example of the arrangement of three types of frames in the sequence. The number of P frames between I frames and the number of B frames between I or P frames need not necessarily be 4 and 2, respectively. Other suitable numbers may be used to encode the sequence. For example, there may be no B frames, or no B and P frames.

FIG. 1 is a block diagram of an interlaced scanning video signal encoding apparatus using the method according to one embodiment of the present invention. The input frame 1 of the sequence is first partitioned by a block partitioning process 2 into blocks of pixel data, each of which undergoes an interfield difference detection process 3 which checks for differences between pixel data from two fields in the block. In one example of the interfield difference detection process, the square error between even and odd line pairs in the block luminance region is calculated and the calculated result is calculated between successive odd and continuous even line pairs in the same region. This is compared with the calculated squared error. If the ratio of the square error between even and odd line pairs to the square error between successive odd and even line pairs is greater than a predetermined threshold value, the block 4 is processed by the field encoding process 5. If encoded, otherwise the block is encoded by a frame encoding process 6. The ratio of the two calculated squared errors is compared to a number of thresholds due to block activity,
A better detection can be made. 16x16
An example of an interfield difference detection process configured as a "C" program procedure for a block size of pixels is given as follows.

[0012]

[Table 1]

The block size, threshold values of 12, 18, 30, and activity values of 8,000, 100,000 in the above "C" program example can be adjusted or extended to other suitable values, if necessary. The result 7 of this interfield difference detection process is encoded as side information.

FIG. 2 shows an embodiment of an apparatus for performing the frame encoding process 6 and the field encoding process 5. In the frame encoding process, the input block undergoes an intra-frame encoding process including a frame reconstruction process 21, a block encoding process 22, and a block decoding process 23 if the input frame is an I-frame. If the input frame is not an I-frame, it undergoes an inter-frame encoding process including an inter-frame motion compensation process 20, a frame reconstruction process 21, a block encoding process 22, and a block decoding process 23. Similarly, in the field encoding process 5, the input block is an intra-field encoding including a field reconstruction process 25, a block encoding process 22, and a block decoding process 23 if the input frame is an I-frame. Process, otherwise, the interfield motion compensation process 26, the field reconstruction process 25, the block encoding process 2
2 and undergoes an interfield encoding process including a block decoding process 23. Frame memory 28
Stores the frame reconstructed from the result of the block decoding process 23, which is used in the interframe or interfield motion compensation processes 20, 26 of the input block from the next frame to be encoded.

FIG. 3 is a block diagram of an apparatus according to an embodiment of the inter-frame motion compensation process. The input block 27 to the inter-frame motion compensation process has one motion compensation process depending on the type of input frame:
If the input frame is a P frame, it undergoes an interframe motion estimation process, and if the input frame is a B frame, it undergoes an interframe motion interpolation process. In the inter-frame motion estimation process, the input block undergoes an inter-frame pre-motion vector detection process 41 for evaluating the previous motion vector 44.
In detecting the motion vector, an ordinary block comparison technique can be used. Next, a pre-motion prediction process 42 is performed, which basically derives a pre-motion displacement block from a previously encoded frame using the detected pre-motion vector. Finally, the result of the previous motion prediction process 42 is compared with the input block 27 in a motion compensation mode selection process 43 to determine which of the two blocks is to be coded. This is because motion compensation is not always successful, especially in regions where very fast changes occur in the video sequence. If it is determined that the input block is to be coded, the actual value of the input block is coded. Otherwise, the difference between the input block and the previous motion displacement block, ie, the previous motion compensated block, is encoded. An example of a procedure in the C language for the motion compensation mode selection process is as follows.

[0016]

[Table 2]

The inter-frame motion interpolation process includes a pre-interframe motion vector detection 41, a post-interframe motion vector detection 46, a pre-motion prediction 42, a pre / post-motion interpolation 47, a post-motion prediction 48, and a motion interpolation mode selection 4.
9 processes. With reference to past and future I or P frames (encoded or unencoded frames may be used), the inter-frame pre- and post-motion vector detection process uses pre- And evaluate the post-motion vector. Using the two estimated motion vectors, the pre- and post-motion prediction processes 42 and 48 find a pre-motion displacement block and a post-motion displacement block in the same manner as described above. In the forward / backward motion interpolation process 47, a forward / backward motion interpolation block is calculated by averaging the forward and backward motion displacement blocks. When averaging the forward and backward motion displacement blocks, weighting based on frame distance is used or not used. The three motion displacement and interpolation blocks described above, along with the input block 27, are compared in a motion interpolation mode selection process 49 to determine which block is to be coded. One way to determine which blocks are to be coded is to use the MPEG Video Simulation Model T
hree, International Organization for Standar
dization, Cod Representation of Picture an
d Audio Information, 1990, ISO-IEC / JTC1 / SC2 / WG8
By comparison of the mean square error described in MPEG90 / 041. If the determined block is not an input block, the difference between the determined block and the input block is encoded.

Since the two fields of data in the input block 29 to the field encoding process 5 are generally uncorrelated or include the motion selected by the interfield difference detection process 3, the two fields of data are based on the data from the two fields. Interfield motion compensation, which performs motion compensation independently, is selected. However, the embodiment of the inter-field motion compensation process shown in FIG. 4 is very similar to the inter-frame motion compensation process (FIG. 3), except for the motion vector detection process and the configuration of the motion displacement block. In the interfield pre / post motion vector detection processes 61 and 62, 1
One motion vector is evaluated for each field in the input block 30 and in the forward and backward directions, so that the front motion vectors 1 and 2 and the back motion vectors 1 and 2 are evaluated. Based on the estimated motion vectors 1 and 2, two motion displacement fields of the block are obtained from the previously coded image, whereby a motion displacement block and further a motion interpolation block are formed in processes 63, 64, 65 Is done. Mode selection processes 43 and 49 are the same processes used in interframe motion compensation process 20. Regarding the encoding of the previous motion vectors 1 and 2 or the subsequent motion vectors 1 and 2, since the motion difference between two fields in the input block and the motion difference between the previous adjacent coded blocks are not considered to be large. An efficient coding method is to delta-encode a one-field (before and / or after) motion vector by referring to the delta difference between the previous coded block and the two-field motion in the input block. It is to be.

The input block 4 or the output of the interframe motion compensation process is the output of the frame reconstruction process (FIG. 2).
21) where it is, for example, DCT
It is spatially partitioned into smaller sub-blocks suitable for a block coding process based on the (discrete cosine transform) method. FIG. 6A shows an example of a 16 × 16 block of luminance data (Y) divided into four smaller 8 × 8 sub-blocks. In the frame reconstruction process, the interlaced fields are not separated so that the block coding process can take advantage of the correlation between the fields. The frame reconstruction process may further include an intra-frame subsampling process that reduces the resolution of the color information (Cr / Cb). FIG. 6 (b)
In the example, an 8 × 16 color block (co-located with a 16 × 16 luminance block in a 4: 2: 2 source) is transformed into an 8 × 8 color block by averaging each even-odd line pair of that color block. Intra-frame sub-sampling is performed on the sub-block. Such sub-sampling of the chrominance signal is performed to reduce the amount of information that needs to be encoded without causing visible differences.

The field reconstruction process (reference numeral 25 in FIG. 2) comprises an interfield motion compensation process (FIG. 2).
Or the input block 4 receives the result. The process partitions the input luminance data (Y) into smaller sub-blocks separated by two fields. FIG. 7A shows that a 16 × 16 input luminance block is represented by 4 pixels.
FIG. 4 illustrates such a process of partitioning into four 8 × 8 sub-blocks, two of which contain even field data and two of which contain odd field data. This is done so that the block coding process (reference numeral 22 in FIG. 2) takes advantage of the high correlation in the field in each sub-block. The field reconstruction process further includes an intrafield subsampling process that reduces the resolution of the color information (Cr / Cb) in each field. In FIG. 7 (b), the example of an 8 × 16 color block is intrafield subsampled into 8 × 8 color subblocks by separately averaging all two consecutive lines in each field. These averaged lines are rearranged such that the two fields are separated in the output sub-block. These averaged lines are also arranged in an interlaced fashion in the output sub-block, but the resulting coding efficiency is not as good as the method described above.

Each sub-block generated by the frame or field reconstruction process is:
By examining the correlations within the sub-blocks, the block encoding process for data compression (reference numeral 2 in FIG. 2)
2) Accepted. The embodiment of the block coding apparatus shown in FIG. 8 performs a discrete cosine transform (DCT) on the sub-blocks into discrete cosine transform (DCT) coefficients.
0, the DCT coefficient is quantized 81 based on a rate control device 84 which monitors the amount of output information 85, and the quantized DC
Zigzag scans 82 the T-coefficients, performs variable length coding (VLC) 83 on the results of the zigzag scan and all the side information needed to describe the sub-blocks 83, coding information for digital storage media or transmission media 8
5 means. The details of the example of the device implementation of the block encoding process shown in FIG.
ideo Simulation Model Three, International
Organization for Standardization, Coded Rep
resentation of Pictureand Audio Information,
1990, found in ISO-IEC / JTC1 / SC2 / WG8 MPEG90 / 041.

The block decoding process (reference numeral 23 in FIG. 2) corresponds to the block encoding process (reference numeral 2 in FIG. 2).
Decode the output of 2) and reconstruct the frame. The frames reconstructed by the block decoding process are stored in a frame memory (reference numeral 28 in FIG. 2) and used by the interframe / interfield motion compensation process (reference numerals 20 and 26 in FIG. 2). . The block decoding process is also used to decode encoded information from digital storage media or transmission media created by the block encoding process and / or the motion compensation process.
FIG. 9 shows an embodiment of a block decoding device corresponding to the block encoding process shown in FIG.
The encoded information is demultiplexed / VLC decoded to reproduce the 90 zigzag scanned sub-blocks and the included side information. The sub-blocks are subjected to inverse zigzag scanning 91 and then inversely quantized 92 to reproduce DCT coefficients. And the coefficient is inverse DC
T93, the result of the inverse DCT is used to calculate the interframe or interfield according to the coding image type (I, P, or B frame) and the coding result of the interfield difference detection process (3 in FIG. 1). The reconstruction sub-block 96 is reconstructed by receiving the inverses 94 and 95 of the motion compensation process (FIGS. 3 and 4), and the inverses 97 and 98 of the frame / field reconstruction process according to the coding result of the interfield difference detection process.
To reproduce the reconstructed block 100 which is then stored in the frame memory 101 for use in the inverse motion compensation process 94 or 95 or for display on a monitor screen. If the block decoding process is to be used in a video signal encoder with the block encoding process, the input to the inverse quantizer and other side information is obtained directly from the corresponding processes in the block encoding process. , The demultiplexer / VLC decoder and the inverse zigzag scanner may be omitted.

[0023]

As described above, an input frame is partitioned into small blocks of pixel data, and two fields within each block are checked for correlation. If the difference between two fields in a block is small (ie, the correlation is high), the block is inter-frame, which is designed by a frame encoding process in which all lines in the block are scanned sequentially, and thus efficiently. It is encoded using a motion compensation process and an intra-frame sub-sampling process for color data. On the other hand, if the difference between two fields in a block is large (i.e., the correlation is low), then the block will have the (intra-field) correlation in each field and the (inter-field) between each field and its neighboring fields.
Correlation studies are coded by an enhanced field coding process, and in such blocks, intra-field subsampling of color data is better. In a typical video sequence, the level of correlation in each image frame is different between regions containing still scenes or objects and regions containing moving scenes or objects. Thus, the present invention provides the effect of encoding these two types of regions more efficiently or adaptively in a way that is more suitable for each type of region. In a fixed bit rate video coding system, the present invention provides the advantage of providing better reconstructed image quality.

[Brief description of the drawings]

FIG. 1 is a block diagram of an interlaced scan video encoding method according to an embodiment of the present invention.

FIG. 2 is a block diagram of a frame encoding process and a field encoding process, showing a part of the embodiment of the present invention;

FIG. 3 is a block diagram of an interframe motion compensation process, showing part of the embodiment of the present invention.

FIG. 4 is a block diagram of an interfield block motion compensation process, showing part of the embodiment of the present invention.

FIG. 5 is an explanatory diagram of an example of the arrangement and relationship of an I image, a P image, and a B image.

FIG. 6A is an explanatory diagram of an effect example of a frame reconstruction process. FIG. 6B is an explanatory diagram showing an example of an intra-frame subsampling process for color data.

FIG. 7A is an explanatory diagram showing an example of the effect of field reconstruction; FIG. 7B is an explanatory diagram showing an example of an intra-field subsampling process for color data;

FIG. 8 is a block diagram of a block encoding process according to one embodiment of the present invention.

FIG. 9 is a block diagram of a block decoding process in one embodiment of the present invention.

[Explanation of symbols]

 2 Block Partitioning Process 3 Interfield Difference Detection Process 5 Field Encoding Process 6 Frame Encoding Process 20 Interframe Motion Compensation Process 21 Frame Reconstruction Process 22 Block Encoding Process 23 Block Decoding Process 25 Field Reconstruction Process 26 Interfield Motion Compensation process

Claims (2)

(57) [Claims] 1. A Indicates were treated either frame encoding treated field coding when it is encoded, the first side information added to each block, the field marks
In each of the encoding process and the frame encoding process,
Whether motion compensation was performed using the previous motion vector or
Motion compensation using the motion vector, or
Motion compensation using both motion vectors
Second side added for each block
By using the information, a block decoder for decoding an interlaced scan digital video signal encoded, and decoding means for decoding a video signal coded for each block, the first and According to the second side information, the decoded video signal is subjected to inverse processing of the inter-field motion compensation processing using the front motion vector, or using the rear motion vector.
Reverse the inter-field motion compensation
Motion Compensation Processing Using One Motion Vector
Reverse processing, or use the previous motion vector
And either the inverse process of the inter-frame motion compensation process, behind
Inverse of inter-frame motion compensation using motion vectors
Process, or use the
A block decoding device for interlaced scanning digital video signals, comprising: a reverse motion compensation processing means for performing either one of a reverse process and a reverse motion compensation process. 2. A first side information added for each block, indicating whether field encoding processing or frame encoding processing has been performed at the time of encoding, and the field code
In each of the encoding process and the frame encoding process,
Whether motion compensation was performed using the previous motion vector or
Motion compensation using the motion vector, or
Motion compensation using both motion vectors
Second side added for each block
By using the information, a block decoding method for decoding interlaced scan digital video signal encoded, decoding a video signal coded for each block, the first and second According to the side information, the decoded video signal is subjected to the inverse processing of the inter-field motion compensation processing using the front motion vector, or the back motion vector is used.
Reverse the inter-field motion compensation
Motion Compensation Processing Using One Motion Vector
Reverse processing, or use the previous motion vector
And either the inverse process of the inter-frame motion compensation process, behind
Inverse of inter-frame motion compensation using motion vectors
Process, or use the
Performing a reverse operation of the frame motion compensation process . A method for decoding a block of an interlaced scanned digital video signal, the method comprising: