JPH1084531A - Method for reducing memory capacity required to decode two-way prediction coded frame during pull-down and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a required memory capacity by storing data from a top field and a bottom field separately to different segments of the memory and re-configuring the top field for a period when re-configuration is extended. SOLUTION: Coded video data are stored temporarily to a rate buffer 804 from a data channel 802 in a bit stream form. A symbol or an event is re-configured into an original frame by a re-configuration device 808, and the data frame are given to a picture buffer 801 via a data channel 810 from the re- configuration device 808, and B-frames are divided into three segments S1, S2, S3 of the picture buffer 801. The memory capacity required to store and display the B frame is smaller than the capacity for one frame. When any of the segments S1-S3 for the B frame is displayed, the segment is released for re-configuration data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to video systems, and more particularly, to memory partitioning to reduce the amount of memory required by a video decoder to decode and display bidirectionally coded frames with pulldown. And iterative operation of field reconstruction.

[0002]

BACKGROUND OF THE INVENTION Video program signals are converted to digital form and then compressed and encoded according to one of several known compression algorithms or methods. This compressed digital system signal (ie, bit stream), including the video portion, the audio portion, and other information portions, is sent to a receiver.
Transmission takes place over existing television channels, cable television channels, satellite communication channels and the like. Next, a decoder is typically used at the receiver to decompress and decode the received system signal according to the compression algorithm used to encode the signal. Next, the decoded video information is output to a display device such as a television (TV) monitor.

[0003] Video compression and encoding are typically performed by video encoders. Typically, a video encoder performs a selected data compression algorithm that conforms to approved standards or specifications agreed upon by the sender and the receiver of the digital video signal. Currently Moving Pic
tours Experts Group (MPEG)
One such standard being developed by the Company is commonly referred to as the MPEG-1 standard. A newer standard, MPEG-2, is similar to MPEG-1, but includes extensions that cover a wider range of applications. More specifically, MPEG-
2 is a standard that may also include high quality encoding of interlaced video signals, but such video includes high definition television (HDTV). It covers a wide range of applications, bit rates, resolutions, signal qualities, and services, but also covers all forms of digital storage media, TV broadcast, and communications.

To compress a video signal, it is typically necessary to sample analog data and represent this data in digital values of luminance and chrominance. M
The PEG standard specifies that a luminance component (Y) of a video signal is sampled at a ratio of 2: 1 (2: 1) with respect to a color difference signal (Cr, Cb). That is, for every two samples of the luminance component Y, there is one subsample for each of the color difference components Cr and Cb. A 2: 1 sampling rate is generally considered reasonable. This is because human eyes are more sensitive to luminance (brightness) components than color components. Video sampling is typically performed in the vertical and horizontal directions.

[0005] Once the video signal is sampled, it is typically formatted into a non-interlaced signal that contains all of the picture content. Specifically, the video signal includes a plurality of pictures (ie, frames),
Each frame includes a plurality of horizontal scan lines for display. In contrast, an interlaced signal is a signal that contains only a portion of the picture content for each complete display scan. In an interlaced signal, each frame is divided into two fields. These two fields are often called even and odd fields, or top and bottom fields. Each field covers the length of the frame, but contains only every other scan line. The purpose of such field division is that most TVs today display video information in an interlaced format, first displaying one field (e.g., the entire top field) and then the bottom field. This is for displaying the whole.

After the video signal has been sampled and formatted, the encoder may further convert it to a different resolution according to the image area to be displayed. In that case, the encoder must decide how to encode each picture. One picture may be considered one frame of motion video or one screen of motion picture film. However, a different coding scheme can be used for each picture. The most commonly used picture coding type is I-picture (intra-code
ed pictures, which are coded without reference to other pictures and are often anchor frames (an
ch picture), P picture (predictively coded picture or predictive-picture).
coded pictures or past I
Or encoded using motion compensated prediction from a P reference picture, also considered an anchor frame), and a B picture (bidirectionally coded picture or b
directionally predictive
-Coded pictures, which are coded using motion compensation from past and future I or P pictures). These picture types will be referred to as I frames, P frames, or B frames.

[0007] A typical coding scheme uses a mixture of I, P, and B frames. Typically, an I-frame occurs once every 0.5 seconds, with two B-frames inserted between each pair of I- or P-frames. I-frames provide a random access point from which decoding can begin within the encoded sequence of pictures, but with moderate compression. P frames are more efficiently encoded using motion compensated prediction from past I or P frames and are generally used for reference for future prediction. B-frames provide the highest degree of compression, but require past and future reference pictures for motion compensation. Generally, B frames are not used as reference pictures for prediction. Within a particular video sequence, the composition of the three picture types is very flexible. The fourth picture type is D picture or DC
The picture is defined by the MPEG standard.
This type is a simple, quality-limited fast forward (F
(ast-Forward) mode.

[0008] Once the picture type is defined,
The encoder can estimate a motion vector for each 16x16 macroblock in a picture. A macroblock (MB) is a basic coding unit of the MPEG standard. One macroblock has a luminance component (Y)
A 16 pixel by 16 line portion containing
It consists of two 8 × 8 line blocks and several spatially corresponding 8 × 8 blocks containing the chrominance components Cr and Cb. The number of blocks of chrominance values depends on the particular format used. As a normal color space sampling method, the highest quality is provided, but the compression degree is relatively small.
And 4: 2: 2 including two Cb chrominance blocks and Cr chrominance blocks, and 4: 2: 0 including two chrominance blocks. Multiple such macroblocks form a horizontal slice in the frame,
A slice is a basic processing unit in the MPEG coding method. A plurality of such slices form each picture or frame, which is the basic display unit.
However, as mentioned above, each frame is typically 2
Interlaced and displayed as two separate fields.

[0009] The motion vector provides displacement information between the current picture and a previously stored picture. P-frames use motion compensation to handle temporal redundancy (ie, absence of motion) between picture frames in the video. Obvious motion between consecutive pictures is caused by the pixels of the previous picture occupying different positions with respect to the pixels in the current macroblock. This displacement between the pixels in the previous macroblock and the current macroblock is represented by the motion vectors encoded in the MPEG bitstream. Typically,
The encoder determines the picture used for each given frame.
Select a type. After defining the picture type, the encoder estimates a motion vector for each macroblock in the picture. Typically, in a P frame,
One vector is used for each macroblock,
In a B frame, one or two vectors are used.

When the encoder processes a B frame,
Typically, the picture sequence is reordered so that the video decoder receiving the digital video signal can process it properly. Normally, the B frame is
As encoded using motion compensation based on frames or P-frames, B-frames can be decoded only after a subsequent anchor picture (I- or P-frame) has been received and decoded. Therefore,
The order of the series of pictures is rearranged by the encoder, and the pictures arriving at the decoder are in an order suitable for decoding the video signal. The decoder can then reorder the pictures into an order suitable for display.

The encoder selects an encoding mode for a given macroblock of video data based on the picture type, the effectiveness of motion compensation in a particular region of the picture, and the nature of the signal within the block. Is programmed as After the encoding mode is selected,
The encoder performs motion-compensated prediction of the block contents based on past or future reference pictures. Next, the encoder generates an error signal by subtracting the prediction from the actual data in the current macroblock. The error signal is similarly divided into 8x8 blocks (four luminance blocks and two chrominance blocks). Next, a discrete cosine transform (DCT) can be performed on each block to perform further compression. The DCT operation is
Transform the 8x8 block of pixel values into an 8x8 matrix of horizontal and vertical coefficients of spatial frequency. The coefficients representing one or more non-zero horizontal or non-zero vertical spatial frequencies are called AC coefficients. 8x consisting of pixel values
8 is later inverse DCTed for spatial frequency coefficients.
It can be reconstructed by a video decoder performing (IDCT).

Since the difference between the average values of adjacent 8 × 8 blocks tends to be relatively small, further compression is performed by predictive coding. Predictive coding is a technique used to increase compression based on blocks of pixel information previously manipulated by an encoder. For the block to be encoded, prediction of pixel values can be performed by the encoder. The difference between the predicted pixel value and the actual pixel value can be calculated and encoded. The difference value represents a prediction error, which can later be used in a video decoder to correct the information in the predicted block of pixel values.

[0013] In addition to the signal compression achieved by the encoding process itself, a significant degree of intentional signal compression can be achieved by choosing the quantization step size. The quantization interval or step is specified by an index. The quantization level of the frequency coefficient corresponding to the higher spatial frequency can easily create a coefficient value of zero by selecting an appropriate quantization step size based on the human visual recognition system. Specifically, the step size is selected so that the human visual recognition system does not recognize the missing spatial frequency unless the coefficient value for the particular spatial frequency exceeds a particular quantization level. Statistically encoding the expected run of successive zero-valued coefficients corresponding to higher order coefficients provides a significant degree of compression.

The non-zero coefficients are collected at the beginning of the series,
In order to encode as many zero coefficients as possible following the last non-zero coefficient in the order, the sequence of coefficients is arranged in a specific arrangement, often referred to as a zigzag order.
The zigzag order concentrates the highest spatial frequencies at the end of the series. Once the zigzag order is performed, the encoder typically performs "run-length encoding" on the AC coefficients. This process reduces each 8x8 block of DCT coefficients to the number of events represented by the number of non-zero and leading zero coefficients. Since the high frequency coefficients are likely to be zero, run length coding will further compress the video.

Next, the encoder can include a variable length decoder (VCD) to perform variable length coding (VLC) on the resulting data. VLC is a reversible data encoding procedure that allows short code
Assign words and use long codes for infrequent events
By allocating words, further video compression is achieved. Huffman coding, especially in the well-known VLC format, reduces the number of bits required to represent a data set without loss of information. Here, the final compressed video data is ready for transfer, and is transferred to a storage device or transmitted to a decoder at a remote place via a transmission medium to be received and decompressed. The MPEG standard defines a specific syntax for compressed bitstreams. The MPEG video syntax includes six layers, each supporting signal processing or system functions. The MPEG syntax layer corresponds to a hierarchical structure. A "sequence" is the highest layer of the video coding hierarchy and consists of a header and several "groups of frames" (GOPs). The sequence header typically initializes the state of the decoder so that the decoder can decode any sequence without being affected by past decoding history. GOP is a random access point. That is, it is the smallest coding unit that can be decoded independently in the sequence. Typically, a GOP consists of a header and some "pictures". GO
The P header contains time and editing information.

As mentioned above, there are at least three types of pictures or frames. That is, an I frame, a P frame, and a B frame. Because of the dependencies of pictures, the order in which frames are transmitted, stored, or retrieved is not necessarily the order of display, but the order necessary for the decoder to properly decode the pictures in the bitstream. For example, a typical frame sequence is as follows in display order:

I BB P B B P P B B P B B I B B P B B P 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

On the other hand, the bit stream order corresponding to the above display order is as follows.

[0019] I P B B P B B P B B I B B P B B P B B 0 3 1 2 6 4 5 9 7 8 12 10 11 15 13 14 18 16 17

The B frame is composed of the following I in the display order.
Depending on the frame or P frame, the I or P frame must be transmitted and decoded before the dependent B frame.

Each "picture" portion of the GOP consists of a header and one or more slices. The picture header contains a time stamp, picture type, and coding information. A slice is made up of an integer number of macroblocks from a picture and can be used by a video decoder to recover from decoding errors. If the bitstream becomes unreadable within a picture, the decoder typically does not need to drop the entire picture and can recover from the anomaly by waiting for the next slice. Further, within the slice there is a header containing the position and quantization scale information. DCT is applied at the block level because blocks are the basic coding unit. Typically, each block is 8x8
Includes 64 component pixels arranged in a matrix. It is not the individual pixel values that are encoded, but the components of the encoded block. In the macroblock header, there is information on quantization scale and motion compensation.

The video decoding process is generally the reverse of the video encoding process and reconstructs a moving image sequence from a compressed and encoded bit stream. The data in the bitstream is decoded according to the syntax, but the syntax itself is defined by a data compression algorithm. First, the decoder must identify the beginning of the encoded picture and identify the type of picture, and then decode the individual macroblocks within that picture. If motion vectors and macroblock types are present in the bitstream (each type of I-frame, P-frame, and B-frame has its own macroblock type), use them to Can construct a prediction of the current macroblock based on past and future reference frames already stored. The coefficient data is then dequantized and manipulated by an IDCT process to convert the macroblock data from frequency domain to time and space domain data.

When all of the macroblocks have been processed by the decoder, the picture reconstruction has been completed. If the reconstructed frame is a reference or anchor frame (eg, an I-frame or a P-frame), it replaces the oldest stored anchor frame and is used as the new anchor for subsequent frames. You. As described above, before the frames are displayed, they may need to be reordered to display order rather than encoding order. After the frames are reordered, they are displayed on a suitable display device.

In general, encoded video data is received and stored in a channel buffer (rate buffer). The data is then pulled from the channel buffer by a decoder or reconstructor performing the decoding process, and the decoded data is stored in a picture storage buffer. Depending on the device configuration, the channel
The buffer and the picture storage buffer are integrated into one memory device. The decoded data is in the form of an I-frame, P-frame, or B-frame,
The display controller retrieves the picture data and an appropriate display device (eg, a TV monitor) displays it. It should be noted that in systems prior to the system according to the invention, the picture storage buffer has the capacity to store at least three frames of video information (ie 3
One "frame store"). Two frame stores were required to store two anchor frames. These two anchor frames were used to reconstruct the B frame into a third framestore area.

Specifically, in order to reconstruct a B frame, the two associated anchor frames must be decoded and available in the picture storage buffer. This is because the B frame is interpolated using both anchor frames during the reconstruction process. Reconstruction of each B frame is performed progressively (i.e., in non-interlaced order) because the macroblock contains even and odd field information for each frame. However, the display of each frame is performed in the interlaced order, and the display of the bottom field is started after the entire top field is displayed first. Therefore, the reconstruction of the B frame needs to be performed at least half a frame before the display of the B frame, and the reconstruction process needs to be completed in order to end the display of the entire B frame. . Because of the differences between progressive and interlaced ordering between reconstruction and display, prior art systems have shown that each B
To complete frame reconstruction, one complete frame had to be available. Therefore,
Prior art systems required at least three memory framestores.

The present invention mainly relates to NTSC (Natio
nal Television Systems Co
International Standards Organization / International Electrotechnical Commission (ISO / IEC) 2-13818 standard supporting
Or PAL (Phase Alternating L)
ine) An MPEG-2 decoder compliant with the standard. NTSC resolution is 720x48 per frame
0 pixels (pixels), PAL resolution is 720 × 5
76 pixels. The picture rate is 24 to 30 frames per second. At a 4: 2: 0 sampling rate, each frame requires on average 12 bits per pixel. In memory devices such as dynamic random access memory (DRAM), PAL
Each frame of the type requires 4,976,640 bits of storage, and each NTSC frame has 4,147,2
Requires 00 bits of storage. The NTSC standard is mainly used in the United States (US), and the PAL standard is mainly used in Europe. Since a minimum of three frame storages are required, the PAL system requires at least 3x
4,976,640 = 14,929,920 (14.9
Mb) requires bit memory. The NTSC system has at least 3 × 4,147,200 = 12,44
1,600 (12.5 Mb) bit memory is required. In addition, additional memory was required for the channel buffers, and overhead storage was required to perform various miscellaneous overhead functions. Such overhead memories have been distributed in decoder systems or incorporated into integrated memories.

Further, if the decoder requires 3: 2 pull-down capability, the problem is complicated. This pull-down capability is currently required in NTSC decoders. The NTSC format specifies a frame rate of 30 frames per second, while the movie frame rate is 24 frames per second. Thus, the ratio of the frame rate between the NTSC format and the movie is 5: 4 frames. It is possible to convert the frame rate with an encoder, but that is undesirable. This is because the degree of compression of the transmitted or stored video data is significantly reduced. Therefore, it is desirable to convert the frame rate at the decoder while the decoder is receiving the encoded video data bitstream. NTS
To convert the movie for display in C, one field is repeated for every four fields of the movie, thereby converting to NTSC format. For example, N
The TSC MPEG-2 video coding standard requires that the data stored in the top field of a frame be displayed more than once if the bit stream signal repeats the top field. Since one frame includes two fields, one of the fields of every other frame is repeated, resulting in three fields, two
Fields, 3 fields, 2 fields, giving rise to the name "3: 2 pulldown". In order for 3: 2 pulldown to occur, one field of every other frame must be long enough in memory to be displayed twice before it is overwritten. Until now, one frame store for the B frame was sufficient to perform 3: 2 pulldown. However, in decoders with one frame store, the decoding or reconstruction process had to be stretched by one field time for every other frame to prevent repeated fields from being overwritten. .

The amount of memory is a major cost item when manufacturing such a decoder. Therefore, it is desirable to reduce the memory requirements of a decoder system in order to reduce the size and cost of the decoder. Since practical memory devices are manufactured in certain convenient discrete sizes, it is important for commercial reasons that the memory requirements be kept within a certain size if possible. For example, it is desirable to reduce memory requirements below a certain memory size (eg, 16 Mb), but using 24 Mb or 32 Mb memory devices adds cost and wastes extra storage. Because. While it is possible to fabricate an NTSC decoder containing three complete storage frames in a 16 Mb memory device, it is still desirable to reduce the memory requirements of the NTSC decoder.

[0029]

SUMMARY OF THE INVENTION A decoder according to the invention is provided.
The system can overcome these limitations and reduce the memory for decoding and displaying B frames during 3: 2 pulldown. In general, B frames are not used to predict other frames, and once B frame data has been pulled for display, it may be discarded. To enable memory re-use, data from the top and bottom fields are separately split into different segments of the memory. Thus, once data retrieval begins within a segment, that segment is released for reconstruction. However, this first-stage approach is not entirely satisfactory when performing 3: 2 pulldown on the top field of every other frame. Because the top
The field must be reusable in some way in order to be redisplayed. Rather than providing additional memory to store the top field of every other frame, the top field is reconstructed again, typically during a period when the reconstruction is prolonged. In this way, the amount of memory required is reduced and 3:
Fields that require a two pulldown redisplay are simply reconstructed again. Note that the term "display"
That means extracting the data for final display. This is because data may be extracted for purposes such as transmission, encoding, storage, etc., and is not always displayed immediately.

[0030]

In the preferred embodiment, the frame is conceptually divided into four different sections.
The sections are above the top, below the top, above the bottom,
Including the field part below the bottom. The memory contains only three segments. Here, each segment is the same size as each of the sections. Thus, each segment of memory is large enough to store any of the four different sections of the frame. The memory portion used for B frames is three quarters of the size of one frame store. During the reconstruction process, the pixel data is divided into a top field and a bottom field and stored in each one of two available segments of memory. Specifically, the data from each macroblock is split into two respective fields and stored in corresponding segments of the memory. Thus, each successive line in each segment corresponds to a successive line of one field. During retrieval, the data in each segment of the memory is retrieved line by line for display until the entire segment has been retrieved. As soon as segment retrieval begins, the entire segment is available for reconstruction. Since each segment includes a plurality of scan lines that are sequentially stored and retrieved, the DRAM typically used to implement the memory is addressed in a standard fashion. Therefore, the memory sectioning keeps the page hit ratio of the DRAM at the same level, and does not slow down the storage and retrieval of data to and from the memory. Thus, such partitioning of the memory preserves the organization of the data and does not cause unwanted fragmentation of the memory.

For each frame, the reconstruction of the top half of the frame is performed on the top and bottom sections. Thus, the top section is stored in one segment and the bottom section is stored in another section. Thus, after half of the frame has been reconstructed, the withdrawal of the top field of that frame begins. The segment storing the section on the top is first retrieved according to the interlaced display. Immediately after segment retrieval begins, the segment may be used to reconstruct other sections of the frame. This is because segment extraction is faster than its reconstruction. Thus, the third segment and the segment being drawn for display are used to store the lower top and lower bottom sections during the reconstruction of the lower half of the frame. When the reconstruction of the lower half of the frame has progressed about halfway, the extraction of the segment holding the top-top section is completed, and the extraction to display the segment containing the bottom-top section is started. However, the reconstruction is not yet finished. However, the reconstruction of each pixel in that segment is completed before drawing for display.

At this point, the two segments contain data from the top and bottom bottom sections of the frame. Therefore, a drawer for displaying the bottom field starts. Since one segment still contains the data of the section under the top of the frame, redisplay of this section is possible due to 3: 2 pulldown. However, the segment that initially contained the section on the top was overwritten and used for the bottom field. Thus, the segment storing the section on the bottom is used to reconstruct the section on the top while being pulled out,
Reconstruction is not delayed. Typically, the reconstruction is performed using two segments, but the data in the section on the bottom is discarded while reconstructing the fields on the top again. In this way, the reconstruction of the top-top section of the frame is reconstructed during the withdrawal of the bottom field for display. Next, after the bottom field has been retrieved for display, the two segments of memory containing the top field are retrieved to redisplay the top field. 2
The reconstruction of the second frame is based on the top frame of the first frame.
Starts while the field is being pulled for redisplay. The operation is the same for the second frame.
However, the top field is not reconstructed twice. Because it is not redisplayed for the second frame.

Typically, a frame that requires four segments of memory only needs three segments, so the memory portion of a B-frame is three quarters of the size of one frame store ( That is, 0.75)
It is. If each pixel contains on average 12 bits,
720 pixels x 4 using 4: 2: 0 sampling
One NTSC frame containing 80 lines is 4,14
Requires a 7,200 bit frame store. Therefore, the B frame is reduced to 3,110,400 bits. Here, each segment of the memory is 1,03
Desirably, it is 6,800 bits. Therefore,
The present invention uses one NTSC picture buffer memory.
1,404,800 bits or about 11.4 Mb. Of course, this particular size of the memory corresponds to the NTSC format. The present invention is not limited to a particular video format or framestore size and can be used to reduce the memory requirements of any video system that uses 3: 2 pulldown capability. Further, the present invention is applicable to different types of video systems that use different rates of pull-down schemes. Such changes are only a matter of design.

Specifically, according to the present invention, a B frame is reconstructed for display including pull down conversion,
The storage and retrieval decoder system includes a reconstructor for reconstructing B frames. Here, the reconstruction device is
Reconstruct the top-top field of every other frame twice. The decoder system includes a memory having only three segments for storing pixel data. Here, the size of each segment is large enough to store any one of the frame sections. A segmenter is provided to receive and divide the pixel data according to the top and bottom fields for each section of each frame. Here, the divider stores pixel data from the top field in one segment of the memory and pixel data from the bottom field in another segment of the memory. First, the segmenter selects any two segments for the top half of the first frame, and then, for the bottom half of the first frame, the segment being drawn for display and the remaining segments select. Thereafter, the segmenter selects a segment that is currently drawn for display and a segment that contains pixel data that has been drawn and will not be redisplayed in the future. In addition, the divider selects one segment to receive the upper portion of the top field that has been reconstructed for pulldown. The segment selected for pull down is the segment that has been drawn for display. Further, the decoder system includes an extraction circuit for extracting pixel data from one of the three segments at the time of the interlaced display.

In accordance with the present invention, a method for reconstructing, storing, and retrieving pixel data for displaying a B-frame while using a pull-down transform uses three segments of memory. Here, each segment is a quarter of the size of one frame store. This method
Reconstructing the upper half of each frame into pixel data; reconstructing the lower half of each frame into pixel data; and reconstructing the upper half of the frame into pixel data every other frame. Comprising configuring. During each reconstruction step, the method further comprises the steps of partitioning the pixel data according to the top and bottom fields of the frame and selecting two available segments to reconstruct the upper half of the frame , Selecting the other two available segments to reconstruct the lower half of the frame, and selecting one available segment to reconstruct the upper half again. . Further, the method stores the top field pixel data in one selected segment while reconstructing the upper half of the frame and the lower half of the frame; Storing the bottom field data in another selected segment and storing the top field pixel data in the selected segment to reconstruct the upper half of the frame again. Finally, from the segment of memory, the stored pixel data is retrieved for interlaced display.

Further, the steps of selecting a segment in the manner described above are selecting any two segments for the top half of the first frame and deriving for the bottom half of the first frame. Selecting a segment and a third segment, and during any reconstruction step, segments that have been drawn for display and pixels that have already been drawn and will not be drawn for future display. While selecting the segment containing the data and reconstructing the top half of the frame again, the top field pixel
Selecting the segment being retrieved to receive the data. Further, the method can include updating a segment table that includes a list of pointers to the segments according to the interlaced representation, and retrieving the pointers from the segment table.

A better understanding of the present invention may be obtained by considering the following detailed description of the embodiments in conjunction with the following drawings.

[0038]

FIG. 1 is a graph illustrating the operation of a decoder system operating in accordance with the prior art without performing 3: 2 pulldown. Each scan line forming a B frame is
Reference is made along the y-axis of the graph, and time is taken along the x-axis. Between times T0 and T4, the first B frame (FR0) is drawn between the origin (O) and point A by a solid line 10
Reconstructed as indicated by 0. Such reconstruction occurs progressively, with each macroblock in each slice being reconstructed one at a time into a picture buffer (not shown). Each macroblock contains data for a continuous 16 pixel x 16 line portion of the frame. Thus, each macroblock contains two fields of data. Successive lines of data are stored at successive locations in the picture buffer. For NTSC and PAL type decoders, 45 such macroblocks are reconstructed for each slice having a width of 720 pixels. In an NTSC system containing a total of 480 scan lines, 30 slices are reconstructed for each frame, for a total of 1,3 per frame.
There are 50 macroblocks. In a PAL system containing a total of 576 scan lines, 36 slices are reconstructed for each frame, for a total of 1,620 macroblocks per frame. Note that the PAL system displays 25 frames per second and the NTSC system displays 30 frames per second. Therefore, each system
On average, about 40,500 macroblocks per second are reconstructed and displayed.

At time T2, as shown by point H1, after the reconstruction of about half of the first frame FR0 is completed,
The display device (not shown) displays the first or top field (FR0) of the first frame FR0.
-FD0) and start displaying. A dash-dot line 102 drawn between points B and A indicates the display of the top field (FR0-FD0) of the first frame FR0. Such a display is completed at about time T4. On average, the reconstruction and display of each pixel occurs at about the same rate. However, while the reconstruction of each frame occurs progressively one line at a time, the display occurs interlaced, i.e., every other line corresponding to a field. Thus, the effective speed of displaying each slice of a scan line is twice as fast as the reconstruction for each macroblock row of pixel data. Since the display of the top field (FR0-FD0) of frame FR0 is half of the entire reconstruction time of frame FR0, the slope of solid line 100 is about half the slope of dash-dot line 102. However, at time T4, only half of the frame FR0 is displayed. It should also be noted that the frame FR0 corresponding to the field (FR0-FD0)
The reconstruction of the last few lines of the last slice of
To complete just before the same line being displayed. In this way, the reconstruction and the display are substantially locked together.

At about time T4, the display device begins to extract and display the second field of the first frame FR0, the bottom field (FR0-FD1).
This is indicated by the dashed line 104 drawn between points C and D. The display of the bottom field (FR0-FD1) of the first frame FR0 is completed at about time T6. Further, between times T4 and T8, a reconstruction of the second frame FR1 occurs, as indicated by the solid line 106 drawn between points C and E. Since one frame store of memory is used, such a reconstruction of the second frame FR1 is written over the data of the first frame FR0. Therefore, the display of the bottom field (FR0-FD1) of the first frame FR0 must start shortly before the reconstruction of the second frame FR1 starts. Otherwise, the data of the bottom field (FR0-FD1) is overwritten by the data of the second frame FR1. However, such delays are relatively short, such as those associated with one macroblock of data. Since the display of the bottom field (FR0-FD1) of the first frame FR0 occurs at twice the effective speed of the reconstruction of the second frame FR1, such display and reconstruction do not interfere with each other. It progresses in parallel between times T4 and T6.

At time T6, the display of the bottom field (FR0-FD1) of the first frame FR0 is completed.
In the meantime, as shown by the point H2, the reconstruction of the second frame FR1 is about half completed. Therefore, at time T6, the entire first frame FR0 is displayed, and half of the second frame FR1 is reconstructed. Between times T6 and T8, the second frame FR
The display of the top field FR1-FD0 of 1 is the point F
Completed as indicated by the dash-dot line 108 drawn between. Therefore, at time T8, the entire reconstruction of the second frame FR1 and the display of the top field (FR1-FD0) of the second frame FR1 are completed. As indicated by the dashed line 110, the bottom field (FR1-F) of the second frame FR1
The display of D1) starts at time T8 and ends at time T10. The reconstruction and display of the third and subsequent frames is
Similarly, the process starts from time T8 and proceeds.

FIG. 2 is a graph showing the operation of a decoder system operating according to the prior art when performing 3: 2 pulldown. Such a system requires at least three frame stores in memory. FIG.
However, each of the scan lines forming the B frame is referenced along the y-axis of the graph, and time is taken along the x-axis. Between times T0 and T4, the first B frame is reconstructed as shown by the solid line 200 drawn between the origin (O) and point A of the graph. Again, such reconstructions occur progressively. At time T2, after the reconstruction of about half of the first frame is completed, the display device begins to pull out and display the top field of the first frame. A dash-dot line 202 drawn between points B and A indicates the display of the top field. The display is completed at about time T4.

At about time T4, the display device begins to extract and display the bottom field of the first frame. This is indicated by the dashed line 204 drawn between points C and D. Here, the display of the bottom field is time T
Happens between 4 and T6. In contrast to FIG. 1, the reconstruction of the second frame does not start at time T4 but is delayed until time T6. Stretching the reconstruction ensures that the top field of the first frame is not overwritten.
The top field is displayed again between times T6 and T8, as indicated by the dash-dot line 206 drawn between points E and F. Further, reconstruction of the next frame begins at about time T6, as indicated by the solid line 208 drawn between points E and G. Note that
Since the reconstruction of the second frame is half of the redisplay rate of the top field of the first frame, these 2
Even if two events occur simultaneously, the data in the top field being redisplayed is not lost before display.
At time T8, the top field of frame 1 is redisplayed and the reconstruction of the second frame has progressed halfway, and the display device starts displaying the top field of the second frame. This is indicated by the dash-dot line 210 drawn between points H and G. The display of the top field of the second frame is completed at time T10.
Next, at time T10, the display of the bottom field of the second frame and the reconstruction of the third frame begin. The representation of the bottom field is shown by the dashed line 212 drawn between points J and K, and the reconstruction of the third frame is shown by the solid line 214 drawn between points J and L. Note that re-display of the top field of the second frame is not required, and thus reconstruction of the third frame is not delayed. In this way, the operation proceeds, the top field of the third frame is redisplayed, and the reconstruction of the fourth frame is delayed. The same applies hereinafter.

As can be seen from FIG. 1 and FIG. 2, even when 3: 2 pull-down is executed or not executed, B
To reconstruct and display the frames, it is sufficient to have at least three frame stores in memory. It is conceivable to further reduce the memory used for B frames. This is because after the data in the memory area is displayed, the memory area can be used for reconstruction.
However, for practical reasons, in prior art systems,
It was not achieved. Reconstruction occurs progressively at a time on a macroblock of data, and the display is interlaced and performed in a raster scan fashion. Therefore,
Reconstructing a subsequent frame in the same area of memory destroys data in the bottom field of the current frame. This is particularly problematic in systems requiring 3: 2 pulldown. This is because the top field data of every other frame must be stored for redisplay.

It is conceivable to reduce the memory used for the B frame by making the data reconstruction twice as fast and performing the reconstruction twice. However, while reconstruction is speeded up by an amount, practical decoder systems cannot currently perform reconstruction at twice the current speed. Further, even if reconstruction at twice the speed is possible, data must be stored at twice the current speed. However, current DRAM devices and similar practical memory devices cannot achieve such storage speeds. Therefore, the reconstruction processing and the display processing are locked to each other, and the B frame requires the entire frame store of the memory. This was the case for decoder systems that used 3: 2 pulldown capability as well as those that did not.

A decoder system according to the present invention can overcome these limitations and reduce the memory for decoding and displaying B frames during 3: 2 pulldown.
Generally, B frames are not used to predict other frames. Thus, once the data of the B frame has been retrieved for display, it may be discarded. To enable memory reuse, data from the top and bottom fields is split into different segments of memory. Thus, once a drawer for display is initiated on the data in a segment, the segment is released for reconstruction. However, this first scheme is 3: 2
The top field of every other frame when the pulldown is performed is not sufficiently complete.
Because the top field must be made available again for redisplay in some way. Instead of providing additional memory to store the top field of every other frame, the top field is reconstructed during a period in which reconstruction is typically stretched. In this way, the amount of memory required is reduced, and fields requiring a 3: 2 pulldown redisplay are simply reconstructed again. It should be noted that the invention is not limited to displaying data immediately. The term "display" refers to extracting data for final display. For example, data from a segment may be extracted and displayed immediately, or it may be extracted for transmission, encoding, storage, or the like. Therefore, it is not always displayed immediately.

In an embodiment, one frame is conceptually divided into four different sections. That is,
Top top, bottom bottom, bottom top and bottom bottom fields. The memory has only three segments. Here, each segment is the same size as each of the sections. Thus, each segment of memory is large enough to store any one of the four different sections of the frame. Thus, the memory portion used for B frames is three quarters of the size of one frame store. During the reconstruction process, the pixel data is divided into a top field and a bottom field, each one of two available segments of memory.
One is stored. Specifically, the data from each macroblock is split into two fields and stored in corresponding segments of memory. Thus, each successive line in each segment corresponds to a successive line of one field. During the fetching of the display process, the data in each segment of the memory is fetched line by line for display, and so on until the entire segment has been fetched. As soon as the retrieval of a memory segment is started, the entire memory segment is available for reconstruction. Each segment is
The DRAM typically used to implement the memory is addressed in a standard fashion because it includes multiple scan lines that are stored and retrieved sequentially. Thus, by partitioning the memory, the page hit rate of the DRAM is maintained at the same level, and the storage and retrieval of data to the memory is not slow. Further, such partitioning of the memory maintains the organization of the data and prevents undesired fragmentation of the memory.

For each frame, the reconstruction of the top half of the frame is performed on the top and bottom sections. Thus, the top top section is stored in one segment and the bottom top section is stored in another segment. In this way, after half of the frame has been reconstructed, the withdrawal of the top field of that frame begins. First, the segment storing the top-top section is extracted according to the interlaced display. Immediately after segment retrieval begins, the segment may be used to reconstruct other sections of the frame. This is because segment extraction is performed faster than its reconstruction. Thus, during reconstruction of the lower half of the frame, the third segment and the segment being drawn for display are used to store the lower top and lower bottom sections. When the reconstruction of the lower half of the frame has progressed almost halfway, the drawing of the segment holding the upper top section is completed, and the drawing for displaying the segment containing the lower top section is started. However, the reconstruction has not yet been completed. However, the reconstruction of each pixel in the segment is completed before drawing for display.

At this point, the two segments contain data for the top and bottom bottom sections of the frame and retrieval begins to display the bottom field. Since one segment still contains data for the top and bottom section of the frame, redisplay of this section is possible for 3: 2 pulldown. However, the segment that originally contained the top-top section has been overwritten and is being used for the bottom field. Thus, the segment storing the bottom top section is used to reconstruct the top top section while it is being pulled, and the reconstruction is not stretched. Typically, the reconstruction is performed using two segments, but while reconstructing the top top field again, the data in the bottom top section is discarded. In this way, reconstruction of the top-top section of the frame is performed while the bottom field is being drawn for display. Next, after the bottom field has been retrieved for display, the two memory segments containing the top field are retrieved to redisplay the top field. Reconstruction of the second frame begins while the top field of the first frame is being pulled for redisplay. The operation is the same for the second frame, except that the top field is not reconstructed twice. This is because for the second frame, the top field is not redisplayed.

It should be noted that the data required to reconstruct the top-top field is already stored in the decoder's channel buffer. The channel buffer is preferably implemented as a circular buffer,
The decoder ensures that the data in the channel buffer needed to reconstruct the top-top field will not be overwritten until it has been reconstructed a second time.

Typically, for a frame that requires four memory segments, only three segments are needed, so the memory portion of a B frame is a quarter of the size of one frame store. 3 (ie, 0.
75). 720 using 4: 2: 0 sampling
An NTSC frame containing 480 pixels by 480 lines (each pixel contains on average 12 bits) is 4,147,
Requires a 200-bit frame store. Therefore, the B frame is reduced to 3,110,400 bits. Here, each of the memory segments is 1,036
Desirably 800 bits. Therefore, the present invention reduces the picture buffer memory for NTSC to 11,404,800 bits (i.e., about 11.4M).
b) can be reduced. Of course, this particular memory size corresponds to the NTSC format. The present invention applies to specific video formats or framestores.
Without being limited to size, the memory requirements of any video system that uses 3: 2 pulldown can be reduced. Further, the present invention is applicable to different types of video systems that use different rates of pull-down schemes. Such changes are only a matter of design.

It should be noted that the decoder system according to the invention can be used to process still frames. Since the reconstructor is stretched even during still frames, it may use processing elements to continuously reconstruct "lost" segments as needed. Further, the decoder system according to the present invention can be implemented in any application, including video capabilities.
The invention is not limited by any particular application. For example, the decoder system can be incorporated into digital video disc (DVD) applications, entertainment systems, communication systems, and the like.
The decoder system may be a set top box, a personal computer (PC), or any one of a larger computer system (eg, a workstation, server, minicomputer, mainframe computer, supercomputer). One or more may be implemented. This decoder system uses a single chip module (SC
M), a multi-chip module (MCM), a board-level product, a box-level product.

The decoder system according to the present invention uses the MP
It is desirable to incorporate it into an EG-2 type system.
However, the present invention applies to other decoder types (eg, M
PEG-1, Wavelet, H .; 261, H .; 32
0, JPEG, etc.). The present invention is suitable for PAL type display formats, but other display formats (eg, NTSC, red / green / blue (RGB))
Format) is also envisaged. The present invention can be used with any type of display device (eg, TV, monitor, liquid crystal display (LCD), plasma screen, video projection device, virtual reality display device). The invention includes a memory device. The memory device is typically implemented using dynamic random access memory (DRAM) for cost reasons. However, static RAM (SRAM), video RAM (VRA
M), or other types of RAM systems will achieve satisfactory performance.

Referring now to FIG. 3A, there are B frames conceptually divided according to the present invention,
This is indicated by the letter F. Each frame F includes a top field T and a bottom field B. The top field T is displayed first, followed by the bottom field B in an interlaced fashion. Each frame F is
It is conceptually divided into four sections. That is, a top top section (TU) 302, a top bottom section (TL) 304, a bottom top section (BU) 306,
And the bottom bottom section (BL) 308. Therefore, the display order is TU, TL, BU, and BL in the interlaced display of each frame F. For 3: 2 pulldown, TU and T following the BL section
The L sections are redisplayed sequentially. FIG. 3 (B)
3 shows a memory M used for decoding (reconstructing) and displaying a B frame when performing 3: 2 pulldown. Memory M is 3
One segment is defined. That is, segment 0
(310), segment 1 (312), and segment 2 (314). Each of these segments is large enough to store any one of the sections TU, TL, BU, and BL.

Referring now to FIG. 4, there is shown graphically the operation of a decoder system that performs 3: 2 pulldown in accordance with the present invention. In contrast to FIGS. 1 and 2, FIG. 4 is shown with respect to segments of memory rather than fields or sections of a frame. 3B (segment 0 to segment 2) of the memory M shown in FIG. 3B are used to display one or more of the B frame F shown in FIG. 3A. Segment 0 is indicated by a dash-dot line, segment 1 is indicated by a dot line, and segment 2 is indicated by a dash line. Reconstruction typically occurs two segments at a time, and the solid line is used to indicate the reconstruction.
The reconstruction of the section is indicated by the letter R and the indication is indicated by the letter D. Therefore, the reconstruction of the TU section is R _TU
And the representation of the same section (or corresponding segment of memory) is denoted by _DTU . As mentioned above, the data need not be displayed immediately, and the term "display" means to extract the data for display.

FIG. 5 is a graph 500 of FIG. 4 showing the operation of the decoder system according to the present invention. Table 500 includes a plurality of columns approximately aligned with the time increments of the graph of FIG. Each column of table 500 represents an access made from segment 0 to segment 2 of memory M during each time increment. Each row corresponds to one of the segments during the time increment, as shown on the left side of table 500. For example, times T0 and T2
The _RTU entry for segment 0 during indicates that the TU section of the frame has been written to segment 0 during reconstruction. Further, the _DTU entry for segment 0 between times T2 and T3 indicates a representation of the data in segment 0 corresponding to section TU. 4 and 5 are useful when reading the following description of the operation.

Reconstruction of the first frame starts at time T0 and ends at time T4. Between times T0 and T2, the first half of the reconstruction of the first frame takes place, segment 0 stores data from the TU section (indicated by _RTU ), and segment 1 stores data from the BU section. Store (indicated by R _BU ). Note that any two from segment 0 to segment 2 can be used to start the reconstruction process. Here, segments 0 and 1 are arbitrarily selected. At time T2, since the first half of the frame is completely reconstructed, the display device (not shown) displays data from segment 0 at time T2 to display the top-top section (denoted by _DTU ). Start pulling out. Top section is indicated by T
Complete with 3. Between times T2 and T4, reconstruction of the lower half of the first frame (including the TL and BL sections) begins. Since segment 0 starts to be displayed at time T2,
It becomes available for reconstruction shortly after time T2. Thus, data in the TL section is written to section 2 (indicated by R _TL ), and data from the BL section is written to section 0 (indicated by R _BL ).
At time T3, at least half of the reconstruction of the TL section has been written to segment 2. Therefore, T
The L section is displayed from segment 2 between times T3 and T4 (indicated by _DTL ).

At time T4, the entire reconstruction of the first frame and the display of the top field of the first frame are completed. Further, segment 1 is the BU of the first frame.
The section 0 includes data of a BL section. Thus, between times T4 and T5, the display of the BU from segment 1 is performed and the time T
Between 5 and T6, the display of the BL section from segment 0 occurs. Sections TU and TL of the first frame
Must be redisplayed for a 3: 2 pulldown. Segment 2 contains the data of the TL section of the first frame. But,
Data from the TU section is no longer in segment 0. Because it was overwritten with data from section BL between times T2 and T4. Thus, between times T4 and T6, the BU section from segment 1 is displayed and segment 0
While the BL section from is displayed, the reconstruction of the TU section is performed again. Here, the data is stored in segment 1. Typically, the reconstruction is performed using two segments of the memory M, but only one segment is used and the data in the BU section is ignored or discarded.

At time T6, the TU section of the first frame exists in segment 1 and the TL section of the first frame
Section exists in segment 2. Thus, the data of the TU section in segment 1 is redisplayed between times T6 and T7, and the data of the TL section in segment 2 is displayed between times T7 and T8 to pull down the top field of the first frame. Will be displayed again. At time T8, the display of the first frame including the redisplay of the top field is completed.

At time T6, the reconstruction of the TU and BU sections of the second frame is performed in section 0, respectively.
To Section 1. Between times T8 and T9, the TU section of the second frame stored in segment 0 is displayed, and between times T9 and T10, the TL section of the second frame stored in segment 2 is displayed. . Between times T8 and T10, the TL and BL sections of the second frame are reconstructed into segments 2 and 0, respectively. No other reconstruction of the TU section of the second frame is required. This is because the top field of the second frame is not repeated. Thus, the BU section of the second frame is displayed from segment 1 between times T10 and T11, and the BU section of the third frame is displayed at time T10
And T12, the same segment 1 is reconstructed. Between times T11 and T12, the TU section of the third frame is reconstructed into segment 2 and the BL section of the second frame is displayed from segment 0. Time T
At 12, the second frame is fully visible and 3
The upper half of the th frame is reconstructed. The operation for the third frame proceeds in the same way as for the first frame. Here, the TU section of the third frame is reconstructed into segment 1 again between times T15 and T16. The above procedure is repeated for frame 5.

Referring now to FIG. 6, there is shown a state diagram 600 illustrating the operation of the present invention. State diagram 600
Shows the operation of a finite state machine (FSM) used to control a decoder system that performs 3: 2 pulldown (eg, an NTSC decoder system). The FSM can be implemented in hardware, software, or any combination thereof. 7 states 60
2,604,606,608,610,612,614
And each state corresponds to one field time of the frame. Furthermore, state diagram 600 corresponds to FIGS. 4 and 5, where each state corresponds to two time (Tn) intervals.
For example, state 602 corresponds to time interval T0-T2,
State 604 corresponds to time interval T2-T4 and state 606
Corresponds to the time interval T4-T6, and so on. The state diagram 600 can be used to form the table 500 of FIG. 5 and the graph shown in FIG. In each state, "R" indicates reconstruction, followed by two segment numbers to which data is being written. The first segment number corresponds to the top field and the second segment number corresponds to the bottom field. The segment number “X” shown in the state 606 and the state 614 indicates the reconstruction of the TU section of the frame for performing the pull-down, during which the data of the BU section is discarded. The state "D" indicates a display, followed by two segment numbers indicating segments that are displayed continuously during that state. A branch line labeled PD indicates that a pull down is performed for the current frame,
A branch line labeled PD * indicates that no pull down is performed for that frame.

Operation starts in state 602. Here, the top field of the first frame is reconstructed and the data is written to segment 0 and segment 1. Operation proceeds to state 604, where the top field is displayed and the bottom field is reconstructed. The data from the reconstruction is written to segment 2 and segment 0,
Meanwhile, segment 0 and segment 2 are displayed. Since a pull-down of the first frame is required, operation proceeds to state 606, where the upper half of the top field of the first frame is reconstructed again while the bottom
The fields are displayed. Operation proceeds to state 608,
There, a pull down of the top field occurs and the reconstruction of the next frame begins. Operation returns from state 608 to state 604. In state 608, the top field of the second frame is displayed and the second bottom field is reconstructed. Since pull down is not required for the second frame, the operation changes from state 604 to state 6
Proceed to 10, where the bottom field of the second frame is displayed and the top field of the third frame is reconstructed. Operation proceeds to state 612, where the top field of the third frame is displayed and the bottom field of the third frame is reconstructed. Since pulldown is required for the third frame, operation proceeds from state 612 to state 614, where the bottom field of the third frame is displayed, while the top field is reconstructed. Operation proceeds to state 610, where a pull down of the top field of the third frame is performed while the top field of the fourth frame is reconstructed. Operation proceeds to state 612, where the top field of the fourth frame is displayed and the bottom field of the fourth frame is reconstructed. Operation returns from state 612 to state 608 since pull down is not required for the fourth frame. The operation proceeds in the same manner as described above until all successive B frames have been reconstructed and displayed.

FIG. 7 is a flowchart illustrating a method according to the present invention. FIG. 7 schematically corresponds to the graph of FIG. 4, the table 500 of FIG. 5, and the state diagram 600 of FIG.
In the first step 702, the upper half of the first frame
Reconstructed into the first and second segments. The operation then proceeds along two paths, display and reconstruction. Each step in the display path corresponds to a corresponding step in the reconstruction path and occurs at about the same time as the corresponding reconstruction step. Therefore, steps 704 and 706 are performed simultaneously, and steps 708 and 710, step 71
2 and 714 are also executed at the same time. The display route is determined in steps 704, 708, 712, 716, 720, 72.
4, 728, 732, 736, and 740, which are executed sequentially. Similarly, the reconstruction paths are steps 706, 710, 714, 718, 722, 726,
730, 734, 738, and 742, which are also executed sequentially. Operation comprises steps 740 and 7
From step 42, the process returns to steps 704 and 706, respectively.

For the display path, the top and bottom fields are determined by steps 704 and 708,7
16 and 720, 724 and 728, and 736 and 740
Are displayed continuously as indicated by. The top field of every other frame is described in steps 712 and 732
Is repeated for pulldown as indicated by. In the reconstruction path, steps 702 and 706, 714 and 71
As shown at 8, 722 and 726, 734 and 738, and 742 and 706, the upper half of the frame is reconstructed first, followed by the lower half. However, if a pull-down is required for the frame, step 71
The upper half of the top field is reconstructed again, as shown at 0 and 730. Note that some steps are repeated but use different segments. For example, reconstruction of the upper half of the frame occurs at steps 714 and 734, whereas step 714 uses the first and second segments,
Step 734 uses the second and third segments.

FIG. 8 is another flowchart illustrating the method according to the present invention. FIG. 8 is similar to FIG.
It is more generalized. The first step 750 is the same as step 702, where the upper half of the first frame is reconstructed using any two segments. next,
Actions are split along a parallel path of display and reconstruction.
For display, operation proceeds to step 752, where the top field of the frame is displayed. Next, the bottom field of the frame is displayed at step 754. If the top field needs to be pulled down and redisplayed for the current frame (decision in step 756), operation proceeds to step 758 where the top field is displayed again. Next, operation returns to step 752, where the top field of the next frame is displayed. If the top field of the current frame does not need to be redisplayed (decision in step 756), operation returns directly to step 752 for the next frame. The operation continues until one or more consecutive B frames have been displayed.

For reconstruction, operation proceeds to step 750
To step 760, where the lower half of the current frame is reconstructed. Step 760 is performed simultaneously with display step 752. In the first iteration of step 760, one segment used is step 7
The first segment, indicated at 52, is the remaining segment that was not used in step 750. Thereafter, the first segment used in step 752 is used as one segment in step 760. Other segments used are:
Of the remaining two segments, those that contain data that has already been displayed and will not be redisplayed in the future due to pulldown. Operation proceeds to step 762, where it is determined whether the current frame requires a pull-down. If necessary, operation proceeds to step 764, where the top-top section of the current frame is reconstructed. If step 764 is performed,
It coincides with step 754, and step 764
Is the first segment used in step 754. From step 764,
Operation proceeds to step 766, where the upper half of the next frame is reconstructed. If the current frame does not require a pulldown (decision in step 762),
Operation proceeds directly to step 766. Step 766 is performed during each iteration, and occurs concurrently with step 754 if the pulldown is not performed on the current frame, and concurrently with step 758 if the pulldown is performed. Step 766 uses the first segment displayed in step 754 or step 758, and selects the remaining two segments that contain data that has already been displayed and will not be redisplayed for future pull-down. use. Operation returns from step 766 to step 760 until one or more consecutive B frames have been reconstructed.

Referring now to FIG. 9, there is shown a simplified block diagram of a decoder system 800 according to the present invention. The decoder system 800 of FIG. 9 primarily illustrates the data flow and configuration. The encoded video data is provided in bit stream form from data channel 802 to a rate (channel) buffer 804, where it is temporarily stored. The encoded video data typically includes image information representing multiple frames of the moving image. Each encoded frame or picture of the video is represented in digital form as a bit sequence. The structure of this sequence depends on the selected video compression standard (eg, MPEG-1 or MPE).
G-2 standard). The video data in the rate (channel) buffer 804 is provided via another channel 806 to a reconstructor 808, where the picture data is decoded into a form suitable for display.

It is desirable that the reconstructing device 808 incorporates a decoder function for converting encoded video data into corresponding symbols or events. Here, these symbols or events are reconstructed into the original frame. For example, the reconstruction device 808 has the ID
It may include a CT pipeline, a motion compensation (MC) pipeline, and merge storage to perform the reconstruction process. The IDCT pipeline coordinates the reconstruction of each macroblock in the frame, and the MC pipeline processes motion compensation information for each macroblock.

Next, the reconstructed frame data is
Passed from reconstructor 808 to picture buffer 812 via data channel 810. The picture buffer 812 has two anchor frames (A1 and A1).
It is desirable to include enough memory to accommodate 2). Here, the anchor frame is an I or P frame according to the MPEG standard. Data channel 810 is bidirectional, and reconstructor 808 uses anchor frame A1 or A2 in picture buffer 812.
Data can be extracted from For example, anchor frame A1 may include an I-frame previously reconstructed by reconstructor 808. Reconstruction device 808
Can retrieve I frames and merge them with data from rate (channel) buffer 804 to reconstruct P frames. Next, the P frame may be stored in picture buffer 812 as anchor frame A2.

Further, picture buffer 812 includes memory for storing data for B frames. As in the prior art systems, anchor frames A1 and A2 are derived by reconstructor 808 to reconstruct the B frames. Therefore, anchor frames A1 and A2 must be fully available in decoded form. However, in prior art systems, the required memory size in the picture buffer for B frames is that the anchor frames A1 and A
It was the same size as 2. Therefore, the picture buffer needed enough memory to provide three full size frame stores (two for anchor frames A1 and A2 and one for B frames). However, in the present invention, in the picture buffer 812, B
The frame is divided into three segments S1, S2 and S3. The amount of memory required to store and display B frames is smaller than one frame store. The size of the segments S1-S3 is one quarter of one frame store, and the total memory size for B frames is 0.75 frame stores. Segment S1-S
Each bit size of 3 depends on the size of the framestore. In an NTSC frame having 4, 147 and 200 bits, the size of the memory M is 3, 110 and 400 bits. When any one of the segments S1-S3 of the B frame is being displayed, that segment is released for reconstructed data. This is achieved without affecting the page hit rate of the DRAM typically used to implement the picture buffer 812, and without affecting the display of B frames. Reducing the amount of memory for storing B frames saves the overall cost of decoder system 800. In an embodiment, the rate (channel) buffer 804, picture buffer 812, and other memory required for the function being performed are combined into a single memory device 814 with a maximum size of 16Mb. As a result, significant cost savings for memory device 814 are achieved.

In accordance with the present invention, reconstructor 808 provides B frame data via data channel 816 to partitioner 820 of partitioner 818. The divider 820 divides the data into a top field SF1 and a bottom field SF2. Next, the data of the top field and the bottom field are stored in the picture buffer 812.
Is written to the B frame portion of Specifically, the data of the top field SF1 is provided to the first segment (for example, the segment S1), and the data of the bottom field SF2 is provided to the second segment (for example, the segment S2). These segments S1
When and S2 are full, the partitioner 820 selects the other two segments and provides the split data to the two segments. The segmenter 820 is a segment pointer that tracks the order of the segments S1-S3.
It is desirable to hold the table 822.

Segment pointer table 822
Need only include four pointer locations to store pointers to the segments. During the reconstruction of the first half of the first frame, the segmenter 820 moves to the first position,
A pointer to the segment containing the top half of the top field is stored, and a pointer to the segment containing the top half of the bottom field is stored in the third position. During the reconstruction of the second half of the first frame, the partitioner 820 stores a pointer to the segment containing the lower half of the top field in the second position and a bottom position in the fourth position. Store a pointer to the segment containing the lower half of the field. For pulldown, the partitioner 820 need only place the pointer to the top half of the top field in the first position. For the second frame, the order of the items is switched. Thus, during the reconstruction of the first half of the second frame, the partitioner 820 stores in the third position a pointer to the segment containing the top half of the top field and in the first position the bottom Store a pointer to the segment containing the top half of the field. During reconstruction of the second half of the second frame, segmenter 820
A pointer to the segment containing the lower half of the top field is stored in the second position, and a pointer to the segment containing the lower half of the bottom field is stored in the second position. This order remains reversed until the next pulldown. On the next pulldown, the order is reversed again. The display controller 824 searches for a pointer according to the sequential order of the pointer positions for interlaced display. That is, the second, third, and fourth are searched from the first pointer, and the process returns to the first pointer. Of course, while display controller 824 searches for pointers in the proper order for interlaced display, segmenter 820
Can put pointers in sequential order.

The display controller 824 extracts the data of the frame stored in the picture buffer 812 via the data channel 826 and provides the data to the display device 828 in a format suitable for display. Display device 82
Reference numeral 8 denotes a monitor, a TV, or a display device equivalent thereto. Display controller 824 and display device 828
Displays information in interlaced format. This is performed in a standard manner for anchor frames A1 and A2. However, for the B frame, the display controller 824 retrieves data one at a time from each of the segments S1-S3. The display controller 824 controls the segment pointer table 8 in the partitioning unit 818.
22 to determine which of the segments S1-S3 to display at a given time. Segment S1-S
The pointer to 3 need not necessarily be listed in the segment pointer table 822 in numerical order. However, the segment pointer is
Preferably, the pointer tables 822 are retrieved in sequential order. Further, each of the segments is used and accessed more than once.

Referring now to FIG. 10, there is shown a block diagram of another decoder system 900 according to the present invention. The decoder system 900 is preferably an NTSC type MPEG-2 decoder. It should be noted, however, that the present invention is not limited to MPEG and can be used with any video standard or configuration that requires 3: 2 pulldown capability. The bit stream of video data is received by a preprocessor 902 connected to the bus 904, which extracts the encoded video data and provides it to the bus 904. The memory controller 906 includes:
Data is extracted from the bus 904 and temporarily stored in a channel buffer portion of the memory device 908. memory·
The controller 906 is connected between the bus 904 and the memory device 908, and controls access to the memory device 908. Memory device 908 is preferably implemented in DRAM and preferably includes a channel buffer and a picture buffer for storing I, P, and B frame pixel data.

Next, the video data from the channel buffer
The data is sent via a bus 904 to a variable length decoder (VL
D). Variable Length Decoder (VLD) 9
10 converts the data into DCT data. The data is then provided to IDCT pipeline 912.
The IDCT pipeline 912 converts DCT data into macroblocks of pixel data. The macroblock is provided to merge storage 914 connected to bus 904. Bus 904 returns data to memory device 908. This procedure is performed for an I frame. The procedure is similar for P-frames, except that data from the IDCT pipeline 912 is merged with I-frame data from the motion compensation (MC) pipeline 916. The motion compensation (MC) pipeline 916 is configured to transfer I-frames from a picture buffer in the memory
Extract data. The procedure is similar for B frame data, except that data from I or P anchor frames is merged to reconstruct the B frame.

Variable length decoder (VLD) 910, IDC
The T pipeline 912, motion compensation (MC) pipeline 916, and merge storage 914 work together to perform frame data reconstruction. In general, I and P frames are stored in a memory device 90 in a standard manner.
8 is stored. However, the B frames are split into memory devices 908 in accordance with the present invention, as described above. Thus, memory device 908 has less than three frame stores of memory (preferably 2.75).
Frame store). The partitioning process is performed by the merge storage device 914, the display device 918, the memory controller 9
06, or any combination of these devices. The memory controller 906 is at a central location that is convenient for storing B-frame data in segments and providing the data to the display 918 in the proper order. However, in an embodiment, B-frame partitioning according to the present invention is performed by a combination of the merge storage 914 and the display 918. Thus, merge storage 914 includes circuitry for storing B-frame data in segments of the picture buffer of memory device 908, and display 918 includes circuitry for extracting the segments in an appropriate order for display.

According to the above description, according to the present invention,
It can be seen that the method and apparatus for reducing the memory required to decode bi-directionally coded frames during pulldown significantly reduces the amount of memory required for a video decoder. Significant cost savings are achieved since memory size is a major cost item when manufacturing such decoders. Generally, once B-frame data has been displayed, it may be discarded because B-frames are not used to predict other frames. To enable memory reuse, data from the top and bottom fields is split into different segments of memory. Thus, once the display of data in a segment begins, the segment is freed for reconstruction. However, this first scheme is
Not entirely sufficient when the top field of every other frame performs a 3: 2 pulldown. This is because the top field must be made available again in some way for redisplay. Rather than providing additional memory to store the top field of every other frame, the top field is reconstructed again during the period during which reconstruction is typically stretched. In this way, memory requirements are reduced and fields that need to be redisplayed for 3: 2 pulldown are simply reconstructed again.

Specifically, one frame is conceptually 4
Divided into two sections. That is, the upper half and lower half of the top field and the upper half and lower half of the bottom field. The memory for storing and displaying B frames includes three segments, each segment being the same size as each of the four sections of the frame. The top half of the first frame contains the top and bottom top sections and is reconstructed into two segments of memory. Next, the segment containing the top-top section is displayed, during which the same segment and the third segment are used to reconstruct the bottom half of the first frame (including the bottom-top and bottom-bottom sections). You. In addition, the lower bottom section is displayed while the lower half of the first frame is being reconstructed. Next, the segment containing the bottom top section is displayed, during which the same section is used to reconstruct the top top section when the top field must be redisplayed. Reconstruction of the top-top section is completed while the bottom field is displayed. In this way, two segments of memory are stored in the top frame after the first frame is completely displayed.
Remember the entire field. To accomplish the pull down, the top field is displayed again from these two segments. The operation is similar for the next frame, except that no pull-down is required.
This is because pulldown is performed only on every other frame.

Since there are only three memory segments,
A decoding device (ie, a reconstruction device) or a segmenter determines the next two available segments, and a display device determines the appropriate display order of the segments,
This is a relatively straightforward procedure. For such a purpose, a segment pointer table may be used.
The segment pointer table is a list of pointers to segments. The segment pointer table is updated during reconstruction to list the segments in display order. Thus, the display need only read the pointers from the segment table in sequential order to identify the next segment to display. Furthermore, even when performing 3: 2 pulldown, only three segments are needed to store and display the B frame (each segment is one-quarter the size of the frame). The frame store for a frame is only 0.75 times the size of a full size frame store. Thus, the size of the picture buffer is reduced to 2.75 frame stores for anchor and B frames.

While the system and method according to the present invention have been described in connection with embodiments, it is not intended that the invention be limited to the specific forms recited herein, but instead, by the claims Alternatives, as long as they are reasonably construed as falling within the spirit and scope of the defined invention,
It is intended to cover amendments and other changes.

[Brief description of the drawings]

FIG. 1 is a graph illustrating the operation of a decoder system according to the prior art.

FIG. 2 is a graph illustrating the operation of a decoder system operating according to the prior art when performing 3: 2 pulldown.

FIG. 3A shows one frame that is conceptually divided according to the present invention. (B) is a diagram showing a memory used for reconstructing and displaying a B frame while performing 3: 2 pulldown.

FIG. 4 is a graph showing the reconfiguration and display operation of a decoder system according to the present invention.

FIG. 5 is a table showing the operation of the decoder system according to the present invention.

FIG. 6 is a state diagram showing an operation according to the present invention.

FIG. 7 is a flowchart illustrating a method according to the present invention.

FIG. 8 is another flowchart illustrating a method according to the present invention.

FIG. 9 is a block diagram illustrating a decoder system according to the present invention.

FIG. 10 is a block diagram illustrating another decoder system according to the present invention.

[Explanation of symbols]

 302 Top top section (TU) 304 Top bottom section (TL) 306 Bottom top section (BU) 308 Bottom bottom section (BL) 310 Segment 0 312 Segment 1 314 Segment 2 500 Table 600 State diagram 800 Decoder system 802 Data channel 804 Rate (channel) buffer 806 Other channels 808 Reconstructor 810 Data channel 812 Picture buffer 814 Memory device 816 Data channel 818 Partitioner 820 Partitioner 822 Segment pointer table 824 Display controller 826 Data channel 828 Display device 900 Decoder system 902 Preprocessor 904 Bus 906 Memory controller 908 Memory device 910 Variable length decoder (VLD) 912 IDCT pipeline 914 Merge storage 916 Motion compression (MC) pipeline 918 Display

Continued on the front page (51) Int.Cl. ⁶ Identification number Reference number in the agency FI Technical display location H04N 9/808 H04N 9/80 B 11/04

Claims (14)

[Claims] 1. A display including a pull-down when a bi-directional predictive coding (B) frame of pixel data includes four sections corresponding to an upper portion and a lower portion of a top field and a bottom field, respectively. A decoding system that efficiently decodes and derives a B-frame from pixel data, reconstructs the B-frame into pixel data, and reconstructs the upper part of the top field twice for every other frame. And three segments for storing pixel data,
A memory in which each of these segments is large enough to store any one of the sections; a reconstructor and a memory connected to the memory for receiving the pixel data and for storing pixel data from the top field; Into one segment and store the pixel data from the bottom field according to the top field and the bottom field for each section of the B-frame, so as to store the pixel data from the bottom field in another segment. A divider connected to the memory, and a extracting circuit for extracting pixel data from one of the three segments at a time for an interlaced display including a pull-down; First select any two of the three segments, then pull out for display Selecting the segment that is being drawn and the remaining segments, and selecting the segment that has been drawn for display and the segment that contains pixel data that has been drawn for display and will not be redisplayed in the future, A decoder system further configured to select a segment being drawn for display to receive pixel data for pulldown. 2. The decoder system according to claim 1, wherein 3: 2 pulldown is performed. 3. The decoder system of claim 2, wherein the decoder operates according to the International Television System Committee (NTSC) format. 4. The decoder system according to claim 3, wherein the B frame includes a plurality of macroblock rows, each macroblock row including data for two sections of the B frame. 5. The decoder system of claim 3, wherein the B frame includes 720 pixels by 480 lines. 6. The B frame averages one pixel per pixel.
2 bits, each of the segments of the memory being 1,
The decoder system of claim 5, wherein the decoder system does not include more than 036,800 bits. 7. The decoder system of claim 6, wherein said memory further includes two frame stores for storing two anchor frames. 8. The memory according to claim 8, wherein said memory is a dynamic random access memory.
The decoder of claim 1, wherein the decoder is an access memory.
system. 9. The decoder system of claim 1, wherein said memory starts storing pixel data from said plurality of segments of said memory after said memory stores half of a decoded B frame. 10. A segment pointer table comprising a list of pointers to said segments, wherein said segmenter places said pointers in an order suitable for an interlaced display of said pixel data. The decoder system of claim 1, wherein the decoder system maintains a pointer table. 11. The decoder system of claim 10, wherein said retrieval circuit retrieves each of said pointers from said segment pointer table in a sequential order to access each of said segments. 12. A bi-directional predictive coded (B) frame voxel data is pulled down using three segments, the size of which is one quarter of the size of the frame store. Reconstructing, storing, and retrieving for display using: reconstructing the upper half of the B frame into pixel data; and reconstructing the lower half of the B frame into pixel data. Reconstructing the upper half of the B frame into pixel data every other frame; and, during each of the reconstructing steps, the pixel according to the top and bottom fields of the B frame. Splitting the data and two available segments for said step of reconstructing the upper half of the B frame And select the other two available segments for the step of reconstructing the lower half of the B frame, and select one available segment for the step of reconstructing the upper half again Storing the top field pixel data in one selected segment and storing the bottom field pixel data between the steps of reconstructing the upper half of the frame and reconstructing the lower half of the frame. For the step of storing the data in another selected segment and reconstructing the upper half of the frame again,
Storing field pixel data in a selected segment; and storing the stored pixel data for interlaced display.
Retrieving pixel data from the segment. 13. The step of selecting a segment comprises the steps of selecting any two segments for the top half of the first frame, and for the bottom half of the first frame, Between the step of selecting the second segment and the step of reconstructing the upper half of the frame and the step of reconstructing the lower half of the frame. During the step of selecting a segment containing pixel data that is not extracted for display and the step of reconstructing the upper half of the frame back into pixel data, the segment being extracted is
Selecting to receive top field pixel data. 14. The method of claim 12, further comprising: updating a segment table containing a list of pointers to the segments according to the interlaced representation, wherein the retrieving step further comprises retrieving a pointer from the segment table. A method for extracting and reconstructing pixel data as described above.