KR100518477B1 - HDTV Down Conversion System - Google Patents

Abstract

A video decoder compliant with the Advanced Television System Standard (ATSC) includes circuitry for decoding ATSC encoded images and employs a down conversion process to generate a standard definition video signal. The video decoder includes a frequency domain filter to reduce the resolution of the ATSC encoded signal. The video decoder down conversion system also performs a resampling process and also includes a formatting section with vertical and horizontal filters to format the decoded and down converted video image for a particular display and aspect ratio. The decoder senses the display format of the encoded video signal and changes the processing provided by the decoder to produce a standard definition output signal regardless of the display format of the encoded input signal. The system also includes a format converter that may be programmed to use a plurality of methods of converting the aspect ratio of the input signal for display on a display device having a different aspect ratio. In one mode, the system is arranged to allow the user to choose one method through the possible methods.

Description

HDTV down conversion system

This patent application is an application involving a priority claim in US Provisional Application No. 60 / 040,517, filed March 12,1997.

The entirety of this US Provisional Application No. 60 / 040,517 is expressly incorporated herein by reference.

The present invention relates to a decoder for receiving a frequency domain encoded signal such as, for example, an MPEG-2 encoded video signal, decoding the signal, and converting the signal into a standard output video signal. A decoder converts and formats an encoded high resolution video signal into a decoded low resolution video signal.

In the American standard, the standard of the Advanced Television System Committee (ATSC) specifies the digital encoding of high definition television (HDTV) signals. Inevitably, some of the standards are identical to the MPEG-2 standard proposed by the Moving Picture Experts Group (MPEG) of the International Orgarnization for Standardization (ISO). The standard is described in the International Standard (IS) publication "Information Technology-Generic Coding of Moving Pictures and Associated Audio, Recommendation H.626" (ISO / IEC 13818-2, IS, 11/94), which is available from ISO. And as a reference for the MPEG-2 digital video coding standard of the present application.

Indeed, the MPEG-2 standard is several different standards. In MPEG-2, different profiles are defined, each corresponding to different levels of complexity of several coded images. For each profile, different levels are defined, each corresponding to a different image resolution. One of the MPEG-2 standards, known as Main Profile, Main Level, is intended to allow video signals to be coded consistent with existing television standards (ie NTSC and PAL). Another standard, known as Main Profile, High Level, is intended to code high resolution television images. An image encoded according to the main profile, high level standard, has 1,152 active lines per image frame and 1,920 pixels per line.

On the other hand, the main profile, main level standard defines a maximum picture size with 720 pixels per line and 567 lines per frame. At a frame rate of 30 frames per minute, signals encoded according to the standard have a data transfer rate of 720 * 567 * 30 or 12,247,200 pixels per minute. On the other hand, the maximum data rate of images encoded according to the main profile, high level standard is 1,152 * 1,920 * 30 or 66,355,200 pixels per minute. The data transfer rate is more than five times the data transfer rate of image data encoded according to the main profile, main level standard. A standard in the United States for HDTV coding is a subset of the standard, with 1,080 lines per frame and 1,920 pixels per line, with a maximum frame rate of 30 frames per minute at this frame size. Still, the maximum data transfer rate for the standard is much greater than the maximum data transfer rate for the main profile, main level standard.

The MPEG-2 standard defines a complex syntax that includes a mixture of data and control information. Part of the control information is used to ensure that signals with several different formats are covered by the standard. The formats define images with different numbers of picture elements (pixels) per line, different numbers of lines per frame or field, and different numbers of frames or fields per minute. In addition, the basic syntax of the MPEG-2 main profile includes a sequence layer, the group of pictures layer, a picture layer, a slice layer, and a macroblock. Defines a compressed MPEG-2 bit stream representing a sequence of images in five layers of a macroblock layer. Each of these layers begins with control information. As a result, other control information, also known as side information (e.g., frame type, macroblock pattern, image motion vector, coefficient zigzag pattern and weight loss information) is distributed over the coded bit stream.

Encoding high resolution main profile, high level pictures low resolution main profile, high level pictures; By format conversion to main profile, main level pictures or other low resolution picture formats, a) provide a single decoder for use with multiple existing video formats, b) main profile, high level signals and personal Provide an interface between computer monitors or existing consumer television receivers, and c) reduce the cost of running HDTV. For example, the conversion is an expensive high-resolution monitor used with main profile, high level encoded pictures, replacing a low cost conventional monitor with low picture resolution, replacing main profile, main level encoded pictures such as NTSC or 525 advanced monitors. Can support Down conversion converts a high resolution input image into a low resolution image for display on a low resolution monitor.

In order to receive digital images effectively, decoders must process video signal information quickly. To be most efficient, the decoding system should be relatively inexpensive and still have enough output to decode the digital signal in real time. As a result, decoders that support conversion to multiple low resolution formats must minimize processor memory.

Is a high level block diagram of a video decoding and format conversion system in accordance with a preferred embodiment of the present invention.

1B is a high level block diagram illustrating functional blocks of an ATV video decoder including an interface to an external memory employed in the preferred embodiment of the present invention.

2A is a high level block diagram of a prior art video decoder.

2B is a high level block diagram of a down conversion system employed in the preferred embodiment of the present invention.

FIG. 2C is a block diagram illustrating the placement of the decoder shown in FIG. 2B used to decode a video signal in 750P format to a 525P / 525I format including down conversion by three factors.

FIG. 2D is a block diagram illustrating the arrangement of the decoder shown in FIG. 2B used to decode a video signal in 750P format including the down conversion by two factors into a 525P / 525I format.

FIG. 3A is a pixel chart showing subpixel positions and predicted pixels corresponding to 3: 1 and 2: 1 in accordance with a preferred embodiment of the present invention. FIG.

3B is a flow diagram of an upsampling process performed for each row of an input macroblock in accordance with a preferred embodiment of the present invention.

4 is a pixel chart showing multiplication pairs for the first and second output pixel values of a preferred embodiment of a block mirror filter.

FIG. 5 is a block diagram illustrating a good performance of a filter for down conversion of a two-dimensional system that processes horizontal and vertical components performed with cascaded one-dimentional IDCT (IDCT). FIG.

FIG. 6A is a macroblock diagram illustrating input and reduced output pixels for a 4: 2: 0 video signal using 3: 1 reduction. FIG.

6B is a pixel block diagram illustrating input and reduced output pixels for a 4: 2: 0 video signal using a 2: 1 reduction.

FIG. 6C is a macroblock diagram illustrating a process of merging two macroblocks into one macroblock to be horizontally down-converted by two and stored in memory; FIG.

6D is a macroblock diagram illustrating a process of merging three macroblocks into one macroblock to be horizontally down-converted by three and stored in memory.

7A is a block diagram of a vertical programmable filter of one embodiment of the present invention.

FIG. 7B is a pixel diagram illustrating the spatial relationship between vertical filter coefficients and pixel sample space of the lines of the vertical programmable filter of FIG. 7A. FIG.

8A is a block diagram illustrating a horizontal programmable filter of one embodiment of the present invention.

Fig. 8B is a pixel diagram showing a spatial relationship between horizontal filter coefficients and pixel sample values in a preferred embodiment of the present invention.

9A is a graph of resampling ratio vs. number of pixels, showing a resampling ratio profile of a preferred embodiment of the present invention.

9B is a graph depicting a first ratio profile that maps a 4: 3 picture to a 16: 9 display.

9C is a graph depicting a second ratio profile that maps a 4: 3 picture to a 16: 9 display.

9D is a graph depicting a first ratio profile that maps a 16: 9 image to a 4: 3 display.

9E is a graph depicting a second ratio profile that maps a 16: 9 picture to a 4: 3 display.

10 is a chart of an image diagram illustrating the effect of using a resampling ratio profile in accordance with a preferred embodiment of the present invention.

Is a high level block diagram of a display section of an ATV video decoder of a preferred embodiment of the present invention.

FIG. 11B is a block diagram illustrating a 27 [MHz] dual output mode of a preferred embodiment of the present invention, wherein video data is 525P or 525I, and a first processing chain is an NTSC encoder (FIG. NTSC encoder) and 27 [MHz] DAC to provide video data.

FIG. 11C is a block diagram showing that in the 27 [MHz] single output mode of the preferred embodiment of the present invention, only 525I video signals are provided to the NTSC encoder. FIG.

FIG. 11D is a block diagram illustrating 74 [MHz] / 27 [MHz] mode of a preferred embodiment of the present invention, where the output format matches the input format and video data is 27 [MHz] DAC or 74 depending on the input format. [MHz] Provided as a DAC.

12 is a high level block diagram of a video decoder with a high bandwidth memory employed by the preferred embodiment of the present invention to decode an ATSC video signal.

The present invention is implemented with a digital video signal processing system to receive, decode and display a video signal encoded in a plurality of different formats. The system may be controlled to include a digital video decoder to decode the encoded video signal and optionally provide a reduced resolution version of the decoded video signal. The system processes the received encoded video signal to determine the format and resolution of the image generated when the signal is decoded. The system includes a controller to receive the determined format and resolution information and to receive information regarding the format and resolution of a display device on which the received image is to be displayed. The controller then generates signals, causing the digital video decoder to provide an analog video signal having a resolution and aspect ratio suitable for the display device.

According to an embodiment of the present invention, an encoded video signal is encoded by using a frequency domain transform operation, and the digital video decoder comprises a low-pass filter that operates on the frequency domain transformed digital video signal. Include.

According to another embodiment of the invention, the digital video decoder is connected to a programmable spatial filter responsive to the control signal provided by the controller, thereby reconstructing the decoded digital video signal provided by the digital video decoder. Sampling generates a digital video signal that matches the aspect ratio and resolution of the display device.

According to another embodiment of the present invention, the digital video signal is encoded in accordance with the use of an encoding technique designated by a moving picture expert group (MPEG), and the aspect ratio and resolution of the encoded video signal are converted by a digital video decoder. It is extracted from the header of the received packetized elementary stream (PES) packet.

According to another embodiment of the present invention, the digital video signal is encoded according to the use of an encoding technique specified by a group of picture experts (MPEG), wherein the aspect ratio and resolution of the encoded video signal are received by the digital video decoder. Extracted from the sequence header of the video bit-stream.

According to another embodiment of the present invention, the system may include a user input device to allow the user to arrange the system to generate an output video signal compatible with the display device.

According to another embodiment of the invention, the system comprises an apparatus for automatically determining the aspect ratio and resolution of the display device.

According to still another embodiment of the present invention, a system includes a device for serially generating video signals corresponding to a plurality of display device types, the system being adapted to a resolution and aspect ratio of the display device in response to a selection signal provided by a user. Identifies one of the corresponding display types.

The above features and advantages of the present invention will be apparent from the accompanying drawings and the description below.

System Overview

A preferred embodiment of the present invention decodes a conventional HDTV signal encoded according to the MPEG-2 standard (especially the Main Profile High Level (MP @ HL) and Main Profile Main Level (MP @ ML) standards), and the decoded signal. It is provided as a video signal having a selected one of a plurality of formats having a lower resolution than the resolution of a received HDTV signal.

The MPEG-2 main profile standard defines a sequence of images at five levels: sequence level, group of pixel levels, pixel level, slice level and macroblock level. Each of these levels can be recorded in a data stream, with the levels listed later being generated as sub-levels in the levels listed earlier. The record for each level includes a header section that holds the data used for decoding the sub-record.

Each macroblock of an encoded HDTV signal contains six blocks, each of which has 64 discrete coefficient values of a discrete cosine transform (DCT) representation of 64 picture elements (pixels) in the HDTV image. Display them.

In the encoding process, prior to the discrete cosine transform may be subject to motion compensated differential coding, the blocks of transformed coefficients are run-length and variable length coding techniques Is further encoded. A decoder that recovers an image sequence from the data stream reverses the encoding process. The decoder employs an entropy decoder (eg, variable length decoder), an inverse discrete cosine transform processor, a motion compensation processor and an interpolation filter.

The video decoder of the present invention is designed to support a number of different picture formats and requires a minimum decoding memory that down converts high resolution encoded picture formats. For example, the memory may be 48 [Mb] parallel Rambus dynamic random access memory (Concurrent RDRAM).

1A shows a preferred embodiment of the present invention for receiving and decoding video information encoded in MP @ HL or MP @ ML and formatting the decoded information into a user-selected output video format (including both video and audio information). A system employing an interface for providing the formatted video output signal to a display device is shown. The preferred embodiment of the present invention is designed to support all ATSC video formats. And, in down conversion (DC) mode, the present invention receives any MPEG main profile video bitstream (forced by the FCC standard) and provides a 525P, 525I or NTSC format picture.

The preferred system of FIG. 1A is a front end interface 100, a video decoder section 120 and decoder memory 130, a main video output interface 140, an audio decoder section 160, an optical computer interface 110. ) And an optional NTSC video processing section 150.

With reference to FIG. 1A, the preferred system includes a preprocessing interface 100 having a transport decoder and a processor 102 having a memory 103. In addition, the preprocessing interface 100 selects, for example, control information received in the IEEE 1394 link layer protocol and a computer-generated image from the computer interface 110, and selects a digital television tuner (not shown). An optional multiplexer 101 to recover the encoded transport stream from < RTI ID = 0.0 > The transport decoder 102 converts the compressed data bitstream received from the communication channel bitstream into compressed video data. For example, it may be a packetized elementary streams (PES) packet according to the MPEG-2 standard. The transport decoder provides all of the PES packets or converts the PES packets into one or more elementary streams.

The video decoder section includes an ATV video decoder 121 and a phase lock loop (PLL) 122. The ATV video decoder 121 receives an element stream or video (PES) packet from the preprocessing interface 100 and converts the packet into an element stream. The pre-processing image processor of the ATV video decoder 121 then decodes the element stream according to the encoding method used to provide luminance and chrominance pixel information for each image image. PLL 122 synchronizes to the audio and video processes performed by the system shown in FIG. 1A.

The ATV video decoder 121 further includes a memory sub-system for controlling a decoding operation using an external memory for providing image picture information, and a display section for processing the decoded picture information into a predetermined picture format. The ATV video decoder 121 employs a decoder memory 130 to process the encoded video signal. Decoder memory 130 includes memory units 131, 132, 133, 134, 135, and 136, each of which may be a 16 [Mb] RDRAM memory. Preferred embodiments of the present invention are described with respect to video decoder section 120 and decoder memory 130.

The primary video output interface 140 includes a first digital-to-analog (D / A) converter (DAC) 141 (luminance signal and C _R and C _B chrominance signals) prior to filter 142 and operating at 74 [MHz]. With three D / A units). The interface generates an analog video signal in 1125I or 750P format. The interface 140 also includes a second D / A converter (DAC) 143 that can operate at 27 [MHz] and prior to the filter 142 for video signals having 525I or 525P. The primary video output interface 140 converts a digitally encoded video signal having a predetermined format, and generates an analog video signal having color difference and luminance components having a predetermined format by using a D / A converter. Filtering the signal removes sampling artifacts of the D / A conversion process.

Audio decoder section 160 includes an AC3 audio decoder 162 for providing audio signals to output ports 163 and 164 and an optional 6-2 channel down mixing processor for providing two channel audio signals to output port 165. A 6-2 channel down mixing processor 161. An audio process for converting MP @ HL MPEG-2 standard audio signal components from encoded digital information to analog output of output ports 163, 164, 165 is well known in the art and is used as decoder 160. A suitable audio decoder is the ZR38500 Six Channel Dolby Digital Surround Processor from Zoran Corporation of Santa Clara, California.

The optional computer interface 110 transmits and receives computer image signals according to, for example, the IEEE 1394 standard. Computer interface 110 includes a physical layer processor 111 and a link layer processor 112. The physical layer processor 111 converts the electrical signal from the output port 113 into computer generated image information or control signal and provides the signals to be decoded by the link layer processor 112 into IEEE 1394 formatted data. . In addition, the physical layer processor 111 converts the received control signal originating from the transport decoder 102 and encoded by the link layer processor 112 into an electrical output signal in accordance with the IEEE 1394 standard.

NTSC video processing section 150 includes an optional ATV-NTSC down conversion processor 151 that converts the analog HDTV signal provided by filter 142 into a 525 I signal. Such conversions between standards are well known in the art and are described, for example, in US Pat. No. 5,613,084 to Hau et al., Incorporated herein by reference, “Interpolation filter selection circuit for converting sample ratios using phase quantization (INTERPOLATION FILTER). SELECTION CIRCUIT FOR SAMPLE RATE CONVERSION USING PHASE QUANTIZATION). In a preferred embodiment of the present invention, the processing section is used only when the decoder processes a 1080I or 1125I signal.

NTSC encoder 152 receives a 525I analog signal from processor 151 or decoder 120 and converts the signal into an NTSC formatted video signal at output ports 153 (S-video), 154 (composite video). Convert.

Video decoder section with decoder memory

1B is a high level block diagram illustrating the functional blocks of ATV video decoder 121 including an interface to external memory 130 employed in the preferred embodiment of the present invention. The ATV video decoder 121 includes a picture processor 171, a macroblock decoder 172, a display section 173 and a memory sub-system 174. The image processor 171 receives, stores and partially decodes the incoming MPEG-2 video bitstream and provides the hatched bitstream, on screen display data and motion vectors, which are under the control of the memory system 174. It may be stored in the memory 130. The macroblock decoder 172 receives the encoded bitstream, the motion vector and the motion compensation reference image data when predictive encoding is used, and stores the decoded macroblock of the encoded video image in memory sub-. To system 174. Display section 173 retrieves the decoded macroblocks from memory sub-system 174 and formats them into a video image picture for display. The operation of the sections is described in detail below.

a) Main profile format support for image processing

The ATV video decoder 121 of the present invention is designed to support all ATSC video formats. That is, the operation of the ATV decoder 121 is called Down Conversion (DC), and the ATV video decoder 121 receives any MPEG main profile video bitstream shown in Table 1 and receives 525P, 525I or NTSC. Provides a format video signal. For the preferred video decoder of FIG. 1A, in DC mode, any HDTV or SDTV signal is decoded and output signal provided at 2 ports (port 1 provides sequential or interlaced image, port 2 provides interlaced image). Display.

TABLE 1

In the DC mode, the low pass filtering of the high frequency components of the main level picture occurs as part of the decoding process to adapt the high resolution picture to a low resolution format. The operation includes horizontal and vertical filtering of high resolution images. In DC mode, the display format conversion can display a 16x9 aspect ratio source on a 4x3 display, and vice versa. The process continues with respect to the display section of video decoder section 120. Table 2 provides the first and second output picture formats for each input bitstream of Table 1:

TABLE 2

b) decoding, down-conversion and down-sampling

1) Overview

2A is a high level block diagram of a typical video decoding system of the prior art for processing MPEG-2 encoded pictures. The general method of decoding an MPEG-2 encoded picture without using subsequent processing, down conversion or format conversion is specified by the MPEG-2 standard. The video decoding system includes an entropy decoder (ED) 211, which may include a parser 209, a variable length decoder (VLD) 210, and a run length decoder. 212. The system also includes an inverse quantizer 214 and an inverse discrete cosine transform (IDCT) processor 218. The controller 207 controls various components of the decoding system in response to the control information retrieved from the input bit stream by the ED 211. For processing of predictive images, the system may include a reference frame memory 222, a summing network 230, a motion vector processor 221, and a half-pixel generator 228. It further includes a compensation processor 206a.

The controller 207 is connected, for example, to an infrared receiver 208 that receives a command signal provided by the user remote control device. Controller 207 decodes the commands and causes the remainder of the system shown in FIG. 2A to perform a particular command. In a preferred embodiment of the present invention, the system shown in FIG. 2A includes a set-up mode in which a user can specify an arrangement for the system. In a preferred embodiment of the invention, the arrangement may comprise a specification of the display device type. The display device type can be specified in terms of display resolution and aspect ratio. The user can specify the display type by selecting a particular display aspect ratio and resolution from a possible selection menu or by causing the system to enter a mode in which signals corresponding to different display formats are continuously provided to the display device, the user being remote The control device can be used to request the display of the best display. During normal operation, the controller 207 also receives information regarding the resolution and aspect ratio of the encoded video signal from the phage 209 of the ED 211. Using the information and stored information regarding the resolution and aspect ratio of the display device, controller 207 causes the system to process the received encoded signal to generate an analog output signal suitable for display on the display device. .

The parser 209 scans the received bitstream for MPEG start codes. The codes include a prefix having the format of a series of 23 bits of zero value followed by a single bit of one value. The value of the start code follows the prefix and identifies the type of record received. In a preferred embodiment of the present invention, when the parser 209 stores the bitstream in the memory 199, the bitstream is then supplied from the memory 199 to the VLD for further processing. In the block diagram shown in FIG. 2A, it is shown that the phage provides the bitstream directly to the VLD.

When the parser 209 finds a start code in the bitstream, the bitstream is transferred to the memory 199, stored in a VBV buffer, and an area of the memory 199 accessed by the controller 207. In the start code therein, a pointer is stored. The controller 207 continues to access the start code pointer and thereby access the record header. When the controller 207 finds the sequence start code, it accesses information in the sequence header that indicates the aspect ratio and resolution of the image sequence represented by the encoded sequence. According to the MPEG standard, this information immediately follows the sequence start code in the sequence header.

Information regarding the display format of the encoded video signal (ie its resolution and aspect ratio) is also received in the header of the packetized element stream (PES) packet. In another preferred embodiment of the present invention, the parser 209 may accept PES packets, detach the headers from the packets, and send header information including the display format of the received video signal to the controller 207. .

As described later, the controller 207 is connected to a decoder system to automatically or semi-automatically adjust information regarding the display format of the received video signal and the process of the received video signal for proper display on a display device. Receive information regarding the display format of the device (not shown).

The VLD 210 receives the encoded bitstream from the parser 209 through a VBV buffer (not shown) in the memory 199 and inverts the encoding process to obtain a macroblock of quantized frequency domain (DCT) coefficient values. to provide. VLD 210 also provides control information including motion vectors. The motion vector describes the relative displacement of the corresponding macroblock in the preceding decoded image relative to the macroblock of the predicted picture that is currently being decoded. Inverse quantizer 214 receives the quantized DCT transform coefficients and reconstructs the quantized DCT coefficients for a particular macroblock. The quantization matrix to be used for the particular block is received from the ED 211.

The IDCT processor 218 uses the reconstructed DCT coefficients in the spatial domain (each block of 8x8 matrix values representing the luminance and chrominance components of the macroblock, and 8x8 representing the differential luminance or differential chrominance components of the predicted macroblock. For each block of matrix values).

If the current macroblock is not predictively encoded, the output matrix values provided by IDCT processor 218 are the pixel values of the corresponding macroblock of the current video image. If a macroblock is inter-frame coded, the corresponding macroblock of the preceding video picture frame is stored in memory 199 for use by motion compensation processor 206. Motion compensation processor 206 receives the predecoded macroblock from memory 199 in response to the motion vector, and then adds the preceding macroblock to the current IDCT macroblock (currently predictively) in summing network 230. Corresponding to the residual component of the encoded frame) to provide a corresponding macroblock of pixels for the current video image, which is stored in the reference frame memory 222.

2B is a high level block diagram of a down conversion system of the preferred embodiment of the present invention employing a DCT filter operation, which operation may be employed by the preferred embodiment of the present invention in DC mode. As shown in FIG. 2B, the down conversion system includes a variable length decoder (VLD) 210, an execution length (R / L) decoder 212, an inverse quantizer 214, and an inverted discrete cosine transform (IDCT) processor. 218. The down conversion system also includes a down conversion filter 216 and a down sampling processor 232 that filter the encoded picture. Although described below relates to a preferred embodiment for MP @ HL encoded input, the present invention applies to any similar encoded high resolution image bitstream.

The down conversion system also includes a motion compensation processor 206b with a motion vector (MV) translator 220, a motion block generator 224 with an upsampling processor 226, and a half pixel. Generator 228 and reference frame memory 222.

The system of the first preferred embodiment of FIG. 2B also includes a Display Conversion Block 280 with a vertical programmable filter (VPF) 282 and a horizontal programmable filter (HZPF) 284. Display conversion block 280 converts the down sampled image into an image displayed on a particular display device having a lower resolution than the original image, as described in detail in section d) (2) of the display conversion.

The down conversion filter 216 performs low pass filtering of high resolution (eg, main profile, high level DCT) coefficients in the frequency domain. The down sampling process 232 generates a set of pixel values that can be displayed on a monitor with a lower resolution than that required for the display of an MP @ HL image by removing spatial pixels by reducing the filtered main profile, high level image. do. The preferred reference frame memory 222 stores spatial pixel values corresponding to at least one previously decoded reference frame having a resolution corresponding to the down sampled picture. For interframe encoding, MV translator 220 scales a motion vector for each block of a received picture that matches the reduction in resolution, and high resolution motion block generator 224 is used for reference frame memory 222. Half-pixel interpolation is required to receive the low resolution motion block provided by, upsample the motion blocks, and provide motion blocks with pixel positions corresponding to the decoded and filtered differential pixel blocks. Perform.

In the down conversion system of FIG. 1B, the down sampled image is stored differently from the high definition image, so that the memory required for storing the reference image is significantly reduced.

The operation of the preferred embodiment of the down conversion system of the present invention for intra-frame encoding is now described. The MP @ HL bitstream is received and decoded by the VLD 210. In addition to the header information used in the HDTV system, the VLD 210 provides DCT coefficients and motion vector information for each block and macroblock. The DCT coefficients are run length coded in R / L decoder 212 and inverse quantized by inverse quantizer 214.

Since the received video image represented by the DCT coefficients is a high resolution image, the preferred embodiment of the present invention employs the low pass of the DCT coefficients of each block before the high resolution video image is reduced. Inverting quantizer 214 provides DCT coefficients to DCT filter 216 that performs low pass filtering in the frequency domain by comparing the DCT coefficients to predetermined filter coefficients before providing them to IDCT processor 218. In a preferred embodiment of the present invention, the filter operation is performed block by block.

IDCT processor 218 performs inverse discrete cosine transform of the filtered DCT coefficients to provide spatial pixel sample values. The down sampling processor 232 reduces the image sample size by erasing the spatial pixel sample values in accordance with a predetermined reduction ratio; Therefore, the storage of a low resolution image uses a small frame memory compared with the frame memory required for the storage of a high resolution MP @ HL image.

Now, the operation of the preferred embodiment of the down conversion system of the present invention on the predicted frame of the coding standard is described. In this case, the currently received image DCT coefficients represent the DCT coefficients of the residual components of the predicted image macroblock, for convenience, which is called a predicted frame (P frame). In the above preferred embodiment, since the low resolution reference pictures of the preceding frames stored in the memory do not have the same number of pixels as the high resolution predicted frame MP @ HL, the horizontal component of the motion vector for the predicted frame is scaled.

2B, the motion vectors of the MP @ HL bitstream provided by the VLD 210 are provided to the MV translator 220, each of which is stored in the reference frame memory 222 of the memory 199. Scaled by MV translator 220 to quote the appropriate predictive block of the reference frame of the preceding image. The size (number of pixel values) in the search block is smaller than the block provided by IDCT processor 218; Thus, the search block is upsampled to form a predictive block having the same number of pixels as the remaining block provided by IDCT processor 218 before the blocks are combined by summing network 230.

The predictive block is upsampled by the upsampling processor 226 in response to a control signal from the MV translator 220, thereby generating a block corresponding to the original high resolution pixel block, and then appropriate spatial alignment of the predictive block. Half pixel values are generated (if generated by the motion vector for the upsampled prediction block in half pixel generator 228) to ensure spatial alignment. The upsampled and ordered prediction block is added to the currently filtered block in summing network 230, eg, the reduced resolution residual component from the prediction block. All processing is done for each macroblock. After the motion compensation process for the current high resolution macroblock ends, the reconstructed macroblock is reduced by the down sampling processor 232. The process does not reduce the resolution of the image, but simply removes the extra pixels from the low resolution filtered image.

If down sampled macroblocks are available for the image, display conversion block 280 filters the vertical and horizontal components of the down sampled image in VPF 282 and HZPF 284, respectively, to display on the low resolution television display unit. Adjust the image as much as possible.

The relationship between the functional blocks of the ATV video decoder 121 of FIGS. 1A and 1B will now be described. The picture processor 171 of FIG. 1B receives the video picture information bitstream. The macroblock decoder 172 includes a VLD 210, an inverted quantizer 214, a DCT filter 216, an IDCT 218, a summing network 230, and motion compensated predictors 206a, 206b. ). The image processor 171 may include a VLD 210. The external memory 130 corresponds to the memory 199 and includes 16 [Mb] RDRAMs 131-136 which accommodate the reference frame memory 222.

2C illustrates the operation of the system in DC mode, where the 1125I signal is converted to the 525P / 525I format. In this scenario, after low pass filtering by the DCT filter 216 as described above with reference to FIG. 2B, the system downsamples the high resolution signal by three factors and stores the image into 48 [Mb] memories of 640H and 1080V. Save them. For the system, the motion compensation process upsamples the stored pictures by three factors (as well as the translation of the received motion vectors) before the achievement of motion-predictive decoding. In addition, the picture is horizontally and vertically filtered for display conversion.

2D shows the relationship between DC mode format down conversion from 750P to 525P / 525I format. The conversion works like the 1125I-525P / 525I conversion except that downsampling for memory storage and upsampling for motion compensation is by a factor of two.

2) Macroblock Prediction for Down Conversion

In the case of the example of the down conversion process, since the size of the reference frame of the preceding image is reduced in the horizontal direction, the received motion vector pointing to the frame is translated according to the conversion ratio. In the following, translation in the horizontal direction with respect to the luminance block is described. Those skilled in the art can also easily perform motion translation in the vertical direction if necessary. When the current macroblock address in the original image frame is represented by x and y, Dx is the horizontal decimation factor and mv _x is the half pixel horizontal motion vector in the original image frame. In this case, the address of the top left pixel of the motion block in the original image frame is represented by XH in the half pixel unit, and the relation is given by equation (1):

The pixel corresponding to the motion convex starts with a down sampled image and has an address represented by x ^* and y ^* determined by equation (2).

The value of equation (2) is an integer value of truncation.

Since the preferred filter 216 and down sampling processor 232 reduce only the horizontal component of the image, the vertical component of the motion vector is not affected. For chrominance data, the motion vector is half of the luminance motion vector in the original image. Thus, the specification for translation of chrominance motion vectors also uses two equations (1) and (2).

Motion prediction is performed by a two-step process: first, pixel accuracy motion estimation in the original image frame is subjected to upsampling of the down sampled image frame in the upsampling processor 226 of FIGS. 2A and 2B. Half pixel generator 228 then averages approximate pixel values to perform half pixel interpolation.

Reference image data is added to output data provided by IDCT processor 218. Since the output values of summing network 230 correspond to an image with multiple pixels that match the high resolution format, the values can be down sampled for display on a display with low resolution. Downsampling in downsampling processor 232 is substantially equivalent to subsampling of image frames, but adjustments are performed based on the conversion ratio. For example, in 3: 1 down sampling, the number of horizontal down sampled pixels is 6 or 5 for each input macroblock, and the first down sampled pixels do not always become the first pixel in the input macroblock.

After obtaining the corrected motion prediction block from the down sampled image, upsampling is used to obtain the corresponding prediction block in the high resolution image. Thus, subpixel accuracy of motion block prediction is desirable in down sampled pictures. For example, using 3: 1 reduction, it is desirable to have 1/3 (or 1/6) sub-pixel accuracy for down-converted pictures for proper motion prediction. In addition to the down sampled motion block, the sub-pixel which is the first pixel obtained by the motion vector is determined. Subsequent sub-pixel positions are then determined by the use of later modulo operations. The sub pixel positions are denoted by x _s , given by equation (3):

Here, "%" represents a modulo manufacturing method.

For example, the range of x _s is 0, 1, 2 for 3: 1 upsampling and 0, 1 for 2: 1 upsampling. FIG. 3A shows the sub-pixel locations and the 17 predicted pixels corresponding to the 3: 1 and 2: 1 cases, and Table 3 shows the legend for FIG. 3A.

TABLE 3

As described above, the upsampling filters can be upsampling polyphase filters, and Table 4 provides the characteristics of the upsampling polyphase interpolation filters.

TABLE 4

Tables 5 and 6 below describe polyphase filter coefficients for good 3: 1 and 2: 1 upsampling polyphase filters.

TABLE 5

TABLE 6

In the fixed point notation, the numbers in parentheses of Tables 5 and 6 are auxiliary signs of 2 in 9 bits corresponding to the left double precession numbers. According to the sub-pixels of the motion prediction block in the down sampled reference image frame, the corresponding phase of the polyphase interpolation filter is used. Also, for the preferred embodiment, the left and right additional pixels are used to interpolate 17 horizontal pixels into the original image frame. For example, in the case of a 3: 1 reduction, up to six horizontal down sampled pixels are provided for each input macroblock. However, in upsampling, since the upsampling filter requires more left and right pixels outside the boundary for the operation of the filter, nine horizontal pixels are used to provide corresponding motion prediction block values. Since the preferred embodiment employs half pixel motion estimation, 17 pixels are needed to obtain 16 half pixels, which are average values of about two pixel samples. A half pixel interpolator performs an interpolation operation to provide a block of pixels of half pixel resolution. Table 7A shows a good idea between the sub pixel positions and the polyphase filter elements and shows the number of left pixels needed in addition to the pixels in the up sampled block for the up sampling process.

TABLE 7A

3B summarizes the upsampling process performed for each row of the input macroblock. First, in step 310, a motion vector for a block of the image frame to be processed is received. In step 312, the motion vector is translated to correspond to a down sampled reference frame in memory. In step 314, the scaled motion vector is used to calculate the coordinates of a predetermined reference image half macroblock stored in memory 130. In step 316 the subpixel point for the half macroblock is determined, and then in step 318 initial polyphase filter values for upsampling are determined. Thereafter, in step 320, the identified pixels for the reference half macroblock of the stored down sampled reference frame are retrieved from the memory 130.

Prior to the first pass of the filtering step 324, the register of the filter is initialized in step 322, where the preferred embodiment includes loading the register with initial 3 or 5 pixel values. Then, at the end of the filter step 324, in step 326 the process determines whether all pixels (17 pixels in a preferred embodiment) have been processed. If all the pixels have been processed, the upsampled block is complete. In the preferred embodiment, the 17x9 pixel half macroblock is returned. The system returns the upper or lower half macroblocks to allow motion prediction decoding of both sequential and interlaced scanned images. If all pixels have not been processed, the phase is updated at step 328 and it is retrieved whether the phase has a value of zero. If the phase is zero, the register is updated for the subsequent set of pixel values. In step 328, updating the phase updates the phase values to 0, 1, and 2 for the filter loop period for 3: 1 upsampling, and sets the phase values 0 and 1 for the filter loop period for 2: 1 upsampling. Update to. If the leftmost pixel is outside the boundary of the image image, the first pixel value in the image image is repeated.

For the preferred embodiment, the upsample filter operation can be performed according to the following guidelines. First, several factors are used: 1) the half pixel motion prediction operation averages two complete pixels, and corresponding filter coefficients are also averaged to provide half pixel filter coefficients; 2) a fixed number of filter coefficients (eg, 5), which may be equivalent to the number of filter taps, may be employed regardless of the particular down conversion; 3) 5 input pixels (LWR (0) -LWR (4)) for each clock transition for each reference block combined with corresponding filter coefficients to provide one output pixel. Parallel input ports may be provided to the upsampling block for each forward and reverse lower and upper block; 4) The sum of the filter coefficients h (0) -h (4) combined with each of the pixels LWR (0) -LWR (4) provides an output pixel of the sampling block.

Since the multiplication instruction is contrary to the normal instruction of the filter coefficients, the filter coefficients are preferably inverted and some coefficients are preferably zero. Table 7B discloses good coefficients for a 3: 1 upsampling filter, and Table 7C discloses good coefficients for a 2: 1 upsampling filter:

TABLE 7B

TABLE 7C

In Tables 7B and 7C, x ^* is the down sampled pixel position defined in equations (1) and (2), and the sub pixel position x _s is redefined by equation (3), such as equation (3 ').

For color difference values of good performance, XH is scaled to 2, and equations (1), (2) and (3 ') apply. In one embodiment, phase and half pixel information (coded with 2 bits and 1 bit, respectively) is used by motion compensation processor 220 and half pixel generator 228 of FIG. 2B. For example, reference block pixels are first provided in U pixels, then V pixels, and finally Y pixels. U and V pixels are clocked in 40 cycles, and Y pixels are clocked in 144 cycles. The reference blocks may be provided for a 3: 1 reduction by providing five first pixels, repeating twice, transitioning the data by one, and repeating until the row is terminated. The same method can be used for a 2: 1 reduction, except that here it is repeated twice rather than twice. Since the reduction depends on the sum of the output from the motion compensation and the half pixel generation of the residual value, the input pixels are repeated. Thus, for a 3: 1 reduction, two of the three pixels are deleted, and dummy pixels for the pixel values are not important.

3) DCT domain aftermath employing weights of DCT coefficients

A preferred embodiment of the present invention includes the DCT filter 216 of FIG. 2A which processes DCT coefficients in the frequency domain, which replaces the low pass filter in the spatial domain. There are several advantages to the DCT domain filter in place of the spatial domain filter for DCT coded pictures, which have been considered by the MPEG or JPEG standards. The DCT domain filter is more efficient in using a computer and requires less hardware than the spatial domain filter applied to spatial pixel sample values. For example, a spatial filter with N taps is used as many as N additional multiplications and additions for each spatial pixel sample value. This compares only one additional multiplication of the DCT domain filters.

The simplest DCT domain filter of the prior art is the truncation of high frequency DCT coefficients. However, breaks in the high frequency DCT coefficients do not result in a smooth filter, and have drawbacks such as "ringing" near the edge of the decoded picture. The DCT region low pass filter of the preferred embodiment of the present invention is derived from a block mirror filter in the spatial region. Filter coefficient values for the block mirror filter can be utilized, for example, by numerical analysis in the spatial domain, and the values are then converted into coefficients of the DCT domain filter.

Although the preferred embodiment only discloses the DCT region filter in the horizontal direction, the DCT region filter can be performed by one of the horizontal or vertical directions or by a combination of horizontal and vertical filters.

4) Derivation of DCT Domain Filter Coefficients

Preferred filters of the present invention are derived from two limitation conditions:

First, to process the image data on a per block basis for each block of the image without using information from the preceding block of the picture; Second, the visibility of the block boundary that occurs when the filter processes the boundary pixel values is reduced.

According to the first constraint, in DCT based on the compression of an MPEG image sequence, for example, N × N DCT coefficients provide N × N spatial pixel values. Thus, the preferred embodiment of the present invention performs a DCT area filter that processes only the current block of the received picture.

According to a second constraint condition, if the filter is simply provided to a block of spatial frequency coefficients, a transition of the filter operation at the block boundary caused by an insufficient number of spatial pixel values beyond the boundary is generated, resulting in the remaining of the filter. Fill your wealth. That is, since the N-tap filter has only values for N / 2 taps, the coefficient values at the edge of the block cannot be filtered properly. There are several ways of supplying missing pixel values: 1) repeating certain constant pixel values across boundaries; 2) repeating the same pixel value as the boundary pixel value; 3) Reflect the pixel values of the block to simulate preceding and subsequent blocks of pixel values adjacent to the processed block. Without prior information on the contents of the preceding and subsequent blocks, the mirroring method of repeating pixel values is considered a good method. Thus, one embodiment of the present invention employs a reflection method for the filter, which is called a "block mirror filter".

The following is a description of the preferred embodiment, which causes the horizontal block mirror filter to cause the low pass filter 8 to input the spatial pixel sample values of the block. If the size of the input block is an 8x8 block matrix of pixel sample values, horizontal filtering can be performed by applying a block mirror filter to each row of eight pixel sample values. The fact that the filtering process is performed by applying filter coefficients to the block matrix, and that multidimentional filtering can be achieved by filtering the rows of the command after filtering the rows of the block matrix. The fact is apparent to those skilled in the art.

4 shows a filter tap for a good mirror filter of eight input pixels employing input pixel values x ₀ to x ₇ (group X ₀ ) and 15 tap spatial filters with tap values represented by h ₀ to h ₁₄ . Good correspondence between them is shown. The input pixels are reflected to the left of group X0 (group X1) and to the right of group X0 (group X2). The output pixel value of the filter is the sum of fifteen products between the filter tap coefficient values and the corresponding pixel sample values. 4 shows multiplication conjugates for the first and second output pixel values.

The following shows that the block mirror filter in the spatial domain is equivalent to the DCT domain filter. The mirror filter is related to the circular convolution of 2N (N = 8) points.

The vector x 'is shown in equation (4).

In this case, N = 8,

x '= (x0, x1, x2, x3, x4, x5, x6, x7, x7, x6, x5, x4, x3, x2, x1, x0)

Rearrange the filter tap values h ₀ to h ₁₄ and mark the rearranged values with h '.

h '= (h7, h8, h9, h10, h11, h12, h13, h14, 0, h0, h1, h2, h3, h4, h5, h6)

Thus, the mirror filtered output y (n) is a circular convolution of x '(n) and h' (n), which is given by equation (5).

This equation is the same as that in equation (6).

Where x '[n-k] is the circular modulo of x' (n),

x '[n] = x' (n); n ≥ 0

x '[n] = x' (n + 2N); n <0.

The circular convolution in the spatial domain represented by equation (5) is equal to the scalar product in the Discrete Fourier Transform (DFT) domain. If the DFT of y (n) is Y (k), equation (5) becomes equation (7) in the DFT region.

Where X '(k) and H' (k) are the DFTs of x '(n) and y' (n), respectively.

Equations (4) to (7) are valid for filters with taps less than 2N. Also, if the filter is a symmetric filter with odd taps, H '(k) becomes a real number. Thus, X '(k), which is the DFT of x' (n), can be weighted with real H '(k) in the DFT frequency domain, instead of 2N multiplication and 2N addition operations in the spatial domain for performing the filtering operation. The values of X '(k) are very closely related to the DCT coefficients of the original N-point x (n), which is x (2N) where the N-point DCT of x (n) is x (n) and its mirror. This is because it is obtained by a 2N-point DFT of x '(n), which is a binding sequence composed of -1-n).

The following describes H '(k), which is the derivative of the DFT coefficients of the spatial filter, assuming a symmetric filter with odd taps 2N-1 (h (n) = h (2N-2-n) , h '(n) = h' (2N-n), h '(N) = 0). In Eq. (8), H '(k) is defined.

Where W _2N ^kn = exp {-2 pi kn / (2N)}; And H '(k) = H' (2N-k).

The inventor has determined X '(k), which is a 2N-point DFT of x' (n), which can be represented by its DCT coefficients as shown in equation (9).

On the other hand, the DCT coefficients of x (n) and C (k) are given by equation (10).

And in other cases C (k) = 0.

The values of X '(k), which are the DFT coefficients of x' (n), can be represented by C (k), which is the DCT coefficients of x '(n), which are represented in equation (11).

The original spatial pixel sample values x (n) may be obtained by IDCT (Inverse Discrete Cosine Transformation), which is represented by equation (12).

Here, α (k) = 1/2 for k = 0, and 1 otherwise.

The values of y (n) for 0 ≦ n ≦ N-1 are obtained by IDFT of X '(k) H' (k), which is shown in equation (13):

The values y (n) of equation (13) are the spatial values of IDCT of C (k) H '(k). Thus, the spatial filter can be replaced by H '(k) by DCT weighting of the input frequency domain coefficients representing the image block, and then performing the IDCT of the weighted values to perform the filter in the spatial domain. Reconstructed pixel values.

One embodiment of the preferred block mirror filter of the present invention is derived by the following steps: 1) a one-dimensional low pass symmetric filter is selected with an odd number of taps less than 2N taps; 2) the filter coefficients are increased to 2N by padding of zeros; 3) the filter coefficients are rearranged so that the original intermediate coefficients go to the 0 th position by the left circular transition; 4) the rearranged filter coefficients are DFT coefficients are determined; 5) DCT coefficients are multiplied by the real DFT coefficients of the filter; 6) Inverse Discrete Cosine Transform (IDCT) of the filtered DCT coefficients is performed to provide a block of low pass filtered pixels prepared for reduction.

The cutoff frequency of the low pass filter is determined by the reduction ratio. For a preferred embodiment, the cutoff frequency is π / 3 for 3: 1 reduction and π / 2 for 2: 1 reduction, where π corresponds to half of the sampling frequency.

The DCT region filter of MPEG and JPEG decoders allows for reduction of memory equipment, which requires that the inverted quantizer and the IDCT of blocks already exist in the prior art decoder, requiring only additional scalar product of DCT coefficients by the DCT region filter. Because. Thus, a single DCT domain filter block product is not physically required for a particular implementation; Another embodiment of the present invention simply combines DCT domain filter coefficients with IDCT processing coefficients and applies the combined coefficients to an IDCT operation.

For the preferred down conversion system of the present invention, reduction of DCT coefficients and horizontal filtering are considered; And the following are two good practices:

1.640 to 1080 interlaced conversion from 1920H to 1080V interlace (3: 1 horizontal reduction)

2.640 to 720 sequential conversion from 1280H to 720V sequential (horizontal 3: 1 reduction)

Table 8 shows the DCT block mirror filter (weighted) coefficients; In Table 8, the numbers in parentheses are the secondary signs of 10-bit 2. "*" In Table 8 indicates beyond the bound value for the secondary indication of 10-bit 2. Because the value exceeds 1; However, it is known to those skilled in the art that multiplying the column coefficients of the block by the value indicated by * can be easily performed by adding the coefficient value to the coefficient multiplied by the fractional value of the filter value. .

TABLE 8

The horizontal DCT filter coefficients weight each column in an 8x8 block of DCT coefficients of the encoded video image. For example, the DCT coefficients of column 0 are weighted by H [0] and the DCT coefficients of the first column are weighted by H [1].

The foregoing has described horizontal filter performance using one-dimensional DCTs. As is known in the digital signal processing art, the processing can also be applied to two-dimensional systems. Equation (12) describes the IDCT for one dimension, and therefore, more general two-dimensional IDCT is described in Equation (12 '):

here Is Or 1 (in other cases).

Where f (x, y) is a spatial domain representation, x and y are spatial coordinates in the sample domain, and u and v are coordinates in the transform domain. Since the coefficients C (u), C (v) are known as the values of the cosine terms, only the transform domain coefficients should be provided to the processing algorithm.

For a two-dimensional system, the input sequence is represented by a matrix of values representing the individual coordinates of the transform region, which matrix consists of a column sequence of period M and a row sequence of period N (M , N may be an integer). The two-dimensional DCT may be performed as a one-dimensional DCT performed on a column of the input sequence, and then the second one-dimensional DCT may be performed on a row of the input sequenced DCT. In addition, as is known in the art, two-dimensional IDCT can be performed as a single process.

5 shows a good performance of the down conversion filter for a two-dimensional system that processes horizontal and vertical components performed as a cascaded one-dimensional IDCT. As shown in FIG. 5, the DCT filter mask 216 and IDCT 218 of FIG. 2 may include a vertical processor including a vertical DCT filter 530 and a vertical IDCT 540. 510 and a horizontal processor 520 that includes the same horizontal IDCT and horizontal DCT filter as performed for vertical components. Since the filter and IDCT processes are linear, the order in which the processes are performed can be rearranged (e.g., horizontal and vertical DCT filters first and horizontal and vertical IDCT later, or vice versa, or vertical processor 520). ) And the horizontal processor 510 later.

In the particular implementation shown in FIG. 5, vertical processor 510 precedes block transpose operator 550, exchanging rows and columns of blocks of vertically processed values provided by the vertical processor. This operation is used to increase the efficiency of computation by arranging blocks for processing by the horizontal processor 520.

For example, an encoded video block, such as an 8x8 block of matrix values, is received by vertical DCT filter 530, which weights each row entry of the block by DCT filter values corresponding to some vertical reduction. Vertical IDCT 540 then performs an inverse DCT on the vertical components of the block. As mentioned above, both processes simply perform matrix multiplication and addition, so that DCT LPF coefficients can be combined with the vertical DCT coefficients for matrix multiplication and addition operations. The vertical processor 510 then provides the vertically processed blocks to the binomial operator 550, which provides the horizontal processor 520 with the binomial block of vertically processed values. The binary operator 550 is not essential unless the IDCT operation is performed solely by row or column. Horizontal processor 520 performs weighting of each column entry of the block by DCT filter values corresponding to a predetermined horizontal filter, and then performs inverse DCT on the horizontal component of the block.

As described above with reference to equation (12 '), only coefficients in the transform domain are provided to the processing algorithm; The operations are linear and allow mathematical manipulation only on the coefficients. As is evident from equation (12 '), the manipulation to the IDCT forms a summation of the products. Thus, hardware introduction requires a group of known coefficients to be stored in a memory, such as a ROM (not shown), and multiplication and addition circuits that accept the coefficients from the ROM as well as selected coefficients from the input transform coordinates. do. For more advanced systems, the ROM accumulator method can be used when the order of mathematical operations is modified in accordance with distributed arithmetic to convert from the summation introduction of enemies to the bit-serial introduction. For example, the technique is described in Stanley A. White's Application of Distributed Arithmetic to Digital Signal Processing: A Tutorial Review, IEEE ASSP Magazine, July, 1989, using symmetry in calculations. Decreases the total gate count of the enemy's summation introduction.

In alternative embodiments of the present invention, the DCT filter operation may be combined with an inverse DCT (IDCT) operation. For this embodiment, since the filter and inverse transform operations are linear, the filter coefficients can be combined with the coefficients of IDCT to form a modified IDCT. As is known in the art, the modified IDCT, the combined IDCT, and the DCT down transform filter can be performed through hardware introduction similar to the introduction of a simple IDCT operation.

c) Memory Subsystem

1) Storage and memory access of image data and bitstream

As shown in FIG. 1B, a preferred embodiment of the present invention is an ATV video decoder 121 having a memory subsystem 174 that controls reading information from or storing information in the memory 130. ) Is adopted. Memory subsystem 174 provides picture data and bitstream data to memory 130 for video decoding operations, and in preferred embodiments, at least two pictures (or frames) are MPEG-2 encoded video data. Is used for proper decoding of. An optional on screen display (OSD) section in memory 130 may be used to support OSD data. The interface between memory subsystem 174 and memory 130 may be a simultaneous RDRAM interface providing 500 [Mbps] channels, and three RAMBUS channels may be used to support the required bandwidth. Embodiments of the present invention having an image processor 171, a macroblock decoder 172, and a memory subsystem 174 that operate in conjunction with an external memory 130 are disclosed in US Pat. 5,623,311, N.V.Philips, " MPEG VIDEO DECODER HAVING A HIGH BANDWIDTH MEMORY " 12 is a high level block diagram of the system of a video decoder with a high bandwidth memory employed by the preferred embodiment of the present invention to decode MP @ ML MPEG-2 pictures.

In summary, as described in FIGS. 1A and 1B, US Pat. 5,623,311 describes a single high bandwidth memory with a single memory port. Memory 130 maintains an input bitstream, first and second reference frames used in the motion compensation process, and image data indicating a field that is currently being decoded. The decoder comprises: 1) circuitry for storing and extracting bitstream data (image processor 171), and 2) extracting reference frame data and storing image data for the current decoded field in the block format (macroblock decoder). 172), the circuitry for extracting the image data for conversion to an interlaced scan format (display section 173). Memory operations are time division multiplexed using a single common memory port with a defined memory access time period (macroblock time (MblkT) for control operations). The digital phase lock loop (DPLL) 122 counts the pulses of the 27 [MHz] system clock signal defined in the MPEG-2 standard to generate a count. The count value is compared with a series of externally supplied system clock reference (SCR) values to generate a phase difference signal used to adjust the frequency of the signal generated by the digital phase lock loop.

Table 9 summarizes the image storage equipment required for DC deployments that support multiple formats:

TABLE 9

For DC mode, 1920 × 720 pictures are horizontally reduced by 3 factors to provide a 640 × 1080 picture; 1280 × 720 images are horizontally reduced by 2 factors to provide a 640 × 720 image. 704x480 and 640x480 images do not need to be reduced in DC mode.

Accommodating a plurality of DC pictures in the memory 130 requires supporting each decoding operation in accordance with the corresponding picture display timing. For example, sequential pictures occur at twice the speed of interlaced pictures (60 or 59.94 [Hz] sequential versus 30 or 29.97 [Hz] interlaced), and therefore, sequential pictures are faster than interlaced pictures (60 or 59.94 frame sequential per minute). 30 or 29.97 frames interlaced). Thus, the decoding rate is limited by the display rate for the format, and if a less stringent decoding rate of 59.97 or 29.97 frames per minute is used rather than 60 or 30 frames per minute, one of every 1001 frames is missing from the conversion. That is, the decoding operation for the format may be measured in units of "macroblock time (MblkT)" defined as a time period in which all decoding operations for the macroblock are completed. As defined in Equation 14, the control signal and memory access operation using the time period can be defined while the MblkT time period occurs regularly.

In addition, a blanking interval may not be used for picture decoding of interlaced pictures, and an 8-line margin for the time period decodes 8 lines (interlaced) simultaneously and 16 Added to decode lines (progressive) simultaneously. Thus, as given in equations (15) and (16), an adjustment factor AdjFact can be added to the MblkT.

Table 10 lists the MblkT for each of the supported formats:

TABLE 10

In a preferred embodiment of the present invention, MblkT of 241 clocks is employed for all formats to meet the requirements of the fastest decoding time including a small margin. For the selected MblkT time period, slow format decoding includes a time period with no decoding activity; Thus, it can be employed to reflect linear decoding with a stall that occurs to stop decoding in the selected MblkT time period.

Referring to FIG. 1B, the memory system 174 may provide an internal image data interface to the macroblock decoder 172 and the display section 173. The decoded macroblock interface accepts decoded macroblock data and stores the decoded macroblock data in the correct memory address location of memory 130 in accordance with a memory map defined for a given format. The memory address is derived from the macroblock number and the picture number. Macroblocks may be received as macroblock rows on three channels, one channel per 16 [Mb] memory device at the system clock rate (131-136 in FIG. 1A). Each memory device may have two partitions for each picture, each partition using a top and bottom address. For interlaced images, one compartment carries Field 0 data and the other carries Field 1 data, and for sequential images, both upper and lower compartments are treated as a single compartment. And data is returned for the entire frame. Every macroblock is decoded and stored for every picture, with the exception of a 3: 2 pulldown mode where decoding is paused for the entire field time period. In 3: 2 pull down mode, a signal with a frame rate of 24 frames per second is displayed at 60 frames per minute (or field) by displaying one frame twice and the subsequent frames three times.

The reference macroblock interface supplies the macroblock decoder 172 with stored and previously decoded picture data for motion compensation. The interface supplies two, one or zero macroblocks corresponding to bidirectional prediction (B) encoding, unidirectional prediction (P) encoding or inner (I) encoding. Each reference block is supplied by the use of two channels, each containing half of the macroblock. For DC mode employing two factor reduction, each retrieved half macroblock 14 × 9 (Y), 10 × 5 (C _R ) and 10 × 5 (C _B ) are provided to allow upsampling and half pixel resolution. .

The display interface provides the retrieved pixel data to the display section 173 and multiplexes the Y, C _R and C _B pixel data into a single channel. Two display channels are provided to support conversion from sequential formats to interlaced formats or from interlaced formats to sequential formats. In the DC mode, the first channel can provide up to four lines of interlaced or sequential data and the second channel can provide up to four lines of interlaced data.

For down conversion, down sampled macroblocks are merged into a single macroblock for storage. The down sampling process in DC mode is described below with reference to FIGS. 6A and 6B. 6C shows a process of merging two macroblocks into one macroblock for two factor horizontal down conversion to store in memory 130. 6D shows a process of merging three macroblocks into one macroblock for three factor horizontal down conversion to store in memory 130.

d) downsampling and display conversion of the display section;

1) Downsampling for Low Resolution Formats

Down sampling is accomplished by the down sampling process 232 of FIG. 2B to reduce the number of pixels in the down converted image. 6A shows input and reduced output pixels for a 4: 2: 0 signal format with a 3: 1 reduction. 6B shows input and reduced output pixels of a 4: 2: 0 color difference type of 2: 1 reduction. Table 11 provides a legend for the luminance and chrominance pixels of FIGS. 6A and 6B. The pixel positions before and after down conversion of FIGS. 6A and 6B are interlaced (3: 1 reduced) and sequential (2: 1 reduced) cases, respectively.

TABLE 11

For down sampling of an interlaced image, which can be a conversion from a 1920x1080 pixel image to a 640x1080 pixel horizontal compressed image, two of every three pixels are reduced on the horizontal axis. For a good 3: 1 reduction, there are three different macroblock types after the down conversion process. In Fig. 6A, the original macroblocks are indicated as MB0, MB1, MB2. The down sampled luminance pixels of MB0 start from the first pixel of the original macroblock, but in MB1 and MB2 the down sampled pixels start from the third and second pixels. In MB0, there are six horizontal down-sampled pixels, while in MB1 and MB2 there are five pixels. The three MB types repeat and thus modulo 3 arithmetic is applied. Table 12 summarizes the number of down sampling pixels and offsets for each input macroblock MB0, MB1, MB2.

TABLE 12

For down sampling of sequential format images, the luminance signal is horizontal subsampled to the sample every minute. For the chrominance signal, the down sampled pixel has a spatial position as low as a half pixel position relative to the pixel position in the original image.

2) display conversion

The display section 173 of the AVT decoder 121 of FIG. 1B is used for the format of picture information (decoded picture information) stored for a specific display format. 11A is a high level block diagram illustrating the display section of an ATV video decoder 121 for a preferred embodiment of the present invention.

Referring to FIG. 11A, two output video signals are supported, the first output signal VID _out1 supporting any selected video format and the second output signal VID _out2 supporting only 525I (CCIR-601). Each output signal is processed by a separate set of display processing elements 1101 and 1102, respectively, which perform horizontal and vertical upsampling / downsampling. This arrangement is desirable when the display aspect ratio does not match the aspect ratio of the input image. An optional on screen display (OSD) section 1104 is included to provide on screen information to one of the supported output signals VID _out1 and VID _out2 to form display signals V _out1 and V _out2 . All processing is performed at an internal clock rate, except for the control of the output signals V _out1 and V _out2 at the output controllers 1126 and 1128, which are at the pixel clock rate. Is performed. For the preferred embodiment, the pixel clock rate may be a luminance pixel rate or twice the luminance pixel rate.

Since the display sets of the processing elements 1101 and 1102 operate similarly, only the operation of the display processing set 1101 is described. Referring to the display processing set 1101, four lines of pixel data are provided from the memory 130 (shown in FIG. 1A) to the vertical processing block 282 (shown in FIG. 2B) in raster size. . Each line supplies 32 bits of C _R , Y, C _B , and Y data at the same time. Vertical processing block 282 then filters the four lines into one line and provides the filtered data to horizontal processing block 284 (shown in FIG. 2B) in a 32-bit C _R YC _B Y format. . Horizontal processing block 284 provides the correct number of pixels for the selected raster format as formatted pixel data. Thus, the filtered data rate entering horizontal processing block 284 need not be the same as the output data rate. In the up sampling case, the input data rate will be lower than the output data rate. In the down sampling case, the input data rate will be higher than the output data rate. The formatted pixel data may have background information inserted by the optional background processing block 1110.

As will be appreciated by those skilled in the art, the elements of display section 173 are controlled by controller 1150, which is set by parameters that are read from or written to the microprocessor interface. The controller generates a signal CNTRL, and the control is necessary to influence and adjust the proper circuit operation, loading and transitioning of pixels, signal processing.

The data from the horizontal processing block 284, the data from the second horizontal processing block 284a, and the HD (non-sequential) video data on the HD Bypass 1122 are multiplexer 118. And a multiplexer to combine the video data with the optional OSD data from the OSD processor 1104 to form a composite output video data by selecting one video data stream under processor control (not shown). (116). The composite video output data is then provided to MUXs 1120 and 1124.

For the first set of processing elements 1101, the MUX 1120 may select composite output video data, HD data provided on the HD bypass 1122, or data from the background insertion block 1110. . The selected data is provided to an output control processor 1126 that also receives a pixel clock. The output control processor 1126 then changes the data clock rate from the internal processing region to the pixel clock rate in accordance with the desired output mode.

For the second processing elements 1102, the MUX 1124 may select composite output video data or data from the background insertion block 1110a. The selected data is provided to an output control processor 1128 that also receives a pixel clock. The output control processor 1128 then changes the data clock rate from the internal processing region to the pixel clock rate according to the predetermined output mode. MUX 1132 provides received selected data (601 data out) of MUX 1124 or optional OSD data from OSD processor 1104.

The raster generation and control processor 1130 also receives a pixel clock, includes counters (not shown) that generate raster space, and sends control commands to the display control processor 1140 line by line. Display control processor 1140 adjusts timing with external memory 130 and initiates processing for each processing chain 1101, 1102 per line in synchronization with raster lines.

11B-11D associate the output modes provided by the display section 173 shown in FIG. 11A of the video decoder 121 to the active blocks of FIG. 1A. 11B shows that if the video data is 525P or 525I, the first processor 1101 (shown in FIG. 11A) provides 525I data (601 data out) to NTSC encoder 152 as well as providing 525P video data at 27 [MHz]. FIG. 27 shows the 27 [MHz] dual output mode provided to the DAC 143. FIG. 11C shows that only 525I data (601 data out) is provided to NTSC encoder 152 in 27 [MHz] single output mode. FIG. 11D shows a 74 [MHz] / 27 [MHz] mode in which the output mode matches the input mode and video data is provided to the 27 [MHz] DAC 143 or 74 [MHz] DAC 141 depending on the output format. One drawing. The 74 [MHz] DAC is used for 1920 × 1088 and 1280 × 720 pictures; The 27 [MHz] DAC is used for all other output formats.

Display conversion of down sampled image frames is used to display an image in a particular format. As described above, the display conversion block 280 shown in FIG. 2B is a vertical processing block (VPF) 282 and a horizontal processing block (HZPF) (which adjusts down-converted and down-sampled images to be displayed on a low resolution screen). 284).

For the preferred embodiment, VPF 282 is a vertical line interpolation processor performed as a programmable polyphase vertical filter, and for the preferred embodiment, HZPF 284 is a horizontal line interpolation processor performed as a programmable polyphase horizontal filter. The filters are programmable, which is a design choice to accommodate display conversions for multiple display formats.

As shown in Fig. 2B, four lines of down-sampled pixel data are input to the VPF 282 in raster size. In a preferred embodiment, the data includes a pair of luminance (Y) and color difference (C _R and C _B ) pixels that are simultaneously input in 32 bits of VPF 282. The VPF 282 filters four lines of data into one line, sends the line to the HZPF 284 as 32-bit values each containing luminance and chrominance data in YC _R YC _B , and then HZPF 284. ) Generates an exact number of pixels to match a given raster format.

7A is a high level block diagram illustrating a preferred filter suitable for use as the VPF 282 of the preferred embodiment of the present invention. In the following, VPF 282 processes the pair of input pixels (each pair of two luminance pixels (Y pixel and luminance _CR and C _B pixels) to generate a pair of output pixels. Facilitate the processing of the 2: 0 format because color pixels can be easily associated with their corresponding luminance pixels, however, those skilled in the art will appreciate that only luminance pixels or only chrominance pixels can be employed in this manner. Also, the VPF 282 generates lines in sequential format Another embodiment employs a dual output and supports both primary and secondary output channels. And a second VPF 282 is added.

7A, VPF 282 includes a VPF controller 702; A first multiplexer network comprising luminance pixel MUXs (LP MUXs) 706, 708, 710, 712 and chrominance pixel MUXs (CP MUXs) 714, 716, 718, 720; A second multiplexer network comprising luminance filter MUXs (LF MUXs) 726, 728, 730, 732 and chrominance filter MUXs (CF MUXs) 734, 736, 738, 740; A luminance coefficient RAM 704; Color difference coefficient RAM 724; Luminance coefficient multipliers 742, 744, 746, and 748; Color difference coefficient multipliers 750, 752, 754, 756; Luminance adders 760, 762, and 764; Color difference adders 766, 768, and 770; Round and clip processors 772 and 776; Demux / Register 774, 778; An output register 780.

Now, the operation of the VPF 282 is described. Vertical resampling is accomplished using two four-tap polyphase filters, one for the luminance pixels and one for the chrominance pixels. Only the operation of the filter for the luminance pixels is described below, since the operation for the chrominance pixels is similar. However, it is pointed out that there are differences in the paths through which they occur. In a preferred embodiment, the vertical filtering of luminance pixels can use up to eight phases in a four-tap polyphase filter, and the filtering of chrominance pixels can use up to sixteen phases in the four-tap polyphase filter. At the beginning of the field or frame, the VPF controller 702 sets up a vertical polyphase filter, provides control timing to the first and second multiplexer networks, luminance coefficient RAM 704 and chrominance coefficient RAM for polyphase filter phases. Select a coefficient set from 724 and include a counter that counts each line of the field or frame being processed.

In addition to adjusting the operation of the network of MUX and polyphase filters, the VPF controller 702 keeps track of the display lines by tracking integer and fractional portions of the vertical position of the decoded picture. The integer part indicates which line should be accessed, and the fraction part indicates which filter should be used. Moreover, the use of modulo N arithmetic can be effective for accurate down-sampling ratios such as 9 to 5 when the fractional portion allows for phases less than 16. The fraction can always truncate one of the N phases with the module being used.

As shown in Fig. 7A, luminance and chrominance pixel pairs from four image lines are separated into a chrominance path and a luminance path. 16-bit pixel conjugate data in the luminance path is further multiplexed in 8-bit even (Y even) and 8-bit odd (Y odd) formats by LP MUXs 706, 708, 710, and 712, and 16-bit in the chrominance path. The pixel pair is in 8-bit C _R and 8-bit C _B formats by CP MUXs 714, 716, 718, 720. Luminance filter MUXs 706, 708, 710, 712 repeat the pixel values of the line at the top and bottom of the decoded image's boundary such that the filter pixel boundary overlaps the polyphase filter operation.

Thereafter, the pixel pairs for four lines corresponding to the luminance pixel information and the chrominance pixel information pass through respective polyphase filters. The coefficients used by the multipliers 742, 744, 746, 748 for weighting the pixels for the filter phase are selected by the VPF controller 702 based on the programmed up or down sampling factor. After combining the weighted luminance pixel information in the adders 760, 762, 764, the value is applied to a round and clip processor 772 providing 8 bit values (because coefficient multiplication occurs with high accuracy). Because). The DEMUX register 774 receives the first 8-bit value corresponding to the interpolated 8-bit even (Y even) luminance value and the second 8-bit value corresponding to the interpolated 8-bit odd (Y odd) value, It provides a pair of vertically filtered luminance pixels of 16 bits. Register 780 provides the vertically filtered pixels in the luminance and chrominance paths and provides them as vertically filtered 32 bit values that receive a pair of luminance and chrominance pixels.

7B is a diagram illustrating the spatial relationship between pixel sample space and coefficients of lines. Each of the coefficients for the luminance and chrominance polyphase filter path has 40 bits assigned to each coefficient set, and there is one coefficient set for each phase. The coefficients are interpreted as fractions with a denominator of 512. The coefficients are located from left to right in a 40-bit word (C0 to C3). C0 and C3 are signed 10 bit 2 auxiliary values, and C1 and C2 are 10 bits with a given range (e.g., -256 to 767), each of which is sequentially converted to 11 bit 2 auxiliary values.

7A includes an optional luminance coefficient adjustment 782 and a chrominance coefficient adjustment 784. The coefficient adjustments 782 and 784 are used to derive an auxiliary number of 11 bits 2 for C1 and C2. If bits 8 and 9 (most significant bit) are both 1, the sine of the 11 bit number is 1 (negative), otherwise the value is positive.

8A is a high level block diagram illustrating a preferred filter suitable for use as the HZPF 284 in one embodiment of the present invention. HZPF 284 receives a pair of luminance and chrominance pixel information (which may be 32-bit data) from VPD 282. HZPF 284 includes HZPF controller 802; A C _R latch 804; C _B latch 806; Y latch 808; Select MUX 810; Horizontal filter coefficient RAM 812; Multiplication network 814; An addition network 816; A round and clip processor 818, a DEMUX register 820, and an output register 822.

Horizontal resampling is achieved by employing an 8 tap, 8 phase polyphase filter. The generation of display pixels is adjusted by the HZPF controller 802 by tracking the integer and fractional portions of the horizontal position in the decoded and down sampled picture. The water purification unit indicates which water purification unit is to be accessed, and the water fountain unit indicates which filter phase should be used. Modulo N arithmetic can be used to calculate the fractions so that less than 8 phases are used. For example, the use of arithmetic may be useful if an accurate down sampling ratio such as 9 to 5 is used. If the down sampling ratio cannot be represented by a simple fraction, the fraction is truncated to one of the N phases. The HZPF 284 of the preferred embodiment of the present invention filters pixel pairs, facilitates processing of a 4: 2: 0 formatted image using alignment on even pixel boundaries, and C _R and C _B pixels (color pixels). Together with the maintenance of the corresponding Y pixels.

The operation of the HZPF 284 is described with reference to FIG. 8A. At the beginning of the horizontal line, the HZPF controller 802 resets the horizontal polyphase filter, provides control timing to the first and second multiplexer networks, and C _R , C _B and Y filter coefficients for each polyphase filter phase. Select coefficient sets from horizontal coefficient RAM 812 for each of them, and select each set of C _R , C _B and Y values for processing. Also, when the horizontal position is near the left or right side of the line, the HZPF controller 802 causes the edge pixel values to be repeated or set to zero for use by the 8 tap polyphase filter. Any distortion in the image caused by this simplification is usually hidden in the overscan portion of the displayed image.

The pixel data received from the VPD 282 is separated into Y, C _R and C _B values, which are individually separated into the C _R latch 804, C _B latch 806 and Y latch 808 for filter purposes. It is latched. HZPF controller 802 then selects Y, C _R and C _B values by appropriate signal to select MUX 810. In a preferred embodiment, there are more Y values than C _R or C _B so that the filter uses additional latches within the Y luminance latch 808. At the same time, the HZPF controller 802 can generate the appropriate filter coefficients for the filter phase and the C _R or C _B or Y values based on the up sampling or down sampling value programmed by the control signal to the horizontal filter coefficient RAM 812. Choose.

The horizontal filter coefficient RAM 812 then outputs the coefficients to respective elements of the multiplication network 814 for multiplication with the input pixel values, generating weighted pixel values, the weighted pixel values being Combined in addition network 816 to provide horizontally filtered C _R , C _B or Y values.

After combining the weighted pixels in the addition network 816, the horizontal filtered pixel value is applied to a round and clip processor providing 8 bit values (since coefficient multiplication occurs at high frequencies). DEMUX register 820 includes a series of 8-bit values corresponding to a C _R value, an 8-bit even (Y even) Y value, an 8-bit C _B value, and an 8-bit odd (Y odd) Y value. Receive a bit value; DEMUX register 820 multiplexes these values into horizontally filtered luminance and chrominance pairs of 32-bit values (Y even, C _R , C _B or Y odd). Register 822 provides the pixel pairs as vertical and horizontal filtered 32-bit pixel luminance and chrominance pixel pairs.

FIG. 8B shows the spatial relationship between pixel sample values of the down sampled image stored in the horizontal filter coefficient RAM 812 and the coefficients used in the polyphase filter. The coefficients for the preferred embodiment are located from left to right (C0 to C7) in 64-bit words. The coefficients C0, C1, C6 and C7 are auxiliary values of signed 7 bit 2, C2 and C5 are auxiliary signals of signed 8 bit 2, C3 and C4 are auxiliary values of signed 10 bit 2,- Display the range from 256 to 767. C3 and C4 are adjusted to derive auxiliary values of 11 bits 2. If bit 8 and bit 9 (most significant bit) are 1, the sine of the 11 bit value is 1 (negative) and in other cases the value is 0 (positive). All coefficients can be interpreted as fractions with a denominator of 512.

Table 13 lists the coefficients for VPF 282 and HZPF 284 for the preferred embodiment of the present invention performing the indicated format conversion.

TABLE 13

In a preferred embodiment of the display conversion system, horizontal conversion is performed in part by the DCT region filter 216 and down sampling processor 232 of FIG. 2B. As such, the conversion provides the same number of horizontal pixels in accordance with 1125I or 750P. Thus, HZPF 284 upsamples the signals to provide 720 active pixels per line and unmodulates the 525P or 525I signals, which signals 720 active pixels per line as shown in Tables 1 and 2. The values of the coefficients of the horizontal filter do not change for conversion to 480P / 480I / 525P / 525I. The horizontal filter coefficients are given in Table 14.

TABLE 14

In addition, the programmable capability of the HZPF 284 allows for nonlinear horizontal scanning. 9A illustrates a resampling ratio profile that may be employed in the present invention. As shown, the resampling ratio of HZPF 284 may vary along a horizontal scan line and may vary linearly. Referring to the preferred embodiment of FIG. 9A, at the beginning of the scan line, the resampling ratio linearly increases (or decreases) to the first point of the scan line, and the resampling ratio will reach a second point. It remains constant until the resampling ratio decreases linearly from the second point. 9A, h initial resampling ratio is the initial resampling ratio for the image, h resampling ratio change is the change per first pixel of the resampling ratio, and -h resampling ratio change is The change per two pixels, h resampling ratio hold column and h resampling ratio hold column are display column pixel points where the resampling ratio is kept constant. The value display width is the last pixel (column) of the pixel line.

9B and 9C show ratio profiles for mapping a 4: 3 picture to a 16: 9 display. The ratios are defined as input values to output values, so 4/3 is downsampled 4 to 3 and 1/3 is upsampled 1 to 3. The non-profiles shown in Figures 9B and 9C map an input picture image with 720 active pixels to a display with 720 active pictures. For example, the mapping of a 4: 3 aspect ratio display to a 16x9 aspect ratio display in FIG. 9B uses 4/3 downsampling, but a 1/1 average on the horizontal line is needed to fill all the samples of the display. Thus, the profile of FIG. 9B has an accurate aspect ratio in the center between display pixels 240 and 480, with the values of the sides being upsampled to fill the display. 9D and 9E show the profile used for resizing from a 16x9 display image to a 4: 3 display, which is the inverse of the profile shown in FIGS. 9B and 9C.

The effect of using resampling ratio profiles according to a preferred embodiment of the present invention is shown in FIG. 10. A video transmission format having a 16x9 or 4x3 aspect ratio may be displayed in 16x9 or 4x3, The original video picture can be adapted to fit the display area. Thus, the original video picture may be full, zoom, squeeze or variable expand / shrink.

The system may allow the user to select a predetermined mapping between the aspect ratio of the received video signal and the aspect ratio of the display device (if the aspect ratios are not compatible). As described above, control processor 207 (shown in FIG. 2A) receives the aspect ratio of the image received from parser 209. The control processor 207 also determines the aspect ratio of the display device (not shown) connected to receive the output signal of the system. For example, if the display device is connected to an S-video output 153 or a composite video output 154 (both of which are shown in FIG. 1A), The aspect ratio will be 4 × 3. However, if the display device is connected to a primary video output port 146, the aspect ratio will be 4x3 or 16x9.

In a preferred embodiment of the present invention, the user can select the aspect ratio of the display device as part of a start-up process that can be generated via a remote control IR receiver 208 (FIG. 2A). It is specified as. The start-up process is executed only when the video decoder system detects that there is a display device including a main output port and the system is connected to the main output port. The start-up process determines the display format (ie, aspect ratio and maximum video resolution) of the display device in several ways. First, the process may present a menu of possible display devices to the user, for example each of which is indicated by the manufacturer name and model number. The user then selects one of the display devices using the remote control device. The decoder system may be arranged with a modem that periodically contacts the central location to receive an updated list of display types as well as other updated lists of programming of the controller 207. Optionally, the type of information can be encoded in user data of the received ATSC video signal and the decoder can be programmed to access the information and update its internal programming.

Alternatively, to determine the aspect ratio of the display device, the user is provided with 4 × 3 rectangles and 16 × 9 rectangles, and the user is required to indicate which is more suitable for the display device. As another example of selection, the user is required to select two menus, one of which is an enumeration of possible video display resolutions, and the other of which is an enumeration of possible aspect ratios.

As another alternative, the control processor 207 may program the on-screen display generator to form (eg, circles) some other signal resolutions (eg, 525I, 525P, 750P, 1180I and 1180P) and some With different aspect ratios (e.g., 4x3 and 16x9), the text may require the viewer to press a button on the remote control (not shown) when the best circle is displayed. The system may provide each of the images sequentially at the main output for several minutes to correlate the display of a particular image with the press of a button on a remote control device. This provides the necessary information regarding the image resolution and aspect ratio for the display device.

With information regarding the display format of the display device, the system can automatically adapt the received video signal for the best possible presentation on the display device. For example, if there is a mismatch between the aspect ratio of the received video signal and the aspect ratio of the display device, it may be displayed to the viewer by the generation of a command using a remote control device (not shown), or the viewer may have a relationship between the two aspect ratios. After seeing all possible transforms sequentially (FIGS. 9A-9E and 10), one can choose to use one of the transforms. This applies when the aspect ratio of the received signal is 16x9 and the aspect ratio of the display device is 4x3, as well as when the aspect ratio of the received video signal is 4x3 and the aspect ratio of the display device is 16x9.

As a final optional example, the system may be arranged to sense the information provided by the display device to determine the display format. For example, the bidirectional path may be provided through one of the output signal lines Y, C _R , C _B of the decoder system from which data in a digital register in the display device can be read. The data in the register can indicate the manufacturer and model number or the maximum resolution and aspect ratio of the display device. Optionally, the display device may apply a direct current (DC) signal to one or more of the lines, which may be sensed by the decoder device as an indication of the display format of the display device.

A multi-sync monitor capable of displaying video signals with several different display formats can be connected to the main output port of the video decoder. In this case, the video resolution component of the display type information reconstructed by the control processor 207 includes an indication that the display is a multi-sync device, so that when the aspect ratio of the received video signal does not match the aspect ratio of the display device, The only format conversion is preferably the aspect ratio adaptation shown in FIGS. 9A-9E and 10.

While the preferred embodiments of the present invention have been described and illustrated above, it should be understood that the above embodiments have been provided by way of example only. Those skilled in the art can devise numerous variations, alternatives, and substitutions without departing from the spirit of the invention. Accordingly, the claims herein are written such that all such modifications fall within the scope of the present invention.

Claims (8)

Baseband coded digital video with a first video display format for a first time interval and a second video display format for a second time interval to produce a decoded video signal suitable for display in a single predetermined video display format. A digital video signal conversion system for decoding and reformatting a signal, wherein the first video display format is different from the second video display format and the first and second video display formats are different from the predetermined video display format, each video In the digital video signal conversion system, the display format has an aspect ratio and a resolution. An input terminal connected to receive the encoded digital video signal, A data retrieval circuit for extracting data identifying said first video format during said first time interval from said encoded digital video signal and extracting data identifying said second video display format during said second time interval; A digital video signal decoder comprising signal processing circuitry for decoding the encoded digital video signal to produce an output video signal in accordance with the single predetermined video display format, wherein the signal processing circuit is configured to perform the first and second time periods. The filtered digital video according to a first down conversion process during the first time interval and a second down conversion process during the second time interval, and a frequency domain filter having first and second cutoff frequencies, respectively, during the intervals. A digital video signal decoder having a down sampling processor for processing a signal; And A controller coupled to receive the identification data identifying the extracted first and second video display formats, the controller causing the decoder to cause the encoded to correspond to the extracted first and second display formats. Generating control signals for the digital video signal decoder to convert a digital video signal to the output video signal having the predetermined video display format during the first and second time intervals. The method of claim 1, The digital video signal decoder is coupled to a format conversion control signal provided by the controller to resample the decoded digital video signal provided by the digital video decoder to generate the output video signal having the predetermined video display format. And a responsive programmable spatial filter. The method of claim 2, The digital video signal is encoded using an encoding technique specified by a moving picture expert group (MPEG) in which the encoded digital video signal is received in a plurality of packetized elementary stream (PES) packets, each PES packet being a header. Wherein the data retrieval circuitry of the digital video signal decoder extracts data identifying the first and second video display formats from headers of respective PES packets respectively received during the first and second time intervals. And means for performing digital video signal conversion. The method of claim 2, The digital video signal is encoded using an encoding technique specified by a Moving Picture Experts Group (MPEG) in which the encoded digital video signal is contained in a digital bitstream containing sequence records, each sequence record having a header, and The data retrieval circuit of the digital video signal decoder includes means for extracting data identifying the first and second video display formats from respective sequence records received during the first and second time intervals, respectively; Digital video signal conversion system. The method of claim 1, And a user input device capable of a user inputting data identifying said single predetermined video display format. The method of claim 1, And further comprising a video display device having said single predetermined video display format, said video display device including a register holding said identification signal, said controller being coupled to said display device to read said identification signal, and And use the identification signal to generate data identifying the predetermined video display format. The video display device of claim 1, further comprising a video display device having the single predetermined video display format, wherein the digital video signal decoder comprises: Analog-digital conversion means for converting an output signal of the signal processing circuit of the video decoder into an analog signal; Transmission means for applying the analog signal to the display device; Means for sensing a direct current (DC) potential connected to said controller and applied to said transmission means by said display device, The controller identifies the single predetermined video display format to be used by the digital video signal decoder in response to the sensing means. The method of claim 5, The video signal decoder further comprises an on screen display processor that can be programmed to generate a plurality of output video signals respectively corresponding to the plurality of different video display formats, respectively; The controller, in response to the first user control signal provided through the user input device, causes the on-screen display device to sequentially provide video signals each having different video display formats, and provided through the user input device. Means for defining, in response to a second selection signal, one of the respective video display formats as the predetermined video display format.