JPH0897725A - Pipeline system - Google Patents

Abstract

PURPOSE: To improve an expansion method and a device for decoding and/or expanding various plural coded input signals. CONSTITUTION: A multi-standard video decoder has plural processing stages 36 connected by a two-wiring interface and arranged as a pipeline processor. A control token and a data token are sent through two-wiring interfaces 31 and 42 for transmitting both control and data in a token format. A token decoder circuit 33 is placed on a specified stage for recognizing any specified token as a control token peculiar to this specified stage and for sending a non- recognized control token along with the pipeline. While being placed on the selected stage, a reconstitution processing circuit reconstitutes such a stage in response to the recognized control token and handles the recognized, identified data token.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to improvements in decompression methods and apparatus that function to decode and / or decompress a plurality of different encoded input signals. The embodiments described below were chosen as illustrative and relate to encoding multiple coded video standards. In particular, this embodiment is JP
It relates to the decoding of one of the known standards known as EG, MPEG and H.261.

The serial pipeline processing method of the present invention uses unique and special interaction interface tokens in the form of control tokens and data tokens for a plurality of adaptive decompression circuits positioned as reconfigurable pipeline processors. It consists of one 2-wire bus used to implement.

[0003]

2. Description of the Related Art One of the prior arts is US Pat.
Some are described in the specification of 6,724. The device consists of a plurality of computing modules, in the preferred embodiment a total of four computing modules connected in parallel. Each computing module consists of a processing unit, a dual port memory, a scratchpad memory and an arbitration mechanism. The first bus connects these computing modules with the host processor. The device has a shared memory connected to a host processor and a computing module via a second bus.

US Pat. No. 4,785,349 discloses a complete format for transmission recorded on a compact disc medium and decoded at conventional video frame rates. During compression, the regions of the frame are individually analyzed to select the optimal fill coding method for each region. The region decoding time is estimated and the compression threshold is optimized. Area description codes indicating the size and position of the area are grouped in the first segment of the data stream.
Region fill codes that indicate the pixel amplitude of a region are grouped by fill code type and placed in other segments of the data stream. Each data stream segment has a variable length coded according to its respective statistical distribution and is formatted for data frame formation. The number of bytes in the frame is reduced by the addition of ancillary data determined by inverse frame sequence analysis, providing an average number selected to minimize pauses during compact disc playback. This avoids the unpredictable seek mode latency characteristics of compact discs. The decoder includes a variable length decoder responsive to the statistical information of the codestream for separately variable length encoding each segment of the data stream. The area position data is obtained from the area description data and is supplied together with the area fill code to a plurality of area specifying decoders selected by detecting the type of the fill code (for example, relative value, absolute value, binomial and DPCM). . The decoded region pixels are then stored in bitmap format for later display.

US Pat. No. 4,922,341 discloses a method of reducing image data for a digital TV signal by using a scene model. Here, the sequentially supplied image signals are coded, and at time t-1.
The previous frame from the scene already coded in 1. exists in the image store as a reference, and the frame-to-frame information consists of an amplification coefficient, a transfer coefficient and an adaptively obtained quad tree partition structure. . When the system is initialized, a uniform, halftone-represented image with a predetermined gradation value or a specified luminance value is displayed in the image storage unit of the coder on the transmitter side and the image storage unit of the decoder on the receiver side. Are written in the same manner for all the pixels.
Both the image store of the coder and the image store of the decoder are fed back to each other so that the contents of the image store in the coder and the decoder are read in variable sized blocks, depending on the coefficient greater or less than the luminance value 1. It is amplified and operates so that it can be written to another address of this image storage unit. Thereby, variable size blocks are organized according to the known quadtree structure.

US Pat. No. 5,122,875 discloses an apparatus for encoding / decoding an HDTV signal. The apparatus provides a high quality video source signal for providing a hierarchical coded word CW representing compressed video data and a corresponding coded word T defining the type of data represented by the coded word CW. And a compression circuit responsive to. A priority selection circuit responsive to the coded words CW and T decomposes the coded words CW into high priority and low priority coded word strings, whereby the high and low priority coded word strings are respectively used for image reproduction. On the other hand, it corresponds to compressed video data of relatively high and low importance. Transfer processors responsive to the high priority and low priority coded word sequences form high priority and low priority transfer blocks for the high priority and low priority coded words, respectively. Each transfer block includes a header, a coded word CW and an error detection check bit. Each transfer block feeds a forward error checking circuit which supplies additional error checking data. The high and low priority data is then provided to the modem to quadrature amplitude modulate the corresponding carrier for transmission.

In US Pat. No. 5,146,325, video signals in odd and even fields are independently compressed in sequence in intraframe compression mode and interframe compression mode, and then interleaved for transmission. A video decoding system for decoding compressed image data is disclosed.
The odd and even fields are decoded independently. During periods when there is no valid decoded odd / even field data, the even / odd field data is substituted for the unavailable odd / even field data. Decoding the data in the even / odd fields independently and substituting the data in the opposite field for the unusable data is effectively used to reduce the image display wait time at system startup and during channel changes. obtain.

US Pat. No. 5,168,356 discloses a video signal coding system for dividing coded video data into transfer blocks for signal transmission. The transport block format improves signal recovery at the receiver side, thanks to the provision of header data that allows the receiver to determine the re-entry point into the data stream when the transmitted data is lost or corrupted. The re-entry point is maximized by providing a secondary transport header that is inserted in the encoded video data in the corresponding transport block.

US Pat. No. 5,168,375 processes a field of image data samples to provide one or more of decimation, interpolation and shaping functions. A method is disclosed. This is realized by the array conversion processor or the like used in the JPEG compression system. The block of data samples is transformed by the Discrete Even Cosine Transform (DECT) in both the decimation and interpolation process,
Then, the number of frequency terms is changed. In the case of decimation, the number of frequency terms is reduced and then an inverse transform is performed to produce a reduced size matrix of sample points representing the original data block. In the case of interpolation processing, a zero-valued additional frequency component is inserted into the frequency component array and the subsequent inverse transform produces a series of expanded data samples without increasing the spectral bandwidth. In the case of shaping, which is realized by convolution or filtering involving multiplication of the transform of the data and the filter kernel in the frequency domain, an inverse transform is provided which produces a series of blocks of processed data samples. The spatial symbol of the kernel is modified by reducing the number of components for the linear phase filter and zero inserted to be the same as the number of samples in the data block. Following this, the Discrete Odd Cosine Transform (DOCT) of the zero-inserted kernel matrix is formed.

US Pat. No. 5,175,617 discloses a system and method for transmitting logmap video images over a telephone line bandlimited analog channel. The pixel configuration in the logmap image is adapted to match the sensor geometry of the human eye with more centered pixels. This transmitter divides the frequency band into channels and assigns one or two pixels to each channel. For example, a 3 KHz voice quality telephone line is divided into 768 channels, each spaced at 3.9 Hz. Since each channel consists of two carriers in quadrature, each channel can carry two pixels. Some channels are reserved for special calibration signals that allow the receiver to detect both the phase and magnitude of the received signal. If the sensor and the pixel are directly connected to the oscillator group and the receiver can receive each channel continuously, the receiver need not be synchronized with the transmitter. F
The FT algorithm performs a fast discrete approximation for the continuous operation case where the receiver is synchronized with the first frame and then gets subsequent frames every frame period. Since the frame period is relatively low compared to the sampling period, it is unlikely that the receiver will lose frame synchronization once the first frame is detected. On a trial video call,
Four frames are transmitted per second, 1440 pixel log map images are orthogonally encoded, and the speed is 40,00 per second.
An effective data transfer rate of 0 bits or more was obtained.

In US Pat. No. 5,185,819, odd and even field video signals are sequentially and independently compressed in intraframe compression mode and interframe compression mode. A video compression system having is disclosed. The odd and even fields of independently compressed data are interleaved for transmission so that the intra-frame even field compressed data occurs halfway between successive fields where the intra-frame odd field compressed. . For the receiver, the interleaved series of fields doubles the number of entry points into the signal for coding without increasing the amount of transmitted data.

US Pat. No. 5,212,742 discloses an apparatus and method for processing video data for compression / decoding in real time. The device consists of a plurality of computing modules, in the preferred embodiment a total of four computing modules connected in parallel. Each computing module consists of a processing unit, a dual port memory, a scratchpad memory and an arbitration mechanism. The first bus connects these computing modules with the host processor. Finally, the device has a shared memory connected to the host processor and the computing module via a second bus. This method performs partial allocation of images for processing by each processing unit.

US Pat. No. 5,231,484 discloses suitable for use with the proposed ISO / IEC MPEG standard. It includes three cooperating components or subsystems. In order to provide the optimum visual quality considering the number of bits assigned to the image, these three components or subsystems adaptively pre-process the incoming digital video sequence and the bits are imaged. , And transform coefficients are adaptively quantized to different regions of the image in video order.

US Pat. No. 5,267,334 discloses a method for removing frame redundancy in a video sequence in a computer system. The method comprises the steps of detecting a first scene change in a sequence of moving pictures and generating a first keyframe containing complete scene information for the first image. In the preferred embodiment, this first keyframe is known as the "forward" keyframe or intraframe and is typically present in CCITT compliant compressed video data. The process comprises the step of generating at least one intermediate compressed frame, the at least one intermediate compressed frame being the first for at least one image temporally following the first image in the sequence of motion pictures. It contains the difference information from one image. This at least one image is known as an interframe. Finally, there are steps of detecting a second scene change in the video sequence and generating a second keyframe containing complete scene information for the image displayed immediately before the second scene change. This second keyframe is known as the "backward" keyframe. The first key frame and at least one intermediate compressed frame are connected for forward reproduction, and the second key frame and the intermediate compressed frame are connected for reverse reproduction. The intra-frame may be used to generate complete scene information when the image is played in the forward direction. When this scene is played back, backward keyframes are used to play the complete scene information.

In US Pat. No. 5,276,513, a first circuit arrangement consisting of a predetermined number of known image / pyramid stages and a second moving picture / vector stage of the same number as the predetermined number are provided. It is disclosed with a circuit device. These devices are cost-effective for hierarchical video analysis (H
MA) in real time with minimal system processing delay and /
Or, by using a minimum system processing delay and / or with a minimum hardware structure. In particular, the first and second circuit devices are responsive to relatively high resolution image data from a continuous input sequence of consecutive predetermined pixel density image data frames occurring at a relatively high frame rate (eg, 30 frames per second). Obtains a continuous output train of continuous predetermined pixel density vector data occurring at the same frame rate after some processing system delay. Each vector data frame represents the motion of the image that occurs between each pair of successive image frames.

In US Pat. No. 5,283,646, a real-time video encoding system encodes an image only once, while precisely delivering the desired number of bits per frame, for example via a communication channel. A method and apparatus for updating a quantization step size used to quantize a coefficient representing a transmitted image is disclosed.
This data is divided into sectors having multiple blocks. This block is coded using, for example, DCT coding, and a coefficient sequence is generated for each block. This coefficient is quantized, and the number of bits required to describe the data varies considerably depending on the quantization step. At the end of transmission of data for each sector, the actual cumulative number of bits used is compared to the desired cumulative number of bits used for the selected number of sectors associated with this particular group of data. . The system readjusts the quantization step size to obtain the final desired number of data bits, for example for multiple sectors representing an image. Various methods have been disclosed for renewing the quantization step size to determine the desired bit allocation.

Wescon Technical Paper
pers) No.2, October / November 1984, Yong M.
A paper by Chong, Yong M, "Data Flow Architecture for Digital Image Processing" (A Data-Flow Archite
Cture for Digital Image Processing) discloses a real-time signal processing system specifically designed for image processing. Specifically, there is disclosed a data flow architecture based on a fixed one-word length token having a fixed length address field.
The token is composed of a data field, a control field, and a tag. The token tag field is further divided into a processor address field and an identifier field. The processor address feed is used to correctly send the token to the data processor, and the identifier field is used to label the data so that the data processor is informed what to do. In this way,
The identifier field acts as an indication to the data processor. The system sends each token to a particular dataflow processor using the module number (MN). If this MN matches a particular stage MN, then perform the appropriate operation on the data. If they do not match, the token is sent to the output data bus.

IEEE J Solid Circuit (Solid-State Circu
Its 1.2) Vol 1.23, No. 1, February 1988, Kimori et al., "An Elastic Pipeline Mechanism by Self-T".
imed Circuits) discloses an elastic pipeline with self-regulating circuits. An asynchronous pipeline comprises multiple pipeline stages. Each pipeline stage is provided following a combinational logic circuit that performs a logical operation specific to the pipeline stage. It consists of input data group. A trigger signal generated from the data transfer control circuit associated with the pipeline stage is simultaneously supplied to the data latch. The data transfer control circuits are connected to each other to form a chain, through which a transmission signal line and an acknowledgment signal line control the data transfer between the pipelines in the subsequent stages in the handshake mode. A decoder is usually provided in each stage to select an operation to be performed on the operand of the current pipeline stage. Further, a decoder can be arranged in the preceding stage in order to precode the complicated decoding process and to reduce the problem of the 1 path which is serious for the logic circuit. The interaction between the sub-modules is based on fully localized decisions, and because each sub-module can independently perform its own data buffering and self-adjusting data transfers at the same time, the flexibility of the pipeline allows Any centralized control is eliminated. Finally, to increase the flexibility of the pipeline, empty pipeline stages are provided between the occupied pipeline stages to ensure reliable data transfer between each stage.

[0019]

SUMMARY OF THE INVENTION The present invention comprises a plurality of processing stages and a universal adaptation unit in the form of an interactive interface control token that performs control and data functions within the processing stages, the processing stages being configured. , Is intended to provide a pipeline system which is provided with improved flexibility in processing.

[0020]

SUMMARY OF THE INVENTION Briefly and generally, the present invention is a pipelined system having an input, an output, and a plurality of processing stages between the inputs and outputs, wherein the processing stages are pipelined. Universally interconnected by a two-wire interface for transmitting tokens along, control tokens, data tokens interface with all processing stages in the pipeline and interact with selected processing stages in the pipeline. In the form of an adaptive unit, the processing stages are provided with increased flexibility in configuration and processing.

Each processing stage in the pipeline is the first
It may have both the secondary and secondary storage means. The processing stage is reconstructed upon recognition of the selected token. The tokens of the pipeline are dynamically adaptive and are located depending on the processing stage performing the function or independent of the processing stage performing the function.

In the pipeline of the present invention, tokens may be modified by interfacing with stages. Tokens may interact with all stages of the pipeline, or only certain less than all stages.
Tokens in the pipeline may interact with adjacent stages, or they may interact with non-adjacent stages. The token reconstructs the processing stage. Such tokens may be subordinate to one function or independent of another function.

Tokens, along with reconfigurable processing stages, provide the basic building blocks for pipeline systems. The interaction between the token and the processing stage of the pipeline is conditioned by the previous processing history of that processing stage. The token may have an address field that characterizes the token. Thereby, the interaction with the processing stage is determined in this address field.

The token may further include an extension bit indicating that there are additional words in this token and identifying the last word in the token. The address field of the token may be of variable length and is typically Hoffman coded.

Tokens are generated by the processing stage. The pipeline token may contain data to be transferred to the processing stage, or the token may have no data. Certain tokens are identified as data tokens and provide data to the processing stage of the pipeline. Other tokens are identified as control tokens and coordinate processing stages. This adjustment involves the reconstruction of the processing stage. In addition, other tokens both supply and coordinate the data to the processing stage. Any of the tokens identifies the encoding standard for the processing stage of the pipeline. Other tokens operate between processing stages independent of any encoding standard. The token can also be changed continuously by the processing stage.

The flexibility of the token interaction in cooperation with the processing stage facilitates a large distribution of the processing stage's functionality with respect to the internal structure, and the token flexibility facilitates system expansion and / or modification. In this regard, tokens may also facilitate multiple functions within the processing stage. The token may be hardware-based or software-based. Tokens provide data and control to the processing stage at the same time.

According to the invention, a plurality of encoded bits as a serial bit stream of digital bits are carried in a single bit stream, each of which has an encoded pair of control code and corresponding data. Start code detection in a pipeline system for handling streams, in which a plurality of processing stages are interconnected by a two-wire interface, in response to a single serial bit stream, generating control tokens and data tokens, and providing the two-wire interface Means and token decoding means provided in a predetermined processing stage for recognizing the token as a control token specific to the stage and passing an unrecognized control token along the pipeline; and for handling the identified data token. Response to the recognized control token. Characterized by comprising a reconstruction decoding parser processing means for reconstructing the stages.

According to the present invention, it further comprises first and second registers, the first register being connected to the input of the decode parser means and the second register being connected to the output of the decode parser means. One of the processing stages is a space decoder means, and the other of the processing stages is a token generation means for generating a control token and a data token and passing through the two-wire interface. The spatial decoder means is provided with token decoding means for recognizing as a control token and for spatially decoding the data token following the control token, which configures the spatial decoder in the first decoding format.

A further processing stage is a temporal decoder means arranged downstream of the spatial decoder of the pipeline, a second token decoder means located in the temporal decoder recognizing the given token as a control token specific to the temporal decoder. Then, the data token following the control token is temporally decoded and the time decoder is configured in the first decoding format. The temporal decoder can utilize a reconstructed prediction filter that can be reconstructed by prediction tokens.

Addressing means are provided for moving data in an 8 × 8 pixel block through a temporal decoder along a two-wire interface to store and retrieve blocks along block boundaries. The addressing means can also store and retrieve data blocks that cross block boundaries. The addressing means may rearrange the data blocks as pixel data for display. The data blocks to be stored and retrieved may have sizes other than the 8 × 8 pixel block. Means may be provided for displaying the output of the temporal decoder or for rewriting the output to the pixel memory. The decoding format may be either a still image format or a moving image format.

In the following description of the embodiments of the present invention, the following terms are frequently used, and a general description of the terms is given below.

Explanation of terms Block: matrix of 8 rows by 8 columns of pixels, or 64D
CT coefficient (source, quantized or dequantized).

Chrominance (component): 1 of two color difference signals relating to the three primary colors as specified in the bit stream
One matrix, block, or one pixel representing an individual. The codes used for the color difference signals are Cr and Cb.

Coded Notation: A data element represented in coded form.

Coded Video Bitstream: A coded representation of a sequence of one or more images, as defined herein.

Coding order: The order in which images are transmitted and coded. This order does not necessarily have to be the same as the display order.

Component: One matrix, block, or one pixel from one of the three matrices (luminance and two chrominances) that make up an image.

Compression: Reducing the number of bits used to represent an item of data.

Decoder: An embodiment of the decoding process.

Decoding (Processing): As defined herein,
The process of reading the input coded bitstream and generating decoded image or audio samples.

Display order: The order in which the decoded images are displayed.
Generally, this is the same order in which the images were input to the encoder.

Encoding (Process): A process, not specified herein, that reads an input image sequence or audio sample and produces a valid encoded bitstream as defined herein.

Intra coding: The coding of a macroblock or image that uses only information from the macroblock or image.

Luminance (component): One matrix, block, or one pixel representing the black and white of a signal, related to the three primary colors as defined in the bitstream. The code used for the luminance signal is Y.

Macroblocks: Four 8 × 8 blocks of luminance data and two from the 16 × 16 portion of the luminance component of the image (for 4: 2: 0 chroma format), four (4: 2: 2) Chroma format) or 8 (4:
Corresponding 8x8 blocks of chrominance data (for 4: 4 chroma format). Macroblocks may also refer to pixel data, and may refer to encoded values of pixel values and other data elements defined in a macroblock header with the syntax defined in this part of the specification. There is also. The use will be clear from the context to those of ordinary skill in the art.

Video compression: Improving pixel value prediction efficiency using video vectors. This prediction uses the motion vector to provide an offset to past and / or future reference images that contain previously decoded pixel values used to create the prediction error signal. Video Vector: A two-dimensional vector used for video compensation that provides an offset from the coordinate position of the current image to the coordinates of the reference image.

Non-intra coding: Coding of a macroblock or image that also uses information from the macroblock or the image itself and macroblocks that occurred at different times and information from the image.

Pixel: Image unit Picture: Source, coded or reconstructed image data The source or reconstructed image each has a luminance and an 8-bit number representing two chrominance signals.
It consists of a square matrix. In advanced video, the image is the same as the frame, and in interlaced video, the image refers to the frame, or the first or last field of the frame, depending on the context.

Prediction: Predicting the currently decoded pixel value or data element using a predictor.

Reconfigurable Processing Stage (RPS): A stage that reconfigures itself and performs various tasks in response to a recognized token.

Slice: A series of macroblocks.

Token: A universal adaptation unit in the form of a package of interactive interface messenger for control and / or data functions.

Start Code (System and Video): A 32-bit code inserted in the unique coded bitstream. These codes are used for several purposes, including identifying some of the structures within the encoding syntax.

Variable Length Coding (VLC): An inverse process for encoding that assigns shorter codewords to frequent events or longer codewords to infrequent events.

Video Sequence: A sequence of one or more images.

[0056]

According to the invention, the processing stage comprises a plurality of processing stages and a universal adaptation unit in the form of an interaction interface control token for performing control and data functions in the processing stages, the processing stages being improved in configuration and processing. A pipeline system is provided that has been given flexibility.

[0057]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of a pipeline system according to the present invention will be described below with reference to the drawings.

As an overview of the most common functions used in the pipeline system used in the preferred embodiment of the present invention, FIG. 1 is a greatly simplified illustration of 6 cycles of a 6 stage pipeline. (As described in more detail below, the preferred embodiment of the pipeline includes some advantageous features not shown in FIG. 1.) Reference will now be made to the drawings, in which the various included in the drawings Throughout the figures, the same reference numerals indicate the same or corresponding elements, and more particularly, FIG. 1 is a six-cycle block diagram for practicing the present invention. Each row of boxes represents one cycle, and each of the different stages is labeled A to F, respectively.
Each hatched box indicates that the corresponding stage holds valid data, i.e. data to be processed in one of the pipeline stages. After processing (which may include nothing more than a simple transfer without manipulation of the data), the valid data is transferred from the pipeline as valid output data.

Note that a real pipeline application may include more or less than six pipeline stages. Of course, it should be noted that the present invention can be used with any number of pipeline stages. Further, the data can be processed in more than one stage, and different stages can have different processing times.

In addition to the clock and data signals (discussed below), the pipeline has two transfer control signals, the "VALID" (valid) signal and the "ACCE" signal.
PT "(tolerance) signal is included. These signals are used to control data transfers within the pipeline. The VALID signal, shown as the upper line of the two lines connecting adjacent stages, is fed forward or downstream from each pipeline stage to the nearest adjacent device. The device can be another pipeline stage, or some other system. For example, the last pipeline stage may feed its data to the next processing circuitry. The ACCEPT signal, shown as the lower line of the two lines connecting adjacent stages, feeds the other upstream device to the previous device.

A data pipeline system of the type used to implement the present invention has one or more of the following characteristics, as shown in the preferred embodiment.

1. A pipeline is "elastic", that is, disturbances caused to other pipeline stages due to delays experienced in a particular pipeline stage are as small as possible. Successive pipeline stages are allowed to continue processing, thus implying a gap in the data flow following the delayed stages. Similarly, the preceding pipeline stage can continue as long as possible. In this case, gaps in the data stream are removed from the data stream as much as possible.

2. The control signal that arbitrates the pipeline is
These signals are organized to propagate only to the nearest adjacent pipeline stage. In the case of signals flowing in the same direction as the data stream, the stage just described is the stage immediately following it. In the case of a signal flowing in the opposite direction of the data flow, said stage is the immediately preceding stage.

3. The data in a pipeline is coded so that many different types of data are processed in the pipeline. This encoding accommodates variable sized data packets, and the size of the packets need not be known in advance.

4. The overhead associated with typing the data is as small as possible.

5. It is possible for each pipeline stage to recognize only the minimum number of data types required for the functionality required for the pipeline stage.
However, the pipeline stage must be able to supply all these data types to successive stages even if it does not recognize the above data types. If this is possible, it allows communication between non-adjacent pipeline stages.

Although not shown in FIG. 1, there is a data line and whether it is a plurality of single lines or several parallel lines forms a data bus to be introduced or derived to each pipeline stage. To do. As described and illustrated in greater detail below, data can be transferred into, out, and out of the pipeline stages via data lines.
And transferred between stages.

Note that the first pipeline stage can receive data and control signals from any form of preceding device. The preceding device in this case means, for example, a receiving circuit of the digital image transmitting system, another pipeline, or the like. On the other hand, the first pipeline stage can generate all or part of the data processed in the pipeline. As a practical matter, as detailed below,
A "stage" may include any processing circuit including a do-nothing system (simply provide data) or an entire system (eg, other pipelines, or multiple systems or multiple pipelines), and , "Stage" can create, modify, and delete data as needed. If a pipeline stage contains valid data to be transferred down the pipeline, the VALID signal indicating data validity need not be transferred further than the next immediately following pipeline stage. Therefore, a two wire interface is included between all pipeline stages that make up the pair of systems. That is, so-called other devices are included, and if data is transferred between this type of device and the pipeline, between the preceding device and the first stage and between the succeeding device and the last stage. A two-wire interface between is included. Each of the signals, ie AC
CEPT and VALID have values of HIGH and LOW. These values are abbreviated as "H" and "L", respectively. In implementing the present invention, the most common application of pipelines is typically digital. When realized as this type of digital, for example, H
The IGH value may be a logical "1" and the LOW value may be a logical "0". However, the system is not limited to being implemented as digital, but when implemented as analog, the HIGH value can be an amount above a certain voltage, or some other similar set threshold, and can be a LOW value. May be indicated by corresponding signals below or above (or above) the same or other thresholds. For digital applications, the present invention can be implemented using any known technology, such as CMOS, bipolar, etc.

It is not necessary to use a separate storage arm and wire to provide storage for the VALID signal.
In the digital embodiment this is true. All that is required is that the "validity" indication of the data be stored with the data. Merely by way of example, in a digital television image represented by a digital value,
Certain values are not allowed, as defined in International Standard CCIR 601. In this system, 8-bit binary numbers are used to represent the samples of the image, and zero and 255 values should not be used.

If an image of this kind needs to be processed in a pipeline incorporated in an embodiment of the present invention, one of these values (for example zero) is a particular value in the pipeline. It will be used to indicate that the data on the stage is not valid. Therefore, any non-zero data should be considered valid. In this example, there is no particular latch that is identifiable and can be said to be storing the "validity" of the associated data. Nevertheless,
The validity of the data is stored with the data.

As shown in FIG. 1, VAL to each stage
The state of the ID signal is indicated as "H" or "L" on the upper right arrow. Therefore, the VALID signal from stage A to stage B is LOW, and VALID from stage D to stage E
The signal is HIGH. The state of the ACCEPT signal to each stage is indicated as "H" or "L" on the lower left arrow. Therefore, the ACCEPT signal from stage E to stage D is HIGH, while the ACCEPT signal from the device connected downstream of the pipeline to stage F is LOW.

Data is transferred from one stage to the other during one cycle whenever the ACCEPT signal going from the upstream adjacent stage to the downstream stage is HIGH (discussed below). . ACCEPT signal between two stages is L
If OW, no data is transferred between these stages.

Referring again to FIG. 1, if one box is shaded, the corresponding pipeline stage is assumed to contain valid output data, for example. Similarly, the VALID signal supplied from that stage to the next stage is HIGH. Stage B,
The pipeline when D and E contain valid data is shown in FIG. Stages A, C, and F contain no valid data. At the starting point, the VALID signal to pipeline stage A is HIGH, which means that the data on the transmission line to the pipeline is valid.

Similarly, at this point, the ACCEPT signal to pipeline stage F is LOW, and thus, whether stage F is valid or not valid.
Does not transfer data. Note that valid and invalid data are transferred between pipeline stages. Invalid data that stores valueless data is overwritten, thereby removing invalid data from the pipeline. However, valid data is for processing or, for example,
Valid data must not be overwritten as it must be stored for use in downstream devices such as pipeline stages, devices connected to the pipeline that receive data from the pipeline, or systems.

In the pipeline shown in FIG. 1, stage E contains valid data D1, stage D contains valid data D2, stage B contains valid data D3, and is connected to the upstream pipeline. The device (not shown) contains the data D4 to be transferred and processed in the pipeline. In addition to the upstream device, stages B, D, and E contain valid data, so the VALID signal from each of these stages or devices to the following device is HIGH. However, VALI from stages A, C, and F
The D signal is LOW because these stages do not contain valid data.

Suppose a device connected downstream from the pipeline is not ready to accept data from the pipeline. Device is compatible with ACC
This state is transmitted to stage F by setting the EPT signal LOW. However, stage F itself does not contain valid data and therefore can accept data from the preceding stage E. Therefore, the ACCEPT signal from stage F to stage E is set HIGH.

Similarly, stage E contains valid data, and stage F is ready to accept this data. Thus, stage E can accept new data as long as valid data D1 is first transferred to stage F. In other words, stage F cannot transfer data downstream, but all other stages can transfer data downstream without overwriting or losing any valid data. Thus, at the end of cycle 1, the data can be "shifted" one step to the right. This state is shown in cycle 2.

In the example shown in the figure, the downstream device is
The ACCEPT signal to stage F is still LOW because it is not ready to accept new data in cycle 2. Therefore, stage F cannot accept new data. The reason is that then the valid data D1 is overwritten and lost. Therefore, since stage E also contains valid data D2, the AC from stage F to stage E is the same as for the ACCEPT signal from stage E to stage D.
The CEPT signal goes LOW. However, it is possible that all stages A to D will accept new data (either if these stages do not contain valid data or if these data shifts those valid data downstream. , And new data can be accepted), and each of these stages sends this condition to its immediately preceding neighboring stage by setting the corresponding ACCEPT signal to HIGH.

FIG. 1 shows the state of the pipeline after cycle 2 during the period classified as row with cycle 3. As an example, assume that the downstream device is not yet ready to accept new data from stage F (the ACCPT signal to stage F is LOW). Thus, stages E and F, which are still "blocked", but in cycle 3, stage D has received valid data D3 and has completed the overwriting of the invalid data previously present in this stage. ing. In cycle 3, since stage D cannot supply the data D3, this stage cannot accept new data, and therefore the ACCEPT signal to stage C is changed to L.
Set to OW. However, stages AC are ready to accept new data, and send it by setting the corresponding ACCEPT signal HIGH. Note that data D4 has been shifted from stage A to stage B.

Let us now assume that the downstream device is ready to accept new data in cycle 4. This downstream device sends this to the pipeline by setting the FACCEPT signal to the stage HIGH. Although stages C-F contain valid data, these stages can shift data downstream in this state, and thus can accept new data. Therefore, since each stage can step the data one step downstream, these stages set their respective ACCEPT signals out of HIGH.

As long as the ACCEPT signal to the final pipeline stage (stage F in this example) is HIGH, the pipeline shown in FIG. 1 operates as a rigid pipeline, and data is stepped one step each cycle. It only shifts. Therefore, in cycle 4, the data D1 contained in stage F is shifted from the pipeline to the next device in cycle 5,
Then all other data is shifted downstream by one step.

Now assume that the ACCEPT signal to stage F goes LOW in cycle 5. This is again explained by the fact that stages DF cannot accept new data and the ACCEPT signal from these stages to the immediately preceding preceding stage is LO.
W. Therefore, data D2, D3, and D4 cannot be shifted downstream, but data D5 is possible. Therefore, the corresponding state of the pipeline after cycle 5 is shown in FIG. 1 as cycle 6.

In accordance with the preferred embodiment of the present invention, the pipeline's ability to "fill up" empty processing stages is highly advantageous. The reason is that this capability frees the processing stages of the pipeline from coupling to each other. In other words, the entire pipeline does not have to stall or wait for a delay stage, even when it is not ready to accept data for the pipeline stage. Not only that, but if a stage cannot accept valid data, it simply forms a temporal “wall” in the pipeline. Even in this state,
The stage downstream of the "wall" is capable of continuing to advance valid data to the circuits connected to the pipeline, and the stage to the left of the "wall" is still
It can accept valid data and forward it downstream. If several pipeline stages are temporarily unable to accept new data, other stages can continue to operate normally. In particular, the pipeline will continue to accept data in its first stage A unless it still contains valid data that cannot be advanced because the next stage is not ready to accept new data. It is possible. As shown in this example, data can be transferred between pipelines and stages even when one or more processing stages are blocked.

In the embodiment shown in FIG. 1, assume that the various pipeline stages do not store the ACCEPT signal it receives from its immediately adjacent stage. Instead, when the ACCEPT signal to a downstream stage goes LOW, this LOW signal is propagated back up to the nearest pipeline stage that contains no valid data. For example, referring to FIG. 1, assume that the ACCEPT signal to stage F goes LOW in cycle 1. In cycle 2, the LOW signal propagates from stage F back to stage D.

In cycle 3, if data D3 is latched to stage D, the ACCEPT signal propagates four stages back in the flow to stage C. ACCE to stage F in cycle 4
When the PT signal goes HIGH, it propagates against the flow to stage C. In other words, AC
Changes in the CEPT signal must propagate backwards through the four stages. However, in the embodiment shown in FIG. 1, the ACCEPT signal need not propagate backwards all the way to the beginning of the pipeline if there is an intermediate stage that can accept new data.

In the embodiment shown in FIG. 1, each pipeline stage allows data to be transferred between stages without performing unintentional overwrite.
It still requires separate input and output data latches. Further, while the pipeline shown in Figure 1 is "compressible", if the downstream pipeline stages are blocked, i.e., they cannot supply the data they contain, then the pipeline is valid. Do not "inflate" to provide a stage that does not contain valid data between stages that contain data. Instead, the ability to compress depends on the cycle corresponding to the period when there is no valid data in the first pipeline stage.

For example, in cycle 4, stage F
The ACCEPT signal to LOW remains LOW, and
If pipeline stages A and 3 are filled with valid data, the pipeline is no longer incompressible and valid input data is lost, as long as valid data remains in stage A. There is.
Nevertheless, the pipeline shown in FIG. 1 reduces the risk of data loss. The reason is that the pipeline is compressible as long as there are pipeline stages that do not contain valid data.

Another embodiment of a pipeline is shown in FIGS. 2 and 3, which allows both compression and expansion in a logical way, and which has a circuit which limits the propagation of the ACCEPT signal to the nearest preceding stage. . A circuit for implementing this embodiment will be described and illustrated in detail below, and FIGS. 2 and 3 serve to illustrate the operating principle of this embodiment.

For ease of comparison, input data and ACCEPT to the pipeline embodiment shown in FIGS.
The signals are the same as in the pipeline embodiment shown in FIG. Therefore, stages E, D, and B are respectively
It has valid data D1, D2, and D3. The ACCEPT signal to the stage F is LOW, and the data D4
Are supplied to the start pipeline stage A. FIG.
FIG. 3 shows three lines connecting each adjacent paired stage of each neighboring pair of pipeline stages. The topmost line, which can be a bus, is a data line. The middle line is the line through which the VALID signal is transferred and the lowest line is the line through which the ACCEPT signal is transferred. Furthermore, as before, the ACCEPT signal to stage F remains LOW except during cycle 4. Furthermore,
In cycle 4, additional data D5 is supplied to the pipeline.

2 and 3, each pipeline stage is divided into two halves to illustrate that each stage in this pipeline embodiment includes a primary and a second data storage element. Represented as one block. 2 and 3, the primary data storage device is shown as the right half of each stage. However, it should be understood that this description is for purposes of illustration only and is not intended to be so limited.

As shown in FIG. 2 and FIG.
As long as the CCEPT signal is HIGH, data is transferred from the primary storage element of that stage to the secondary storage element of the next stage during any given cycle. Therefore, ACCEPT to stage F
Signal is LOW, but ACC to all other stages
The EPT signal is HIGH, and as a result, the data D1,
D2 and D3 are shifted forward one stage in cycle 2 and data D4 is shifted to the first stage A.

Up to this point, the pipeline embodiment shown in FIGS. 2 and 3 operates in a manner similar to the pipeline embodiment shown in FIG. However, ACC to stage F
Even if the EPT signal is LOW, the ACCEPT signal from the stage F to the stage E is HIGH. Because of the secondary storage element, as described below,
Propagation beyond stage F upstream is not necessary for the LOW ACCEPT signal. Further, by leaving the ACCEPT signal to stage E HIGH, the stage F signal signals that it is ready to accept new data. Stage F cannot transfer data D1 in its primary storage element downstream in cycle 3 (the ACCEPT signal to stage F is LOW), so stage E transfers data D2 to stage Transfer to F's secondary storage element. Both the primary and secondary storage elements of stage F contain valid data that cannot be supplied in this state, so that A from stage F to stage E
The CCEPT signal is set LOW. This therefore means that the LOW ACCEPT signal propagates backwards by one stage with respect to cycle 2, in which case this ACCEPT signal travels backwards through all the way to stage C in the embodiment shown in FIG. Must have been propagated to. Stages A-E can supply their data, so the ACCEPT signal from that stage to their immediately preceding stage is set HIGH. Therefore, the data D3 and D4 are shifted to the right by one stage. As a result, in cycle 4, the data is
They are loaded into the primary data storage elements of stage E and stage C, respectively. Stage E now contains valid data D3 in its primary storage element, but its secondary storage element stores other data without the risk of overwriting any valid data. It can still be used to

Here, as in the previous case, the ACCEPT signal to stage F goes high in cycle 4.
Assume IGH. This indicates that the downstream device to which the pipeline supplies data is ready to accept data from the pipeline. However, since the ACCEPT signal of stage F is set LOW, it indicates to stage E that stage F is not ready to accept new data. The ACCEPT signal in each cycle period tells what will happen in the next cycle, that is, whether data will be supplied (AC
CEPT HIGH) or whether the data must remain in place (ACCEPT LO)
W) Please consider showing. Thus, during cycle 4 to cycle 5, data D1 is provided from stage F to the next device and data D2 is shifted from secondary storage in stage F to primary storage, but at stage E. The data D3 in is not transferred to the stage F. The next stage ACCE
Since the PT signal is HIGH, the data D4 and D5 can be transferred to the next pipeline stage as usual.

By equipping the secondary storage element by comparing the pipeline states in cycle 4 and cycle 5, the pipeline embodiment shown in FIGS. 2 and 3 is inflatable. That is, it can be seen that the data storage element is released so that valid data can be advanced into it. For example, the data in these data blocks cannot be transferred until the ACCEPT signal to the stage F becomes HIGH, so that the cycle 4
At, the data blocks D1, D2 and D3 form a "solid wall". However, this signal is once HIGH
Then, the data D1 is shifted from the pipeline,
The data D2 is shifted into the primary storage element of stage F, and if the next device cannot receive the data D2, and the pipeline also has to "compress" once, the second data of stage F is deleted. The next storage element is free to accept new data. This is shown in cycle 6. During this cycle, data D3 is shifted into the secondary storage element of stage F, and data D4 is supplied from stage D to stage E as usual.

4, 5, 6, and 7 generally show preferred embodiments of the pipeline. This preferred embodiment implements the structure shown in FIGS. 2 and 3 using two phase non-overlapping clocks with phases φ0 and φ1. Although a two phase clock is preferred, it should be understood that various embodiments of the invention can be driven with more than one phase clock.

As shown in FIGS. 4, 5, 6 and 7, each pipeline stage is illustrated as having two separate boxes representing the primary and secondary storage elements. Furthermore, the VALID signal and data lines connect the various pipeline stages as before, but for ease of explanation, FIGS.
FIG. 7 shows only the ACCEPT signal. ACCEP
The change in state during one clock phase of the T signal is
The change from LOW to HICH is shown in FIGS. 4, 5, 6 and 7 using the up arrow. Similarly, the change from HIGH to LOW is indicated by a down arrow. The transfer of data from one storage element to one of the other storage elements is indicated by a large open arrow. The VALID signal from any given stage primary or secondary storage element is assumed to be HIGH whenever the storage element contains valid data.

In FIGS. 4, 5, 6, and 7, each cycle is shown to consist of one full cycle of non-overlapping clock phases φ0 and φ1. As described in more detail below, data is transferred from the secondary storage element (shown as the left box in each stage) to the primary storage element (shown as the right box in each stage) during clock cycle φ1. Transferred to
On the other hand, data is transferred from the secondary storage element of one stage to the primary storage element of the sono next stage during the clock cycle φ0. Similarly, FIGS. 4, 5, 6, and 7 show that the primary and secondary storage elements in each stage are A in the same manner as the ACCEPT signal is provided from stage to stage.
It is shown to be further connected via an internal accept line to provide the CCEPT signal. In this way, the secondary storage element should know when it can supply its data to the primary storage element.

FIGS. 4, 5, 6, and 7 show the φ1 phase of cycle 1. During this cycle, the data D1, D2, and D3 already shifted to the secondary storage elements of stages E, D, and B, respectively, are shifted to the primary storage elements of their respective stages. Therefore, the configuration of the pipeline during the φ0 phase of cycle 1 is the same as for cycle 1 of FIG. As before, the ACCEPT signal to stage F is assumed to be LOW. However, FIG. 4, FIG.
As shown in FIGS. 6 and 7, the ACCEPT signal to the primary storage element of stage F is LOW, but since this storage element does not contain valid data, this storage element sends the ACCEPT signal to its secondary storage element. Set to HIGH inside.

The ACCEPT signal is also set HIGH into the primary storage element of stage E from the secondary storage element of stage F because the secondary storage element of stage F does not contain valid data. As before, the primary storage elements of stage F can accept data so that the data in all upstream primary and secondary storage elements can be shifted downstream without any data being overwritten. is there. 1
The shifting of data from one stage to the next is done during the next φ0 phase period in cycle 2. For example, the valid data D1 contained in the primary storage element of stage E is shifted to the secondary storage element of stage F and the data D4 is in the pipeline, ie
Shifted into the secondary storage element of stage A, and so on.

The primary storage element of stage F does not yet contain valid data during the φ0 phase period in cycle 2, so the ACCEPT signal from the primary storage element to the secondary storage element of stage F remains HIGH. is there. Therefore, during the φ0 phase period in cycle 2, the data can be shifted one step further to the right, ie from the secondary storage element in each stage to the primary storage element. However, if the valid data is
Loaded to primary storage element of stage F, ACCEPT from downstream devices to stage F is still LOW
, The data cannot be shifted from the secondary storage element of stage F without overwriting and destroying valid data D1. Therefore, the ACCEPT signal from the primary storage element to the secondary storage element of stage F goes LOW. However, the secondary storage device does not contain valid data, and its ACCEPT
Since the signal output was HIGH, the data D2 can still be shifted to the secondary memory of stage F.

During the φ1 phase of cycle 3, data is shiftable in all previous stages, but data D2 cannot be shifted into the primary storage element of stage F. However, once valid data has been loaded into the secondary storage element, stage F cannot provide this data. Stage F sends this event to its ACCEPT signal at LO.
Send by setting it outside W.

The ACCEPT signal to the stage F is LOW.
Assuming that it remains, the data upstream of the F stage is shifted during and within the stages of each clock phase until the next valid data block D3 reaches the primary storage element of stage E. It is possible to As shown in the figure, φ in cycle 4
This state is reached during one phase period.

During the φ phase period of cycle 5, the data D3
Loading of the stage E into the primary storage element is completed. This data cannot be further shifted, so that the ACCEPT signal from the primary storage element of stage E is L
Set to OW. Upstream data can be shifted normally.

Here, as shown in cycle 5 of FIG.
Suppose a device connected downstream in the pipeline can accept pipeline data. This device drives the ACCEPT signal into the pipeline stage F during the φ1 phase period of cycle 4 to HIG.
Send this event by setting it to H.
Here, the primary storage element of stage F shifts the data to the right, and the primary storage element of stage F is also capable of accepting new data. Therefore, since the data D1 was shifted during the φ1 phase period of cycle 5, the primary storage element of stage F no longer contains the data that has to be saved. Thus, during the φ1 phase of cycle 5, data D2 is shifted from secondary storage element to primary storage element within stage F. The secondary storage element of stage F can accept new data and signals by setting the ACCEPT signal HIGH in the primary storage element of stage E. Within one stage, ie during the transfer of data from its secondary to its primary storage element, both sets of storage elements contain the same data, but the data in the secondary storage element is in the primary storage element. Since it is also retained, this data in the secondary storage element can be overwritten without data loss. The same is true for data transfers from the primary storage element of one stage to the secondary storage element of the next stage.

Here, during the φ1 phase period of cycle 5, the ACCE to the primary storage element of the stage F thereof is performed.
Assume that the PT signal goes LOW. This means that stage F cannot transfer data D2 from the pipeline. Therefore, the stage F ACCs from its secondary storage element to its temporary storage element to prevent overwriting of valid data D2.
Set the EPT signal LOW. However, stage F
The data D2 stored in the secondary storage element of the memory cannot be overwritten without loss, and as a result, the data D3 can be stored in the secondary storage of the stage F during the φ0 phase of cycle 6. Transferred within the element. The data D4 and D5 cannot be shifted downstream as normal. Once valid data D3 has been stored in stage F along with data D2, as long as the ACCEPT signal to the primary storage elements of stage F is LOW,
No secondary storage element can accept the new data, and the secondary storage element sends this by setting the ACCEPT signal to stage E LOW.

AC from downstream device to pipeline
If the CEPT signal changes from LOW to HIGH, or vice versa, this change must not propagate further upstream in the pipeline than the immediately preceding preceding storage element (either in the same stage or earlier). Inside the pipeline stage). Furthermore, this change propagates upstream in the pipeline, one storage element block per clock phase.

As illustrated in this example, FIGS.
The concept of "one stage" in the pipeline structure shown in FIGS. 6 and 7 is a perceptual problem to some extent. If the data is located between stages, it is transferred within the stage (secondary storage element to primary storage element) (upstream stage primary storage element to adjacent downstream stage secondary storage element). Thus, one stage may be considered to consist of a "primary" storage element followed by a "secondary storage element" instead of the case shown in FIGS. 4, 5, 6 and 7. Therefore, the concept of "primary" and "secondary" storage elements is mostly a labeling issue. 4, 5, 6, and 7, when data is transferred from one stage to the next stage or device, the primary storage element is the element from which the data is retrieved, and
The "primary" storage element can also be referred to as the "output" storage element, as the "secondary" storage element can be the "input" storage element for the same stage.

As shown in FIGS. 1 and 3, in the description of the above-mentioned embodiments, only the transfer of data under the control of the ACCEPT and VALID signals has been mentioned. In addition, before feeding between the internal storage elements of each pipeline stage, or before feeding to the next pipeline stage, each pipeline stage must also process the data it arbitrarily received. Please understand that is possible. Therefore, as shown in FIG.
Referring again to FIGS. 6 and 7, pipeline stages include input and output storage elements and can be defined as part of a pipeline that optionally processes data stored in the storage elements.

Furthermore, the "device" downstream from the pipeline stage F need not be some other type of hardware structure, but rather may be another pipeline itself or part thereof. As described below, a pipeline stage requires multiple clock phases to complete processing of that data, not only if all of the downstream storage elements are filled with valid data. Even if that ACCEP
The T signal can be set LOW. This can similarly occur if the pipeline stage produces valid data in one or both of its storage elements. In other words, it is not necessary for the stage to simply supply the ACCEPT signal based on whether the storage element immediately downstream contains valid data that cannot be supplied. Rather, the ACCEPT signal itself can be modified either within the stage or by circuitry external to the stage to control the data path between adjacent storage elements. The VALID signal can be similarly processed in a similar manner.

2-wire interface (VALID and A
(One wire for each CCEPT signal)
The great advantage of is that it has the ability to control a pipeline without the control signals required to propagate the entire path backwards through the pipeline to its start stage. Referring again to cycle 3 of FIG. 1, for example, stage F "tells" stage E that it cannot accept data, but stage E tells stage D, and stage D tells stage C Tell. As a practical matter, this signal may propagate further back along the pipeline if there are more stages than the valid data involved. In the embodiment shown in FIGS. 4, 5, 6 and 7, cycle 3, the LOW ACCEPT signal does not propagate upstream from stage E and stays up to its primary storage element.

As will be described below, this embodiment can achieve this flexibility without specifically attaching to the silicon portion needed to implement the design.
In general, each latch in the pipeline used for data storage requires only one extra transistor (extremely efficient silicon fit). In addition, the ACCEP associated with the data latch in each half stage
It is preferable to add two extra latches and a few gates to handle the T and VALID signals.

FIG. 8 shows a hardware structure for realizing the stages as shown in FIGS. 4, 5, 6 and 7.

By way of example only for illustration, transfer 8-bit data in parallel through a pipeline (with or without further operation in optional combinatorial logic). Suppose.
However, it should be understood that either 1-bit data or less than 8-bit data can be used to implement the invention. Furthermore, the 2-wire interface according to this embodiment is suitable for use with data buses of any width, and the width of the data bus may vary from stage to stage, depending on the needs of a particular application. possible. The interface according to this embodiment can also be used to process analog signals.

As discussed above, other conventional timing arrangements can be used, but the interface is preferably controlled by a two-phase non-overlapping clock. 8-16, these clock phase signals are referred to as PH0 and PH1. In FIG. 8, one line is shown for each clock phase signal.

The input data is the multi-bit data bus I.
It is input to the pipeline stage via N data, and transferred to the next pipeline stage or the next receiving circuit via the output data bus OUT data. Input data is first loaded into a series of input latches (one for each input data signal), collectively referred to as LDIN, which make up the secondary storage element described above, in the manner described below.

In the example of the present embodiment shown in the figure, the Q outputs of all latches follow their D inputs, ie when the clock input is HIGH, ie at a logic "1" level all latches. Q output is loaded. In addition, the Q outputs hold their last value.
In other words, the Q outputs are "latched" on the falling edge of their corresponding clock signals.

Each latch has, for its clock, one or two non-overlapping clock signals PH0 or PH1 (as shown in FIGS. 9 and 10), or one of these clock signals PH0, PH1. And a logical AND combination of one and one logic signal. However, the present invention provides a latch that latches on the rising edge of the clock signal, or other known latching arrangements, as long as conventional methods are applied to ensure proper timing of the latch operation. Works equally well with.

The output data from the input data latch LDIN is provided via an optional and optional combinational logic circuit B1, which is provided for converting the output data from the input latch LDIN into intermediate data. Of course, this intermediate data is then later loaded into the output data latch LDOUT, which comprises the primary storage element described above.

The output from the output data latch LDOUT can likewise be supplied via an optional and optional combinational logic circuit B2 before being supplied downstream to the next device as OUT data. The next device may be another pipeline stage or another device connected to the pipeline.

In the implementation of the invention, each stage of the pipeline is likewise validated by the input latch LVIN,
Validation output latch LVOUT, acceptance input latch L
Includes AIN and an acceptance output latch LAOUT. Each of these four latches is preferably a simple single stage latch. Latch U / IN, LVOUT,
The outputs from LAIN and LAOUT are respectively Q
VIN, QVOUT, QAIN, QAOUT. The output signal QVIN from the validation input latch is connected to the validation output latch LVOUT either directly as an input or via an intermediate logic device or a circuit that modifies the signal.

Similarly, the output validation signal QVOUT of a given stage may be connected to the input of the validation input latch QVIN of the next stage, either directly or through an intermediate device or logic circuit. . The device in this case may be a device that modifies the validation signal. This output QVIN is likewise connected to a logic gate (described below), the output of which is connected to the input of the receiving input latch LAIN. Output QAOUT from acceptance output latch LAOUT
Are optionally connected to similar logic gates (discussed below) via other logic gates.

As shown in FIG. 8, the output validity checking signal Q
VOUT is IN VALI depending on the subsequent stage.
OUT VAL that can be received as a D signal
It forms an ID, or simply points valid data to subsequent circuits connected to the pipeline. The next circuit or stage is ready to accept data and the signal OUTA
It is instructed to each stage as CCEPT, and this signal is
It is preferably connected as an input to the receiving output latch LAOUT via the logic circuit described below. Similarly, the output QAUT of the receiving output latch LAOUT, preferably via the logic circuit described below, receives the receiving input latch LOUT.
Connected as an input to AIN.

In practicing the present invention, the output signals QVIN, QVOUT from the plausibility check latch LVIN are used to form acceptance signals QAOUT, OUT ACC to form inputs to the acceptance latches LAIU, LAQUT, respectively.
Combined with EPT1 respectively. In the embodiment shown in FIG. 8, these input signals are formed as a logical NAND combination of the respective validation signals QVIN, QVOUT, as the logical inverse of the respective received output signals QACUT, OUT ACCEPT. Conventional logic gates NAND1 and UAND2 implement a NAND operation, and inverters INV1 and INV2 form the logical inverse of their respective acceptance signals. As is well known in the digital design art, if any or all of the input signals to a NAND gate are in a logic "0" state,
The output from the NAND gate is a logic "1". Therefore, the output from the NAND gate will be a logical "0" only if all inputs to the NAND gate are in a logical "1" state.
Is. As is also well known in the art, the output of a digital inverter, such as INV1, is a logical "1" when its input signal is "0" and its input signal is "1". Is “0”.

Therefore, when "NOT" indicates binary inversion, the input to the NAND gate NAND1 is QVI.
N and NOT (QACUT). Using known techniques, the input to the receiving latch LAIN can be decomposed as follows.

NAND (QVIN, NOT (QAOUT)) = NOT (QVIN) OR QAQUT In other words, the signal QVIN is “0”, or
If the signal QAQUT is "1" or both, the inverter INV1 and the NAND gate NAND
The combination of 1s is a logical "1". Therefore, the gate N
The AND1 and the inverter INV1 have one single O whose one input is directly coupled to the QAOUT output of the receiving latch LAOUT and one of which is coupled to the inversion of the output signal QVIN of the validation input latch LVIN.
It can be realized by an R gate.

Similarly, as is well known in the digital design art, many latches suitable for use as validation and acceptance latches have two outputs Q, and NOT.
(Q), that is, Q, and its logical inverse. So if you choose this type of latch,
One input to the OP gate (and hence) can be directly coupled to the NOT (Q) output of the validation latch LVIN. Gate NAND1 and inverter INV
1 can be realized using known conventional techniques. However, depending on the architecture of the latch used, a latch that does not invert the output is used, and instead, a gate NAND1 and an inverter IN are used.
In some cases it may be more efficient to provide V1. Both of these can likewise be efficiently realized in silicon devices. Therefore, any known arrangement can be used to generate the Q signal and / or its logical inverse.

Both clock signals (PH on the output side
0, and PH1 on the input side) and the output from the accept latch on the same side is a logic "1", the data and validation latches LDIN, LDOUT, LVIN, and LVOUT will pull their respective data inputs. To load. Thus, the clock signal (PH0 for input latches LDIN and LVIN), and the output of each accepting latch (LAIN in this case) is used in a logical AND method, and they are both logic "1".
Data is loaded only if

In a particular application, such as a CMOS implementation of a latch, the logical AND operation that controls the loading of the latch (via the CK shown, or via the activation "input") will cause the respective activation input signal ( For example, PH for the latches LVIN and LDIN
0 and QAIN) can be easily realized in the conventional way by connecting the gates to the MOS transistors connected in series to the input line of the latch. Therefore,
It is necessary to provide the actual logical AND gate,
This can cause timing problems due to propagation delays in high speed applications. Therefore, the AND gates shown in the figure merely dictate the logic function to be performed to generate the activation signals for the various latches.

Therefore, the data latch LDIN loads the input data only when PH0 and QAIN are both "1". This data latch latches this data when either of these two signals becomes "0".

One of the clock phase signals PH0 or PH1 to clock the data and validation latches on the input (and output) side of the pipeline stage.
Only one is used, but the other clock phase signal is used directly to clock the receiving latch on the same side. In other words, the receiving latches on either side (input or output) of the pipeline stage are:
It is preferably clocked "out of phase" with data and validation latches on the same side. For example, PH0 is used to generate the clock signal CK for the data latch LDIN and the validation latch LVIN, while PH1 is used to collock the receiving input latch.

As an example of the operation of a pipeline augmented by a two-wire validation and acceptance circuit, the input to the circuit is initially valid from either the preceding pipeline stage or the transmission device. Assume no data is provided. In other words, the validation input signal IN VALID to the stage shown is still "1" because the system was reset very recently.
Suppose that is not. Further, assume that several clock cycles have occurred since the last time the system was reset, and as a result, the circuit has reached steady state. Therefore, during the next positive period of clock PH0, the validation input signal QVIN from the validation latch LVIN is loaded as "0". Therefore, during the next positive period of clock signal PH1, the input to the accept input latch LAIN (via gate NAND1, or other equivalent gate) is loaded as a "1". That is, because the data in the data input latch LOIN is not valid, the stage (since it does not hold any data value store) will signal that it is ready to accept input data.

Note that in this example, the signal IN ACCEPT is used to enable the data and validation latches LOIN and LVIN.
At this point, since the signal IN ACCEPT is a "1", these latches effectively act as conventional transparent latches, so that any data on the IN data bus will have a clock signal PHD of "1". Is immediately loaded into the data latch LOIN. Of course, as long as the output QAOUT from the accept latch is "1", this invalid data will be loaded into the next data latch LDOUT of the next subsequent pipeline stage as well.

Therefore, unless the data latch contains valid data, during the next positive period of each clock signal, this latch will accept, or "load," any data provided. On the other hand, such invalid data will not be loaded into any stage for which the acceptance signal from the corresponding acceptance latch is low (ie, "0"). In addition, the output signal from the validation latch (forming the validation input signal to the next validation latch) as long as the corresponding INVALID (or QVIN) signal to the validation latch is low. Remains "0".

If the input data to the data latch is valid, this is indicated by the validity checking signal INVALID rising to "1". The output of the corresponding validation latch then rises to "1" at the next rising edge of each clock phase signal. For example, at the next rising edge of clock phase signal PH0, the corresponding IN VALID signal goes high (ie,
If it goes to "1"), the validation input signal QVIN of the latch LVIN goes to "1".

Instead, here, the data input latch L
Assume that the DIN contains valid data. If the data output latch LDOUT is ready to accept new data, its acceptance signal QAOUT
Is "1". In this case, during the next positive cycle of clock signal PH1, data latch LDOUT and validation latch VOLT are enabled, and data latch LDOUT loads the data located at its input. Since the clock signals are non-overlapping, the above operation occurs before the next rising edge of the other clock signal PH0. Therefore, on the next rising edge of PH0, the preceding data latch (LDIN) will not latch in the new input data from the preceding stage until the data output latch LOOUT has safely latched the data transferred from the latch LDIN. .

Therefore, because it operates on alternating phases of the clock, the same sequence is continued by all adjacent pairs of data latches (in stage or between adjacent stages) capable of receiving data. Any data latch that is not ready to accept new data because it contains valid data that is not yet available,
Has an output acceptance signal (QA output from its acceptance latch LA) which is OW, and its data latch LDI
N or LDOUT is not loaded. Therefore, the acceptance signal (output from the acceptance latch) of a given stage or one side (input or output) of a certain stage is L
As long as it is OW, its corresponding data latch is not loaded.

The reset function included in the preferred embodiment is shown in FIG. In the example shown in this figure, the reset signal N
OTRESET0 is a validation output latch LVOUT
Inversion reset input (inversion is conventionally indicated by a small circle). As is well known, this means that the validation latch LVOUT is forced to output "0" whenever the reset signal NOTRESET0 goes to "0". One advantage of resetting the latch when the reset signal goes low (goes to "0") is to reset the latch by interrupting the transmission. In this case, these latches are in the "empty" or reset state whenever a valid transmission begins and the reset signal goes high. Therefore, the reset signal NOTRESET0
Operates as a digital "on / off" switch. Therefore, this signal must be a HIGH value to activate the pipeline.

Note that it is not necessary to reset all the latches holding valid data in the pipeline. As shown in FIG. 8, the validity check input latch LVIN is not directly reset by the reset signal NOTRESET0 but is reset indirectly.
Assume that the reset signal NOTRESET0 drops to "0". Reception output latch LAQUT (gate N
When the input to (via AND1) goes HIGH, regardless of its previous state, the validation output signal QVOUT
Also drops to "0". Similarly, the acceptance output signal QAOUT also rises to "1". Next, this QAOUT
The value "1" is set to "1" and the validity check input signal QVI
Transferred to the accept input latch input LAIN, regardless of the N state. Next, the acceptance input signal QAI rises to "1" at the next rising edge of the N clock signal PH1. Assuming the validation signal IN VALID has been correctly reset to "0", if the validation latch LVIN had been reset directly, then the next rising edge of the clock signal PH0, when implemented. At, the output from the validation latch LVIN goes to "C".

As this example shows, resetting all the validation latches requires only resetting the validation latch on only one side of each stage (including the final stage). is there. In fact, in many applications it is not necessary to reset all other validation latches. That is, the reset signal NOTOSET0 is the phase PH of both phases of the clock.
0, PH1 can be guaranteed to be present for more than one complete cycle, an "auto-reset" (backpropagation of the reset signal) is a validation latch in the preceding pipeline stage. Occurs for a period. As a practical matter, if the reset signal is held for at least the total number of cycles of both phases of the clock with the same number of pipeline stages as there are,
It is only necessary to directly reset the validation output latch in the final pipeline stage.

9 and 10 show the non-overlapping clock signal PH.
0, the relationship between PH1, the effect of the reset signal, the retention of data, and the illustrated two sides of the pipeline stage in the embodiment shown in FIG. 8 for different sequences of validation and acceptance signals, and The transfer of data during that time is shown. In the example shown in the timing diagrams of FIGS. 9 and 10, it is assumed that the outputs from the data latches LDIN, LDOUT are supplied without further operation by the intervening logic blocks B1, B2. This is shown as an example and does not have a restrictive meaning. Every combinatorial logical structure is
It should be understood that it can be included between the data latches of successive pipeline stages or between the input and output of one single pipeline stage.
The values that are actually shown for the input data (for example,
The HEX data words “aa” or “04”) are likewise merely illustrative. The width of the input data bus is arbitrary (and can be analog, as described above), as long as the data latch or other storage device can accommodate and latch or store the value of each bit or input word. Absent). Preferred Data Structure— “Token” In the sample application shown in FIG. 8, there is no control circuit that excludes stages from being able to supply input data through combinatorial logic blocks B1, B2, etc., so each stage is all Process the input data of. To provide even greater flexibility, the present invention has a data structure in which "tokens" are used to distribute data and control information throughout the system.
Each token consists of a series of binary bits divided into one or more token word blocks. Further, the bit corresponds to one of the following three types: address bit (A), data bit (D), or extension bit (B). Simply by way of example, the data can be
Suppose it is transferred as a word over a 0-bit bus with a 1-bit extended bit line. An example of a 4-word token is shown below in order of transmission. 1st word E A A A A D D D D D D 2nd word E D D D D D D D D D 3rd word E D D D D D D D D D D 4th word E D D Note that the DDDDDDDD extension bit E is used as an addition (preferably) to each data word. Furthermore, the length of the address field can be variable and is preferably transmitted immediately after the extension bit of the first word.

Therefore, the token is (2
Hex) consists of one or more words of digital data.
Each of these words is preferably transferred in parallel and in sequence. However, this transfer method is not always necessary, and serial data transfer is also possible using a known technique.
For example, in a moving image parsing system, control information is transmitted in parallel, in which case data is transmitted in series.

As illustrated by way of example, each token is
Preferably at start-up, it has one address field (a string of A-bits) that identifies the type of data contained in the token. In most applications, a single word, or part of a word, is sufficient to transfer the entire address field. However, even logic circuits are included in the corresponding pipeline stages that are capable of receiving the entire address field and storing some representation of a partial address field that is long enough to be decoded. If so, the above conditions are not always necessary for the present invention.

Note that no dedicated lines or registers are needed to carry the address fields. Note that the address field is transmitted using data bits. As explained below,
A pipeline stage is not slowed down if it is not intended to be activated by a particular address field, ie if the stage can be fed along the token without delay.

The rest of the data in the token that follows the address field is not constrained by the use of the token.
These D data bits can take any value, and the meaning attached to these bits is not important here. That is, the meaning of the data can change, for example, depending on where in the system the data was located at a particular time. The number of data bits D appended after the address field can be as long or as short as required, and the number of data words can vary significantly with different tokens. The address field and extension bits are used to carry control signals to the pipeline stages. Since the number of words (strings of D bits) in the data field can be arbitrary, the information carried in the data field can also change accordingly. Therefore, the following description is directed to the use of address and extension bits. In the present invention, a token is a particularly useful data structure when a large number of blocks in a circuit are connected together in a relatively simple structure. The simplest configuration is, for example, a pipeline of processing steps, one of which is shown in FIG. However, the use of tokens is not restricted to use in pipeline structures only.

It is again assumed that each box represents a complete pipeline stage. In the pipeline of Figure 1, data flows from left to right in the figure. The data enters the machine and feeds into processing stage A. This stage may or may not modify the data, and feeds the data to stage B. It can be complex if modified, and in general,
The number of data items flowing into a given stage is not the same as the number of data flowing out. Stage B modifies the data again,
Then supply it to Stage C, and so on. In this type of design, it is impossible for the data to flow in the opposite direction. Thus, for example, stage C cannot supply data to stage A. This constraint is often entirely acceptable.

On the other hand, it is highly desirable that stage A can transfer information to stage C even when these two blocks are not directly connected. The communication between the stages A and C is limited to the stage B. One advantage of tokens is that they have the ability to accomplish this type of communication. Any processing stage that does not recognize the token simply supplies the next block with a token that does not change.

According to this example, one extension bit is transmitted with the address and data fields in each token, so that the processing stage does not need to decode the address at all, and the token ( Can be of any length).
According to this example, every token that contains an extension bit that is HIGH ("1") is followed by a trailing word that is part of the same token. This subsequent word also has an extension bit, which indicates whether there are more token words in the token. If the stage encounters a token word whose extension bit is LOW (one "0"), it knows that token word is the last word of the token. Therefore, the next word is assumed to be the first word of the new token.

It should be noted that although a simple pipeline of processing stages is particularly useful, tokens can be applied to processing elements of more complex configurations. An example of a more complex processing element is shown below.

In the case of the present invention, it is not necessary to use the state of the extension bit to send the last word of a given token by providing the extension bit set to "0". One alternative to the preferred implementation plan is to move the extension bit to point to the first word of the token instead of the last word of the token. This can be achieved by appropriate modification of the encoding / decoding hardware.

The advantage of using the extension bit of the present invention to send the last word of the token rather than the first word is that the block of circuitry depends on whether the token has the extension bit. It is often useful to modify the mode of operation. An example of the above argument is a token that activates a stage that processes a video quantization value stored in a quantization table (typically a memory device). For example, a table containing 64 8-bit arbitrary binary integers.

To load a new quantizer table into the quantizer stage of the pipeline, a "QUANT table" token is sent to the quantizer. In such cases,
For example, a token consists of 65 token words.
The first word contains the code "QUANT table", i.e. creates the quantization table. This is followed by 64 words, which are integers in the quantization table.

When encoding moving picture data, it is sometimes necessary to transmit a quantization table of this kind. To accomplish this function, a QUANT table token without extension words can be sent to the quantizer stage.
If we look at this token and notice that the extension bit of the first word is LOW, the quantizer stage reads its quantization table and QUANT containing 64 quantization table values.・ Table token can be created. The extension bit is HIGH until the new end of the token is indicated by the LOW extension bit corresponding to the value of the 64th quantization table, and the token continues in the state of the HIGH extension bit. , The extension bit of the first word (which was LOW) is changed. This proceeds in a general way through the system and is encoded in the stream of bits.

Continuing with this example, the quantizer determines whether the first word of the QUANT table token has its extension bit set or not.
It is possible to either load the new quantization table into its own memory device or read the table.

The choice of whether to use the extension bit to send the first token word or the last token word in a token depends on the system in which the pipeline is used. Both alternatives are possible according to the invention.

Another alternative to the preferred extension bit scheme is to include a length count at the start of the token.
For example, this type of device may be efficient if the token is very long. For example, assume that a typical token length in a given application is 1000 words. Using the proposed extended bits (with each token word attached bit), the token requires 1000 additional bits to include all the extended bits. However, to encode the token length in binary form, only 10
Need a bit.

Therefore, although there are uses for long tokens, experience has shown that there are many uses for short tokens. Here, the preferred extension bit scheme is advantageous.
If the token is only one word long, then only one bit is needed to send it. However, the counting proposal requires the same 10 bits as in the above case.

The drawbacks of the length counting scheme are: 1) Inefficient for short tokens. 2) Restrict the maximum length of the token (if it is less than 1023 words, it can be counted with only 10 bits) 3)
The token length must be known before the count is generated (probably at token start). 4) All circuit blocks that handle tokens need to have hardware to count words. 5) If the count is corrupted (due to a data transmission error), it is not clear if recovery is achievable.

The advantages of the proposed extended bit according to the invention are the following: 1) Unrecognized tokens can be supplied correctly by considering only the extension bits, so the pipeline stage does not need to have a circuit block to decode all tokens. 2) The extension bit encoding is the same for all tokens. 3) There is no limit on the length of the token. 4) This scheme is efficient for short tokens (in terms of overhead for representing token length). 5) Error recovery is naturally achieved. If the extension bit is corrupted, then one random token is generated "(for corrupt extension bits from" 1 "to" 0 ") or the token is lost (corrosion extension bit is" 0 "). To "1"). Furthermore,
The problem is localized only to the relevant token. After the token, correct operation is automatically resumed.

Furthermore, the length of the address field may change. This is highly advantageous as it allows the most common tokens to be squeezed into a very small number of words. This ensures that all processing stages can be operated continuously at full bandwidth and, consequently, is very important in a video data pipeline system.

According to the invention, in order to allow variable length address fields, addresses are chosen such that short addresses followed by random data are never confused with longer addresses. The preferred technique for encoding the address field (also serving as the "code" for activating the intended pipeline stage) is the well-known technique first disclosed by Huffman and commonly referred to as the "Huffman Code". However, it should be understood by one of ordinary skill in the art that other coding schemes will succeed.

Huffman coding is well understood in the field of digital design, but the following example is provided to provide a general background.

A Huffman code consists of words organized by a sequence of symbols (for example, in the context of digital systems such as the present invention, symbols are generally binary digits). The length of the codeword is variable, and a special property of the Huffman codeword is that the codeword is chosen such that no longer codeword can begin with the symbols forming the shorter codeword. In accordance with the present invention, the token address field is preferably (but not necessarily) selected using known Huffman coding techniques.

Further, in the present invention, it is preferable that the address field starts at the most significant bit (MSB) of the first word token. (The manifestation for the MSB is arbitrary, and the proposal can be modified to accommodate various manifestations for the MSB.) The address field continues via the less significant contiguous bits. In a particular application, if the token address requires more than one token word, the address field, which is the least significant bit in any given word, continues in the most significant bit of the next word. The minimum length of the address field is 1 bit.

Any one of several known hardware structures can be used to generate the tokens used in the present invention. One such structure is a microprogrammed state machine. However, known microprocessors or other devices may be used.

The main advantage of the token scheme according to the invention is its adaptability to unforeseen needs. For example, when a new token is introduced, this introduction typically affects only a few pipeline stages. The most likely case is that only two stages, or circuit blocks, are affected. That is, one block first creates a token, and the other block or stage is newly designed or modified to handle this new token. Note that the other pipeline stages do not need to be modified at all. On the contrary, these stages do not recognize this new token and therefore supply unmodified tokens, so that new tokens can be handled without modifying the design of these stages.

It is a distinct advantage of the present invention that it has the ability to keep an existing designed device substantially unaffected. Keeping some semiconductor chips in one chip set completely unaffected by improving some of the other semiconductor chips in the chip set in design. Is possible. This is an advantage from both the customer and chipmaker's perspective. Reuse the same design, even if the modifications mean that all chips are affected by the design change (conditions that progressively increase the level of integration and reduce the number of chips in the system) As such, it remains a significant advantage in terms of time to market before being achievable.

Note in particular the conditions that arise when it is necessary to extend the token set to include a two word address. Even in this case, it is not necessary to modify the existing design. The token decoder in the pipeline stage attempts to decode the first word of this kind of token, and concludes that the first word does not recognize the token. The token decoder then supplies the token modified by using the extension bits to properly perform this operation. Since this token decoder "assumes" that the second word is part of the data field of the token that it does not recognize, this token decoder (even if it contains address bits) Do not try to decode the second word of

In many cases, pipeline stages, or connected circuit blocks, modify the token. This will generally, but not as a requirement, take the form of modifying the cocoon data field. Furthermore, it is common to change the number of data words in the token to be remodeled by either removing specific data words or adding new data words. In some cases, tokens are completely removed from the token stream.

In most applications, the pipeline stage generally only decrypts a few tokens (actuated by tokens). That is, the stage does not recognize the other token and supplies the other token unchanged. Very often only one token is decrypted, the data token itself.

In many applications, a particular stage of operation depends on the results of its own past operations. Therefore, the "state" of the stage
Depends on the previous state. In other words, the stage relies on stored state information, which means that it must retain some information about its own history in one or more previous clock cycles. Another way to describe. The invention is well suited for use in pipelines that include this type of "state machine" stage, as well as in applications where the latches in the datapath are simple pipeline latches.

A two-wire interface according to the invention is
Compatibility with this type of "state machine" circuit is an important advantage of the invention. This is especially true when the datapath is being controlled by the state machine. In this case, the two wire interface technique described above can be used to ensure that the "current state" of the machine remains in step with the data it is controlling in the pipeline.

A simplified block diagram of one example of circuitry included in the pipeline stage for decoding the token address field is shown in FIG. This figure shows a pipeline stage with "state machine" characteristics.
Each word of the token contains one "extension bit," which is HIGH if there are more words in the token, or LOW if this is the last word of the token. If this is the last word of the token, then the next valid data word is
It is the start of a new token, so its address must be decrypted. Therefore, the decision as to whether to decode the token address in any given word depends on knowing the value of the previous extension bit.

For the purpose of simplifying the drawing only, the 2-wire interface (with accept and validation signals, and latches) will not be shown, and for all details dealing with resetting the circuit It will be omitted. As before, the 8-bit data word is
It is assumed as an example only, not due to constraints.

This exemplary pipeline stage delays the data bits and extension bits by one pipeline stage. In addition, this stage decrypts the data token. At the time when the first word of the data token is supplied to the circuit output, the signal "data A
DDR "is created and set HIGH.
The data bits are delayed by latches LDIN and LDOUT, each of these latches being in this example (8 inputs 8
It is repeated eight times for the eight data bits used for (corresponding to the output latch). Similarly, extension bits are delayed by extension bit latches LEIN and LEOUT.

In this example, the latch LEPREV is
It is provided to store the most recent state of the extension bit. The value of the extension bit is loaded into LEIN, and
LEOUT is loaded on the next rising edge of the non-overlapping clock phase signal PH1. Therefore, the latch LEOUT contains the value of the current extension bit, but only during the second half of the non-overlapping two-phase clock. However, the latch LEPREV loads this extension bit value at the next rising edge of the clock signal PH0, the same signal that enables the extension bit input latch LEIN. Therefore, the output QEP of the latch LEPPEV
REV holds the value of the extension bit during the previous PH0 clock phase. The result of adding the non-inverted MD [2] of the latch LDIN to the 5 bits of the data word output from the inverted Q output is a series of logic gates NAND1, NAND2,
, And the previous extension bit value QEPPEV in NOR1. These operations are well known in the field of digital design technology. Nickname "NM
D [m] "indicates the logical inversion of bit m of mid data word MD [7: 0]. Using known Boolean algebra techniques, the previous extension bit is "0" (QPPEV =
This logical block (NOR1
Output signal SA is HIGH (“1”)
And can be shown to be
It can be seen that the data word structure at the output of the latch (original input word) LDIN is "000001xx". That is, five high-order bits, MD [7]
-MD [3] bits are all "0" s, and bits MD [2] are "1" s, and the bits in the zero-one positions have any arbitrary value.

The four possible data words (there are four permutations of "xx") therefore drive the output of the address signal latch LADDR whose SA, and thus the input SA, is connected to its input. In other words, this stage is such that if one of the four possible correct tokens is provided and if the previous extension bit was zero, i.e. the previous data word is the last in the previous series of token words. A word, meaning that the current token word is the first token word in the current token, the activation signal (data AD
DP "1") is supplied.

Therefore, the signal Q from the latch LEPREV
When PREV is LOW, the value at the output of latch LDIN is the first word of the new token. The gates NAND1, NAND2, and NOR1 decode the data token (000001xx). However, the address decoding signal SA is delayed in the latch LADOP so that the signal data ADDR has the same timing as the output data OUT data and OUTEXTN.

Another simple example of a state dependent pipeline stage according to the present invention is shown in FIG. This example shows the signal LA to indicate the value of the previous output extension bit OUTEXTN.
Generate STOUTEXTN. 2 to the current and final extension bit latches LEOUT and LEPPEV, respectively
One of the two enable signals (at the CK input) indicates that the data is valid and is being accepted (the output validation and Q outputs from the accept latches LVOUT and LAOUT, respectively). HI
GH), so that these latches load only the new values for these latches from gate AND1. In this way, these signals only hold valid extension bits, and non-true values associated with invalid data are not loaded into these signals. In the embodiment shown in FIG. 12, the 2-wire valid / acceptable logic includes OR1 and OR2 gates that are provided with a downstream accept signal and an input signal consisting of the non-inverting output of each of the validation latches LVIN and LVOUT. This illustrates one way in which the gates NAND1 / 2 and INV1 / 2 in FIG. 8 can be replaced if the latch has an inverting output.

This is a very simple example of a "state-dependent" pipeline stage, ie one that depends on the state of only one single bit, but the data is actually transferred between the pipeline stages. Only in all cases is it true that all latches holding state information are updated. In other words, only if the data are both valid and are being accepted by the next stage. Therefore, care must be taken to ensure that this type of latch is properly reset.

Generating and using tokens in accordance with the present invention offers numerous advantages over known encoding techniques for transferring data through pipelines.

First, as already mentioned, the length of the token's address field can be variable (and, for example, Huffman coding) in order to provide an efficient representation of a general token. Available).

Second, by consistently encoding the length of the token, the end of the token (thus Allows the next token's start to be processed correctly (including simple, non-operational transfers).

Third, one stage and the nearest non-adjacent downstream stage in the pipeline due to the rules and hardware structure for handling unrecognized tokens (ie, for supplying tokens unchanged). Allows communication with. In addition, this allows future changes to the token set without requiring a major redesign of the existing pipeline stages, thus increasing the scalability and efficient adaptability of the pipeline. The tokens of the present invention are particularly useful when used with a two wire interface, as described above and below.

The function of the exemplary pipeline stage shown in FIGS. 13 and 14 will now be described. If a stage is processing a scheduled token (referred to as a data token in this example), the stage
All words except the first word, including the address field of the data token in this token, are duplicated. On the other hand, if the stage is processing any other type of token, the stage deletes all words. The overall effect is that only data tokens will appear at the output, and each word within these tokens will be repeated twice.

Many of the components of the system illustrated herein can be the same as those depicted in the much simpler structure shown in FIGS. 4, 5, 6 and 7.
This illustrates an important advantage. A little modification,
Alternatively, even the more complex pipeline stages have the same advantages in terms of flexibility and elasticity as the same 2-wire interface can be used without any modification.

The data duplexing stages shown in FIGS. 13 and 14 are but one example of the myriad of different types of operations that pipeline stages can perform in a given application. However, this “duplexing stage” can form a “bottleneck” and, as a result, show tages such that the pipeline according to this embodiment “packs together”.

A "bottleneck" is that it takes a relatively long time to perform its operation, or
It can be any stage in the pipeline that produces more data than it receives. Furthermore, this example shows that the two-wire accept / valid interface according to this embodiment can be very easily adapted to different applications.

The duplexing stage shown in FIGS. 13 and 14 is
As shown in the example of FIG. 11, two latches L for latching the states of extension bits at the input and output of each stage.
It has EIN and LEOUT. As shown in FIGS. 13 and 14, the input expansion latch LEIN is the input data latch L
Synchronously clocked with DIN and the validation signal INVALID. For ease of reference, the various latches included in the duplexing stage form a pair with each unique output signal.

In the duplication stage, the data latch L
The output from DIN forms intermediate data called MID data. This intermediate data word is loaded into the data output latch LDOUT only when the intermediate accept signal (see "MID ACCEPT" in FIGS. 13 and 14) is set HIGH.

The receiving latch LAI shown in FIGS. 13 and 14.
A circuit part under N, LAOUT is used to generate various internal control signals used to duplicate data.
It is a circuit added to the basic pipeline structure. These signals include a "data token" signal indicating that the circuit is processing a valid data token, and a NOT DUP used to control data duplication.
The LICATE signal is included. If the circuit is processing a data token, the NOT DUPLICATE signal toggles between HIGH and LOW states, which causes each word in the token to be duplicated once (and only once). . NOT DUPLICA if the circuit is processing a valid data token.
The TE signal is held in the HIGH state. Therefore, this means that the token word being processed is not duplicated.

As shown in FIGS. 13 and 14, the upper 6 bits of the 8-bit intermediate data word and the output signal QI1 from the latch LI1 are transmitted from the form input to the logic gate N.
It forms the input to the group of OR1, NOR2, NAND18. The output signal from the gate NAND 18 is labeled S1. Using the well-known Boolean algebra, the output signal QI
It can be shown that the 1 is a "1" and that the signal S1 is a "0" only during the time period when the MID data word has the following structure. “000001xx”, that is, all the upper 5 bits are “0”, the bit MID data [2] is “1”, and the bit in the MID data [1] and the MID data [0] position are any arbitrary values. have. Therefore, the signal S1 has a structure in which the MID data signal is scheduled, and the latch LI
Acts as a "token identification signal" that is low only if the output from 1 is "1". The nature of latch LI1 and its output QI1 is further described below.

Latch LO1 performs the function of latching the last value of the intermediate extension bit (denoted as "MID EXTN" and signal S4) and latches this value on the next rising edge of clock phase PH0.
Load to I1. The output of this latch is bit QI1 and is one of the inputs to the token decoding logic group forming signal S1. As already mentioned,
The signal S1 just drops to "0" if the signal QI1 is "1" (and the MID data signal has the expected structure). Therefore, the signal S1 just drops to "0" at any time if the last extension bit was "0", indicating that the previous token has ended. Therefore, M
The ID data word is the first data word in the new token.

Latches LO2 and L12, along with NAND gates NAND20 and NAND22, form a storage device for the signal data token. Under normal conditions, signal QI1 at the input to NAND 20 and signal S1 at the input to NAND 22 are both logic "1". Again by Boolean algebra techniques, it can be seen that under these conditions these NAND gates behave in the same way as inverters. That is, the signal Q12 from the output of the latch LI2 is inverted in NAND20, and then this signal is NAND2.
It is inverted again by 2 to form the signal S2. In this case, since there are two logical inversions in this path, the signal S
2 has the same value as QI2.

It can be seen how the signal data token at the output of latch LO2 forms the input to LI2. As a result, QH1 and S1 are both HI
As long as it remains GH, the signal data token retains its state (“0” or “1”).
Clock signals PH0 and PH1 are latched (each L
This is also true when clocking 12 and LO2). The value of the data token is only changeable if one or both of the signals QI1 and S1 are “0”.

As already explained earlier, the signal QI1 is "0" if the previous extension bit was "0".
Therefore, whenever the NID data value is the first word of the token, the previous signal is "0" (and the address field for that token is included). Under this condition, the signal S1 is either "0" or "1". As I explained earlier,
If the MID data word has the expected structure, which in this example represents a data token, the signal S1 is "0". If the MID data word has another structure (indicating that the token is another token that is not a data token), S1 is "1".

When QI1 is "0" and S1 is "1", this indicates that there is a token other than the data token. As is well known in the field of digital electronics, the output of NAND 20 is "1".
NAND gate NAND22 inverts this (as described above) and signal S2 is a "0". As a result, this "0" value is loaded into the latch LO2 at the beginning of the next PH1 clock phase, and the data token signal goes to "0", indicating that the circuit is not processing a data token.

If QI1 is "0" and S0 is "0", thereby indicating a data token (despite the other input from NAND20 output to NAND22). The signal S2 is "1". As a result, this "1" value is loaded into latch LO2 at the beginning of the next PH1 clock phase, and the data token signal goes to "1", indicating that the circuit is processing a data token. .

The NOT DUPLICATE signal (output signal QO3) is loaded into latch L13 at the next rising edge of clock PH0. Latch LI
The output signal QI3 from 3 is combined with the output signal Q12 at gate NAND 24 to form signal S3. Using Boolean algebra as before, it can be seen that signal S3 is "0" only if both signals QI2 and QI3 have the value "1". When the signal QI2 becomes “0”, that is, when the data token signal is “0”,
The signal 53 becomes "1". In other words, the signal S3 goes high if there is no valid data token (QI2 = 0) or if the data word is not duplex (QI3 = 0).

Here, it is assumed that the data token signal remains HIGH for one clock signal or more. The NOT DUPLICATE signal (QO3) is "feeded back" to latch LI3, and gate N
Since it is inverted by AND24 (the other input QI2 is held at HIGH), the output signal QO3 is "0".
Toggle between "1" and "1". However, if there is no valid data token, the signal QI2 is "0" and the signal S3 and output QO3 are forced HIGH until the DATE token signal becomes "1" again. The output QO3 (NOT DUPLICATE signal) is fed back in the same way, and a series of logic gates whose output is "1" (together together if and only if the values of both signals QA1 and QO3 are "1") Combined with the output QA1 from the receiving latch LAIN in NAND16 and INV16) forming an AND gate. As shown in FIGS. 13 and 14, an AND gate (gate NAND1 followed by gate INV16
Similarly, the output from 6) is the acceptance signal IN ACCEPT.
To form. This acceptance signal, as already mentioned,
Used in a two-wire interface structure.

The acceptance signal IN ACCEPT is used as the enablement signal for the latches LDIN, LEIN, and LVIN. As a result, NOT DUPLI
If the CATE signal is low, the acceptance signal IN
ACCEPT is also low, and all three latches are disabled and hold the value stored at their outputs. NOT DUPLICATE signal is H
The stage will not accept new data until it becomes IGH. This is in addition to the above requirement to force the output from the accept latch LAIN high.

As long as there is a valid data token (data token signal QO2 is "1"), signal QO3 is:
Toggle between HIGH and LOW states. As a result, the input latches are activated and, at a maximum, data is available for every other complete cycle of both clock phases PH0, PH1. "HIG
The additional condition that the next stage is ready to accept data, as indicated by the "H" OUT ACCEPT signal, must of course still be satisfied. Thus, for at least two full clock cycles, the output latch LDOUT places the same data word on the output bus OUT data. The OUT VALID signal is "1" only if there is a valid data token (QO2 HIGH) and the validation signal QVOUT is HIGH.

The extension bit corresponding to the MID data, signal QEIN, is combined with signal S3 in a series of logic gates (INV10 and NAND10) to form signal S4. During the display period of the data token,
Each data word MID data outputs it to the output latch LD
Iterate by loading OUT twice. During these first periods, S4 is forced to "1" by the action of NAND10. The signal S4 is MOUT data LDOUT in order to form OUT data [7: 0].
At the same time it is loaded into the latch LEOUT to form OUTEXTN.

In this way, the predetermined MID data is initially loaded into LEOUT, and the associated OUTEXT
N is forced high, but second, OUTEXTN
Is the same as the signal QEIN. Now consider the condition during the last word of one token, where QEIN is known to be low. During the first period, MID data is loaded into LDOUT and O
UTEXTN is "1", and during the second period,
OUTEXTN is "0" and indicates the true end of the token.

The output signal QVIN from the validation latch LVIN is combined with the signal QI3 in a similar gate combination (INV12 and NAND12) to form the signal S5. Using the well known Boolean technique, signal S5 goes HIGH either when validation signal QVIN is HIGH or when signal QI3 is LOW (indicating that the data is duplexed). It can be seen that it is. At the same time as the MID data is loaded into LDOUT and the intermediate extension bit (signal S4) is loaded into LEOUT, the signal S
5 is loaded into the validation output latch LVOUT.
Similarly, the signal S5 is the output validation signal OUT VA.
Combined with signal QO2 (data token signal) at logic gate NAND30 and INV30 to form a LID. As already mentioned earlier, OUT VALID is HIGH only if there is a valid token and the validation signal QVOUT is high.
Is.

In the present invention, the MID ACCEPT signal is the signal S used as one of the two enable signals for the latches LOl, LO2, and LO3.
6 is combined with signal S5 on a series of logic gates (NAND26, and INV26) which perform the well-known AND function to form 6. MID ACCEPT signal is H
IGH and the validation signal QVIN is high or the token is dual (QI3
Signal S6 rises to "1" if there is a deviation. If the signal MID ACCEPT is HIGH, then whenever the input of the stage is loaded with valid input data, if the clock signal PH1 is high, or if the latched data is a duplicate, then the latch LO1- LO3 is enabled.

From the above discussion, it can be seen that the stages shown in FIGS. 13 and 14 receive and transfer data between stages under the control of validation and acceptance signals. However, as in the previous embodiment, the exception is that the output signal from the accept latch LAIN on the input side is combined with a toggling duplex signal, so that the data word is accepted by the new word. It was output twice before.

Various logic gates such as NAND16 and INV16 can, of course, be replaced by equivalent logic circuits (one single AND gate in this case). Similarly, inverters INV10 and INV12 are not needed, for example, if the outputs of latches LEIN and LVIN are being inverted. Alternatively, the corresponding inputs to gates NAND10 and NAND12 can be directly coupled to the inverting outputs of these latches. The stages operate in the same way as long as the appropriate logical operations are performed. The data word and extension bits are still duplicated.

Until the first data word of the token places a "1" in the third position of the word and a "0" in the five high order bits, the duplication function performed by the stages shown in the figure is Note that it will not be implemented. (Of course, other logic gates and NOR1 and NOR shown in the figure
2. By selecting a connection other than the NND18 gate, the required pattern can be easily changed and set. In addition, as shown in FIGS. 13 and 14, unless the first data word has the described structure, the OUT VALID signal is forced low throughout the entire token. This has the effect of removing all tokens from the token stream except the one that causes the duplication process. The reason is that the output terminal (OU
T data, OUTEXTN, and OUT VALID)
This is because the device connected to does not recognize these tokens as valid data. As before, both validation latches LVIN, L at that stage are
VOUT is one single conductor NOTRESET
O, and one single reset input R of the downstream latch LVOUT can be reset with a backward propagating reset signal to force the upstream validation latch to be low on the next clock cycle. Is.

In the examples shown in FIGS. 13 and 14, the circuit included in the data token is duplicated when the circuit is in the ACCE state.
Note that the PT and VALID signals can be manipulated, and as a result, serve only as an example of leaving more data from the pipeline stage than arriving at that input. Similarly, merely to illustrate how the circuit may manipulate the VALID signal to remove data from the stream,
In the example shown in FIGS. 13 and 14, all non-data tokens are removed. However, in most typical applications, a pipeline stage simply supplies any tokens it does not recognize without modification. As a result, other stages further down the pipeline can act on the token as needed. 15 and 16 show
13 shows an example of a timing diagram for the data duplexing circuit shown in FIGS. 13 and 14. This timing diagram shows the relationship between the two-phase clock signal, the various internal and external control signals, and the method of clocking the data between the input and output of the stage and duplicating the data. Reference is now made in more detail to FIG. 17, which shows a reconfigurable process stage according to one aspect of the present invention.

The input latch 34 receives an input via the first bus 31. The first output from input latch 34 is provided to token decode subsystem 33 via line 32. The second output from the input latch 34 is provided as a first input via the line 35 to the processing unit 3
6 is supplied. The first output from token decode 33 is provided as a second input to processing unit 36 via line 37. Token decode 3
The second output from 3 is supplied to the action identification unit 39 via line 40. Further, the action identification unit 39 receives input from the registers 43 and 44 via line 46. Registers 43 and 44 are
Overall, it keeps the machine state. This state is determined by the history of previously received tokens. The output from the action identification unit 39 is provided as a third input to the processing unit 36 via line 38. The output from the processing unit 36 is supplied to the output latch 41. The output from the output latch 41 is supplied via the second bus 42.

Referring now to FIG. 18, the start code detector (SCD) 51 receives input via the 2-wire interface 52. This input is either in the form of a data token or is a data bit in the data stream. The first output from the start code detector 51 is supplied via line 53 to a first logical first-in first-out buffer (FIFO) 54. The output from the first FIFO 54 is line 55
Via the input to the Huffman decoder 56 as a first input. The second output from the start code detector 51 is, via line 57, as a first input, D
It is supplied to the RAM interface 58. Furthermore, D
RAM interface 58, via line 60
It receives input from the buffer manager 59. The signal is
It is transmitted and received by a DRAM interface 58 to an external DRAM (not shown) via line 61. The first output from the DRAM interface 58 is provided to the Huffman decoder 56 via line 62 as the first physical input. Huffman decoder 56
Output is routed through line 63 as an input to the index up to data (ITOD) 64. The Huffman decoder 56 and the ITOD 64 work together as one logic unit. The output from the ITOD64 is
It is supplied to arithmetic logic unit (ALU) 66 via line 65. The first output from ALU 66 is line 70
Via read-only memory (ROM) state machine 68. The output from ROM state machine 68 is provided to Huffman decoder 56 via line 69 as a second physical input. Second from ALU66
Is supplied to the token formatter (TF) 71 via the line 70.

First output 71 from TF according to the invention
Are supplied to the second FIFO 73 via line 72. The output from the second FIFO 73 is provided as a first input to the inverse modeler 75 via line 74. The second output from the T / F 71 is provided as a third input via line 76 to the DRAM interface 58.
Is supplied to. The third output from the DRAM interface 58 is provided as a second input to the inverse modeler 75 via line 77. The output from the inverse modeler 75 is
It is provided via line 78 as an input to the dequantizer 79. The output from the inverse quantizer 79 is supplied as an inverse input to the inverse zigzag 81 (IZZ) via the line 80. The output from IZZ 81 is provided as an input to discrete inverse cosine transform 83 (IDCT) via line 82. The output from the IDCT 83 is line 84.
To the time decoder (FIG. 19).

Referring now in more detail to FIG. 19, a temporal decoder according to the present invention is shown in this figure. Fork 91 receives the output from IDCT 83 (FIG. 18) as input via line 92. As a first output from the fork 91, a control token, such as a motion vector, is fed to the address generator 94 via line 93. In addition, the data tokens are provided to the address generator 94 for counting purposes. This data is provided as a second output from fork 91 to FIFO 96 via line 95. The output from FIFO 96 is then provided as a first input to adder 98 via line 97. The output from address generator 94 is provided as a first input to DRAM interface 100 via line 99. Signals are sent and received by the DRAM interface 100 to external DRAM (not shown) via line 91. The first output from the DRAM interface 100 is provided to the predictive filter 103 via line 102. The output from the predictive filter 103 is provided as a second input to the adder 98 via line 104. The first output from adder 98 is provided to output selector 106 via line 105. The second output from adder 98 is line 1
The second input is supplied to the DRAM interface 100 via 07. DRAM interface 1
The second output from 00 serves as the second input on line 108.
Is supplied to the output selector 104 via. The output from the output selector 104 is supplied to the moving picture formatting unit (FIG. 20) via the line 109.

Referring now to FIG. 20, fork 111 receives input from output selector 106 (FIG. 19) via line 112. As a first output from the fork 111, the address generator 114 is supplied with a control token via line 113.
The output from the address generator 114 is, as the first input,
DRAM interface 116 via line 115
Is supplied to. The data as the second output from the fork 111 is sent to the D input via the line 117 as the second input.
It is supplied to the RAM interface 116. The signal is
DRAM interface 11 via line 118
6 sends and receives to and from an external DRAM (not shown). The output from the DRAM interface 116 is output via line 119 to the display pipe 120.
Is supplied to.

It is apparent from the above description that each line may have a plurality of lines as necessary.

Here, referring to FIG. 21, one picture 131 is encoded as one or a plurality of slices 132 in the MPEG standard. Each slice 132 has a plurality of blocks 133 and is coded column by column from left to right in each column. As shown, the span of each slice 132 is block 13
There may be exactly one row 132 in 3 or less than one row B, C in block 133, or multiple rows C in block 133.

With reference to FIG. 22, JPEG and H.264. In the H.261 standard, a common intermediate format (CIF) is used, and a picture 141 is encoded as 6 columns, each column containing two groups of blocks (GOB) 142. GOB 142 is composed of an unlimited number of blocks 143, either 3 or 6 columns. Each GOB 142 is zigzag encoded in the Z-direction shown by arrow 144. As a result, GO
B142 is processed from left to right in each column.

Referring now to FIG. 23, it can be seen that the encoder output takes the form of a data stream 151 for both MPEG and CIF. The decoder receives this data stream 151. The decoder can then reconstruct the image depending on the format used to encode the image. In order for the decoder to be able to recognize the start and end points for each standard, the data stream 151 is divided into 33 blocks 152 of length.

The Venn diagram shown in FIG. 24 represents a region of values selectable by the table from the Huffman decoder 56 (FIG. 18) of the present invention. These values are used by the MPEG decoder and H.264. Two
This is a value that can be duplicated for a H.61 decoder, and a single MPEG and a specific H.264 can be selected by one single table selection.
Both 261 formats are to be decoded.
Similarly, these values are duplicate values for the MPEG and JPEG decoders, indicating that one single table selection will decode both specific MPEG and specific JPEG formats. Furthermore,
H. The 26l value and the JPEG value do not overlap, indicating that there is no single table selection that alone decodes both formats.

Now referring to FIG. 25 in more detail, this figure shows an overview of variable length picture data according to an implementation of the present invention. First picture 1 to be processed
61 has a first picture start token 162, first picture information 163 of unlimited length, and a first picture end token 164. The second picture 165 to be processed is the second picture start token 166, the second picture information 16 of unlimited length.
7 and a second picture end token 168. The picture start tokens 162 and 166 are
4B shows a start for the processor of pictures 161 and 165. Similarly, picture end tokens 164 and 168 signify the end of pictures 161 and 165 to the processor. This allows the processor to
It is possible to process variable length picture images 163 and 167.

In FIG. 26, split 171 receives input via line 172. Split 171
The first output from is supplied to the address generator 174 via line 173. The address generated by the address generator 174 is transferred to the DRA via line 175.
It is supplied to the M interface 176. Signals are transmitted and received by the DRAM interface 176 to external DRAM (not shown) via line 177. The first output from the DRAM interface 176 is provided via line 178 to the prediction filter 17
9. The output from the prediction filter 179 is supplied as a first input to the adder 181 via the line 180. The second output from split 171 is sent via line 182 to a first-in first-out buffer (FIF).
O) provided as an input to 183. FIFO183
Is provided as a second input to adder 181 via line 184. The output from the adder 181 is supplied to the write signal generator 186 via the line 185. The first output from the write signal generator 186 is provided to the DRAM interface 176 via line 187. The second output from the write signal generator 186 is supplied to the read signal generator 1 via the line 188.
It is provided as the first input to 89. The second output from DRAM interface 176 is provided as a second input to read signal generator 189 via line 190. The signal from the read signal generator 189 is supplied to the line 19
1 to video formatting (not shown in FIG. 26).

FIG. 27 shows the prediction filter process.
The forward prediction filter 201 receives the first input as
It is supplied to the adder 203 via 202. The backward prediction filter 204 receives the line 2 as the second input.
It is supplied to the adder 203 via 05. Adder 20
The output from 3 is provided via line 206. As shown in FIG. 28, the slice 211 has one or a plurality of macroblocks 212. Consequently, each macroblock 212 has four luminance luminance blocks 213,
, And two chrominance blocks 214, and information about the original 16 × 16 block of pixels. The size of each of the four luminance blocks 213 and the two chrominance blocks 214 is 8 × 8 pixels. The four luminance blocks 213 are 1
Pixel Pixel has a pixel by pixel mapping of the original 16x16 block of luminance (Y) information. One chrominance block 214 contains an indication of the chrominance level of the blue signal (Cu / b), and another chrominance block 21 contains an indication of the chrominance level of the red signal (Cv / r). Each chrominance level is such that each 8 × 8 chrominance block 214 contains the chrominance level of its color signal for the entire original 16 × 16 block of pixels.
Subsampled.

With reference to FIG. 29, the structure and function of the start code detector should be apparent. The value register 221 receives the image data via the line 222. The width of line 222 is 8 bits, allowing parallel transmission of 8 bits at a time. The output from the value register 222 is sent via the line 223.
It is supplied in series to the decode register 224. The first output from decode register 224 is provided to detector 225 via line 226. The width of line 226 is 24 bits, allowing parallel transmission of 24 bits at a time. The detector 225 detects the presence or absence of an image corresponding to a 23 "zero" value standard independent start code followed by one "1" value. The 8-bit data value image follows the valid start code image. When the presence of the start code image is detected, the detector 225 transmits the start image to the value decoder 228 via line 227.

The second output from the decode register 224 is serially supplied to the value decode shift register 230 via the line 229. The value decode shift register 230 can hold one data value image having a bit length of 15. Partial area 23
The 8-bit data value following the start code image is shifted to the right of the value decode shift register 230, as indicated by 1. As discussed below, this process eliminates duplicate start code images. The first output from the value decode shift register 230 is via line 232
It is supplied to the value decoder 228. The width of line 232 is 15 bits, allowing parallel transmission of 15 bits at a time. The value decoder 228 decodes the value image using a first look-up table (not shown). Value decode shift register 23
The second output from 0 is provided via line 235 to a value decoder 228 which provides a flag to the index / token converter 234. In addition, the value decoder 228 receives the index / line via line 236.
It supplies information to the token converter 234. Information is
It is either a data value image obtained from the first lookup table or a start code index image. The flag indicates the form to which the information is supplied. The width of line 236 is 15 bits,
Allows parallel transmission of 15 bits at a time. In the present invention, 15 bits was chosen as the width in this case, but it should be understood that bits of other lengths may be used. The index / token converter 234 uses a second lookup table (not shown) similar to that described in Table 12-3 of the user's manual.
To convert the information into a token image. The token image generated by the index / tokens converter 234 is output via line 237.
The width of the line 237 is 15 bits, and parallel transmission of 15 bits is possible at one time.

In FIG. 30, a data stream 241 consisting of individual bits 242 is input to a start code detector (not shown in FIG. 30). The first start code image 243 is detected by the start code detector. Next, the start code detector receives the first data value image 244. Prior to processing the first data value image 244, the start code detector has a second start code image 24 that overlaps the first data value image 244 in length 246.
5 may be detected. When this occurs, the start code detector does not process the first data value image 244, and instead receives and processes the second data value image 247.

In FIG. 31, the flag generator 251 is
Receives data as a first input via line 252. The width of line 252 is 15 bits, 15 at a time.
Allows parallel transmission of bits. Similarly, the flag generator 251 receives the flag as a second input on line 253 and the input valid image on the first two-wire interface 254. The first output from the flag generator 251 is provided on line 255 to an input valid register (not shown). The second output from flag generator 521 is provided to decode index 257 via line 256. Decode index 257 produces the following four outputs:
That is, the picture start image is output via the line 258 and the picture number image is output via the line 259.
Output via line 260
, And the replace image is output via line 261. The data from the flag generator 251 is provided on line 262a. The header generator 263 uses a look-up table to generate the replacement image, which is supplied via 262b. The extra word generator 264 uses the MP to generate the insert image.
Using U, the insert image is provided via line 262c. Line 262a has output latch 265
Combine to form 262 lines which are the first input to. The output latch 265 outputs the data via the line 266. The width of line 266 is 15 bits,
Allows parallel transmission of 15 bits at a time.

An input valid register (not shown) provides the image on line 268 as the first input to the first OR gate 267. The insert image is provided as a second input to the first OR gate 267 via line 269. The output from the first OR gate 267 is provided as the first input to the first AND gate 270 via line 271. The logical negation of the remove image is applied to the first AND gate 27 via line 272.
Supplied as a second input to 0 and then line 273
Is provided as a second input to the output latch 265. The output latch 265 provides the output valid image via the second two-wire interface 274. The output accept image is the second 2-wire interface 2
It is received by the output accept latch 275 via 74. The output from the output accept latch 275 is
It is provided to an output accept register (not shown) via line 276.

Output Accept Register (not shown)
Via line 278, as a first input, a second OR
The image is supplied to the gate 277. The logical negation of the output from the input valid register is output via line 279 to the second
It is provided as the second input to the OR gate 277. The removed image is provided as a third input to the second OR gate 277 via line 280. The output from the second OR gate 277 is output via the line 282 to the second A gate.
It is provided as the first input to ND gate 281. The logical negation of the insert image, via line 283,
It is provided as a second input to the second AND gate 281. The output from the second AND gate 281 is the line 28
4 to the input accept latch 285. The output from the input accept latch 285 is provided via the first 2-wire interface 254. Table 600 Format Received Image Token 1 H.V. 261 Sequence start Sequence start MPEG picture start Group start No JPEG picture start Picture data 2 H.264 261 No Picture End MPEG No Padding JPEG No Flash Stop After Picture Image detection by start code detector 51, as shown in Table 600, which shows the relationship between the presence and absence of standard signals in a specific machine independent control token. Generates a series of machine independent control tokens. Each image listed in the "Received Images" column initiates the generation of all machine independent control tokens listed in the group in the "Generate Tokens" column. As shown in the first row of table 600, H.264. 261 "Sequence start" during processing period
Whenever an image is received or a "picture start" image is received during PEC processing, an entire group of four control tokens is generated, each with its corresponding data or data. Followed by value. Further, as shown in the second row of table 600, the second group of four control tokens is generated at the appropriate time regardless of the image received by start code detector 51. Table 601 Display Order I1 B2 B3 P4 B5 B6 P7 B8 B9 I10 Transmission Order I1 P4 B2 B3 P7 B5 B6 I10 B8 B9 Shows one row of table 601 showing the timing relationship between transmitted pictures and displayed pictures. As such, the picture frames are displayed in numerical order. However,
To reduce the number of frames that must be stored in memory, the frames are transmitted in different orders. It is useful to start the analysis with an intra frame (I frame). The I1 frames are transmitted in the order they should be displayed. Next, the next predicted frame (P frame) P4 is transmitted. Then, any bi-directionally interpolated frames (B-frames) to be displayed between the I1 and P4 frames are transmitted. These frames are represented by frames B2 and B3. This allows a transmitted B frame to refer to a previous frame (forward prediction) or a future frame (backward prediction).
After transmitting all B frames to be displayed between the I1 frame and the P4 frame, the next P frame, P7, is transmitted. Then all B frames corresponding to B5 and B6 to be displayed between P4 and P7 frames are transmitted. Then, the next I frame, 110, is transmitted. Finally, P7 and 1
All B frames corresponding to frames B8 and B9 to be displayed between 10 frames are transmitted. Sending frames in this order requires only two frames to be held in memory at any one time, and then sending the next P or I frame to display the intermediate B frame. It does not require the decoder to wait.

Further information regarding the structure and operation of the invention, as well as its features, objects and advantages, can be found in
It will be readily apparent to one of ordinary skill in the art from the additional continuing detailed description of the illustrative embodiments of the present invention. For the sake of clarity and convenience, the detailed contents are summarized in the following sections.

1. Multi-standard configuration 2. 2. Decoding of JPEG still image Video extension 4. RAM memory map 5. Bitstream characteristics 6. Reconfigurable processing stage 7. Multi-standard coding 8. Multi-standard processing circuit-second operation mode 9. Start code detector 10. Token 11. DRAM interface 12. Prediction filter 13. Accessing register 14. Microprocessor interface (MPI) 15. MPI read timing 16. MPI write timing 17. Keyhole address location 18. Picture end 19. Flushing operation 20. Flash function 21. Stop after picture 22. Multi-standard search mode 23. Reverse modeler 24. Inverse quantizer 25. Huffman decoder and parser (syntax analyzer) 26. Various separate cosine transformers 27. Buffer manager 1. Multi-Standard Configurations As shown in the aforementioned US Pat. No. 5,212,742, various compression standards, namely JPEG, MPEG,
And H .; Since H.261 is well known, the detailed specifications of these standards will not be repeated here.

As mentioned previously, the present invention is capable of decompressing differently coded different picture data bitstreams. Different coding standards deal with the data provided to the output of the spatial decoders operating alone or to the serial output of the spatial decoders and temporal decoders operating in combination (as described in more detail below). ), And the reformatting of this output for use with other display systems, including displays in computers or moving image display systems, requires some form of regulated output formatting. The manner in which this formatting is accomplished varies significantly between coding standards and / or the type of display chosen.

In a first embodiment according to the present invention,
As previously described with respect to FIGS. 17-19, blocks of formatted data, namely a first decoder (spatial decoder) or a first decoder (spatial decoder) and a second decoder (temporal decoder). ) And the output from either of the combinations is written and the decoded information is written to the memory in raster order and / or
Alternatively, an address generator is used to write out.
The video formatter described below provides a wide range of output signal combinations.

In the preferred multi-standard video decoder embodiment of the present invention, the spatial and temporal decoders are MPEG coded signals and H.264 / AVC signals. It is necessary to realize both of the H.261 moving picture coding / decoding system. The DRAM interface of both of these devices uses small picture formats and is configurable to reduce the amount of DRAM required when operating at low coded data rates. The reconfiguration of these DRAMs will be further described below in connection with the DRAM interface. Generally, one single 4 Mbyte DRAM is required by each temporal decoder and spatial decoder circuit.

The present current spatial decoder performs all the processing required within one single picture. This reduces the redundancy within a picture.

The temporal decoder reduces the redundancy between the subject picture that is related to the picture that arrives before the arrival of the subject picture and the pictures that arrive after the arrival of the subject picture. . One aspect of the temporal decoder addresses the complex address functions required to read the data associated with all these pictures, using as few circuits as possible, and with high speed and improved accuracy. It is to provide a decoding network.

As previously described with respect to FIG. 18, the data is a start code detector preceding the Huffman decoder and parser, a FIFO, a second FIFO register, an inverse modeler, an inverse quantizer, an inverse zigzag and an inverse DCT.
Arrive via. The two FIFOs do not have to be on-chip. In one embodiment, data does not flow through a FIFO located on the chip. The data is D
Supplied to RAM interface and FIFO
-IN storage register and FIFO-DUT register
In both cases, it is located off-chip. These registers, whose operation is completely independent of the standard, will be described in more detail subsequently.

The majority of subsystems and stages shown in FIG. 18 are actually independent of the particular standard used, and the DRAM interface 58, DRAM
It includes a buffer manager 59 that generates addresses for the interface, a reverse modeler 75, a reverse zigzag 81, and a reverse DCT 83. The standard-independent unit in the Huffman decoder and parser includes the ALU 66 and the token formatting unit 71.

In FIG. 19, a unit independent of the standard includes a DRAM interface 100 and a fork 9
1, a FIFO register 96, an adder 98, and an output selector 106 are included. The standard dependent unit is an H.264 standard. 26
1 and MPEC, different address generators 94, and H.264. The prediction filter 103 is reconfigurable so as to be compatible with both H.261 and MPEG. JPEG data is
It flows through the entire machine, which never changes.

FIG. 20 is a high level block diagram of the moving picture formatting section chip. Most parts of this chip are standard independent. The few items that are affected by the standard are H.264, which is different from MPEG or JPEG. In the case of H.261, data is written in the DRAM, and H.264 is used. At 261 it is not necessary to code every single picture. There is some timing information that provides some information as to when the picture is intended to be displayed, and is also referred to as a time reference that is also addressed by the logic's address generation type in the video formatting section.

The rest of the circuitry embodied in the video formatting section includes all of the color space conversion, upsampling filters, and gamma correction RAM, and is completely independent of the particular compression standard used. There is.

The start code detector of the present invention relies on a compression standard in which the detector must recognize for each of the standards different start code patterns in the bitstream. . For example, H.264. 261 has a 16-bit start code, MPEC has a 24-bit start code, and JPEG uses a marker code that is significantly different from other start codes. Once the start code detector recognizes these different start codes, its operation is essentially independent of the compression standard. For example, apart from the circuitry that recognizes various different categories of markers, many of the operations are very similar between the three different compression standards.

The following unit is a state machine 518 located in the Huffman decoder and parser (see FIG. 1).
8). In this case, the actual circuit configuration is almost the same for each of the three compression standards. In fact, the only element in operation that is affected by the standard is the machine reset address. If the parser is simply reset, it will jump to a different address for each standard. In fact, there are four recognized standards. These standards are based on H.264. 261, JP
For the EG, MPEG, and one of the other standards, and the last standard, the parser enters one code used for testing. The fact is that the circuit is the same in almost every aspect, only the microcode programming for each standard is different. Therefore, one program is H.264. 261 is running and another different program is running,
There is no overlap between these programs. This is JP
It is also true of EG and is the third fully independent program.

The next unit shown is the Huffman decoder 56 which works in conjunction with the data index unit 64. These two units work together to perform Huffman decoding. In this case, the algorithm used for Huffman decoding is the same regardless of the compression standard. The change is which table is used and whether the data coming into the Huffman decoder is inverted. In addition, the Huffman decoder itself includes a state machine that understands some aspects of the coding standard. These different operations are selected in response to instructions coming from the parser state machine. The Parser State Machine works by a different program for each of the three compression standards,
And at different times that meet the operating standards,
Supply the correct instruction to the Huffman Decoder.

The last unit on the chip that depends on the compression standard is the dequantizer 79, where the math performed by the dequantizer is different for each of the different standards. At this time, the coding standard token is decoded, and the dequantizer 79 remembers under which standard it is operating. Then, any subsequent data tokens that occur after the event and may occur before other coding standards are handled by the method indicated by the coding standard stored in the dequantizer. The tables mentioned in the detailed description show the different parameters in the different standards and which circuits correspond to these different parameters or mathematics.

H. Address generation based on H.261 is shown in FIG.
9 and not the same for each subsystem shown in FIG. 2 before or after the Huffman decoder shown in FIG.
The address generation method for generating addresses for one FIFO does not change according to the coding standard. H. 261
In this case, the address generation that occurs on the chip does not change. In essence, the difference between these standards is M
The difference between PEG and JPEG, the organization of macroblocks is arranged in linear rows that extend horizontally across the picture. As shown in FIG. 21, the first macroblock A
Covers one line. Macroblock B covers less than one row. Macroblock C covers multiple lines. M
In PEG, it is divided into slices, and one slice may be one horizontal row A, one slice may be a part of horizontal row B, or one slice may be There can also be C if it extends from one row to the next. Each of these slices consists of a row of macroblocks.

H. In the case of 261, the picture is divided into groups of blocks (GOB), so the organization is somewhat different. A group of blocks corresponds to a macroblock with a height of 3 columns and a width of 11 macroblocks. In the case of CIP pictures, there are 12 block groups of this kind. However, they are not organized like one above the other. Instead, groups of 6 blocks in height are arranged in the front and rear 2 groups. That is, 6 GOBs and 2 GOBs are arranged vertically.

In all other standards, when performing addressing, the macroblocks are addressed in sequence, as already indicated. More specifically, the address progresses along the line, and at the end of the line the next line is started. H. In the case of 261, the order of the blocks is the same as the order described within the group of blocks, but when moving to the next group of blocks it is almost zigzag.

The present invention provides a circuit for handling the above effects. It describes the address generation in the spatial decoder and the video formatting unit according to H.264. It is a method of changing to 261. This is accomplished whenever information is written to DRAM. As it is filled in with knowledge of the address generation sequence described above, the physical location in RAM is the same size MPEG.
Exactly the same as for pictures. Therefore, the physical arrangement of the information in the memory is the same as it was in the MPEG sequence, so that all address generation circuits for reading from the DRAM can, for example, generate H.264. It is not necessary to understand that this is the case for the H.261 standard. Thus, in all cases only the entry of data is affected.

In the temporal decoder, the H.264 circuit appears to be something different from what is actually happening. Two
Abstraction is performed on 61. That is, each group of blocks is conceptually expanded so that instead of a rectangle that is an 11 × 3 macroblock, the macroblock is
It is expanded to a block group having a height of 1 macroblock and a length of 33 blocks (FIG. 23). By doing this, the exact same counting technique used in the temporal decoder to count through groups of blocks is also used for MPFG.

H. of blocks. 261 Group and MPEG
The circuit is designed so that there is a match with the slice. H.264 after the start code detector. When 261 data is processed, each block group is preceded by a slice start code. The next block group is preceded by the next slice start code. When counting is done in the temporal decoder to count through this structure, it looks as if it is a group of one macroblock in height and 33 macroblocks in length. Similarly, the circuit counts every 10 intervals, which is sufficient. Counting the 11th macroblock or the 22nd macroblock resets some counters. This is accomplished by a simple circuit with another counter that counts up each macroblock and resets to zero when it reaches 11. Microcode interrogates it and performs its action. All circuits in the temporal decoder of the present invention are essentially independent of the compression standard regarding the physical placement of macroblocks.

There are a number of different tables for adaptability of multiple standards, and the circuit selects the appropriate table to adapt the appropriate standard at the appropriate time. Each standard has multiple tables, and the circuit selects from a set of tables at any given time. Within any one standard, the circuit selects one table at a time and another table at another time. Within different standards, the circuit selects different sets of tables. As noted earlier in the discussion regarding FIG. 24, there is some crossing between these tables. For example, one of the tables used in MPEG is used in JPEG as well. These tables are not completely separate sets. FIG. 26
One set, an MPEG set, and a JPEG set are shown. H.
Note that there is a very large overlap between the 261 set and the MPEG set. The tables used by these standards are quite common. There is a slight overlap between MPEG and JPEG, and H.264. Since there is no overlap between 261 and JPEG, these standards have completely different sets of tables.

As already pointed out, most system units are compression standard independent. If a unit is standard independent, then this type of unit
There is no need to remember which coding standard is being processed. All standards-dependent units remember the compression standard as the coding standard token flows through these units. First
Information encoded / decoded in the H.264 coding standard is distributed throughout the machine, and if the machine is changing standards, the pre-microprocessor controlled machine is generally H.264. We choose to work according to the H.261 compression standard. MPU in this kind of previous machine
Produces signals at several different locations within a machine whose compression standard is changing. The MPU makes changes at different times and flushes throughout the pipeline.

In accordance with the present invention, the first in pipeline
This change in compression standard can be easily handled by causing a change in the coding standard token in the start code detector arranged as a unit of. The token indicates that a coding standard is being initiated, and the control information is
Flows down the machine and configures all other registers at the appropriate times. The MPU does not have to program each register.

Prediction tokens tell how to use bits in the bitstream to form a prediction. Depending on which compression standard is in operation, the circuit transforms the information found in that standard from a bitstream into a prediction mode token. This processing is performed by the Huffman decoder and the parser state machine, and in this case, it is easy to manipulate the bit based on the specific condition.
The start code detector produces this prediction mode token. Then the token goes down the machine,
Flow to the circuit of the time decoder, which is the device responsible for prediction formation. Since the bits in a standard do not change in the three different standards, the circuitry of the spatial decoder interprets the token without knowing which standard it operates in. The spatial decoder only does what was told in response to the token. Owning these tokens and using them properly simplifies the design of other units in the machine. The program can be somewhat complex, but there are benefits to using hardwired logic in this case, which is difficult to design for multiple standards.

2. Decoding JPEG Still Images As previously pointed out, the present invention relates to signal decompression, and more particularly to decompression of coded video signals independent of the compression standard used.

One aspect of the present invention is a pipelined processing system, wherein a first encoded signal (MPE) is used.
G, or H. H.261 encoded video signal) in combination with a second decoder circuit (temporal decoder) to decode the H.261 encoded video signal).
To provide a first decoder circuit (spatial decoder) for decoding the. No temporal decoder is required for JPEG code decoding.

In this regard, the present invention facilitates decompression of multiple signals encoded in different ways by using one single pipeline decoder and decompression system. The coder / decoder and decompressor pipeline processor is a special configuration that enables the handling of multi-standard coded video signals using technology that is unique and fully compatible with single pipeline decoders and processing systems. Is organized in. The spatial decoder is combined with the temporal decoder, and the video formatter is used to drive the video display.

Another aspect of the invention is the use of a combination of spatial decoder and video formatter for use with still images only. A compression standard independent spatial decoder performs all data processing within the boundaries of one single picture. This kind of decoder
Spatial decompression of internal picture data that passes through the pipeline and is distributed within the address generation circuitry independent of the standards for handling information storage and retrieval for the associated constant velocity call storage, memory. deal with. The still image data is decoded at the output of the spatial decoder, and this output is used as the input of a multi-standard configurable video formatter that provides the output to the display terminal. In a first sequence of similar images,
Until the image reaches the output of the spatial decoder, each decompressed image at the output of the spatial decoder is the same bit length. The second sequence of images can be of completely different image size and therefore different lengths when compared to the first length. Again, until this kind of image reaches the output of the spatial decoder, all such second sequences of similar images have the same bit length.

Another aspect of the present invention is that the incoming standard dependent bitstream is reconfigured and ordered and arranged and arranged to operate as a standard independent reconfigurable pipeline processor. Internally organizing into a series of control tokens and data tokens combined with possible processing stages.

Off-chip DRA for JPEG decoding
One single spatial decoder without M can quickly decode the baseline JPEG image. The spatial decoder supports all features of the baseline JPEG encoding standard. However, the decodable image size may be limited by the size of the equipped output buffer. The spatial decoder circuit includes a constant speed call store circuit having a machine dependent, standard independent address generation circuit for handling storage of information in memory.

As already pointed out, no temporal decoder is required to decode a JPEG encoded moving picture. Therefore, the signal carried by the data token is
If the temporal decoder is configured for JPEG operation, it is fed directly through the temporal decoder without further processing.

Another aspect of the present invention is a Huffman decoder /
Providing in the spatial decoder a pair of memory circuits, such as buffer memory circuits, for operation in combination with the video demultiplexer circuit (HD & VDM). , The first buffer storage is HD & V
Located in front of the DM and the second buffer store is
It is placed after HD & VDA. KD & VDM
Decodes a bitstream consisting of binary 1's and 0's. These numbers are located in the standard encoded bitstream, and the bitstream is
Convert to the number used downstream. The advantage of the two-buffer system is that it can be used to implement multiple decompression systems. For these two buffers,
It will be described in detail later in connection with the verified embodiment of the Huffman decoder.

Yet another aspect of the present multi-standard decompression circuit is that it is associated with a start code detector circuit located upstream of the first forward buffer which operates in combination with the Huffman decoder. One advantage of this combination is that it increases the flexibility that must be added to the bitstream when dealing with the input bitstream, especially padding. The placement of these verified components, start code detectors, memory buffers, and Huffman decoders enhances the handling of specific sequences in the input bitstream.
Further, the off-chip DRAM is used to decode a JPEG-coded moving image in real time.
The size and speed of the buffer used with DRAM is
Depends on the video coded data rate.

The encoding standard identifies all information of the standard dependent type needed for storage in a DRAM associated with a spatial decoder using circuit independent of the standard.

3. Video Decompression In the present invention, when the video is being expanded through the decoding process, a temporal decoder is further required. The temporal decoder combines the decoded data in the spatial decoder with an already decoded image intended for display after the picture image currently being decoded. The temporal decoder receives information for identifying this temporally displaced information in the image coded data stream. The temporal decoder addresses the temporally and spatially arranged information, retrieves the information, is currently being decoded, and is complete and transmitted to the video formatting section to drive the display screen. The information is combined in such a way that it decodes the information located in one image with the resulting image terminating the appropriate image.
Alternatively, the resulting image may be stored for later use for temporal decoding of subsequent images.

In general, a temporal decoder performs inter-picture processing either early and / or late on the picture currently being decoded. The temporal decoder reintroduces the uncoded information into the coded representation of the image. The reason is that the information is redundant and already available at the decoder. More specifically, any given image can contain similar information both temporally surrounding it, both before and after. This similarity can be further magnified if motion compensation is applied. Temporal decoders and decompression circuits likewise reduce the redundancy between related pictures.

In another aspect of the invention, the temporal decoder is used to handle the standard dependent output information from the spatial decoder. This standard dependent information about one single image is the spatially decoded image information in which the decompressed output information processed by the spatial decoder is temporally arranged with respect to the temporal position of the first image. Other spatially-decoded information packets to combine one image by another constant-speed call store having address generation circuitry that is further machine dependent and standard independent.
DR in such a way that it is stored in a RAM register
It is distributed among several areas of the AM.

In a multi-standard circuit capable of decoding an MPEG encoded signal, a larger logical D
The RAM buffer may be required to support a larger image format compatible with MPEG.

Image information is moving through a serial pipeline in 8 pixel by 8 pixel blocks. In one form of the invention, the address decoding circuitry handles (stores and retrieves) these pixel blocks along block boundaries. In addition, the address decoding circuit handles the storage and retrieval of 8x8 pixel blocks across boundaries. This versatility will be explained more fully below.

Similarly, the second temporal decoder may supply the output of the first decoder circuit (spatial decoder) directly to the moving picture formatting section in order to handle it without any signal processing delay.

In addition, the temporal decoder reorders the blocks of image data due to delays caused by the display circuit. The address decoding circuit enables this reordering, as will be explained later.

As already mentioned, one important feature of the temporal decoder is that it sums image information that arrives earlier or later than the image being processed resulting from the selection of images.

When describing a picture in this context, the picture means any of the following contents.

1. It is a coded data representation of a picture.

2. The result, ie the final decoded picture resulting from the addition of the process steps performed by the decoder.

3. It is a picture that has already been decoded and has been read from the DRAM.

4. It is the result of spatial decoding. That is, the range of data between the picture start token and the next picture end token.

After the picture data information has been processed by the temporal decoder, the information is either displayed in the picture storage location or written back. This information is then retained as a further criterion used in processing another different coded data picture. For rearrangement of MPEG encoded pictures for visual display, the required scrambled picture is
The capabilities achieved by varying the rearrangement function of the temporal decoder are included.

4. RAM Memory Map Spatial decoder, temporal decoder, and video formatting unit all use external DRAM. The same DRAM is preferably used for all three devices. Even if all three of these devices use DRAM, and all three devices also use the DRAM interface with the address generator,
What each implements in DRAM is not the same. That is, each chip, eg, the spatial decoder and the temporal decoder, has a different DRAM interface and address generation circuit even if the same physical external DRAM is used.

In short, the spatial decoder is a common DRAM.
Implements two FIFOs inside. Referring again to FIG. 18, one FIFO 54 is the Huffman decoder 5
6 and the parser, and the other after the Huffman decoder and parser. FIFO
Is implemented in a relatively straightforward way. A special part of the DRAM is that for each FIFO there is a FIFO
It is deducted as a physical memory for realizing.

Spatial decoder DRAM interface 5
The address generator associated with 8 uses two pointers to manage tracks of FIFO addresses. 1
One pointer points to the first word stored in the FIFO and the other pointer points to the last word stored in the FIFO. Therefore, it enables a read / write operation to a predetermined word. The address generator "wraps around" to the start of physical memory if the end of physical memory is reached during the course of a read or write operation.

In short, no matter which coding standard (MPEG or H.261) is specified, the temporal decoder according to the invention must be able to store two complete pictures or frames. For simplicity of explanation, D tries to store two frames in it.
The physical memory of the RAM shall be divided into two halves, each half dedicated (with appropriate pointers) to a particular one of the two pictures.

MPEG uses three different types of pictures: intra (I), predictive (P), and bidirectional interpolation (B). As already mentioned, B pictures are based on predictions from two pictures. One picture comes from the future, and the other one comes from the past. I-pictures do not require further decoding by the temporal decoder, but must be stored in one of two picture buffers for later use in decoding P and B pictures. I have to. Decoding a P picture requires forming a prediction from already decoded P or I pictures. The decoded P picture is 1 for use in decoding P and B pictures.
Stored in one picture. B-pictures can require prediction from both picture buffers. However, the B picture is stored in the external DRAM.

When I and P pictures are decoded,
Note that there is no output from the time decoder. Instead, the I and P pictures are 1 of the picture buffer.
One and the next I or P picture is read for decoding only if it arrives. In other words, the temporal decoder relies on its next P or I picture to flush the previous picture from the two picture buffers, as described further below in this section on flushing. In essence, the spatial decoder can provide a false I or P at the end of the video sequence to flush a P or I picture. Consequently, this fake picture is flushed when the next video sequence starts. Peak memory bandwidth loading occurs during the decoding of B-pictures. The worst case is when all predictions are made with half-pixel accuracy and a B-frame is formed from this kind of prediction supplied by both picture buffers.

As previously described, the temporal decoder is
It can be configured to provide reordering of MPEG pictures. This picture reordering causes P and I until the time when the decoding of the next P or I picture in the data stream by the temporal decoder starts.
The output of the picture is delayed.

When P or I pictures are reordered, certain tokens are temporarily stored in chips as the pictures are written to the picture buffer. These stored tokens are retrieved when the picture is read for display. At the output of the temporal decoder, the data tokens of the newly decoded P or I picture are replaced with the older P or I picture.

On the other hand, H.264. 261 produces the prediction only from the picture just decoded. As each picture is decoded, it is filled in one of the two picture buffers and is available for the next picture decoding. The only DRAM memory operation required is 8x
To write 8 blocks, and to form a prediction with integer precision motion vectors.

In summary, the moving picture formatting section
Store three frames or pictures. 3 to accommodate features such as picture repetition or skipping
One picture needs to be stored.

5. Bitstream Characteristics With particular reference to the spatial decoder according to the invention,
It is useful to review the bitstream characteristics of the encoded data stream. The reason is that these properties have to be handled by the circuitry of the spatial and temporal decoders. For example, under one or more compression standards, the compression ratio of that standard is achieved by varying the number of bits used to encode multiple pictures of a picture. The number of bits can vary over a wide range. In particular, this means that the length of the bitstream used to encode the reference picture of one picture can be identified as one unit length, and the other picture can have many unit lengths. However, the third picture can be a part of the unit.

Current standards (MPEG1.2, JPEG,
H. 261) does not define how to end the picture image, and it is tolerated that the current picture ends when the next picture starts. Furthermore, the standards (especially H.261) allow incomplete pictures to be generated by the encoder.

In accordance with the present invention, there is provided a method of indicating the end of a picture by using the one picture end of a token. The static coded data leaving the Wurtt code detector consists of a picture that starts with a picture start token and ends with a picture end token, but with significantly varying length. There may be other information transmitted here (between the first and second pictures), but it is known that the first picture has ended.

The data stream at the output of the spatial decoder has the same length (number of bits) for a given sequence and still consists of picture start and picture end pictures. The length of time between the picture start and picture end can vary.

The video formatter receives these pictures, which are non-uniform in time, and displays them on the screen at a fixed picture rate determined by the type of display being driven. For example, different display rates are used around the world, such as the PAL-NTSC television standard. This is accomplished in a unique way by selectively dulling or repeating the Kuqa. Ordinary "frame rate converter",
For example, 2-3 pulldown operates at a fixed input picture rate, but the video formatter can handle a variable input picture rate.

6. Reconfigurable Processing Stage Referring again to FIG. 17, the reconfigurable processing stage (RPS) is a token decode circuit 32 used to receive incoming tokens from a two wire interface 33 and an input latch 34. Have. The output of the token decoding circuit 33 is the 2-wire interface 36.
And is supplied to the processing circuit 35 via the action identification circuit 39. The processing circuit 35 is suitable for data processing under the control of the action identification circuit. After processing is complete, processing circuit 36 connects the completed signal to the output of 2-wire interface bus 40 via output latch 41.

The action identification decoding circuit 39 is supplied from the token decoding circuit 33 via the 2-wire interface bus 40 and / or from the memory circuits 43 and 44 via the 2-wire interface bus 46, respectively. With input. The token from the token decoding circuit 33 is simultaneously supplied to the action identification circuit 39 and the processing circuit 36. The action identification function as well as the RPS will be described in more detail in the next part of this specification using tables and figures.

The functional block diagram in FIG. 17 illustrates stages that are not standard independent circuits described in FIGS. 18, 19 and 20. The data flow passes through the token decoder 32 and the processing stage 35.
And to the two-wire interface circuit 42 via the output latch 41. If the control token is recognized by the RPS, it is decrypted in the token decoding circuit 33 and the appropriate action is taken. If it is not recognized, 2 is output via the output circuit 41.
It is supplied unchanged to the line interface circuit 42. The present invention functions as a pipeline processor with a 2-wire wire interface circuit for controlling the movement of control tokens through the pipeline. This feature of the invention has been disclosed in more detail in previously filed European patent application No. 92306038.8.

In the present invention, the token decoding circuit 3
3 is used to identify whether the token currently incoming via the 2-wire interface 42 is a data token or a control token. If it is recognized that the token is being examined by the token decoder circuit 33, it is output to the action identification circuit 39 by an appropriate index signal or flag signal indicating the action to be performed. At the same time, the token decode circuit 33 supplies an appropriate flag or index signal to the processing circuit to warn the processing circuit of the presence of the token being handled by the action identification circuit 39. The control token may be processed in the same manner.

The various types of tokens that can be used in the present invention are described in further detail below. For this part of the specification, it suffices to note that the address carried by the control token is decoded in the decoder 33 and used to access the register contained in the action identification circuit 39. . If the token under investigation is a recognized control token, the action identification circuit 39 uses its reconfiguration state circuit to distribute control signals throughout the state machine. As already mentioned, this activates the state machine of the action identification decoder, which reconstructs the decoder itself. For example, the decoder may change the coding standard. In this way, the action identification circuit 39 decodes the required action for handling the specific standard being supplied via the state machine shown in FIG.

Similarly, the processing circuit 36, which is controlled by the action identification circuit 39, is ready to process the information contained in the data field of the data token when appropriate to trigger this event. Often, the control token arrives first, reconfigures the action identification circuit 39, and is immediately followed by a data token that is processed by the processing circuit 36. The control token exits the output latch circuit 41 via the output 2-wire interface 42 which immediately precedes the processed data token in the processing unit 36.

In the present invention, the action identification circuit 39
Is a state machine that holds the history state. Registers 43 and 44 hold the information decrypted from token decoder 33 and stored in these registers. This type of register can be either on-chip or off-chip, as desired. These plurality of state registers contain action information currently being identified in the action identification circuit 39 connected to the action identification. This action information can affect the action that is stored and selected from the previously decrypted token. Connection 40 connects linearly from token decode 33 to action identification block 39. this is,
It is intended to show that the action may also be affected by the token currently being processed by the token decode circuit 33.

7 illustrates token decryption and data processing according to the present invention. The data processing is performed by the action identification circuit 39.
Is made up of: The action is affected by the number of conditions and by the information obtained entirely from the already decrypted token, or more specifically, stored from the already decrypted token in registers 43 and 44. Information, the current token being processed, and the state and history information that the action identification unit 39 has acquired on its own. It indicates the distinction between control tokens and data tokens.

In every RPS, some tokens are seen by the RPS unit as control tokens. In this case, these tokens should affect the operation of the RPS at some later time. Another set of tokens is seen by RPS as data tokens. This kind of data token contains information processed by the RPS in such a way as to be determined by the design of the particular circuit, the already decoded token and the state of the action identification unit 39.

A particular RPS identifies, as data, one particular set of tokens and another set of tokens for a particular RPS control, which is a view of that particular RPS. Another PPS can have another different view on the same token. On the one hand, some tokens can be seen by one RPS unit as data tokens, and on the other hand, determined by another RPS unit as actually control tokens. For example, the quantization table information is data, at least as far as the Huffman decoder and the state machine are concerned. The reason is that the information arrives as coded data at its input and is formatted into a series of 8-bit words,
Then, it is formed into a token called a quantization table token (QUANT table) that moves downstream in the processing pipeline. As far as the machine is concerned, all information would have been data, which was handling data, converting one kind of data into another kind of data, which obviously Is the processing function performed by that part of the. However,
When the information reaches the inverse quantizer, the inverse quantizer stores the information in the token, which is a plurality of registers. In fact, since there are 64 8-bit numbers, and a large number of registers, there can be a large number of registers as a whole. This information is considered control information,
Therefore, since the information affects the number by which each data word is multiplied, the control information affects the processing performed in the next data token. There is an example where one stage considers the token to be data and the other stage considers the token to be control.

Token data according to the present invention is most often considered to be data that passes through a machine. One of the important aspects is that in general each stage of a circuit with a token decoder is looking for a particular set of tokens, and tokens that the stage does not recognize pass through the stage unchanged, and To move the pipeline downstream. As a result, downstream stages following the current stage have the advantage of seeing these tokens and are able to respond to them. This is an important feature. That is, communication between blocks is possible by using the token mechanism for blocks that are not adjacent.

Another important feature of the present invention is that each stage of the circuit must have processing capability within a range capable of performing the operations required for each standard, and must be performed at a predetermined time. It is possible to execute control as a token for operations that must be performed. There is one different processing element between different stages to provide this capability. In the Parser state machine ROM, there are three completely different isolated programs. Each program corresponds to each standard handled. Which program is executed depends on the coding standard token.
In other words, each of these three programs has the capability within it to handle both decoding and coding standard tokens. If each of these programs understands which encoding standard is to be interpreted next, these programs literally jump to the start address for that program in the microcode ROM. This is how the stage handles multiple standards.

Two things are affected by different standards. First, it affects which pattern of bits in the bitstream is recognized as the start code, or marker code, to reconfigure the shift register for detecting the length of the start marker code. Second, one piece of information is printed in the microcode that indicates what the start or marker code means. Recall that the encoding of bits differs between the three standards. Therefore, the microcode refers in the table specific to that compressor standard to the one that is independent of the standard, ie the type of token that represents the incoming code. In most cases, each of the various standards
This token is generally independent of the standard as it provides the specific code that it generates.

The inverse quantizer 79 has mathematical capability. The quantizer has the ability to perform multiplications and additions, and to implement all three compression standards configured by parameters. For example, a flag bit in ROM in the controller tells the dequantizer whether to add the constant K. Another flag tells the dequantizer whether to add another constant. The inverse quantizer stores the coding standard token in a register if the token flows by the quantizer. On subsequent passes of the data token, the dequantizer remembers what the standard is and looks for the parameters that need to be supplied to the processing element to perform the appropriate operation. For example, the inverse quantizer looks for K set to 0 or 1 for a particular compression standard and feeds the result to its processing circuitry. In a similar sense, the Huffman decoder 56 has multiple tables in it, some of which are JP
For EG, some of them are H.264. 261 and some of them are for MPEG. In fact, the majority of these tables serve one or more of these compression standards. Which table is used depends on the syntax of the standard. The Huffman Decoder works by receiving a command from the state machine that tells it which table to use. Therefore, the Huffman decoder does not itself directly have one state input to it, stored, and asserting which encoding is being performed. Instead, it is a combination of a parser state machine and a Huffman decoder that together contain the information in them.

For the spatial decoder of the present invention, address generation has been modified and is similar to the case shown in FIG. 17, where multiple pieces of information are decoded from tokens, such as encoding standards. Similarly, coding standards and additional information are recorded in registers and affect the progression of the address generator state machine if the information passes through macroblocks in the system and counts one by one. . The last stage has two modes, H.H. H.261, or MPEG, and is an easily identified predictive filter 407-9 (FIG. 26).

[0309] 7. Multi-Standard Encoding More specifically, the control token according to the present invention has a plurality of words in the token. In this case, one bit, known as the extension bit, is set to specify the use of additional words in the token to carry additional information. Some of these additional control bits include an index that indicates the information used in the corresponding state machine to create a set of standard-independent index signals. The remainder of the token is used to identify and identify internal processing control functions that are standard for all data streams passing through the pipeline processor.

In one form of the invention, the token extension is used to carry the current coding standard that is decoded by the relative token decoding circuitry distributed throughout the machine, and, and Used to reconfigure stage-wide action identification circuitry 39 throughout the machine whenever it is appropriate to operate under the new encoding standard. In addition, the token encoding circuit can indicate whether the control token is relevant to one of the selected standards the circuit is designed to handle.

More specifically, the MPEG start code,
The JPEG marker is followed by an 8-bit value. H. The 261 start code is followed by a 4-bit value. In this context, the start code detector 5
A 1 indicates that the next 8 bits contain the value associated with the start code by detecting either the MPEG start code or the JPEG marker. The detectors are then independently either MPEG start codes or JPEG markers, and H.264.
A signal indicating that it is not a 261 start code can be created. In this first case, the 8-bit value is input to the decode circuit, and part of the decode circuit creates a signal indicating the index and flag used in the current circuit to handle the token passing through the circuit. . It is also used to insert the part of the control token that is subsequently referenced to determine which standard is being addressed. In this sense, the control token includes the MPEG standard, as well as the part that indicates that it is related to the part that indicates the type of operation that operates on the accompanying data. As already discussed,
This information is used in the system to reconfigure the processing stages used to perform the functions required by the various standards created for that purpose.

For example, H.264. For the 261 start code, this code immediately follows the start code 4
Associated with bit value. The start code detector is
Supply this value to the token generator state machine. This value is fed to an 8-bit decoder that produces a 3-bit start number. The number of starts is used to identify the picture start of the number of pictures, as indicated by the value. Furthermore, the system is based on the previously mentioned 2
It includes multiple parallel processing pipelines operating under the principle of line interface. Each stage is generally shown in FIG.
Including machines that take the form shown in. The token decode stage 33 is used to guide the action identifying circuit 39 or the token which is causing the processing unit 36 to appropriately enter the state machine. The processing unit has been reconfigured into the form necessary to handle the current encoding standard by the next previous control token, which is currently being entered in the processing stage and carried by the next data token. Be done. Furthermore, in accordance with this aspect of the invention, successive state machines in the processing pipeline have one encoding standard:
H. 261, while the previous stage can operate under a proprietary standard such as MPEG. The same 2-wire interface is used to carry both control and data tokens.

Furthermore, the system of the present invention uses control tokens required to decode multiple coding standards with a fixed number of reconfigurable processing stages. More specifically, the picture end control token is used because it is important to indicate when the picture actually ends. Therefore, when designing a multi-standard machine, it is necessary to create an additional control token in the multi-standard pipeline that indicates which standard decoding technique to use. This type of control token is a picture end token. This picture end token is used to indicate that the current picture is finished, forcing the buffer to flush and pushing the current picture through the decoder to the display.

8. Multi-Standard Processing Circuits-Second Mode of Operation The compression standard dependent circuits already described in the form of the start code detector are suitably interconnected via independent buses to circuits independent of the compression standard. The standard dependent circuit is connected to the dependent independent combinational circuit via the same bus and an additional bus. The standard independent circuit provides an additional input to the standard dependent independent circuit, while at the same time the standard dependent independent circuit provides information back to the standard independent circuit. Information from the standards independent circuit is provided to the output via another suitable bus. Table 600 shows that the multiplex standard applied to the input to the standard-dependent start code detector 51 includes a particular bitstream having a standard-dependent meaning within each encoded bitstream.

9. Start Code Detector As mentioned above, the start code detector according to the present invention includes MPEG, JPEG, and H.264. A series of ownership tokens can be generated from the stream that handle the 261 bit stream and are meaningful to the rest of the decoder. As an example where multi-standard decoding is achieved, MPEG (1 and 2) picture start code, H.264. 261 picture start code and J
The PEG start of scan (SOS) marker is treated as equivalent by the start code detector, and
All of these generate an internal picture start token. In a similar manner, the MPEG sequence start code and the JPSG 501 (start of image) marker both generate a machine sequence start token. However, H. The H.261 standard does not have an equivalent start code. Therefore, the start code detector is the first H.264. A sequence start token is generated in response to the 261 picture start code.

None of the above images are used directly except for SCD. Rather, for example, a machine picture start token is believed to be equivalent to the picture start image contained in the bitstream. In addition, keep in mind that the machine picture start itself is the picture
It is not a direct image of the start. Rather, it is a control token that is used in combination with other control tokens to provide a standard-independent decoding function that emulates the operation of the image in each compression coding standard. The combination of control tokens combined with the reconfiguration of the circuit based on the information carried by the control tokens, and also the combination with the index and / or flag generated by the token decoding circuit portion of each state machine is unique. is there. A typical reconfigurable state machine will be described subsequently.

Referring again to the table 600, the name of the standard image group is shown in the left column.
In the right column, does it exist in the standard image?
Alternatively, a machine dependent control token used for emulation of unused standard encoded signals is shown.

In table 600, it can be seen that, as already mentioned, when decoding any one of the standard signals listed in table 600, the machine sequence start signal is generated by the start code detector. The start code detector produces sequence start, group start, sequence end, slice start, user-data, extra-data, and picture start tokens to feed the 2-wire interface used throughout the system. . Each of these stages operating with these control tokens is configured by the contents of the token, or by an index created by the contents of the token, and
The picture data token is ready to handle the data expected to be received if it arrives at the station.

As described above, for example, H.264. One of the compression standards, such as 261, does not have a sequence image start in its data stream, nor a picture end image in its data stream. The start code detector is used to
Indicates the endpoint and PICTUtZEEN
Make a D token. In this regard, the system of the present invention contemplates carrying a fully packed data word to contain one bit of information in each register location selected for use in implementing the present invention. For this purpose, 15 bits were chosen as the number of bits supplied between the two start codes. Of course, one of ordinary skill in the art will appreciate that any choice greater or less than 15 bits can be made. In other words, it is required for proper operation that all data words supplied from the start code detector to the DRAM interface are 15 bits. Therefore, the start code detector creates an extra bit called padding that is inserted in the last word of the data token. For illustration purposes, 15 data bits were chosen.

To perform a padding operation according to the present invention, a number of binary 1's are followed by 2's.
The base 0 is automatically inserted to complete the 15-bit data word. This data is then fed through a coded data buffer and to a Huffman decoder which removes padding. Therefore,
Any number of bits can provide a fixed size and width buffer.

In one embodiment, slice start control tokens are used to identify slices of a picture. The slice start control token is used to divide the picture into smaller areas. The size of this region is selected by the encoder, and the start code detector then slice-starts to divide the received picture into smaller regions for machine dependent state stages located downstream from the start code detector. Identify this unique pattern of code. The size of the region is identified by the start code detector and selected by the recombining circuit and the encoder used by the control token to decompress the encoded picture. The slice start code is mainly used for error recovery.

The start code provides the only way to start the decoder, which will be described in more detail subsequently. If the start code detector is arranged in front of the coded data buffer, the number of start code detectors can be increased after the coded data buffer and before the Huffman decoder and the moving picture demultiplexer. The advantages of If the start code detector is installed in front of the first buffer, 1)
Tokens can be assembled, 2) standard control signals such as start codes can be decoded, and 3)
It is possible to pad the bitstream before the data goes into the buffer, and 4) make an appropriate sequence of control tokens to empty the buffer and put the available data from the buffer into the Huffman decoder. Is possible.

Most of the control tokens output by the start code detector directly reflect the syntax elements of various pictures and video coding standards. The start code detector translates syntax elements into control tokens. In addition to these natural tokens, some unique,
And / or machine dependent tokens are generated. A unique token is specifically designed for use by the system of the present invention and is unique both internally and externally,
And the tokens that are used to help with the multi-standardity of the present invention are included. Examples of unique tokens of this kind include picture ends and coding standards.

In addition, tokens are introduced to remove some of the syntactic differences between coding standards and to work in concert with error conditions. Automatic token generation occurs after the sequential analysis of the standard dependent data. Therefore, the spatial decoder equally responds to the tokens provided directly at the input of the spatial decoder, ie the SCD, as well as the tokens generated following the detection of the start code in the encoded data. A series of extra tokens is inserted into the 2-wire interface to control the multi-standardity of the present invention.

MPEG and H.264. The 26l-encoded video stream contains standard-dependent non-data identifiable bit patterns, one of these patterns is hereinafter referred to as the start image and / or standard-dependent code. J
In PEG, similar functionality is provided by the marker code. These start / marker codes are
Identify important parts of the syntax of the encoded data stream. The analysis of the start / marker code performed by the start code detector is the first stage of coded data analysis.

The start / marker code pattern is designed so that the pattern can be identified without decoding the entire bitstream. Therefore, those patterns can be used in accordance with the present invention to aid error recovery and decoder start. The start code detector provides functions to detect errors in the coded data structure and to aid in starting the decoder. The error detection capability of the start code detector will be described in detail later as the decoder starts.

The above description was primarily concerned with the properties of the machine dependent bitstreams and their relationship to the addressing properties of the present invention. The following is a description of the characteristics of the standard dependent coded data bitstream for a start code detector.

Each standard compression encoding system uses a unique start code configuration, or image, chosen to identify a particular compression specification. Each start code has a start code value. The start code value is used within the language of the standard to identify the type of operation associated with the start code. In the multi-standard decoder of the present invention, as described above, the compatibility is
Based on control token and data token configurations. The index signal including the flag signal is circuit-generated in each state machine, and will be described below as appropriate.

The start and / or marker code included in the standard, and other standard words as opposed to data words, are controls used in the machine.
And / or with respect to the content of the data token, a code,
And / or is often identified as an image to avoid confusion with the use of machine dependent code. Similarly, the term start code refers to JPEG marker code as well as MPEG and H.264. Often used as a generic term for 261. Marker code and start serve the same purpose. Furthermore, "flash" (f
The term lush) has both meanings when it refers to the usage flush token and as a verb, for example when flushing the start code detector shift register (including the "flushed" signal). Used. Flash tokens are always capitalized to avoid confusion. All other uses of the term (verb or noun) are in lower case.

Standards Dependent Coded Input Pictures The input stream contains data of varying length and a start image. The start image carries a value that tells which operation should be performed on the data immediately following it according to the standard. However, in the multi-standard pipeline processing system of the present invention where compatibility with multiple standards is required, the system is optimized to handle all functions in all standards. Therefore, in many conditions, the unique start control token is not only compatible with respect to the values contained in the values of the encoded signal standard image, but also for each standard as is well known in the art. The various stages must be controllable in order to emulate the operation of the standard according to the indication by the parameters specified. All such standards are incorporated herein by reference.

It is important to understand the relationships between tokens that emulate non-data information contained in a standard bitstream, either alone or in combination with other control tokens. A separate set of index signals, including flag signals, is generated by each state machine to handle some processing within that state machine. The values carried in the standard can be used to access the dependent control signals of the machine to emulate the handling of standard data and non-data signals. For example, the slice start token is a two word token and is therefore input to the two wire interface as previously described.

The data input to the system of the present invention may be any suitable data source, such as a disk, tape, etc., that provides 8 bits of data to the first functional stage within the start code detector 51 (FIG. 18). It can be a data source from a source. The start code detector has three shift registers, namely the first
Shift registers are 8 bits wide, followed by 24
It is one bit wide, and then 15 bits wide. Each register is part of a two wire interface. The data from the data source is loaded into the first register as one single 8-bit byte during one timing cycle. After that, the contents of the first shift register are decoded (second) one bit at a time.
It is shifted to the shift register. After 24 cycles, 2
The 4-bit register is full. Every 8 cycles, an 8 bit byte is loaded into the first shift register. Each byte is loaded into the value shift register 221 (FIG. 29), and an additional 8 cycles empty the value shift register and shift register 23.
Used to load 1. Shift register 23
Eight cycles are used to empty one, so
After these 3 operations, or 24 cycles, there are still 3 bytes in the 24-bit register. The value decode shift register 230 is still empty.

Assuming there is now one picture start word in the 24-bit shift register, the detect cycle recognizes the picture start code pattern and provides the start signal as its output.
Once the detector detects a start, the following byte is the value associated with the start code, and this is currently sitting in the value register 221.

Since the contents of the detect shift register have been identified as the start code, these 3 bytes are used to remove this content from the 2-wire interface to ensure that no further processing occurs. Must. The decode register is emptied and the value decode shift register 230 waits for the value to be shifted to this kind of register.

At this stage, the contents of the low-order bit position of the value decode shift register contains the value associated with the picture start. The spatial decoder equivalent to the standard picture start signal is called the SD picture start signal. Here, the SD picture start signal itself is about to be included in the token header,
This value is about to be included in the extension word for the token header. 10. Token In the implementation of the present invention, a token is a universal adaptation unit in the form of an interactive interface messenger package for control and / or data functions, and for performing various operations in response to one recognized token. Reconfigurable processing stage (RP
S) adapted for use with. Tokens can be either position dependent or position independent depending on the processing stage to achieve various functions. In addition, tokens are changed by the processing stage,
It can then be altered, as it is fed down the pipeline to achieve yet another function. Tokens interact with all or less than all of the stages and, in this respect, may be considered as interacting with adjacent and / or non-adjacent stages. Tokens may be position dependent for some functions and position independent for other functions, and a particular interaction with a stage may be prior to that stage. It can be adjusted by the processing history in.

The picture end token is a method of transmitting a picture end in a multi-standard decoder.

Multi-standard tokens are MP-capable using a mixture of standard-dependent and standard-independent hardware and control tokens.
EG, JPEG, and H.264. 261 data streams to 1
This is a method of mapping to one single decoder.

Search Mode Tokens are MPEG, which enable random access and enhanced error recovery.
JPEG, and H.264. 261 is a technique for searching a data stream.

The stop-after-picture token is
A method to achieve the clear end of decoding, sending the end of the picture and clearing the decoder pipeline, i.e. the change of channel.

Further, padding tokens is a method of supplying an arbitrary number of bits through a fixed size fixed width buffer.

The present invention is directed to a pipeline processing system having a variable configuration using a token and a two wire system. The use of control tokens and data tokens in combination with a two-wire system facilitates the achievement of a multi-standard system that allows to have extended operating capabilities compared to systems that do not use control tokens.

Control tokens are generated by circuitry in the decoder processor and emulate the operation of a number of different types of standard dependent signals that are fed into the serial pipeline processor for handling.
A technique is used to consider all parameters of the multi-standard selected for processing by the serial processor and consideration of the following items. That is, 1) their similarities, 2) their differences, 3) their needs, and requirements, and 4) of the correct token functions to effectively process all standard signals fed into the serial processor. Selection. The function of the token is to emulate the standard. The control token function is used, in part, as an emulation / translation between standard-dependent signals and as an element to convey control information through the pipeline processor.

In the prior art system, a dedicated machine is designed according to well known techniques to identify the standard and to create a dedicated circuit by using a microprocessor interface. Signals from the microprocessor are used to control the flow of data through dedicated downstream components. The selection, timing and organization of this decompression function is under the control of fixed logic circuits assisted by signals coming from the microprocessor.

In contrast, the system of the present invention
Configure the downstream functional stage under the control of the control token. Options are provided to obtain the necessary and / or alternative controls from the MPU.

Tokens provide and create an excellent format for communicating information through a decompression pipeline processor. In the design chosen and used in the preferred embodiment described below, the width of each word of the token is a minimum of 8 bits,
And a single token can extend beyond one or more words. The width of the token is variable, and the number of bits can be arbitrarily selected. The extension bit indicates whether the token extends beyond the current word, i.e. whether the bits in all words of the token except the last word of the token are set to binary one. If the first word of the token has zero extension bits, this indicates that the token is only one word long.

Each token is identified by an address field starting at bit 7 of the first word of the token. The address field is variable in length and potentially capable of extending beyond multiple words. In the preferred embodiment, the address length is up to 8 bits. However, this does not imply a limiting condition in the present invention, but rather the size of the processing steps chosen to be achieved by the use of these tokens. 1 with one extension bit in words 1 and 2
Note that, and the meaning of the trailing of the additional word is indicated by the extended bit identification label. The extension bit in word 3 is zero, thus indicating the end of the token.

In addition, tokens are of variable bit length.
For example, a total of 10 bits is used by adding extension bits to a 9-bit token word. In the design of the present invention, the width of the output bus is variable. The width of the output from the spatial decoder is 9 bits, or 10 bits if an extension bit is included. In the preferred embodiment,
The only token that makes use of these extra bits is the data token, and all other tokens ignore this extra bit. It should be understood that this is not a limiting condition and is merely an example.

By using data tokens and control token configurations, it is possible to vary the length of data represented by the number of bits in a word being carried by these data tokens. For example, a data bit in a word of a data token may be combined with a data bit in another word of the same data token to be used in accessing the constant velocity call store used throughout this serial decompression processor to create 11 bits. , Or 10
It has been considered that an address of bits can be formed. This facilitates widespread versatility and adds variability.

As already mentioned, the data token is 1
Carries data from one processing stage to the next. Therefore, the characteristics of this token change as it passes through the decoder. For example, at the input to the spatial decoder, the data tokens carry bit-serial encoded video data packed into 8-bit words. Here, there is no limit on the length of each token. However, to illustrate the versatility of this aspect of the invention (at the output of the spatial decoder circuit), assume that each data token has exactly 64 words and each word is 9 bits wide. More specifically, standard encoded signals allow for encoding different intensities and details of the image for messages of different lengths. The first image of the group usually has the longest number of data bits as it has to provide the most processing unit information. As a result, the first image can start decompressing as much information as possible. Subsequent words are generally short in length because they contain a different signal when compared to the first word with respect to the second position in the scan information field.
As required by standard coding systems, the words are interspersed with each other so that a variable amount of data is provided at the input of the spatial decoder. However, after the spatial decoder is functional, information is provided at its output at a picture format rate suitable for on-screen display. For example, the spatial decoder's temporal output rate can be varied to interface globally with various display systems such as NTSC, PAL, and SECAM. The moving picture formatting unit converts this variable picture rate into a constant picture rate suitable for the display. However, the picture data is still carried by a data token consisting of 64 words.

11. DRAM Interface One single high performance configurable DRAM interface is used for each of the three decoder chips. Generally, the DRAM interface on each chip is substantially the same. However, the interfaces differ in the way they handle channel priorities. This interface is
It is designed to directly drive the external DRAM used by the spatial decoder, the temporal decoder, and the video formatter. Generally, no external logic, buffers, or components are required to connect the DRAM interface to the DRAM of these systems.

According to the present invention, the interface can be configured in the following two ways. 1. The detailed timing interface can be configured to accommodate different DRAM types.

2. The width of the data interface to the DRAM can be configured to provide cost / performance tradeoffs for different applications.

Generally, the DRAM interface is a standard independent block implemented on each of the three chips in the system. Again, these chips are the spatial decoder, temporal decoder, and video formatting unit. Referring again to FIGS. 18, 19 and 20, which are block diagrams illustrating the relationship between the DRAM interface and the remaining blocks of the spatial decoder, the temporal decoder, and the video formatting unit, respectively. is there. In each chip, the DRAM interface connects the chip to external DRAM. External DRAMs are used because it is currently not practical to build the relatively large amount of DRAM needed on a chip. (Note: Each chip has its own external DRAM and its own DRAM interface.)
If the M interface is compression standard independent, then a multi-standard H.M. The interface must be configured to implement each of H.261, JPEG, and MPEG. How to reconfigure the DRAM interface for multi-standard operation is described in more detail later.

To understand the operation of the DRAM interface, it is necessary to understand the relationship between the DRAM interface and the address generator and how to communicate between them using a 2-wire interface. is necessary.

In general, as the name implies, an address generator is used to address DRAM (eg, D
When reading and writing to a specific special address in RAM) Generates an address required by the DRAM interface. DR when using 2-wire interface
Reading and writing will only occur if the AM interface has both data (from the previous stage in the pipeline) and a valid address (from the address generator). The use of a separate address generator simplifies the construction of both the address generator and the DRAM interface, as will be discussed further below.

In the present invention, the DRAM interface is operable from a clock that is synchronous to both the address generator and the clock of the stage through which the data is supplied. Special techniques are needed to handle the asynchronous nature of operations.

The data is generally transferred between the DRAM interface and the rest of the chip in blocks of 64 bytes (the only exception being the predictive data in the temporal decoder). The transfer is done by a device known as a "swing buffer".
The swing buffer essentially fills or empties one RAM, while at the same time another component of the chip empties or fills the other ram.
A pair of RAMs operated in a double buffered configuration using the M interface. A separate bus carrying the address from the address generator is associated with each swing buffer.

In the present invention, each chip has four swing buffers, but the functions of these swing buffers differ in each case. In the spatial decoder,
One swing buffer D
One swing buffer is used to read the coded data from the DRAM, and a third swing buffer is used to transfer the tokenized data to the DRAM. , And then
The fourth swing buffer D stores the tokenized data.
Used to read from RAM. However, in a temporal decoder, one swing buffer is used to write intra or predictive picture data to DRAM, a second swing buffer is used to read inra or predictive data from DRAM, and the other. Two are used to read forward and backward prediction data. In the video formatting section,
One swing buffer is used to transfer the data to the DRAM, and the other three are used to read the data from the DRAM, the luminance (Y), and the red and blue color difference data (respectively, One for Cr and one for Cb). For the operation of one virtual DRAM interface with one write swing buffer and one read swing buffer,
This will be explained in the next section. Essentially this is the same operation of the DRAM interface of the spatial decoder. The operation is shown in FIG.

FIG. 32 shows the address generator 301, DRA.
The control interface between the M interface 302 and the remaining stages of the chip that supplies the data is shown to be all 2-wire interfaces. The address generator 301 may generate an address as a result of receiving a control token, or may simply generate a fixed sequence of addresses (eg, for a spatial decoder FIFO buffer). The DRAM interface handles the 2-wire interface associated with the address generator 301 in a special way. Instead of holding accept high when the accept line is ready to receive an address, the address line waits for a valid address supply by the address generator, processes the address, and waits for one clock period. Then set the accept line high. In this way, a request / acknowledgement (PEQ / ACK) protocol is implemented.

A unique feature of the DRAM interface 302 is the ability to communicate independently of the address generator 301 and the stages that supply or receive data. For example, the address generator may generate an address associated with the data in the write swing buffer (FIG. 33), but the write swing buffer will not have a block of data ready for writing to the external DRAM. No action is taken until the time of sending the existence. Similarly, the write swing buffer may contain blocks of data that are ready for writing to the external DRAM, but at addresses
No action is taken until the address generator 301 feeds the bus. In addition, once one of the RAMs in the write swing buffer has been filled with data, the other RAM can be used before the data input is stalled (2-wire interface accept signal set low).
Can be completely filled and "swinged" to the DRAM interface side.

In understanding the operation of the DRAM interface 302 according to the present invention, in a properly configured system, the DRAM interface is between the swing buffer and the external DRAM 303.
It is important to note that it is possible to transfer data at a data rate at least equal to the sum of all average data rates between the swing buffer and the rest of the chip.

Each DRAM interface 302 determines which swing buffer will serve the next. Generally, this is a "round robin".
n) (i.e. the next swing buffer to be serviced is
Either the next available swing buffer that was in the least recent order) or a priority encoder (ie, in this case some swing buffers have higher priority than others). is there. In both cases,
Additional requests with a higher priority than all other requests come from the Refresh Request Generator. The refresh request is generated from a refresh counter that is programmable via the microprocessor interface.

Referring now to FIG. 33, which shows a block diagram of the write swing buffer.
The write swing buffer interface has a first R
Two blocks of the AM 311 and the second RAM 312 are included. As discussed further herein, data is written to RAM 331 and RAM 312 from the previous stage under the control of write address 303 and controller 314. Data is written to the DRAM 315 from the RAM 311 and the RAM 312. During data entry into the DRAM, as described further herein, the DRAM
The row address is provided by the address generator and the column address is provided by the write address and controller. In operation, valid data is provided at input 316 (data in). In general,
The data is received from the previous stage. Each datum is written into RAM 311 as it is received by the DRAM interface, and the write address controller increments RAM 311 and
The next one data can be written in the RAM 311. Writing data into the RAM 311 continues until there is no more data or the RAM 311 is full. If RAM 311 is full, the input side gives up control and sends a signal to the read side to indicate that RAM 311 is ready for reading. Therefore, this signal feeds between two asynchronous clock regimes and feeds through three synchronizing flip-flops. Assuming the RAM 312 is empty, the next item of data that arrives at the input side is the RAM 312.
Will be filled in. Otherwise, this will happen if RAM 312 is empty. If the round robin or priority encoder indicates the swing buffer to hit in the order in which it is read (depending on which is used by the particular chip), the DRAM interface will
Read the contents of AM311 and read them to external DRA
Fill in M. The signal is then sent back through the asynchronous interface, indicating that RAM 311 is ready to be refilled.

The DRAM interface is the RAM 311
Empty and then "swing" it before the input side fills the RAM 312, continuous acceptance of the data by the swing buffer is possible. Otherwise, if RAM2 is full, the swing buffer sets its acceptance signal low until RAM311 is "swinged" backwards for use by the input side.

The operation of the read swing buffer according to the present invention is the same except that the input data and output data are inverted.

The DRAM interface of the present invention is designed to maximize the available memory bandwidth. Each 8x8 block of data is the same DRAM
Remembered on page. In this way, the DRAM fast page access mode can be fully utilized. In this case, 1
One row address is provided, followed by many column addresses. In particular, the row address is provided by the address generator, while the column address is the DR, as will be discussed later.
Supplied by the AM interface. In addition, a feature is provided that allows the width of the data bus to the external DRAM to be 8, 16 or 32 bits. Therefore, DRAM usage can be matched to specific application size and bandwidth requirements.

In this example (more precisely, how the DRAM interface on the spatial decoder works), the address generator supplies the DRAM interface with the block address for each of the read and write swing buffers. .. This address is used as the row address for the DRAM. The 6-bit column address is provided by the DRAM interface itself, and these bits are also used as the address for the swing buffer RAM.
The width of the data bus to the swing buffer is 32 bits. Therefore, if the bus width to the external DRAM is less than 32 bits, then 2 or 4 external DRAM accesses will either be before the next word is read from the write swing buffer or the next word will be read to the read swing buffer. It must be done before it is written (read and write refer to the direction of transfer to the external DRAM).

In the case of the temporal decoder and the moving picture formatting section, the conditions are more complicated. The temporal decoder addressing is more complicated due to its predictive aspects as discussed further in this section. Addressing of video formatting is more complicated due to multiple video output standard aspects, as discussed further in this section regarding the video formatting part.

As already mentioned, the temporal decoder has four swing buffers. Two of them are used to perform reading and writing of decoded int and predictive (I call P) picture data. They are,
It works as described above. The other two are used to receive prediction data. These buffers are even more interesting.

In general, the prediction data deviates from the position of the processed block, as specified in the motion vectors as x and y. Therefore, since the data block to be retrieved is encoded (and written to DRAM), it generally does not match the block boundary of the data. In this, as shown in FIG. 34, a shaded portion represents a block that is being formed, and an outline indicated by a dotted line represents a block that is being predicted based on it. The address generator translates the address specified by the motion vector into offset blocks (total number of blocks), as indicated by the large arrows, and into offset pixels, as indicated by the small arrows.

At the address generator, the frame pointer, base block address, and vector offset are added to form the address of the block to be retrieved from DRAM. If the pixel offset is zero, then only one request is generated.
If there is an offset in either the x or y dimension,
Two requests are generated, the original block address and the block address immediately below. If there are offsets in x and y, four requests will be generated. For each block to be searched, the address generator calculates the start and stop addresses, as best shown in the illustrated example.

Consider the pixel offset (1,1) corresponding to the shaded portion of FIG. The address generator makes four requests, labeled A to D in the figure. The problem to be solved is how fast to supply the required sequence of row addresses. The answer is to use the "start / stop" technique as shown below.

Consider block A of FIG. The reading must start at position (7,7) and end at position (1,1).
At this stage, assume that one byte is being read at a time (ie, 8-bit DRAM interface). The x value in the coordinate pair is the three LSs of the address
Form B and the y value forms 3 MSBs. x and y
The start values are both 1 and provide 9 for the address. The data is read from this address and the x value is incremented. The process is repeated until the x value reaches its stop value, at which point the y value is incremented by 1 and the x start value is reloaded by 1.
Give one address 17. As each byte of data is read, the x value is incremented again until the stop value is reached. until both x and y values reach their stop value,
The process is repeated. In this way, 9, 10,
11, 12, 13, 14, 15, 17. ． ． 23, 2
, 31, 33, ..., 57, ..., 63 address sequences are generated.

In a similar manner, the start and stop coordinates for block B are (1,0) and (7,0).
And for block C (0, 1) and (0,
7), and for block D (0,
0) and (0, 0).

The next question is where to enter this data. Obviously, observing block A, the data read from address 9 must be written to address 0 of the swing buffer and the data from address 10 must be written to address 1 of the swing buffer. .. And so on. Similarly, the data read from address 8 in block B must be written to address 15 in the swing buffer and the data from address 16 must be written to address 15 in the swing buffer. This feature is very easy to implement, as outlined below.

Consider block A.
At the start of reading, the swing buffer address register is loaded with the reverse value of the stop value. The reverse stop value of y forms 3 MSBs, and x
The reverse stop value of forms three LSBs. In this case, the DRAM interface is reading address 9 in the external DRAM, but the swing buffer address is zero. As the external DRAM address register is incremented, the swing buffer address register is also incremented, as shown in Table 500 "Predictive Addressing".

The description so far has focused on the 8-bit DRAM interface. For 16 or 32 bit interfaces, a few minor modifications must be made. First, the pixel offset vector must be "clipped" so that it points to a 16 or 32 bit boundary. In the example used here, for block A, the first read
DRAM points to address 0, and data from addresses 0 to 3 are read. Second, unwanted data must be discarded. This is accomplished by writing all of the data to a swing buffer (where the swing buffer must be physically larger than was necessary in the 8-bit case) and then reading it along with the offset. When implementing MPEG half pixel interpolation, 9 bytes in x and / or y must be read from the DRAM interface. In this case, the address generator provides the appropriate start and stop addresses. Some additional logic is used in the DRAM interface, but there is no fundamental change in the way the DRAM interface operates. A final point to note regarding the temporal decoder DRAM interface of the present invention is that additional information must be provided to the predictive filter to indicate what processing is required to be performed on the data. is there. The contents of the necessary processing are shown below.

Transfer (64, 72, or 81 bytes)
A "last byte signal" that indicates the last byte of the.

H. 261 flag.

Bidirectional prediction flag.

Two bits indicating the dimension of the block (8 or 9 bytes in x and y).

One 2-bit number indicating the order of blocks.

As the data is read from the swing buffer, the final byte flag can be generated. The other signal is obtained from the address generator and is piped through the DRAM interface so that the signal is combined with the correct block of data as the data is read from the swing buffer by the predictive filter block.

In the moving picture formatting section, the data is written in the external DRAM in the block,
However, they are read in raster order. The writing is exactly the same as already described for the spatial decoder,
However, reading is slightly more complicated.

The data in the moving picture format section and the external DRAM are organized such that at least eight blocks of data fit into one single page. These eight blocks are eight consecutive horizontal blocks. Upon rasterization, eight bytes from each of eight consecutive blocks need to be read and written to the swing buffer (ie, the same column in each of the eight blocks).

Considering the top row (and assuming a 1 byte wide interface), the x address (3L) is the same as for the y address (3MSBS).
SBS) is set to zero. The x address is then incremented as each of the first 8 bytes is read. At this point, the top part of the higher part of the address (bit 6 and above-LSB = bit 0) is incremented and x address (3 LSBS)
Is reset to zero. This process is repeated until 64 bytes have been read. If the interface width to the external DRAM is 16 or 32 bits, the x-address is incremented by 2 or 4 respectively instead of being incremented by 1.

In the present invention, although multiples of 8 bytes are always read, the address generator can signal to the DRAM interface that less than 64 bytes must be read (this is the start of a raster line). Or may be required at the end). This is achieved by using start and stop values. The start value is used for the top part of the address (bit 6 and above) and the stop value is compared to the start value to generate a signal indicating when the read must be stopped.

The DRAM interface timing block in the present invention uses a timing chain to place the edges of the DRAM signal to an accuracy of one quarter of the system clock period. Two quadrature clocks derived from a phase clocked loop are used. These clocks are combined to form a conceptual 2x clock. Therefore, every single chain is made up of two shift registers with opposite phase with 2x clocks connected in parallel.

First of all, there is one chain for the page start cycle, and another one for the read / write / refresh cycle. The length of each cycle is programmable through the microprocessor interface, after which the page start chain has a fixed length, and during the page start the length of the cycle chain changes accordingly. .

When reset, the chain is cleared,
Then a pulse is created. The pulse travels along the chain and is directed by the state information from the DRAM interface. The pulse generates the DRAM interface clock. Each DRAM interface clock period corresponds to one DRAM cycle. Therefore, when the DRAM cycle lengths are different, the DRAM interface clock is not constant speed.

In addition, the additional timing chain combines the pulses from the above chains with the information from the DRAM interface, eg notcas, notr.
Generate output strobes and enables such as as, notwe, notbe. 12. Prediction Filter Referring again to FIGS. 19, 20 and 21, and more particularly to FIG. 40, a block diagram of a temporal decoder is shown. This includes prediction filters. The relationship between the prediction filter and the remaining elements of the temporal decoder is shown in FIG.
6 shows in more detail. The essence of the structure of the prediction filter is shown in FIGS. A detailed description of the operation of the predictive filter is given in the "More Detailed Description of the Invention" section of this section.

In general, the prediction filter according to the present invention is
MPEG and H.264. Used in 26l mode,
Not used in JPEG mode. Recall the case in JPEG mode. That is, the time decoder
It only allows the data to be fed through the movie format section without any substantial decoding beyond the range achieved by the spatial decoder. Referring again to FIG. 27, in MPEG mode, the forward and backward prediction filters are the same, and filter each MPEG forward and backward prediction block. However, H. Since H.261 does not use backward prediction, H.261 does not use backward prediction. In the 261 mode, only the forward prediction filter is used.

Each of the two predictive filters of the present invention is substantially the same. 17 and 505 will be referred to again, and FIG. 37 will be referred to for further details. In the figure, a block diagram of the structure of the prediction filter is shown. Each prediction filter has four stages arranged in series. The data is input to the format stage 505-7 and formatted into a format that can be easily filtered. In the next stage 505-2, the ID prediction is
It is performed on the X coordinate. After the required transport is done by the dimension buffer stage 505-3, the ID prediction is done on the Y coordinate at stage 505-4. A more detailed description of how the stage performs the filtering action will be given. The conditions required for the filtering action are defined by the compression standard. H. 26
In the case of 1, the actual filtering effect is similar to that of a low pass filter.

Referring again to FIG. 26, the multi-standard operation is MPEG or H.264. The predictive filter needs to be reconfigurable to perform any of the H.261 filtering, or to perform no filtering in JPEG mode. Like many other reconfigurable aspects of a 3-chip system, the predictive filter
Reconstructed by token. The token is also used to inform the address generator about a particular mode of operation. In this case, the address generator is M
Addresses of the required data that vary significantly between PEG and JPEG can be provided to the predictive filter.

13. Accessing Registers Most registers in a microprocessor interface (MPI) can be modified only when the stage to which the interface is associated is stalled. Therefore, groups of registers are generally associated with access registers. If the value in the access register is zero, the group of registers associated with the particular access register in question should not be modified. Writing a 1 to the access register requests that the stage be stopped. However, since the stage cannot be stopped immediately, the stage access register therefore holds the value zero until stopped.

Any user software associated with MPI and used to perform functions through MPI must wait "after writing a 1 to the request access register" until a 1 is read from the access register. I won't. If the user writes a value to the configuration register, at the same time, the result is unknown until the access register is set to zero.

14. Microprocessor Interface A microprocessor interface (MPI) with a standard byte width is used in all circuits in the spatial and temporal decoders. MPI operates asynchronously with various space and time decoder clocks. Referring to Table 37, this table shows the various MPI signals used in this interface.
The character of the signal is shown in the input / output column, the name of the signal is shown in the signal name column, and a description of the signal function is shown in the description column. MPI electrical specifications are shown with respect to Table 38. All specifications are categorized by type, and the symbol column indicates three types. The meaning of these symbols is shown in the parameter column. The actual specifications are shown in their respective columns, with minimum, maximum, and units.

DC operating conditions are shown with respect to Table 39. The column headings are the same as in Table 38. The DC electrical characteristics are shown for Table 40, and the same headings are used as for Table 38 and Table 39.

15. MPI Read Timing The AC characteristics of the MPI Read Timing Diagram are shown with respect to FIG. Each line in the figure is labeled by the corresponding signal name, and the timing is shown in nanoseconds. The read timing characteristics of the complete microprocessor interface are shown in Table 41. The column number indicates the signal corresponding to the signal name described in the characteristic column. The columns identified by MIN and MAX provide the minimum length of time that indicates the maximum amount of time the signal is available. The Units column shows the unit of measurement used to describe the signal.

16. MPI Write Timing A general description of the MPI write timing diagram is shown with respect to FIG. This figure shows each individual signal name associated with MPI write timing. Names, signal characteristics, and various other physical characteristics are shown with respect to Table 42.

17. Keyhole Address Location In the present invention, the memory map with low access frequency is placed after the keyhole register. The keyhole register has two registers associated with it. The first register is a keyhole address register and the second register is a keyhole data register. The keyhole address specifies a location within the extended address space. Read or write operations to the keyhole data register access the location specified by the keyhole address register. After accessing the keyhole data register, the associated keyhole address register is incremented. Random access within the expanded address space is only possible by writing a new value to the keyhole address register for each access.
Circuits within the scope of the present invention may have multiple keyhole maps. Nevertheless, there is no dialogue between different keyholes.

18. PICTURE END Referring again to FIG. 18, which shows a general block diagram of the spatial decoder used in the present invention. The description of the picture end function applies throughout this block diagram. The picture end function is based on the H.264 standard. 261
It has an advantage in the multiplex standard that it can handle coded picture information, MPEG and JPEG signals.

As already mentioned, the overall block diagram of FIG. 18 is interconnected by the already described two wire interface. Each functional block is arranged to operate based on the state machine configuration shown with respect to FIG.

In general, the picture end according to the invention starts at a start code detector which produces a picture end control token. The picture end control token is supplied unchanged to the DRAM interface via the startup control circuit. In this case, the token is used to flush the write swing buffer in the DRAM interface. Recall that the contents of the swing buffer are written to RAM only when this buffer is full. However, it may end at the point where the buffer is not full, causing sticking of picture data. The picture end token forces the data out of the swing buffer.

Since the present invention is a multi-standard machine, this machine behaves differently for each compression standard. More specifically, the case of operating according to a machine dependent action cycle will be fully described. For each compression standard, a combination of control tokens and / or output signals from the MPU can select the total number of all available action cycles, or
The total number of all available action cycles can also be selected by the design of the control token itself. In this regard, the present invention is organized to defer information from entering the next block until all the information has been collected in the upstream block. The system waits until it is ready to feed the data to the next stage. In this way, the picture end signal is supplied to the coded data buffer, and the control portion of the picture end signal causes the reading of the contents of the data buffer and to the Huffman decoder and video demultiplexer circuit. Supplied.

Another advantage of the picture end control token is its general expected full range of signals supplied to the Huffman decoder and video demultiplexer circuits for use by the Huffman decoder demultiplexer, and / or
Alternatively, this control token identifies the end of the picture, even if it has never had a number. In this condition, the information held in the coded data buffer is supplied to the Huffman decoder and the video demultiplexer as a complete picture. In this way, the Huffman decoder and video demultiplexer state machine can still handle the data, depending on the system design.

Another advantage of the picture end control token is its ability to completely empty the coded data buffer so that floating information does not inadvertently remain in the off-chip DRAM, or swing buffer.

Yet another advantage of the picture end function is:
Its use in error recovery. Assuming, for example, that the amount of data held in the coded data buffer is less than the amount commonly used to indicate spatial information for one single image, the last image will fill the swing buffer. Is held in the data buffer until, but by definition the buffer is never full. At some point, the machine decides that the error condition leaves. Thus, the picture end token is decoded, and to the extent that it forces the data in the coded data buffer to be supplied to the Huffman decoder and video demultiplexer, the last image is decodable, and , Information is emptied from the buffer. As a result, the machine does not enter error recovery mode and continues to process the encoded data successfully.

Yet another advantage of the use of the picture end token is that the serial pipeline processor continues to process uninterrupted data. By using the picture end token, the serial pipeline processor is configured to handle less than the expected amount of data and, as a result, continues processing. Generally, prior art machines shut down due to error conditions. As already mentioned, the coded data buffer counts macroblocks as they come into its storage area. Furthermore, Huffman decoders and video demultiplexers generally know the amount of information expected when decoding each image, that is, the state machine part of the Huffman decoding and video demultiplexer is known during each image recovery cycle. Know how many blocks to process. If the correct number of blocks does not arrive from the coded data buffer, an error recovery routine will generally result. However, by using a picture end control token that reconstructs the Huffman decoder and video demultiplexer, this control token is handling the amount of information that is actually appropriate for the Mandecoder and video demultiplexer by reconstruction. So you can continue to work.

Referring again to FIG. 17, the token decoder portion of the buffer manager detects the picture end control token generated by the start code detector. In normal operation, as already mentioned, for normal operation of the swing buffer, the buffer register is full and emptied. Again, a swing buffer that is partially filled with data, until the swing buffer is completely filled,
And / or not empty until you know when it will be empty. The picture end control token is decoded in the token decoder portion of the buffer manager, and forces the partially full swing buffer to empty the coded data buffer. This is ultimately fed to the Huffman Decoder and Video Demultiplexer, either directly or via the DRAM interface.

19. Flushing Operation Another advantage of the picture end control token is its function in relation to the flash token. Flash tokens are not associated with either the state machine reconfiguration control or the data path to the system. On the contrary, this token completes the previous partial signal for handling by the machine-dependent state machine. Each state machine recognizes the flash control token as information that cannot be processed. Therefore, the flash token is used to fill all of the remaining empty parts of the coded data buffer, and to allow a sufficient set of information to be sent to the Huffman decoder and video demultiplexer. To be done. In this way, the flush token acts like padding on the buffer.

The token decoder in the Huffman circuit recognizes the flash token and ignores the spurious data that the flash token pushed into it.
The Huffman Decoder then acts only on the data content of the last picture buffer if it exists before the arrival of the picture end token and the flash token. A further advantage of the picture end token alone or in combination with the flash token is the reconfiguration and / or reorganization of the Huffman decoder circuit. Upon the arrival of the picture end token, the Huffman decoder circuit knows that it has less information than would normally be expected to decode the last image. The Huffman decoding circuit finishes processing the information contained in the last image,
Then, this information is output to the inverse modeler via the DRAM interface. When identifying the last image,
The Huffman Decoder goes into cleanup mode and re-adjusts for the arrival of the next image information.

20. Flush Function A flush token is provided according to the present invention to serve throughout the pipeline processor, and the buffer is empty and other circuitry is reconfigured to wait for new data to arrive. Used to guarantee. More specifically, the present invention comprises a combination of picture end tokens, padding words, and flash tokens that indicate to the serial pipeline processor that image processing for the form of the current picture is complete. After that, various state machines
It needs to be reconfigured to wait for new data to arrive for new handling. Also note that the flash token acts as a special reset to the system. The flash token resets each stage as it passes. However, it allows processing to continue in the next stage. This prevents data loss. In other words, flash tokens are variable resets as opposed to absolute resets.

21. Stop After Picture The stop after picture function is used to close the decompression circuit of the serial pipeline at a logical point in its operation. At this point
A picture end token is generated to indicate that the data has finished coming in from the data input line and the padding operation is complete. The padding function is
Partially fills an empty data token. The flush token is then generated, fed through the serial pipeline system, and pushes all the information out of the registers, forcing the registers to return to their neutral standby state. Then a stop-after-picture event is generated and no input is accepted until either the user or the system clears this condition. In other words, the picture end token sends the end of picture, while the stop-after-picture operation sends the end of all current processing.

22. Multi-Standard Search Mode Another feature of the invention is the use of the SEARCHNODE control token which is used to reconstruct the input to the serial pipeline processor to look at the incoming bitstream. If the search mode is set, the start code detector will
Alternatively, search only for markers used in any one of the compression standards. However, it should be appreciated that other images from other data bitstreams can be used for this purpose. Therefore, these images are modified throughout the present invention into other embodiments where it is possible to use data tokens with reconstruction circuitry to provide combinations of control tokens, and similar processing. Can be used for

The use of search mode in the present invention is convenient in many conditions, including the following: 1) when a data bitstream break occurs.
2) When the user intentionally changes the channel to disconnect the data bit stream, for example, the arrival of data by a cable carrier compressed digital video, or
3) By fast forward or reverse user activation from a controllable data source, eg an optical disc, or a motion picture disc. In general, search mode is useful when the user interrupts the normal processing of the serial pipeline in that the machine does not anticipate an interrupt.

If any of the search modes is set, the start code detector searches for a suitable incoming start image to make a machine independent token. All data coming into the start code detector prior to the identification of the standard dependent start image is meaningless and is abandoned as if the machine were idling if it was waiting for this information.

The start code detector can assume any one of the numeric 0 configuration configurations. For example, one of these configurations allows a search for groups of pictures or higher start codes. With this pattern, the start code detector discards all its inputs and looks for a group start standard image. If an image of this kind is identified, the start code detector produces a group start token and the search mode is automatically reset.

It is important to note that one single circuit, the Huffman Decoder, and the Video Demultiplexing Circuit, works with a combination of input signals, including standard independent setup signals, and coding standard signals. . The coding standard signal is transmitting information directly from the incoming bitstream as required by the Huffman decoder and video demultiplexing circuitry. However, on the other hand, the Huffman decoder and the video demultiplexing circuit function under the operation of the standard independent sequence signal.

Since this mode of operation is most effective and could be designed such that when carrying standard dependent inputs to the Huffman Decoder and Video Demultiplexer a special control token is used instead of carrying the actual signal itself. This operation mode was selected.

23. Inverse Modeler Inverse modeling is a feature of all three standards,
And it is the same for all three standards. In general,
The data token in the token buffer contains information about the value of the quantized coefficient and about the number of zeros between the displayed coefficients (1 of run length encoding).
Shape). The inverse modeler of the present invention is adapted for use with tokens, and simply extends the information about execution of zero so that each data token contains 64 values as a prerequisite. After that,
The values in the data token are quantized coefficients that can be used by the dequantizer.

24. Inverse Quantizer The inverse quantizer of the present invention is an element required in the decoding sequence, but was implemented in such a way that the entire IC set could handle multi-standard data. Further, the inverse quantizer has been adapted for use with tokens. The inverse quantizer has an inverse modeler and an inverse D.
It is arranged between CT (IDCT). For example, in the present invention, the adder in the inverse quantizer is
Used to add a constant to the pixel decode number before moving to OT.

IDOT uses a pixel decoding number that varies depending on the standards used to code the information.
To correctly decode the information, the value 1024 is added to the decode number by the dequantizer before the data continues into the IDCT.

Circuitry or software in the IC to handle data compressed by various standards by using an adder that is already located in the dequantizer to standardize the data before it reaches the IDCT. Eliminates the need to add. Other operations to prepare for multi-standard operation are performed in the middle of the "post-quantization function" and will be considered later.

The various standardization routines in which the control tokens associated with the data need to be decoded and performed by the dequantizer are identified in detail below. All of these "post-quantization" functions are implemented to avoid circuit duplication and allow the IC to handle multi-standard decoded data.

25. Huffman Decoder and Parser (Parser) Referring again to FIGS. 18 and 36, the spatial decoder comprises a Huffman decoder for decoding Huffman coded data according to various compression standards.
Standards JPBG, MPEG, and H.264. Each of the H.261s requires that the particular data be Huffman coded, while the differences in the Huffman decoding methods required by each standard are significant. In the spatial decoder of the present invention, without designing and creating a total of three individual Huffman decoders, one for each standard, the present invention identifies the common aspects of each Huffman decoder, and Saving valuable die space by creating aspects only once. In addition, a judicious multi-part algorithm is used, i.e., an algorithm that further enhances the commonality of aspects of each Huffman decoder that have commonality to other standards. In summary, the Huffman decoder 32
1 works with the other units shown in FIG. These other units are the parser state machine 322,
In shifter 323, data index unit 32
4, ALU 325, and token formatting unit 326. As already mentioned, the connection between these blocks is managed by a two wire interface. How these units work will be described in further detail below, focusing on specific aspects that support multi-standard operation of the Huffman decoder according to the present invention. .

The parser state machine of the present invention is a programmable state machine that acts to integrate the operations of the other blocks of the video parser. The parser state machine controls other blocks acted upon by the control word by generating control words arranged in parallel with the data and supplied to other blocks in response to the data. Providing the control word in juxtaposition with the associated data is not only useful, but essential, as the blocks are connected via a two wire interface. Thus, data and control arrive at the same time. The passage of control words is under the data lines connecting the blocks, as shown in FIG. 36 by the control lines. In particular, this codeword identifies the particular standard that is being decoded.

The Huffman Decoder also performs certain control functions. In particular, the Huffman decoder 321 includes a state machine that can control specific functions of the data index unit 324 and the ALU 325. Control of these units by the Huffman Decoder is necessary to correctly decode the block level information. Having the parser state machine 322 make these decisions takes too long.

An important aspect of the Huffman decoder of the present invention is the ability to invert the encoded data bits when reading the data bits into the Huffman decoder. This ability is 261 Huffman style code required to decode. The reason is H.264. The particular type of Huffman code used by H.261 (essentially MPEG) has the opposite polarity as the code used by JPEG. Therefore, the use of an inverter makes the same table as that used by the Huffman decoder practically usable for all three standards.
Other aspects of how the Huffman Decoder implements all three standards are described in more detail in the "More Detailed Description of the Invention" section.

The data index unit 324 performs the second part of the multiple part algorithm. This unit contains a look-up table that provides the actual Huffman decoded data. Entries in this table are organized based on the number of indexes generated by the Huffman decoder.

The ALU 325 implements the rest of the multi-part algorithm. In particular, the ALU handles sign extension. In addition, the ALU includes a register file that holds vector and DC predictions. Uses in this file are described in the section related to predictive filters. In addition, the ALU includes a counter that counts through the structure of the picture that is being decoded by the spatial decoder. In particular, the picture dimensions are programmed into registers associated with the counter to facilitate detection of "picture start", and macro-block code start.

In accordance with the present invention, the token-matching unit 326 (TF) assembles the decrypted data into data tokens. These tokens are then provided to the remaining stages or blocks in the space-time decoder.

In the present invention, the inshifter receives data from the FIFO buffering data that supplies the start code detector. There are generally two types of data that the inshifter receives: a data token, and a start code that is exchanged with each token by a start code detector, as discussed further in the "Tokens" section. . Note that most of the data are data tokens that require decryption.

The inshifter supplies data to the Huffman decoder serially, while supplying control tokens in parallel. In the Huffman decoder, the Huffman encoded data is decoded according to the first part of the multi-part algorithm. In particular, the particular Huffman code is identified and then replaced with the index number.

In addition, the Huffman Decoder identifies specific data that requires special handling by the other blocks shown in FIG. This data includes block ends and escapes. In the present invention, time is saved by detecting these at the Huffman decoder, not at the data index unit. This index number is then provided to the data index unit 324. The data index unit is
It is essentially a lookup table. The look-up table based on one aspect of the algorithm is JP
It is not much different from the Huffman code table specified by EG. Generally, it is JPEG alternative JPE
The format specified for transferring the G-table is the compressed data format.

The decoded index number, or other data, is provided from the data index unit 324, along with the associated control word, to the ALU 504-5 which performs the previously described operations.

The data and control word are supplied from the ALU 325 to the token format unit 326 (TF). In the token format section, the data is optionally combined with control words to form tokens. The token is then brought to the next stage of the spatial decoder. Note that at this point there are as many tokens used by the system.

26. Discrete Inverse Cosine Transform The Discrete Inverse Cosine Transform (IDCT) according to the present invention expands the data related to the frequency of the DC component of the picture. When a particular picture is being compressed, the light frequency (frequency) in the picture is quantized, reducing the overall amount of information that needs to be stored. IDCT is
Using this quantized data, it is expanded and restored to frequency information.

The IDCT operates on parts of the picture that are 8 × 8 pixels in size. The mathematics applied to this data is governed primarily by the particular standard used to encode the data. However, an important application in the present invention is to create common mathematical functions between standards to avoid unnecessary duplication of circuits.
By using a specific scaling order, common math functions that increase the symmetry between the top and bottom of the algorithm, thereby eliminating the need for additional circuitry, can be reused.

The IDCT responds to multiple multi-standard tokens. The first part of the IDCT checks the incoming data to ensure that the size of the data token is correct for processing. In fact, if the error is not very large, the token stream can be conditionally corrected.

27. Buffer Manager The buffer manager of the present invention receives incoming video information and then the timing of data arrival, display,
And information about the frame rate to the address generator. Multiple buffers are used to allow both display and display rate changes. Display and display rates generally vary according to the coded data and the monitor on which the information is being displayed. The data arrival rate generally varies depending on the errors involved in the raw materials used to encode, decode, or create the data. When the information arrives at the buffer manager, it is decompressed. However, the data is arranged in an order that is useful to the decompression circuit but not to the particular display unit in use. When a block of data is input to the buffer manager, the buffer manager supplies information to the address generator so that the block of data can be placed in an order usable by the display device. This causes the buffer manager to use the necessary frame rate conversion to adjust the incoming data block so that it can be displayed on the particular display device in use.

In the present invention, the buffer manager primarily supplies information to the address generator. However, it also requires interfacing with other elements of the system. For example, there is an interface with the input FIFO which transfers the tokens to the buffer manager which results in supplying these tokens to the write address generator.

In addition, the buffer manager also interfaces with the display address generator to receive information as to whether the display device is ready to display new data. Similarly, the buffer manager confirms that the display address generator has cleared the information for display from the buffer.

The buffer manager of the present invention stores whether a particular buffer is empty, full, ready for use, or in use, and
The number of displays associated with the particular data in each buffer is stored. In this way, the buffer manager determines, in part, the state of the buffer by having only one buffer ready for display at a time. Once the buffer is displayed, the buffer is in the "empty" state. When the buffer manager receives a picture start, flush, valid token, or access token, the state of each buffer,
And make preparations for receiving new data. For example, a picture start token causes the buffer manager to cycle through each buffer to find a buffer that can accept new data.

In addition, the buffer manager is configurable to handle the multi-standard requirements dictated by the tokens it receives. For example, H.264. In the 26l standard, data can be skipped during display. When such a token arrives at the buffer manager, the data to be skipped is flushed from the buffer in which it is stored.

Thus, by managing the buffer, the data is effective according to the compression standard used to encode the data, the speed at which the data is decoded, and the particular type of display device in use. Can be displayed at.

The foregoing description is generally sufficient for a person of ordinary skill in the art to implement the present invention with all the attendant features, objects and advantages. It is believed that it adequately describes the concepts, system implementations, and operation of various aspects of the present invention. However, in order to facilitate a more thorough understanding of the present invention, as well as additional details relating to more specific and commercial implementations of various embodiments of the present invention, a further description and It is preferable to continue the description below.

The following is a more detailed description of a multi-standard video decoder chip set. The description is divided into three parts, A, B, and C.

For the purpose of organizing, clarifying, and simplifying the description, additional disclosure will be presented again in the following section.

Description of features common to the chips in the chip set.

-Token-2-wire interface-DRAM interface-Microprocessor interface-Clock-Spatial decoder chip description-Time decoder chip description Section A. 1 “How to Use This Description” The first description section covers most of the electrical design issues associated with the use of chip sets.

A. 1.1 "Printing conventions" Some printing conventions are used to emphasize certain classes of information. An example is shown below.

Token Name Wirename Active High Signal Wirename Active Low Signal Register Name Section A. 2 "Video Decoder Family" * 30MHz operation * Decode MPEG, JPEG & H.264. 261 * Coded data rate for 25 Mb / s * Video data rate for 21 MB / s * MPEG resolution 704 x 480, 30 Hz, 4:
Up to 2: 0 * Flexible chroma sampling format * Overall JPEG baseline decoding * Glueless page mode DRAM A interface * 208 pin PQFP package * Independent coded data and decoder clock * Reorder MPEG picture sequence Video decoder family Provides a low chip count solution for implementing a high resolution digital video decoder. Currently, the chip set has three different video and picture coding systems: JPEG, MPE.
G, and H.G. H.261 is configurable.

Full JPEG baseline picture decoding is supported. 720 x 480, 30 Hz, 4: 2: 2 J
A PEG encoded moving image can be decoded in real time.

CIF (Common Exchange Format), and Q
CIF H. The H.261 video can be decrypted. 740x
All special-function MPEG moving pictures having formats up to 480, 30 Hz and 4: 2: 0 can be decoded.

Note: The above values are given as examples only, and do not necessarily represent the limiting conditions of one embodiment of the present invention. Therefore, it should be understood that other values and / or ranges may be used.

A. 2.1 "System Configuration" A. 2.1.1 "Output Formatting" In each of the following examples, the shape of the output formatting section is, optionally, using the data provided at the output of the spatial or temporal decoder, and the computer or display. Reformatting is required for the system. The details of this formatting are application dependent. In the simple case, as an example, one only needs an address generator for using the Brookformatted data output by the decoder chip and for writing the data output in raster order.

The image formatter is a single chip VLSI device that provides output formatting functions over a wide range.

A. 2.1.2 "JPEG still image decoding" One single spatial decoder equipped with a non-off-chip DRAM can decode a baseline JPEG image at high speed. The spatial decoder supports all features of baseline JPEG. However, the decodable image size may be limited by the size of the output buffer provided by the user. The characteristics of the output formatter may limit the chroma sampling format and the color space that it can support.

A. 2.1.3 “JPEG moving image decoding” By adding an off-chip DRAM to the spatial decoder, it is possible to decode a JPEG encoded moving image in real time. The size and speed of the buffer required depends on the video and coded data rate. The temporal decoder does not have to decode the JPEG encoded video. However, if the time decoder is on a multi-standard decoder chip set and the system is configured for JPEG operation, it simply supplies the data through the time decoder without any changes or modifications. Absent.

A. 2.1.4 “H.261 decoding” The spatial decoder and the temporal decoder are H.261. Both are needed to implement a H.261 video decoder. The DRAM interface on both devices requires the D required for proper operation when operating at low coded data rates in small picture formats.
It is configurable so that the amount of RAM can be reduced. In general, each spatial and temporal decoder is
It requires one single 4Mb (eg 512k × 8) DRAM.

A. 2.1.5 "MPEG Decoding" The configuration required for MPEG operation is H.264. It is the same as the case of H.261. However, as will be appreciated by those of ordinary skill in the art, larger DRAM buffers may be required to support the larger picture formats available in MPEG.

Section A. 3 "Token" A. 3.1 "Token Format" In accordance with the present invention, tokens provide an extensible format for carrying information through the decoder chip set. In the present invention, the minimum width of each word of the token is 8 bits, but it should be understood by one of ordinary skill in the art that the width of the token can be arbitrary. Further, a token can be extended over one or more words, which is accomplished with extension bits in each word.

The extension bit indicates whether the token continues in other words. All words except the last word of the token are set to one. If the first word of the token has zero extension bits, this indicates that the token is only one word long.

Each token is identified by an address field where it starts at bit 7 of the first word of the token. The length of the address field is variable and is potentially expandable over multiple words (although there are no addresses in the current chip that exceed 8 bits in length, so there is no One of ordinary skill in the art should understand that the length of the address can be arbitrary in this case as well).

Depending on the interface, 8 of data
Transfer more bits than. For example, the width of the output of the spatial decoder is 9 bits (10 bits including extension bits). The only tokens that make use of these additional bits are data tokens. A data token can have as many bits as needed to perform processing at a particular location in the system. All other tokens ignore the extra bit.

A. 3.2 "Data token" A data token carries data from one processing stage to the next. Therefore, the characteristics of this token change as it passes through the decoder. Furthermore, the meaning of the data carried by a data token varies depending on where it is in the system, i.e. the data is position dependent. In this regard,
The data may be either frequency domain data or pixel domain data depending on where the data token is located in the spatial decoder. For example, at the input of the spatial decoder, the data tokens have bit-serial coded video data packed into 8-bit words. In this respect, there is no limit on the length of each token. However, in contrast, at the output of the spatial decoder, each data token has exactly 64 words, each word being 9 bits wide.

A. 3.3 "Use of token-formatted data" Some applications require that the circuit be directly connected to the input or output of a decoder or chipset. In most cases, it will be sufficient to collect the data tokens and detect a few tokens that provide synchronization information (such as PICTURE START). In this regard, Section A. 16 "Connection to the output of the spatial decoder" and A. See 19 "Connect to Output of Time Decoder".

As already mentioned, observing the activity on the extension bits is sufficient to identify when each new token starts.
Again, the extension bit sends the last word of the current token. In addition, the address field can be tested to identify the token. Unnecessary or unrecognized tokens can be consumed (and destroyed) without knowing their contents. However, the recognized token causes the appropriate action.

Furthermore, the data input to the spatial decoder can be provided as bytes of coded data or in data tokens (section A.10).
See "Coded Data Entry"). Providing the token via a coded data port or a microprocessor interface allows many features of the decoder chip set to be constructed from the data stream. This offers an alternative to implementing the configuration via a microprocessor interface.

[0471]

[Table 1]

[0472]

[Table 2] A. 3.4 Token Description This section details a token implemented in a spatial decoder and a temporal decoder chip according to the present invention. Note: "r" means the currently reserved bit,
The value is 0. Unless otherwise noted, all integers are unsigned.

[0473]

[Table 3]

[0474]

[Table 4]

[0475]

[Table 5]

[0476]

[Table 6]

[0477]

[Table 7]

[0478]

[Table 8]

[0479]

[Table 9]

[0480]

[Table 10]

[0481]

[Table 11] A. 3.5 "Number sent in token" A. 3.5.1 "Component Identification Number" The component ID number according to the present invention is a 2-bit integer designating a color component. This 2-bit field is generally
It is placed as part of the header in the data token. MPEG and H.264. In the case of 261, the interrelationship is very simple.

[0482]

[Table 12] Since the color components that can be used in JPBG are not limited, J
In the case of PEG, the conditions are more complicated. The decoder chip allows up to 4 different color components in each scan. IDs are assigned sequentially as the color component specifications arrive at the decoder.

A. 3.5.2 "Number of horizontal and vertical samplings" For each of the four colors, there is a specification regarding the number of blocks arranged horizontally and vertically in a macroblock. This specification includes a 2-bit integer that is one less than the number of blocks containing a 2-bit integer. For example, 4: 2: 0 in MPEG (or H.26l)
Table 1 shows the arrangement of component IDs for chroma sampling (Fig. 45).
3 shows.

[0484]

[Table 13] For JPEG and 4: 2: 2 chroma sampling,
The component allocation to the component ID differs depending on the use. Section A. See 3.5.1.

Note: When processing 4: 2: 2 data, JPEG will be 2: 1: for that macroblock.
Requires a 1: 1 structure.

[0486]

[Table 14] A. 3.6 “Special Token Format” Based on the present invention, for example, data token and QUAN
T-table tokens are used for their "extended form" within the decoder chip set. In the expanded form, the token contains some data.
In the case of data tokens, such tokens may contain coded data or pixel data. Q
In the case of UANT table tokens, these tokens contain quantizer table information.

Furthermore, the "non-expanded" form of these tokens
Is defined as "empty" in the present invention.
This token format provides a place in the token stream that is subsequently filled by an extended version of the same token. This format is mainly applicable to encoders, so it will not be described in further detail.

[0488]

[Table 15]

[0489]

[Table 16] A. 3.7 "Use of Tokens for Different Standards" Each standard uses different subsets of tokens defined in accordance with the present invention.

Section A. 4 "2-wire interface" A. 4.1 "2-Wire Interface and Token Port" In the chipset, a simple 2-wire enable / accept protocol is used to control the flow of information.
Data is only transferred between blocks if it is observed that the transmitter and receiver are ready to generate clocks.

1) Data transfer 2) Not ready for receiving side 3) Not ready for sending side If the sending side is not ready (3 above), ready for sending side If not), the input of the receiving side must wait. If the receiver is not ready (2 above, the receiver is not), the sender continues to supply the same data at its output until it is accepted by the receiver.

When token information is transferred between blocks, the two wire interface between blocks is called the token port.

A. 4.2 "Place of Use" The decoder chip set according to the invention uses a two-wire interface to connect three chips. In addition, the coded data input to the spatial decoder is a two wire interface as well.

A. 4.3 "Bus Signal" The width of the data words transferred by the 2-wire interface varies according to the needs of the relevant interface (see Figure 44 "Tokens on interfaces wider than 8 bits"). For example, a 12-bit coefficient has only 9
Discrete inverse cosine converter (ID
CT).

[0495]

[Table 17] In addition to the data signals, there are three other signals that are sent via the 2-wire interface.

* Valid signal (valid) * Accept signal (accept) * Extension signal (extension) A. 4.3.1 "Extension signal" The extension signal corresponds to the token extension bits already mentioned.

A. 4.4 “Design considerations” The 2-wire interface is for short-distance, chip-to-chip serial communication.

The decoder chips should be placed close to each other to minimize the length of the PCB tracks between the chips. Wherever possible, the length of the truck should be kept below 25 mm. The PCB track capacitance should be kept to a minimum.

The clock distribution must be designed to minimize clock through between chips. If there is any clock through, the "receiver chip" must be arranged so that it sees the clock before the "transmit chip".

All chips that communicate via the 2-wire interface must operate from the same digital power supply.

A. 4.5 "Interface Timing"

[0502]

[Table 18] Note: Figure 47 shows a two wire interface between the system demux chip and the spatial decoder coded data port operating from the main decoder clock. This is optional because the 2-wire interface can operate from a coded data clock that can be asynchronous to the decoder clock. Section A. See 10.5, "Coded Data Clock". Similarly, the display interface of the image formatting section can operate from a clock that is asynchronous to the main decoder clock.

A. 4.6 "Signal Level" The 2-wire interface uses a COMS input and output. VIHmin is about 70% of VDD, and VILmax is V
It is about 30% of DD. The values shown in Table 19 are the values of VIH and VIL when VDD is the worst. VDD = 5.0
± 0.25V.

[0504]

[Table 19] A. 4.7 "Control Clock" Generally, the clock that controls the transfer via the two-wire interface is the decoder clock of the chip. The exception is the coded data port input to the spatial decoder. This is controlled by the encoding clock. The clock signal will be described further herein.

Section A. 5 “DRAM Interface” A. 5.1 "DRAM Interface" One single high performance, configurable DRAM interface is used in each video decoder chip. Generally, the DRAM interface on each chip is substantially the same. However, the interfaces differ from each other in the way they handle channel priorities. The interface is the DRA used by each decoder chip
Designed to drive M directly. In general,
The external logic, buffer, or component is DR
DRA in most systems with AM interface
Not required to connect to M.

A. 5.2 "Interface signal"

[0507]

[Table 20] According to the invention, the interface can be configured in two ways:

* For the detailed timing of the interface,
It can be configured to accommodate a variety of different types of DRAM.

* The "width" of the DRAM interface is
It can be configured to provide cost / performance tradeoffs in different applications.

A. 5.3 "DRAM Interface Configuration" In general, there are three groups of registers associated with the DRAM interface: Interface Timing Configuration Registers, Bus Configuration Registers, and Refresh Configuration Registers. . Refresh Configuration Register The registers shown in Table 23 must be configured last.

A. 5.3.1 "Post-reset conditions" After reset, the DRAM interface according to the invention starts operation with a set of default timing parameters (corresponding to the slowest operating mode). Initially, the DRAM interface continuously performs a refresh cycle (except for all other transfers). This continues until a value is entered in the refresh interval. The DRAM interface can then perform other types of transfers between refresh cycles.

A. 5.3.2 “Bus Configuration” Registers in bus configuration 22 should be done only when no data transfer is attempted by the interface. The interface is
Immediately after reset, this condition is placed before the value is written to the refresh interval. The interface can be reconfigured later, if necessary, only if no transfer has been attempted. Temporal Decoder Chip Access Register (A.18.3.1) and Spatial Decoder Buffer Manager Access Register (A.13.1.
See 1).

A. 5.3.3 "Configuration of Interface Timing" In accordance with the present invention, modification of interface timing configuration information is controlled by the interface timing access register. (Enter 1 in this register to show it in the interface timing register 21)
Can be modified. While Interface Timing Access = 1, the DRAM interface continues to operate with its previous configuration. 1
The user must wait until one can be read back from the interface timing access before filling in any interface timing registers after writing.

When the configuration is completed, the interface timing access must be filled with 0. The new configuration is then transferred to the DRAM interface.

A. 5.3.4 "Refresh Configuration" The refresh interval of the DRAM interface according to the present invention can be configured only once after reset. Until the refresh interval is configured
The interface continuously performs refresh cycles. This prevents any other data transfer. After the refresh interval is filled in, the data transfer can start.

As is well known in the art, DRAMs are typically 1
A "pause" is required between 00 and 500 microseconds and is followed by a number of refresh cycles before normal operation is possible. Therefore, before filling in the refresh interval,
These DRAM start requirements must be met.

A. 5.3.5 "Read Access to Configuration Register" All DRAM interface registers of the present invention are readable.

A. 5.4 "Interface Timing (Tick)" DRAM interface timing is derived from a clock that runs at four times the input clock rate of the device (decoder clock), which is generated by an on-chip PLL.

For the sake of brevity, this high speed clock period is called a tick.

A. 5.5 "Interface Register"

[0521]

[Table 21]

[0522]

[Table 22] A. 5.6 "Interface Operation" The DRAM interface uses the fast page mode. Three different types of access are supported.

* Read (Read) * Write (Write) * Refresh Each read or write access transfers one burst from 1 to 64 bytes to one single DRAM page address. Read and write transfers are not mixed within one single access, and each successive access is treated as a random access to a new DRAM page.

[0524]

[Table 23] A. 5.7 "Structure of Access" Each access has two parts.

* Access Start * Data Transfer In the present invention, each access starts with an access start and is followed by one or more data transfer cycles. In addition, there are read, write, and refresh variants (variants) for both access start and data transfer cycles.

When the last data transfer for a particular access is complete, the interface enters its default state (see A.5.7.3) and stays in this state until a new access is ready to start. When the last access ends and the new access is ready to start, the new access starts immediately.

A. 5.7.1 "Access Start" Access start provides the page address for read or write transfers, and establishes some initial signal conditions. According to the invention, there are three different access starts.

* Start of reading * Start of writing * Start of refreshing

[0529]

[Table 24] In each case, the timing of RAS and row address is
It is controlled by the register RAS falling and the page start length. The states of OE and DRAM data [31: 0] are RA from the end of the previous data transfer.
It is secret until S falls. The three different types of access starts differ only in the way they drive OE and DRAM data [31: 0] when RAS falls. See FIG. 52.

A. 5.7.2 "Data Transfer" In the present invention, there are different types of data transfer cycles.

* Fast Page Read Cycle * Fast Page Late Write Cycle * Refresh Cycle The start of refresh can only be followed by one single refresh cycle. The start of a read (or write) can be followed by one or more fast page read (or write) cycles. At the start of the reading cycle, C
AS is driven high and the new column address is driven.

In addition, an early write cycle is used.
WE is driven low at the start of the first write transfer and stays low until the end of the last write transfer. The output data is driven by the address.

CAS before RAS refresh cycle
Is initiated by the start of the refresh cycle, so the interface signal is inactive during the refresh cycle. The purpose of the refresh cycle is to meet the minimum RAS low period required by the DRAM.

A. 5.7.3 "Default State of Interface" Interface signals in the present invention enter the default state at the end of access.

RAS, GAS, and WE are high.

* Data and OEs remain in their previous state.

* Addr remains stable.

A. 5.8 “Data bus width” 2-bit register, DRAM data width is DR
Allows the width of the AM interface data path to be configured. This allows minimizing DRAM cost when used with small picture formats.

[0539]

[Table 25] A. 5.9 "Row Address Width" The number of bits taken from the middle section of the 24-bit internal address to provide the row address is configured by the row address bit register.

[0540]

[Table 26] A. 5.10 "Address Bits" An on-chip 24-bit address is generated. How this address is used to form the column and row addresses depends on the width of the data bus and the number of bits chosen for the row address. Some configurations do not allow all internal address bits to be used, so a "hidden bit" is created.

Similarly, the row address is extracted from the central part of the address. Thus, this maximizes the rate at which the DRAM naturally refreshes.

[0542]

[Table 27] A. 5.10.1 “Lower Order Row Address Bits” The lowest 4 to 6 bits of the row (column) address are used to provide up to 64 bytes of fast page mode transfer address. The number of address bits required to control these transfers depends on the width of the data bus (see A.5.8).

A. 5.10.2 "More DRAM
Row Address Decoding for Accessing Banks "If only one single bank of DRAM is used, the width of the row address used is the DRAM used
Depends on the type of. For applications that require more memory than can generally be provided by one single DRAM bank, it is possible to configure a wider row address and thus one single DRAM bank. It is possible to decode some row address bits for selection.

Note: Row addresses are extracted from the center of the internal address. To select the DRAM bank,
If some bits of the row address are decoded,
All possible values for these "bank select bits" must select one bank of DRAM. Otherwise, holes will be left in the address space.

A. 5.11 "DRAM Interface Activation" In the present invention, there are two ways to make all output signals on the DRAM interface high impedance. That is, a method of setting a DRAMenable register and a method of using a DRAMenable signal. Both these registers and signals must be logic ones for the driver on the DRAM interface to work. If either is low, the interface is high impedance.

Note: On-chip data processing cannot be terminated if the DRAM interface is high impedance. Therefore, if the chip attempts to access the DRAM, an error occurs and at the same time the interface is high impedance.

In accordance with the present invention, the ability to make the DRAM interface high impedance tests other devices or, if the spatial decoder (or temporal decoder) is not in use, the spatial decoder (or temporal decoder). Provided to use a DRAM controlled by. It is not intended to allow memory sharing by other devices during normal operation.

A. 5.12 "Refresh" Unless disabled by filling in the no-refresh register, the DRAM interface will be at the interval / RAS / before the refresh cycle / at the interval determined by the register refresh interval.
Use CAS / to automatically refresh the DRAM. Here, / signal name / indicates an inverted signal of the signal.

The value in the refresh interval specifies the interval between refresh cycles within the period of 16 decoder clock cycles. The values in the range 1.255 are configurable. The value 0 is automatically loaded after reset and
The DRAM interface forces continuous refresh cycles to be performed (once enabled) until a valid refresh interval is configured. It is recommended that the refresh interval be configured only once after each reset.

When / reset / is asserted, at the same time, D
The RAM interface makes it impossible to refresh the DRAM. However, the reset time required by the decoder chip is short enough, so that
Reset these decoder chips, and then
It should be possible to reconfigure the DRAM interface before the contents of the DRAM are corrupted.

A. 5.13 “Signal strength” The drive strength of the DRAM interface output is 3
It is user configurable using bit registers CAS strength, RAS strength, addr strength, DRAM data strength, and OEWE strength. This 3-bit value MSB selects either the fast or slow edge rate. The two less significant bits constitute the output for different load capacitances.

The default strength after reset is 6,
And this is GND when loaded by 24pF
Configure the output so that it requires about 10 ns to drive the signal between V and VDD.

[0553]

[Table 28] If the output is properly configured for the load it is driving, it will meet the AC electrical characteristics specified in Tables 32 through 35. When properly configured, each output will roughly match its load, and thus minimal overshoot will occur after a signal transient.

A. 5.14 “Electrical Specifications” All the information presented in this section is merely an example of the present invention and is presented by way of example and not necessarily in a limiting sense.

[0555]

[Table 29] Table 29 only shows maximum ratings for the examples.
Especially for this embodiment only, stresses below the values shown in this table must be used to ensure operational reliability.

[0556]

[Table 30]

[0557]

[Table 31] A. 5.14.1 “AC characteristics”

[0558]

[Table 32]

[0559]

[Table 33]

[0560]

[Table 34]

[0561]

[Table 35] When reading from DRAM, the DRAM interface samples the DRAM data [31: 0] as the / CAS / signal rises.

[0562]

[Table 36] Section A. 6 "Microprocessor Interface (MPI)" Standard byte width microprocessor interface (M
PI) is used for all chips in the video decoder chip set. However, it should be understood by one of ordinary skill in the art that microprocessor interfaces of other widths can be used as well. MPI operates synchronously with various decoder chip clocks.

A. 6.1 "MPI signal"

[0564]

[Table 37] A. 6.2 "MPI electrical specifications"

[0565]

[Table 38]

[0566]

[Table 39]

[0567]

[Table 40] A. 6.2.1 “AC characteristics”

[0568]

[Table 41]

[0569]

[Table 42] A. 6.3 "Interrupt" In the present invention, "event" is a term used to indicate an on-chip condition that a user may wish to observe. The event may indicate an error, or may be beneficial to the user's software.

There are two single bit registers associated with each interrupt, or "event". These are the condition event register and the condition mask register.

A. 6.3.1 “Condition Event Register” The condition event register is a 1-bit read / write register, and its value is set to 1 by the condition generated in the circuit. The register is set to 1 even if the condition is only temporary and now has gone by. Therefore, the register is guaranteed to remain set to 1 (or the entire chip is reset) until reset by the user's software.

* Registers are set to zero by filling in the value one.

* By entering zero in the register,
Keep the register unchanged.

* The register must be set to zero by the user software before this condition will ever occur again.

* Register will be reset to zero on reset.

A. 6.3.2 “Condition Mask Register” The condition mask register is a 1-bit read / write register that enables generation of an interrupt request when the corresponding condition event register is set. If a condition event has already been set, a 1 is entered in the condition mask and an interrupt request is issued immediately.

* A value of 1 enables interrupts.

* Registers are cleared to zero upon reset.

Unless stated otherwise, blocks are
The operation is stopped after generating an interrupt request, and then restarted after either the condition event or the condition mask register is cleared.

A. 6.3.3 "Event and Mask Bits" Event bits and mask bits are always placed as a group in corresponding bit positions in consecutive bytes in the memory map (see Table 61, Table 141). this is,
To identify which event generated the interrupt,
Allows interrupt service software to use the value read from the mask register as a mask as a mask for the value in the event register.

A. 6.3.4 "Chip Event and Mask" Each chip has one single global event bit that summarizes the event activity on the chip. The chip event register provides the OR of all on-chip events that have a 1 in their mask bits.

A 1 in the chip mask bit enables the chip to generate an interrupt. A 0 in the chip mask bit prevents any on-chip event from generating an interrupt request.

Filling the chip event from 1 to 0 has no effect. The chip is cleared only if all events (activated by writing a 1 to either mask bit) are cleared.

A. 6.3.5 "irq signal" The / irq / signal is asserted if both the chip event bit and the chip event mask are set.

The / irq / signal is an active low open collector output that requires an off-chip pull-up resistor. If active, / irq / output is 1
Pulled down by impedance less than 00 ohms.

It should be appreciated that for most applications a pull-up resistor of about 4 kilohms will be suitable.

A. 6.4 "Access to Registers" A. 6.4.1 "Accessing Stopping Circuit" Most registers in the present invention can be modified only if the block to which the register is associated is stopped. Therefore, groups of registers are generally associated with access registers.

A value of 0 in an access register indicates that the group of registers associated with that access register should not be modified. Writing a 1 to the access register requests that the block be stopped. However, the block does not have to stop immediately, and the access register of the block holds the value 0 until the block stops.

Therefore, the user software should wait (after filling in a 1 to request access) until a 1 is read from the access register. When the user enters a value in the configuration register with the access register set to 0,
The results are uncertain.

A. 6.4.2 "A register holding an integer" The least significant bit of any byte in the memory map is the bit associated with the signal data [0].

A register holding an integer value greater than 8 bits is divided over any two or four consecutive byte positions in the memory map. The byte order is "big endian" as shown in FIG. However, no assumptions are made regarding the order in which the bytes that are not made for the order are written into the multibyte register.

Unused bits in the memory map except unused bits in the register holding the signed integer return 0 when read. In this case, the most significant bit in the register is sign extended. For example, a signed 12-bit register is sign-extended to fill a 16-bit memory map location (2 bytes). A 16-bit memory map location holding an unsigned 12-bit integer returns 0 from its most significant bit.

A. 6.4.3 “Keyhole Address Location” In the present invention, the memory map location with low access frequency is located behind the “keyhole”. A "keyhole" has two registers associated with it: a keyhole address register and a keyhole data register.

The keyhole address specifies a location within the expanded address space. A read or write operation to the keyhole data register will access the location specified by the keyhole address register.

The associated keyhole address register increments after accessing the keyhole data register. Random access within the expanded address space is possible only by writing a new value value into the keyhole address register for each access.

A chip according to the present invention may have more than one "keyhole" memory map. There is no interaction between different keyholes.

A. 6.5 "Special Register" A. 6.5.1 Registers or bits described as "unused registers""notused" are locations in the memory map that were not used to implement the device. Generally, a value of 0 can be read from these positions. Entering 0 in these positions has no effect.

As will be appreciated by those of ordinary skill in the art, in order for this type of product to be compatible with future variants, the user's software may read from an unused location. It is recommended that the values given should not be relied upon. Similarly, this type of location should be avoided or
Should be set to the value 0.

A. 6.5.2 "Reserved Registers" Similarly, registers or bits described as "Reserved" in the present invention have no documented effect on the way the device operates.

A. 6.5.3 "Test Register" Furthermore, registers or bits described as "test register" control various aspects of the device under test. Therefore, these registers are not used for the standard application of the device and need not be accessed by standard device configuration and control software.

Section A. 7 "Clocks" In accordance with the present invention, many different clocks are identifiable in a video decoder system. Clock example,
It shows in FIG.

The data is resynchronized (on-chip) to each new clock as it is fed between different clock regimes within the video decoder chip set.
In the present invention, the maximum frequency of any input clock is 30 MHz. However, one of ordinary skill in the art should, of course, understand that other frequencies including frequencies above 30 MHz can be used as well. In each chip, the microprocessor interface (MPI) operates asynchronously to the chip clock. Furthermore, the image formatting unit can generate a low frequency audio clock synchronized with the picture rate of the decoded moving image. Therefore, this clock can be used to provide audio / video synchronization.

A. 7.1 "Spatial Decoder Clock Signal" The spatial decoder has two different (and potentially asynchronous) clock inputs.

[0604]

[Table 43] A. 7.2 "Time Decoder Clock Signal" The time decoder has only one clock input.

[0605]

[Table 44] A. 7.3 "Electrical Specifications"

[0606]

[Table 45]

[0607]

[Table 46] A. 7.3.1 "CMOS Level" The clock input signal is a CMOS input. VIHmin is VD
D is about 70% and VILmax is about 30% of VDD
Is. The values shown in Table 46 are the values of VIH and VIL under the worst condition of VDD, respectively. VDD = 5.0 ± 0.2
5V A. 7.3.2 "Clock Stability" In the present invention, the clock used to drive the DRAM interface and the chip-to-chip interface is derived from the input clock signal. The timing specifications for these interfaces assume that the input clock timing is stable above ± 100 ps.

Section A. 8 As the density of "JTAG" circuit boards increases, it is not possible to test connections between components by conventional methods, such as in-circuit testing using "bed-of-nails" techniques. It gets harder and harder. The Joint Test Action Group (JTAG) was formed in an attempt to solve access problems and standardize methodologies. The culmination of this group of work is the "Standard Test Access Port and Boundary Scan Architecture", which is currently adopted by the IEEE as standard 1149.1. Spatial and temporal decoders comply with this standard.

The standard utilizes a boundary scan chain that connects each digital signal pin on the device in series. The test circuit is responsive (transparent) under normal operating conditions, but in test mode the boundary scan chain allows the test pattern to be shifted in, and
Allows you to feed pins on the device. The resulting signal appearing on the circuit board at the input to the JTAG device can be scanned and checked by relatively simple test equipment. This method allows inter-component connections to be tested as well as areas of logic on the circuit board.

All JTAG operations are performed through the Test Access Port (TAP), which has five pins. The trst (test reset) pin resets the JTAG circuit to ensure that the device does not power up in test mode. tck
The (test clock) pin clocks the clock serial test pattern to the tdi (test data input) pin,
It is then used to clock from the tdo (test data output) pin. Finally, the operating mode of the JTAG circuit is set by clocking the sequence of bits of interest to the tms (test mode select) pin.

The JTAG standard is extensible to provide additional features at the chipmaker's discretion. The spatial and temporal decoders have 9 user instructions (instructions) including 3 JTAG required instructions. The additional instruction (Extra Line Instruction) is
It allows a certain degree of internal device testing to be performed and provides additional external testing flexibility. For example, all device outputs can be made to float by a simple JTAG sequence.

For details on the available functions and how to use the JTAG port, please refer to the individual JTAG application note.

A. [0613] 8.1 “JTAG Pin Connection in Non-JTAG Systems”

[0614]

[Table 47] A. 8.2 "Level of conformance to IEEE 1149.1" 8.2.1 “Rules” The following clauses are noted, but all regulations are relevant.

[0615]

[Table 48]

[0616]

[Table 49] A. 8.2.2 “Recommended conditions”

[0617]

[Table 50]

[0618]

[Table 51] A. 8.2.3 “Permission conditions”

[0619]

[Table 52] Section A. 9 "Spatial Decoder" * 30 MHz, Operation * Decoder MPEG, JPEG, and H.264. 26l * Coded data rate up to 25Mb / s * Video data rate up to 21MB / s * Flexible chroma sampling format * All JPEG baseline decoding * Glueless DRAM interface * Single + 5V power supply * 208-pin PQFP package * Maximum consumption Power 2.5W * Independent coded data and decoder clock * Uses standard page mode DRAM Spatial decoders are available in various JPEG, MPEG and H.264.
Configurable VLSI decoder chip for use in H.261 picture and video decoding applications.

In the minimum configuration, which does not use off-chip DRAM, the spatial decoder is one single chip, high speed JPEG decoder. DRAM
To allow the spatial decoder to decode the JPEG encoded video. 720 x 480, 3
0 Hz, 4: 2: 2 “JPEG moving image” can be decoded in real time.

By using the temporal decoder, the spatial decoder can 26l, and MPEG (also JPE
G) can be used to decrypt. 704 x 480,
30 Hz, 4: 2: 0 MPEG moving pictures can be decoded.

It should be restated that the above values are merely examples and are given as examples and are not necessarily intended to be limiting and are typical values for one embodiment according to the present invention. Make a note. Therefore, one of ordinary skill in the art will appreciate that other values and / or values can be used.

A. 9.1 "Spatial decoder signal"

[0624]

[Table 53]

[0625]

[Table 54]

[0626]

[Table 55]

[0627]

[Table 56]

[0628]

[Table 57] A. 9.1.1 ““ nc ”unconnected pins” In Table 57, the pins labeled nc represent pins that are not currently used. These pins must be left unconnected.

A. 9.1.2 “VDD and GND Pins” As will be understood by one of ordinary skill in the art, all installed V and GND pins must be connected to their respective power sources. I won't. Proper operation of the device cannot be guaranteed unless all V and GND pins are used correctly.

A. 9.1.3 “Connecting Test Pins for Normal Operation” Nine pins of the spatial decoder are reserved for internal testing.

[0631]

[Table 58] A. 9.1.4 “JTAG Pin for Normal Operation” Section A.6. See 8.1.

A. 9.2 “Spatial Decoder Memory Map”

[0633]

[Table 59]

[0634]

[Table 60]

[0635]

[Table 61]

[0636]

[Table 62]

[0637]

[Table 63]

[0638]

[Table 64]

[0639]

[Table 65]

[0640]

[Table 66]

[0641]

[Table 67]

[0642]

[Table 68]

[0643]

[Table 69]

[0644]

[Table 70]

[0645]

[Table 71]

[0646]

[Table 72]

[0647]

[Table 73]

[0648]

[Table 74]

[0649]

[Table 75]

[0650]

[Table 76]

[0651]

[Table 77]

[0652]

[Table 78]

[0653]

[Table 79]

[0654]

[Table 80]

[0655]

[Table 81] Section A. 10 "Coded Data Entry" The system according to the invention must know which video standard is being entered for processing. From now on, the system can accept either pre-existing tokens or raw byte data. This raw byte data can then be placed in the token by the start code detector.

Therefore, the coded data and the configuration token can be supplied to the spatial decoder via the following two routes.

* Coded Data Input Port * Microprocessor Interface (MPI) The choice of route to use depends on the application and system environment. For example, if the data rate is low, both for controlling the decoder chip set and for multiplexing the system bitstream,
It is possible to use one single microprocessor. In this case, it is also possible to input coded data via MPI. Alternatively, if the coded data rate is high, the coded data must be provided via the coded data port.

Depending on the application, it may be appropriate to use a mixture of MPI and coded data port inputs.

A. [0659] A. 10.1 "Coded Data Port"

[0660]

[Table 82] The coded data port according to the invention can be used in two modes: token mode and byte mode.

A. 10.1.1 “Token Mode” In the present invention, when the byte mode is low,
The coded data port operates as a token port in the normal way and accepts tokens under the control of code valid and code accept. See Section A.3 for details on the electrical operation of this interface. See 4.

Signal byte mode is data [7: 0],
At the same time as coding extn and coding valid, that is,
Sampled on the rising edge of the coded clock.

A. 10.1.2 “Byte Mode” However, if the byte mode is high, then one byte of data is data valid under the control of 2-wire interface control signal coding enable and coded accept [7:
0]. In this case, the coded extn
Is ignored. The bytes are then assembled into data tokens on-chip until the input mode is changed.

1) The first word of the token supplied in token mode (head) 2) The last word of the token supplied (coded ex)
tn goes low) 3) First byte of data supplied in byte mode. New data tokens are automatically created on-chip.

A. 10.2 “Supply Data via MPI” Tokens can be provided to the spatial decoder via MPI by accessing the coded data input register.

A. 10.2.1 "Token Entry via MPI" The coded data registers of the present invention are grouped into 2 bytes in the memory map to allow efficient data transfer. Eight data bits, coded data [7: 0] are located in one place, and control registers, coded busy, enable mpi input, and coded extn are located in another place. Table 62,
See Table 63).

When configured for token input via MPI, the current token is extended with the current value of coded extn each time a value is entered in the coded data [7: 0]. . Software is responsible for setting the coded extn to 0 before the last word of every token is written into the coded data [7: 0].

For example, the data token is coded ex
1 for tn and 0x04 for coded data [7: 0]
Start by filling in. This new data token start is then fed to the spatial decoder for processing.

The current token is extended each time a new 8-bit value is entered in the coded data [7: 0]. The coded extn only needs to be accessed again if the current token is terminated, for example to introduce another token. The last word of the current token is coded ext followed by the entry of the last word of the current token into the coded data [7: 0].
Indicated by a 0 entry in n.

[0670]

[Table 83] Prior to the coded data [7: 0], each time the coded busy has to be examined to determine if the interface is ready to accept more data.

A. 10.3 "Switching between input modes" It is possible to change the data input mode dynamically, provided that appropriate precautions have been taken. In general, the transfer of tokens via any one route must be completed before switching mode.

[0672]

[Table 84] With the first byte supplied in byte mode, the data token header can be generated on-chip. All further bytes transferred in byte mode will be attached to this data token in the future until the input mode changes. Recall that a data token can contain as many bits as needed.

MPI register bit, coded busy,
And the signal, Code Accept, indicates at which interface the spatial decoder is willing to accept data. Correct observation of these signals ensures that no data is lost.

A. 10.4 "Acceptance Rate of Coded Data" In the present invention, the input circuit supplies the token to the start code detector (see Section A.11). The start code detector analyzes the data serially as data token bits. The normal processing rate of the detector is
One bit per clock cycle (of the coded clock). Therefore, the decoder is coded clock
1 byte of coded data is decoded every 8 cycles of. However, additional processing cycles may be required, such as when a non-data token is provided or when a start code appears in the coded data. If an event of this kind occurs, the start code detector will be unable to accept any further information for a short time.

After the start code detector, the data is provided to the first logic coded data buffer. If this buffer is full, the start code detector will be unable to accept any more information.

Therefore, more coded data (or other token) is accepted at the coded data port or via MPI. in this case,
The start code detector remains unacceptable for further information. This is indicated by the states of signal coded accept and register coded busy.

The use of Coded Accept and / or Coded Busy ensures to the user that no coded information is lost. However,
One of ordinary skill in the art, if the spatial decoder cannot accept the data, the system either buffers the newly arrived coded data (or
You should understand that (stopping the arrival of new data).

A. 10.5 "Coded Data Clock" In accordance with the present invention, the coded data port, input circuitry, and other functions in the spatial decoder are coded cloc.
controlled by k. Furthermore, this clock can be asynchronous to the main decoder clock. The data transfer is synchronized with the on-chip decoder clock.

Section A. 11 "Start Code Detector" A. 11.1 "Start Code" As is well known in the art, MPEG and H.264. The H.261 coded video stream contains an identifiable bit pattern called a start code. JPEG
In, similar functionality is provided by the marker code. The start / marker code identifies a significant portion of the syntax of the coded data stream. The analysis of the start / marker code performed by the start code detector is the first step in purging coded data. The start code detector is the first block in the spatial decoder that follows the input circuit.

The start / marker code pattern is designed to be identifiable without decoding the entire bitstream. Therefore, the start / marker code pattern can be used to aid error recovery and decoder start in accordance with the present invention. The start code detector detects an error in the coded data structure,
It also provides functions to help the decoder start.

A. 11.2 "Start Code Detector Registers" As previously discussed, a number of start code detector registers are always used by the start code detector. Accessing these registers is therefore unreliable when the start code detector processing the data is processing the data. It is the user's responsibility to ensure that the start code detector is stopped before accessing the register.

Register start coded detector
access is used to stop the start code detector and thus allow access to its registers. The start code detector stops after generating an interrupt.

Regarding when it is possible to start the start code search and discard of all data modes,
There are further constraints. For these constraints,
A. 11.8, and A. It has been described in 11.5.1.

[0684]

[Table 85]

[0685]

[Table 86]

[0686]

[Table 87]

[0687]

[Table 88]

[0688]

[Table 89]

[0689]

[Table 90] A. 11.3 “Start Code to Token Conversion” The function of the start code detector in normal operation is to identify the start codes in the data stream and then convert these codes into the corresponding start code tokens. It is to be. In the simplest case, the data is supplied to the start code detector as one long single data token. The output of the start code detector is a number of shorter data tokens interrupted by the start code token.

Alternatively, in accordance with the present invention, the input data to the start code detector can be split into a number of shorter data tokens. Regarding how to split the coded data into data tokens, where n is an integer:
There are no restrictions other than that each data token must contain 8xn bits.

Other tokens can be fed directly to the input of the start code detector. In this case, the token is supplied to the other stages of the spatial decoder via the start code detector without processing. These tokens can be inserted only in the coded data, just before the start code position.

A. 11.3.1 "Start Code Format" According to the start code detector of the present invention, three different start code formats are recognized. It is constructed via the Register Start Code Detector Coding Standard.

[0693]

[Table 91] A. 11.3.2 “Start Code Token Equivalent” When a start code is detected, the start code detector examines the value associated with that start code and generates the appropriate token. In general, tokens are
Named after the related MPEG syntax. However, one of ordinary skill in the art will understand that tokens follow an additional naming format. The currently selected coding standard constitutes the relationship between the start code value and the generated token. This relationship is shown in Table 92.

[0694]

[Table 92] A. 11.3.3 "Extension of Coding Standards" Coding standards provide a number of mechanisms that allow data embedding into data streams whose use is not currently defined according to the coding standards. It can also be considered as a specific "user data" application that provides additional usability for a specific manufacturer, and can instead be considered as "extended data".
The coding standard authority has reserved the right to use extended data to add functionality to the coding standard in the future.

Two obvious mechanisms are used. J
In PEG, a marker code precedes a block of user and extension data. However, H. At 261, the "additional information" indicated by the additional information bit in the coded data is inserted. Both of these techniques can be used in MPEG.

According to the invention, MPEG / user and extension data preceded by a start / marker code
The JPEG block can be detected by the start code detector. H. "Additional information" of 261 / MPEG
Is detected by the Huffman decoder of the present invention.
A. See 14.7 “Receiving Additional Information”.

The register, the discext extension data, and the discard user data can configure the start code detector to discard the user data and the extension data. If this data is not discarded by the start code detector, it can be accessed when this data reaches the video demux. A. l4.6, "Receiving User and Extended Data"
Please refer to.

The spatial decoder of the present invention supports the JPEG baseline function. The non-baseline function of JPEG is considered as extended data by the spatial decoder. Therefore, for non-baseline JPEG, all JPEs that precede the data
The G marker code is treated as extended data.

A. 11.3.4 "JPEG Table Definition" JPEG supports downloaded Huffman and quantizer tables. In JPEG data, the definitions of these tables are preceded by the marker codes DNL and DQT. The start code detector generates tokens DHTMARKER and DQTMMARKER when these marker codes are detected. These tokens indicate to the video demux that the subsequent data tokens contain coded data describing the Huffman or quantizer table (using the format described in JPEG).

A. 11.4 "Error Detection" The start code detector is capable of detecting specific errors in the coded data and provides some functionality to allow recovery by the decoder after an error is detected. (See A.11.8, "Search for Start Code").

A. 11.4.1 “Illegal JPEG Length Counts” Most JPEG marker codes have 16 associated with them.
It has a bit length count field. This field indicates how much data the data is associated with this marker code. Length counts of 0 and 1 are illegal. Inappropriate length occurs only if it follows a data error. In the present invention, if the illegal length count mask is set to 1, the occurrence of an improper length will generate an interrupt.

It is presumed that recovery from errors in JPEG data requires additional application specific data because it is difficult to retrieve the start code in JPEG. Paragraph A. 11.8.1
reference).

A. 11.4.2 “Start / Marker Code Duplication” In the present invention, the start code duplication occurs only when the data error follows. FIG. 73 shows an MPEG array byte overlap start code. In this case, the start code detector first looks at a pattern that looks like a picture start code. The start code detector then sees that this picture start code overlaps the group start. Therefore, the start code detector produces a duplicate start event. Furthermore,
If the overlapping start mask is set to 1, the start code detector will generate an interrupt and a stop.

It is impossible to determine which of the two start codes is correct and which cause of the data error. However, the start code detector according to the invention discards the first start code,
Then, after the duplicate start code event has been serviced, "as the second start code is correct" the second
Continue decoding the start code of. If a sequence of duplicate start codes is present, the start code detector discards all these codes except the last code (generates an event for each duplicate start code).

Similar errors are possible in unaligned byte systems (H.261, or perhaps MP).
EG). In this case, Ignore / non-aligned states must be considered as well. FIG. 74 shows that the first start code found is an array byte, but
An example of overlapping with a non-sequence start code is shown. Ignore
When non-aligned is set to 1, the second duplicate start code is treated as data by the start code detector. Therefore, the duplicate start code event does not occur. This hides any data communication errors that may have occurred. If the ignor non-align is set to 0, the start code detector looks at the second unaligned start code. And see that it overlaps with the first start code.

A. 11.4.3 “Unrecognized Start Code” The start code detector can generate an interrupt if an unrecognized start code is detected (if unrecognized start mask = 1). The start code value that caused this interrupt can be read from the register start value.

The start code value 0xB4 (sequence error) is used in the MPEG decoder system to indicate a channel error or media error.
For example, this start code may be inserted into the data by the ECC circuit when an uncorrectable error is detected.

A. 11.4.4 "Event Generation Sequence" In the present invention, a particular coded data pattern (often indicating an error condition) causes one or more of the above error conditions to occur within a short period of time. Therefore, the following is the order in which the start code detector examines the coded data for error conditions.

1) Non-sequenced start code 2) Duplicate start code 3) Unrecognized start code Therefore, if the non-sequenced start code overlaps with the other start code, the first event generated is , Associated with non-sequenced start codes. After this event has been serviced, the start code detector continues to operate, detecting a duplicate start code after a short period of time.

After all non-sequenced and duplicate start codes have been tested, the start code detector simply attempts to recognize the start code.

A. 11.5 Decoder Start and Stop The start code detector provides the capability to allow the current decoding task to be completely completed and to allow a new task to be started.

The use of these techniques with JPEG-encoded animation is limited because the data segment can contain values that emulate a marker code (see A.11.8.1).

A. 11.5.1 Clear End of Decoding Once the data for the current picture is complete, the start code detector can be configured to generate interrupts and stops. This is done by setting stop after picture = 1 and stop after picture mask = 1.

Once the end of the picture supplies the start code detector, a flash token is generated (A.
11.7.2), an interrupt is generated and the start code detector is stopped. Note that the picture just completed is decoded in the usual way. In some applications it may be appropriate to detect the flash arriving at the output of the decoder chip set. The arrival of this output indicates the end of the current video sequence. For example, the display could stick to the last picture output.

When the start code detector is stopped, there may also be data from the "old" video sequence that has been "trapped" in a buffer implemented by the user between the media and the decode chip. Register discar
Setting the dall data causes the spatial decoder to consume and discard this data. This continues until either the flash token reaches the start code detector or the discardall data is reset via the microprocessor interface.

After discarding any data from the "old" sequence, the decoder is now ready to begin operation according to the new sequence at this stage.

A. 11.5.2 "Discard all modes (dis
card) Start time ”Immediately after 1 is written in the discardall data register, discarding of all modes starts. If this is done when the start code detector is actively processing data, the results are unpredictable.

All mode discards can safely be initiated after any of the start code detector events (such as a non-aligned start event) generate an interrupt.

A. 11.5.3 "Start New Sequence" If it is unknown where in a coded data a new coded video sequence starts, a start code search mechanism can be used. This mechanism discards any unnecessary data that precedes the start of the sequence. A. See 11.8.

A. 11.5.4 "Jumps between sequences" This section presents applications of some of the above techniques. The purpose is to "jump" from one part of one coded video sequence to another. In this example, the filing system only allows access to "blocks" of data.
This block structure can also be derived from the sector size of the disc or block error correction system. Therefore, the positions of the entrance and the exit in the coded moving image data may not be related to the filing system block structure.

Stop after picture and disca
The rdall data mechanism allows the discarding of unnecessary data from old video sequences. After the end of the last filing system data block, inserting a flush token resets the discardall data mode. The start code search mode is then used to discard any data in the next data block preceding the appropriate entry.

A. 11.6 "Byte Alignment" As is well known in the art, different coding schemes have quite different views regarding byte alignment of start / marker codes in a data stream. For example,
H. 261 sees the communication as a serial bit. Therefore, there is no concept regarding byte alignment of the start code. By setting Ignore / Non-Align = 0, the start code detector can detect start codes of arbitrary bit alignment. By setting non-aligned start mask = 0, start code non-aligned interrupts are suppressed.

However, in contrast to this, JPEG
Designed for computing environments where byte alignment is guaranteed. Therefore, the marker code should be detected only if the bytes are aligned. The encoding standard is JP
If configured as EG, the ignor non-aligned register is ignored and the misaligned start event is never generated. However, it is recommended to set Ignore / Non-Align = 1 and Non-Align Start Mask = 0 in order to guarantee compatibility with future products.

On the other hand, MPEG was designed to meet the needs of both communication (serial bit) and computer (byte oriented) systems. The start code in the MPEG data should normally be aligned bytes. However, the standard is designed so that a serial bit search is possible with respect to the start code (the MPEG bit pattern does not look like a start code unless it is a start code, whatever the bit alignment). ). Therefore, MPEG decoders can be designed to tolerate the loss of byte alignment in serial data communications.

When a non-aligned start code is found, it usually means that a communication error has already occurred.
If the error is a "bit-slip" in a serial bit communication system, the data containing this error has already been supplied to the decoder. It is speculated that this error may cause other errors in the decoder. However, new data arriving at the start code detector can subsequently be decoded after this alignment is lost.

An interrupt can be generated when an unaligned start code is detected by setting Ignore / Non-Align = 0 and Non-Align Start Mask = 1. Responsibility depends on the application. All subsequent start codes are unaligned (until the byte alignment is restored). Therefore, it may be appropriate to set the non-aligned start mask = 0 after the byte alignment is lost.

[0727]

[Table 93] A. 11.7 "Automatic token generation" In the present invention, most of the tokens output by the start code detector directly reflect the syntax elements of various picture and video coding standards. In addition to these "natural" tokens, some useful "invented" tokens are generated. Examples of tokens that can claim their ownership are picture ends, and coding standards. In addition, tokens are introduced to remove some syntactic differences between coding standards and to tidy up under error conditions.

This automatic token generation is performed after serial analysis of the coded data (see Figure 70 "Start Code Detector"). Thus, the system equally responds to the tokens provided directly to the input of the spatial decoder via the start code detector and the tokens generated by the start code detector following the detection of the start code in the encoded data.

A. 11.7.1 "End of picture indication" In general, coding standards do not explicitly send the end of a picture. However, if the start code detector of the present invention detects information indicating that the current picture is complete,
Generate a picture end token. The tokens that generate the picture end are: Sequence start, group start, picture start, sequence end, and flash.

A. 11.7.2 "Stop after end of picture option" If the Register Stop After Picture is set, the start code detector will stop after the picture end token is supplied. However, the flash token is inserted after the picture end to "push" the end of the coded data through the decoder and reset the system. A. 11.
See 5.1.

A. 11.7.3 “Introduction of sequence start for H.261” 261 has no syntax element equivalent to sequence start (see Table 92). If the insert sequence start register is set, the start code detector guarantees that there is one sequence start token before the next picture start, i.e., the start code detector before the picture start. If no sequence start is seen at, one token is introduced. If one already exists, sequence start is not introduced.

This function must not be used with MPEG or JPEG.

A. 11.7.4 “Set coding standard for each sequence” Every sequence start token leaving the start code detector is always preceded by a coding standard token. This token is loaded with the current coding standard of the start code detector. this is,
Set the coding standard for the entire decoder chip set for each new video sequence.

A. 11.8 "Search for Start Codes" The start code detector according to the present invention can be used to search for a particular type of start code in a coded data stream. This allows the decoder to restart decoding from a specified level within the syntax of some coded data (after discarding any data that precedes it). The application for this is shown below.

* Start decoder after jumping to coded data file at unknown location (eg for random access).

* Search for known points in the data to aid recovery after a data error.

For example, Table 94 shows M searched for start code searches for different configurations.
Indicates a PEG start code. Equivalent H. 26l and JP
The EG start / marker code can be found in Table 92.

[0738]

[Table 94] If the start code search is filled with a non-zero value, the start code detector will begin to discard all incoming data until the specified start code is detected. The start code search register is then reset to 0 and normal operation continues.

The start code search starts immediately after a non-zero value is written in the start code search register. If this is done while the start code detector is actively processing the data, the results are unpredictable. Therefore, before starting the start code search, the start code detector must be stopped so that no data is processed. The start code detector will always be in this condition if any start code detector event (such as a misaligned start event) has just completed generating an interrupt.

A. 11.8.1 "Limitations When Using Start Code Search With JPEG" Most JPEG marker codes have 16 associated with them.
It has a bit length count field. This field indicates the length of the data segment associated with the marker code. This segment may contain values that emulate the marker code. In normal operation, the start code detector does not look for start codes in these segments of data.

If a random access to some JPEG encoded data "lands" on such a segment, the start code search mechanism cannot be used reliably. In general, JPEG-encoded video requires additional external information to identify the entrance for random access.

Section A. 12 "Decoder start control" A. 12.1 "Overview of Decoder Start" In a decoder, the video display is usually displayed with a short delay after the coded data is first made available. During this delay, the coded data accumulates in a buffer within the decoder. This prefilling of the buffer ensures that the buffer will never be empty during the decoding period, and thus that the decoder will be able to decode new pictures at regular intervals. .

Generally, two functions are required to properly start the decoder. First, there must be a mechanism to measure how much data was supplied to the decoder. Second, there must be a mechanism to prevent the display of new video streams. The spatial decoder of the present invention comprises a bit counter near its input to measure how much data has arrived, and to prevent a new video stream being output from starting. And an output gate near its output.

The complexity of controlling these functions is 3
There are two levels.

* Output gate is always open * Basic control * Advanced control By keeping the output gate always open, the picture output is as early as possible after the coded data starts to arrive at the decoder. Will start. this is,
Suitable when decoding still images or when the display is being delayed by some other mechanism.

The difference between basic control and advanced control is how many short video streams can fit in the decoder's buffer at any given time. Basic control is sufficient for most applications. However, advanced control allows user software to assist the decoder in managing the start of several very short video streams.

A. 12.2 "MPEG Video Buffer Verifier" MPEG describes "Video Buffer Verifier (VBV)" for constant data rate systems. VBV
The information allows the decoder to prefill its buffer before starting to display the picture. Again, this prefill ensures that the decoder's buffer is never empty during the decoding period.

In summary, each MPEG picture is vbv
It has a delay parameter. This parameter is the first
Defines the time required for the "ideal decoder" coded data buffer to be filled with coded data before the picture in is decoded. Noting the start delay for the first picture, it will automatically meet the requirements of all subsequent pictures.

Therefore, MPEG defines a requirement for a start as a delay. However, in constant bit rate systems, this delay can be easily converted to a bit count. This is the basis of the start control operation of the spatial decoder of the present invention.

A. 12.3 “Definition of Streams” In this application, the term stream is used to avoid confusion with the term sequence in MPEG. Therefore, the stream means the amount of moving image data of interest to the application. Therefore, one stream can be multiple MPEG sequences or one picture.

The decoder start function described in this chapter is the VBV for the first picture in one stream.
Relevant to meeting requirements. It automatically meets the requirements of subsequent pictures in the stream.

A. 12.4 “Start Control Register”

[0753]

[Table 95]

[0754]

[Table 96] A. 12.5 "Normally Open Output Gate" The output gate can be configured to remain open. This configuration is appropriate if the still image is being decoded, or if some other mechanism is available to manage the start of the video decoder.

The configuration required after reset is shown below (accessing the start control logic is obtained by writing a 1 to the startup access).

* Set off-chip queue = 1 * Set enable stream = 1 * Ensure that all decoder start event mask registers are set to 0, disabling interrupts in these registers (this Is the default state after reset). (See A.12.7.1 for an explanation of why this holds the output gate open).

A. 12.6 "Basic Operation" In the present invention, basic control of start logic is sufficient for most MPEG video applications. In this mode, the bit counter communicates directly with the output gate. The output gate automatically closes when the end of the video stream supplies the output gate, as indicated by the flash token. The gate remains closed until one enable is provided by the bit counter circuit when the stream reaches its start bit count.

The configuration required after reset is shown below (filling in 1 to the start-up access gives access to the start control logic).

Set bit count prescale to approximately predictive range of coded data.

* Counter flushed tooearly mask = 1 to enable detection of this error condition
Set.

The following two interrupt service routines are required.

* Video demultiplexing service to obtain the vbv delay value for the first picture in each new stream.

* Counter fl for reacting to this condition
used to early service.

When decoding a vbv delay for a new video stream (ie, when the first picture arrives at the video demultiplexer after a flash), this video demultiplexer (aka the video parser) can generate an interrupt. . The interrupt service routine must calculate and fill in the appropriate value for the bit count target. When the bit counter reaches this target, it inserts an enable in a short queue between the bit counter and the output gate. When the output gate opens, it removes one enable from this queue.

A. 12.6.1 “Start of a new stream immediately after another stream ends” As an example, the MPEG stream which is about to end is A
, And the MPEG stream about to start is called B. A flush token must be inserted after A's end. This token pushes the last piece of coded data through the decode, alerting various sections of the decoder that a new stream is expected.

Normally, the bit counter has been reset to zero, and A has already met its start condition. After the flush, the bit counter begins counting the bits in stream B. When the video demux finishes decoding the vbv delay from the first picture in stream B, an interrupt is generated to allow the bit counter to be configured.

The gate closes as the flash marking the end of stream A passes through the output gate. The gate, until B meets its starting conditions,
Keep closed. Depending on a number of factors, such as the start delay for stream B and buffer depth, it is possible that B has already met its start condition when the output gate is closed. In this case, one enable is waiting in the queue and the output gate opens immediately. Otherwise, stream B must wait until it meets its start requirements.

A. 12.6.2 The capacity of the queue located between the "Short Stream Consecutive" bit counter and the output gate to allow the three individual video streams to meet their start conditions and to finish the decoding Is sufficient to allow waiting for the previous stream. In the present invention, this condition occurs only if a very short stream is being decoded, or if the off-chip buffer is very large compared to the picture format being decoded.

In FIG. 78, stream A is being decoded and the output gate is open. Streams B and C meet their start conditions and are completely contained within the buffer managed by the spatial decoder. Stream D is still arriving at the input of the spatial decoder.

The enable for streams B and C is in the queue. Thus, stream A is complete and B can start immediately. Similarly, C can immediately follow B.

1 if A is still passing through the output gate if D meets its start goal
One enable is added to the queue to fill it.
If no enable is removed from the queue by the time D's end passes through the bit counter (ie, A is passing through the output gate), the new stream goes through the bit counter. I can't start. Since the output gate is open so that B can supply it, the coded data is held up at the input until A is complete and one enable is removed from the queue.

A. 12.7 “Advanced Operation” In accordance with the present invention, advanced control of the start logic is described in A.
Allows user software to extend the length of the enablement queue without restriction, as described in 12.6 "Basic Operations". The video decoder is A.
This level of control is required only if it must accommodate a series of short video streams that are longer than those described in 12.6.2 "Sequence of Short Streams".

In addition to the configuration configuration required for the basic operation of the system, after reset, the following configuration is required (by writing a 1 to the start up access to the start control logic: Access is obtained).

* Set off-chip queue = 1.

* If one enable is removed from the queue, set AcceptEnableMask = 1 to enable interrupts.

* If the stream's bit count target is reached, set target met mask = 1 to enable interrupts.

The following two additional interrupt service routines are required.

* Accept Enable Interrupt * Target Achievement Interrupt Target Achievement Interrupt When a target occurs, the service routine must add one enable to its off-chip enablement queue.

A. 12.7.1 "Operation of Output Gate Logic" When 1 is written in the enable stream register, one enable is loaded into the short queue.

When a flush (marking the end of the stream) supplies the output gate, the gate is closed. If there is one enable available at the end of the queue, the gate will open and generate an accept enable event. Accept enable mask is 1
If set to, an interrupt can be generated and one enable is removed from the end of the queue (register enable stream is reset).

However, if the accept enable mask is set to zero, no interrupt is generated following the accept enable event, and the enable is not removed from the end of the queue. This mechanism is described in A. It can be used to keep the output gate open, as described in 12.5.

A. 12.8 "Bit Count" The bit counter starts counting after the flash token has passed this counter. This flash token indicates the end of the current video stream. In this regard, the bit counter is the bit count tar.
Continue counting until the bit count target set in the get register is met. Next, a Goal Match event is generated, the bit counter resets to zero, and waits for the next flush token.

Further, when the bit counter reaches the maximum count (255), it stops incrementing.

A. 12.9 "Bit Count Prescale" In the present invention, in order to increment the bit counter once, 2 (bit count prescale + 1) th power x 512 bits are required. In addition, the bit count prescale is
It is a 3-bit register that can hold a value between 0 and 7.

[0785]

[Table 97] Since some elements of the video stream have already been tokenized (eg start code), the bit count is approximate and thus includes non-data tokens.

A. 12.10 "Premature Counter Flash" If a flash token arrives at the bit counter before the bit count goal is reached, an event that could cause an interrupt is generated (counter flushhe).
d too early mask = 1). If an interrupt is generated, the bit counter circuit is stopped, preventing further data input. It is the responsibility of the user software to decide when to open the output gate after this event occurs. The output gate can be made to open by entering 0 as the bit count target. These situations should occur only if one tries to decode a video stream that lasts only a few pictures.

Section A. 13 "Buffer Management" Spatial Decoder has two logical data buffers:
It manages the coded data buffer (CDB) and the token buffer (TB).

The CDB buffers coded data between the start code detector and the input of the Huffman decoder.
This provides a buffer for low data rate coded video data. TB is the output of the Huffman decoder and the spatial video decoding circuit (inverse modeler, quantizer, and DC).
Buffer the data to and from the input of T). This second logical buffer allows processing time to include expansion to accommodate processing of pictures with varying amounts of data.

Both buffers are physically held in one single off-chip DRAM array. The addresses for these buffers are generated by the buffer manager.

A. 13.1 "Buffer Manager Register" The Spatial Decoder Buffer Manager is intended to be configured only once immediately after the device is reset. During normal operation, there are no requirements for buffer manager reconfiguration.

After the reset is removed from the spatial decoder, the buffer manager is stopped (buffer manager access register set to 1) and awaits configuration. After the register is configured, the buffer manager access can be set to 0 and decoding can begin.

During the operation of the buffer manager, the registers used by the buffer manager are in most cases inaccessible with high reliability. If you want to access every buffer manager register, you must first make one buffer manager access.
Must be set to. This makes it mandatory to follow the wait protocol until the value 1 is read from the buffer manager access. To monitor the condition of the buffer, for example cdbfull, and cdbempty
When polling registers such as, the time taken to secure and relinquish access must be considered.

[0793]

[Table 98] A. 13.1.1 “Value of pointer of buffer manager” Generally, data is a space decoder and off-chip DRAM.
Is transferred as a 64-byte burst between (using the fast page mode of DRAM). All buffer pointers and length registers refer to these blocks of 64 bytes (512 bits) of data. Therefore, the buffer manager's 18-bit registers are 25
6k block linear address space (ie 128M
Describe b).

The 64-byte transfer is independent of the width of the DRAM interface (8, 16 or 32 bits).

A. 13.2 "Use Buffer Manager Registers" The spatial decoder buffer manager has two sets of registers that define two similar buffers. The buffer limit register defines the physical upper limit of memory space. All addresses are calculated modulo this number.

Within the limits of available memory, the range of each buffer is two registers: buffer base (cdbbase and tbbase), and buffer length (cdblength and tblengt).
h). All the registers described so far must have been configured before the buffer can be used.

The current state of each buffer can be seen in four registers. The buffer read registers (cdbread and tbread) indicate the offset from the buffer base from which the next data is read. Buffer number register (cdbnumb
er, and tbnumber) indicate the amount of data currently held by the buffer. Status bit cdb
full, tbfull, cdbempty, and tb
empty indicates whether the buffer is full or empty.

A. As stated in 13.1.1, the unit for all the above registers is a 512 bit data block. Therefore, to obtain the number of bits in the coded data buffer, cdbnumbe
It must be multiplied by the value 512 read from r.

A. 13.3 "Zero Buffer" Still image applications (e.g., using JPEG) where "real time" requirements do not apply do not require large off-chip buffers supported by the buffer manager. In this case, the DRAM interface can be configured (by filling the register zero buffer with 1) to ignore the buffer manager to provide the 128 bit stream on-chip FIFO for the coded data buffer and the token buffer.

Furthermore, the zero buffer option may be suitable for applications that operate with small picture formats at low data rates.

Note: Register Zero Buffer is a DRAM
Being part of the interface, it must be set only during the post reset configuration of the DRAM interface.

A. 13.4 "Buffer Operation" The data transfer performed through the buffer is controlled by the handshake protocol. Therefore, it is guaranteed that no data error will occur if the buffer is full or empty. If the buffer is full, the circuits that are trying to send data to the buffer are stalled until space is available in the buffer. If the buffer continues to be full, further processing stages located upstream of the buffer will stall until the spatial decoder can no longer accept data at its input port. Similarly, if the buffer is empty, circuits that try to remove data from the buffer will stall until the data is available.

A. The location and size of the coded data and token buffers are specified by the buffer base and length registers, as shown in 13.2. It is the responsibility of the user to configure these registers and to ensure that the memory usage between the two buffers is consistent.

Section A. 14 "Video Demultiplexer" A video demultiplexer, also called a video parser, completes the task of converting coded data into tokens started by a start code detector. The video demultiplexer has four main processing blocks,
That is, parser state machine, Huffman decoder (IT
(Including OD), macroblock counter, and ALU.

The parser or state machine follows the syntax of the coded video data and points to other units. The Huffman decoder converts variable length coded (VL0) data to integers. The macroblock counter keeps track of the picture section currently being decoded. The ALU performs the necessary arithmetic calculations.

A. 14.1 “Video Demultiplexer Register”

[0807]

[Table 99]

[0808]

[Table 100]

[0809]

[Table 101]

[0810]

[Table 102]

[0811]

[Table 103] A. 14.1.1 "Register Loading and Token Generation" A number of registers in the video demultiplexer hold values that are directly related to the parameters normally conveyed in coded picture / video data. For example, the horizontal pixel register corresponds to MPEG sequence header information, horizontal size, and JPEG frame header parameter, X. These registers are loaded by the video demultiplexer when the coded data is decoded. Similarly, these registers are also associated with tokens.

For example, it is related to the horizontal pixel register, the token, and the horizontal size. The token is generated by the video demultiplexer when the encoded data is decrypted (or shortly thereafter). Similarly, tokens can be fed directly to the input of the spatial decoder. In this case, the value of the token constitutes the video demultiplexer associated with the token.

[0813]

[Table 104]

[0814]

[Table 105]

[0815]

[Table 106]

[0816]

[Table 107]

[0817]

[Table 108]

[0818]

[Table 109]

[0819]

[Table 110]

[0820]

[Table 111]

[0821]

[Table 112] A. 14.2 "Picture Structure" The picture dimensions in the present invention are described to the spatial decoder as two different units: pixels and macroblocks. Both JPEG and MPEG
Communicates picture dimensions as pixels. By conveying the dimensions as pixels, we determine the area of the buffer that contains valid data. In this case, the area may be smaller than the total buffer size. Delivering the dimensions as macroblocks determines the size of the buffer needed by the decoder. The macroblock dimension must be determined by the user from the pixel dimension. The spatial decoder registers associated with this information are: horizontal pixels, vertical pixels, horizontal macroblocks, and vertical macroblocks.

Spatial decoder register, number of blocks h
n, vn, max h, max v, and max c
The component id is a macroblock composite (J
Specify the minimum encoding unit in PEG. Each is a 2-bit register that can hold a value in the range 0 to 3. max compo
All registers except lent id specify block counts from 1 to 4. For example, register ma
If x h holds 1, the width of the macroblock is 2 blocks. Similarly, max component
id specifies the number of different color components involved.

[0823]

[Table 113] A. 14.3 "Huffman Table" A. 14.3.1 "JPEG Style Huffman Table Description" In the present invention, the Huffman table description is provided to the spatial decoder via the format used by JPEGO to convey the table description between the encoder and the decoder. . Each table description has two elements, namely BITS
And HUFFVAL. The user is guided to the JPEG specification for a complete description of how the table is coded.

A. 14.3.1.1 "BITS" BITS is a table showing values that describe how many different symbols are coded using each length of VLC.
Each entry is one 8-bit value. JPEG is V
A maximum of 16 bits is allowed as the LC length. Therefore, there are 16 entries in each table.

BITS [0] describes how many different 1-bit VLCs exist, while BITS [1].
Describes how many different 2-bit VLCs exist, and so on.

A. 14.3.1.2 "HUFFVAL" HUFFVAL is a table of 8-bit data values arranged in increasing order of VLC length. The size of this table is VL
It depends on the number of different symbols that can be coded by C.

The JPEG specification describes in more detail how the Huffman coding table is coded or decoded for this format.

A. 14.3.1.3 “Token Configuration” In the JPEG bitstream, the Huffman table description used to encode the AC and DC coefficients is D.
The HT marker precedes. Start code detector is DHT
When recognizing a marker, the detector generates a DHT marker token and then places the Huffman table description in the data token that follows (see A.11.3.4).

The construction of the AC and DC coefficient Huffman tables within the spatial decoder can be achieved by supplying the data and DHT marker tokens at the input of the spatial decoder, while at the same time the spatial decoder is configured for JPEG operations. . This mechanism can be used to construct the DC coefficient Huffman table needed for MPEG operation, provided that the table is downloaded and the spatial decoder coding standard is JP
Must be set to EG.

[0830]

[Table 114] A. 14.1.4 "Configuration with MPI" AC and DC coefficient Huffman tables can likewise be entered directly into registers via MPI. Table 104
Please refer to.

* Register dc bits0 [15: 0]
And dc bits1 [15: 0] hold the BITS values for tables 0x00 and 0x01.

* Register ac bits0 [15: 0]
And ac bits1 [15: 0] hold the BITS values for tables 0x10 and 0x11.

* Register dc huffval0 [1
1: 0] and dc huffval1 [11: 0] are
Holds HUFFVAL values for tables 0x00 and 0x01.

* Register ac huffval0 [16
1: 0] and ac huffval1 [161: 0]
Holds the HUFFVAL values for Tables 0x10 and 0x11.

A. 14.4 "Structures for Different Standards" Video demultiplexers are MPEG, JPEG, and H.264. 261 requirements are supported. The encoding standard is
CODINC generated by the start code detector
It is automatically constructed by the STANDARD token.

A. 14.4.1 "H.261 Huffman table" All Huffman tables needed to decode the H.261 are stored in ROM in the spatial decoder, and more specifically in the parser state machine of the video demultiplexer, requiring user intervention. Not.

A. 14.4.2 “Structure of H.261 picture” H.261 is defined as supporting only two picture formats, CIF and QCIF. The picture format in use is transmitted in the PTYPE section of the bitstream. When this data is decoded by the spatial decoder, it becomes H
261 picture type register and picture type token. Moreover, all picture and macroblock configuration registers are automatically configured.

The information in the various registers is placed in their associated tokens, see Tables 109-112), and this ensures that other decoder chips (eg, time decoders) are properly configured. Guarantee.

A. 14.4.3 "MPEG Huffman Table" The majority of the Huffman coded tables needed to decode MPEG are kept in ROM in the spatial decoder (again, restate that it is in the parser state machine). It does not require user intervention. The tables required to decode the DC coefficients of intra macroblocks are an exception. Two tables are needed, one for chroma and one for luma. These tables must be constructed by the user software before decoding begins.

[0840]

[Table 115] Table 117 shows the sequence of tokens needed to construct the DC coefficient Huffman table in the spatial decoder. Alternatively, writing this information to the register via MPI will achieve the same result.

Register dc huffn controls which DC coefficient Huffman table is used with each color component.
Table 116 shows how it must be configured for MPEG operation. This is MPI
Can be done directly, or by using an MPEG DCH table token.

[0842]

[Table 116]

[0843]

[Table 117]

[0844]

[Table 118]

[0845]

[Table 119]

[0846]

[Table 120] A. 14.4.4 "Structure of MPEG picture" The structure of a macroblock defined for MPEG is described in H.264.
The structure used by 261 is the same. The picture dimensions are encoded in the encoded data.

For standard 4: 2: 0 operation, the characteristics of the macroblock must be configured as indicated in Table 115. This can be done by writing in a register as instructed or by supplying the equivalent token to the input of the spatial decoder (see Tables 109-112).

The method used to construct the picture dimension is application dependent. If the picture format is known before the start of decoding, Table 115
The picture configuration register shown in can be initialized with appropriate values. Alternatively, the picture dimension can be decoded from the coded data, and
It can be used to construct a spatial decoder. In this case, the user has to support the parser error MPEG sequence. A. 14.8
See "Changes in the MPEG Sequence Layer".

A. 14.4.5 "JPEG" Within Baseline JPEG, there are numerous encoder options that significantly change the complexity of the control software required to operate the decoder. In general, a spatial decoder
It is designed to require minimal support when the following conditions are met: * The number of color components per frame is less than 5 (Nf ≦ 4).

A. 14.4.6 "JPEG Huffman Table" In addition, JPEG allows Huffman coded tables to be downloaded. These tables are used when decoding the VLC that describes the coefficients. Two tables are allowed per scan to decode the DC coefficients, and two tables are also allowed for the AC coefficients.

There are three different types of JPEG files: interchange format, shortened format for compressed image data, and shortened format for tabular data. The interchange format file contains both the compressed image data and all the table definitions (Huffman, quantization, etc.) needed to decode the image data. The shortened image data format file omits the table definition. The shortened table format file contains only table definitions.

The Spatial Decoder accepts all three formats. However, if all the required tables are defined, only the shortened image data file can be decrypted. The other two types of JPEG
This definition can be implemented via either of the files, or, alternatively, the table can be set up by user software.

If each scan uses a different set of Huffman tables, the definition of the table is placed (by the encoder) in the coded data before each scan. These definitions are automatically loaded by the spatial decoder for use during the scan and subsequent scans.

To improve the performance of Huffman decoding, certain commonly used symbols are specially cased. These symbols are DC coefficients of size 0, block A
The end of the C coefficient and the run of 16 zero AC coefficients. Values for these special cases must be placed in the appropriate registers.

A. 14.4.6.1 “Table selection” registers dc huff n and ac huff n
Controls which AC and DC coefficient Huffman tables are used with which color components. During the JPEG operation, these relationships are represented in the scan header syntax TDj.
And Taj fields.

A. 14.4.7 “Structure of JPEG Pictures” There are two distinct levels of baseline JPEG decoding supported by the spatial decoder: up to 4 components per frame (Nf ≦ 4), and per frame. The level is larger than the four components (Nf> 4). Nf
When using> 4, the required control software becomes more complex.

A. 14.4.7.1 “Nf ≦ 4” For the frame component specification parameter included in the JPEG frame header, refer to Table 115 which constitutes the macroblock configuration register when decoding these). No user intervention is required as all the specifications needed to decode the four different color components are defined.

For more details on the options provided by JPEG, please study the JPEG specification. Similarly, regarding the JPEG picture format, see Section A. It is described briefly in 16.1.

A. 14.4.7.2 "JPEG with more than 4 components" Spatial Decoder supports up to 256 different color components (JPEG
It is possible to decrypt a JPEG file including the maximum number allowed by However, when trying to decode more than four color components, additional user intervention is required. In JPEG, only a maximum of 4 components is allowed in every scan.

A. 14.4.8 "Non-standard variant" As already mentioned, the spatial decoder is based on JPEG and MP.
Supports more picture formats than defined by EG.

JPEG limits the minimum coding unit so that it does not contain more than 10 blocks per scan. The spatial decoder uses the number of blocks hn, the number of blocks v n, max h, and max.
This limit does not apply to spatial decoders, since any minimum coding unit that can be described by v can be processed.

In MPEG, see Table 115, which is defined only for 4: 2: 0 macroblocks). However, the spatial decoder can process three other component macroblock structures (eg, 4: 2: 2).

A. 14.5 "Video Events and Errors" The video demultiplexer can generate two types of events: parser events and Huffman events. For a description of how to handle events and interrupts, see A. See 6.3 “Interrupts”.

A. 14.5.1 "Huffman Event" A Huffman event is generated by a Huffman decoder. The Huffman event and the event indicated in the Huffman mask determine whether an interrupt is generated. If the Huffman mask is set to 1,
An interrupt is generated and the Huffman Decoder stops. The register Huffman error code [2: 0] holds a value indicating the cause of the event.

If the Huffman Event is filled with 1 after servicing the interrupt, the Huffman Decoder will
Try to recover from the error. Similarly, if the Huffman mask is set to 0 (mask interrupts and do not stop the Huffman decoder), the Huffman decoder automatically attempts to recover from the error.

A. 14.5.2 "Parser Events" Parser events are generated by parsers. This event is designated in the parser event. After that, whether or not an interrupt is generated is determined by the parser mask. If the parser mask is set to 1, an interrupt is generated, and
The parser stops. The register parser error code [7: 0] holds a value indicating the cause of the event.

After servicing the interrupt, if the Huffman event is filled with 1, the Huffman decoder
Try to recover from the error. Similarly, if the Huffman mask is set to 0 (mask interrupts and do not stop the Huffman decoder), the Huffman decoder automatically attempts to recover from the error.

After servicing the interrupt, if the parser event is filled with 1, the parser will resume operation. If the event indicates a bitstream error, the video demultiplexer attempts to recover from the error.

If the parser mask is set to 0, the parser sets its event bit but does not generate an interrupt or stop. The parser continues operation and attempts automatic recovery from the error.

[0870]

[Table 121]

[0871]

[Table 122]

[0872]

[Table 123]

[0873]

[Table 124]

[0874]

[Table 125]

[0875]

[Table 126] Each standard uses a different subset of defined parser error codes.

[0876]

[Table 127]

[0877]

[Table 128] A. 14.6 "Reception of user data and extension data" MPEG and JPEG use a similar mechanism for embedding user data and extension data. The data is preceded by the start / marker code. If the application is not interested in the data,
The start code detector can be configured to delete this data (see A.11.3.3).

A. 14.6.1 "Data Source Identification" Parser Event, ERREXTENSION Token,
And ERR user token indicate the arrival of EXTENSION data or user data token at the video demultiplexer. If these tokens are generated by the start code detector (A.11.
(See 3.3), these tokens have the values of the start / marker code that caused the start code detector to generate the token) (see table 92). This value can be read by reading the rom revision register while supporting parser interrupts. The video demultiplexer remains stopped until the parser event is filled with 1 (see A.6.3, "Interrupts").

A. 14.6.2 "Reading Data" EXTENSION data, and user data tokens are expected to be immediately followed by data tokens with extended or user data. The video demultiplexer will receive E
Generate either RREXTENSION data or ERR user data parser event. The first byte of the data token is r while servicing the interrupt.
It can be read by reading the om revision register.

The continuous state of the video demultiplexer register determines the behavior after the event is cleared. If this register holds the value 0, then any remaining data in the data token is consumed by the video demultiplexer and no event is generated. If the continuation is set to 1, an event is generated as each byte of extension or user data arrives at the video demultiplexer. This continues until either the data token is exhausted or the continuation is set to zero.

Notes: 1) The first byte of extended / user data is always presented via the romrevision register, regardless of the state of continuation.

2) There is no event indicating that the last byte of the extension / user data has been read.

A. 14.7 “Receiving additional information” H.261 and MPEG allow information that extends the coding standard to be embedded within pictures and groups of blocks (H.261) or slices (MPEG). This mechanism differs from the mechanism for extended data and user data (described in Section A.14.6). Since the start code does not precede the data, it can be deleted by the start code detector.

H.H. Parser events ERR PSPARE and ERR GSPA during the 261 operation period
The RE directs the detection of this information. Corresponding event during MPEG operation is ERR EXTRA
Picture and ERR EXTRA slice.

When a parser event is generated, the first byte of additional information is presented via the register romrevision.

The continuous state of the video demultiplexer register determines the behavior after the event is cleared.
If you keep this register value 0, any remaining additional information will be consumed by the video demultiplexer and no event will be generated. If the continuation is set to 1, an event is generated as each byte of additional information arrives at the video demultiplexer. This continuation continues until either the additional information is exhausted or the continuation is set to zero.

Notes: 1) The first byte of extended / user data is always presented via the romrevision register, regardless of the state of continuation. 2) There is no event indicating that the last byte of extended / user data has been read.

A. 14.7.1 "FIELD INF
Generation of O token "During MPEG operation, register field i
If nfo is set to 1, then any ex
The first byte of the tra information picture is placed in the FIELD INFO token.
This behavioral aspect is not covered by the MPEG standardization activities. The definition of the FIELD INFO token is shown in Tables 4-11.

If the field info is set to 1, no parser event is generated for the first byte of the extra information picture. However, extra information
An event is generated for every subsequent byte of the on picture. If there is only one single byte of the extra information picture, no parser event will occur.

A. 14.8 “Changes in the MPEG Sequence Layer” The MPEG sequence header describes the following characteristics of the video that is about to be decoded.

* Horizontal and vertical size * Aspect ratio of pixels * Picture rate * Coded data rate * Buffer size of video buffer verifier If any of these parameters change when the spatial decoder decodes the sequence header Generates a parser event ERR MPEG sequence.

A. 14.8.1 "Picture Size Change" If the picture size changes, the user's software will read the values in holizpels and vert vertical pels and then register holizma
croblocks, and vertmacroblock
Compute a new value to load into ks.

Section A. 15 "Spatial Decoding" In accordance with the present invention, spatial decoding occurs between the output of the token buffer and the output of the spatial decoder.

There are three main units responsible for spatial decoding: the inverse modeler, the inverse quantizer, and the inverse discrete cosine transformer. At the input of this section (from the token buffer), the data token contains a run and level representation of the quantized coefficients. Output (inverse D
In CT), the data token is 8 of pixel information.
Includes x8 blocks.

A. 15.1 "Inverse Modeler" The data token in the token buffer contains information about the value of the quantized coefficient and the number of zeros between the displayed coefficients. The inverse modeler expands the information about runs of zeros so that each data token contains 64 values. At this stage, the value in the data token is a quantized coefficient.

The inverse modeling process is the same regardless of the coding standard currently in use. No configuration is required.

To better understand all the requirements for the modeling and de-modeling functions, the reader can explore any picture coding standard.

A. 15.2 "Dequantizer" In an encoder, the quantizer splits the DCT output to reduce the resolution of the DCT coefficients. The function of the inverse quantizer in the decoder is to multiply these quantized DCT coefficients in order to restore them to approximate their original values.

A. 15.2.1 “Outline of Standard Quantization Plans” There are important differences in the quantization plans used by each of the different coding standards. To gain a detailed understanding of the quantization schemes used by each standard, the reader should study the relevant coding standard documents.

The register iq coding standard configures the operation of the inverse quantizer to meet the requirements of different standards. In normal operation, this coded register is automatically loaded by the Coding Standard Token. For more information on the configuration of coding standards, see Section A.3. See 21.1.

The main difference between quantization schemes is the source of the numbers by which the quantized coefficients are multiplied. These are outlined below. Further, although not described here, there are detailed differences in the required arithmetic operations (rounding, etc.).

A. 15.2.1.1 "Overview of H.261 IQ" At 261, a single "scale factor" is used to adjust the coefficients. The encoder is
This scale factor can be changed periodically to adjust the data rate created. A slightly different rule applies to "DC" coefficients within intra-coded blocks.

A. 15.1.2.1.2 “Overview of JPEG IQ” Baseline JPEG allows pictures containing up to four different color components in each scan. A 64-entry quantization table can be designated for these four color components. Each entry in these tables can be used as a "scale" factor for one of the 64 quantized coefficients.

The values for the JPEG quantization table are coded JP
It is included in the EG data and automatically loaded into the quantization table.

A. 15.2.1.3 “Overview of MPEG IQ” MPEG is based on H.264. 261 and JPEG quantization technique. As with JPEG, four quantization tables with 64 entries each are available. However, the usage of the table is completely different.

The two "types" of data are intra and non-intra. Different tables are used for each data type. Two "default" tables are MPEG
Defined by One is for intra data,
The other is Table 130 and Table 131 for non-intra data). These default tables must be filled in the quantization table memory of the spatial decoder before MPEG decoding is possible.

Similarly, MPEG allows two "download" quantization tables. One is for use with intra-data and another is for use with non-intra-data. The values for these tables are
It is included in the MPEG data stream and is automatically loaded in the quantization table memory.

The values output from the table are modified by the scale factor.

A. 15.2.2 “Inverse Quantizer Register”

[0910]

[Table 129] In the present invention, the iq access register must be set before the quantization table memory can be accessed. If a table read is attempted while the iq access is set to 0, the quantization table memory returns the value zero.

[0911] A. 15.2.3 Inverse Quantizer Configuration In normal operation, there is no need to configure the inverse quantizer coding standard, as it is automatically configured by the coding standard token.

H.092. The quantizer table is not used for H.261 operations. No special configuration is required. For JPEG operations, the tables needed by the inverse quantizer must be automatically loaded with the information extracted from the encoded data. MPEG operation requires that a default quantization table be loaded. This loading must be done while iq access is set to 1. The values in table 130 are the locations of the dequantizer extended address space 0x00 to 0x3F.
Must be filled in (accessible via keyhole register iq keyhole address and iq keyhole data). Similarly, the values in Table 131 are from the dequantizer extended address space locations 0x40 to 0x.
Must be completed by 7F.

[0913]

[Table 130]

[0914]

[Table 131] A. 15.2.4 Table construction from tokens Instead of constructing the dequantizer tables via MPI, it is possible to initialize these dequantizer tables with tokens. These tokens can be supplied either by a coded data port or MPI.

For the QUANT table token,
Described in Tables 3-11. The token is 4 in the table
It has one 2-bit field tt that specifies which of the (0-3) locations is defined by the token. For MPEG operations, the default definitions in Tables 0 and 1 need to be loaded.

A. 15.2.5 "Quantization Table Values" For both JPEG and MPEG, the quantization table entries are 8-bit numbers. Values from 255 to 1 are legal. A value of 0 is illegal.

A. 15.2.6 Quantization Table Number Order Quantization table values are used in a "zigzag" scan order (see encoding standard). The table should be considered as a one-dimensional array with 64 values (not an 8x8 array). The table entry at the lower address corresponds to the lower frequency DCT coefficient.

If the quantization table value is carried by a QUANT table token, the first value after the token header is the table entry for the "DC" coefficient.

A. 15.2.7 “Inverse Quantizer Test Register”

[0920]

[Table 132] A. 15.3. "Discrete Inverse Cosine Transform" The inverse discrete transform processor of the present invention is based on CCITT Recommendation H.264.
26l, meets the requirements specified in IEEE specification P1180, and meets the requirements described in the current revision of MPEG.

The discrete inverse cosine transform process is the same regardless of which coding standard is used.
No user configuration required.

There are two events associated with the inverse discrete transform processor.

[0923]

[Table 133] In order to better understand the DCT and inverse DCT functions, the reader can explore any picture coding standard.

Section A. 16 "Connect to Spatial Decoder Output" The output of the Spatial Decoder is a standard token port for a 9-bit wide data word. See Section A.3 for more information on the electrical behavior of the interface. Four
Please refer to.

The token at the output depends on the encoding standard being adopted. As an example, this section of the disclosure considers the output of a spatial decoder when configuring for JPEG operation. Since the temporal decoder does not modify the resulting token sequence of JPEG decoding, this section describes the token sequence observed at the output of the temporal decoder during the JPEG operation.

However, MPEG and H.264 are not supported. 261 both require the use of temporal decoders. MPEG and H.264.
For information on connecting to the time decoder output when configured for H.261 operation, see Section A.2. See 19. In addition, this section identifies which tokens are available at the output of the spatial decoder and which are most useful in the circuit design for displaying that output. There are other tokens, but their output is not required for display and will not be considered here. The following are the items to focus on in this section.

* A method for identifying the start and end of a sequence.

* A method for identifying the start and end of a picture.

* How to identify when to display a picture.

* Method of identifying picture data arrangement location in display A. 16.1 JPEG Picture Structure This section gives an overview of the characteristics of the JPEG syntax. See the coding standard for details.

JPEG provides various mechanisms for encoding individual pictures. JPEG does not attempt to describe how to code a collection of pictures together to provide a mechanism for coding a video.

The spatial decoder according to the present invention is a JPEG
Support sequential modes of operation as the baseline of
There are three main levels in this syntax: image, frame, and scan. One sequential image contains only one single frame. One frame is
It may contain between 1 and 256 different image (color) components. These image components can be collected in the scan in various ways. 1 for each scan
Image components between 4 and 4 can be included (see Figure 90 "Overview of Sequential Structure as JPEG Baseline"). If the scan contains one single image component, it is a non-interleaved scan; if it contains multiple image components, it is an interleaved scan. A frame can contain a mixture of interleaved cans and non-interleaved scans. The number of scans a frame can contain is determined by the 256 limit on the number of image components a frame can contain. Within one interleaved scan, the data is MPEG and H.264. The minimum coding unit (M
CU). These MOUs are arranged in raster order within the picture. The MCU in non-interleaved scan is one single 8x8 block. Again, the MCU is raster organized.

The spatial decoder can easily decode JPEG data containing 1 to 4 different color components. Files that describe a large number of components can be similarly decoded. However, some reconstruction may be required during the scan to accommodate the next set of components to be decoded.

A. 16.2 "Token Sequence" The JPEG marker code is converted by the start code detector into a similar MPEG designated token, see Table 92, and Fig. 91 "Tokenized JPEG Pictures").

Section A. 17 "Time Decoder" * 30, MH7 operation.

* MPEG and H.264. Provides temporal decoding for a H.261 video decoder.

* H. 261 CIF and QCIF formats.

* MPEG moving image resolution up to 704 × 48
0, 30 Hz, 4: 2: 0.

* Flexible chroma sampling format.

[0940] * MPEG picture sequence can be rearranged.

* Glueless DRAM interface.

* Single + 5V power supply.

* 208-pin PQFP package.

* Maximum power consumption is 2.5W.

* Use standard page mode DRAM.

The Temporal Decoder is a companion chip to the Spatial Decoder, and 261 and M
It provides the temporal decoding required by PEG.

Temporal decoders are MPEG, H.264 and H.264. 26
1. Perform all prediction formation functions required by 1. One single 4Mb DRAM (eg 512k x
8), the temporal decoder uses CIF and QCIF
H. The H.261 video can be decrypted. 8 Mb D
Using RAM (eg, two 256k x 16),
704 × 480, 30 Hz, 4: 2: 0 MPEG video can be decoded.

Intra-coding scheme (eg JPEG)
Does not need a time decoder. If the temporal decoder is included in a multi-standard decoder, it delivers the decoded JPEG picture to its output. Note: The above values are merely an example of one embodiment of the present invention and are not necessarily intended to be limiting. It should be understood that other values and ranges can be used without departing from the scope of the present invention.

A. 17.1 “Time Decoder Signal”

[0950]

[Table 134]

[0951]

[Table 135]

[0952]

[Table 136]

[0953]

[Table 137]

[0954]

[Table 138] A. 17.1.1 “Unconnected Pins“ nc ”” The pins labeled “nc” in Table 138 are pins that are not currently used in the invention and are reserved for future products. . These pins must be left unconnected. These pins are VDD, GND,
Do not connect to each other or any other signal.

A. 17.1.2 “VDD and GND Pins” It should be understood that all VDD and GND pins must be connected to the appropriate power supply. If all VDD and GND pins are not used correctly, The device does not work properly.

A. 17.1.3 Test Pin Connections for Normal Operation Nine pins of the time decoder are reserved for internal testing.

[0957]

[Table 139] A. 17.1.4 [JTAG Pin for Normal Operation] Section A. See 8.1.

[0958]

[Table 140]

[0959]

[Table 141]

[0960]

[Table 142]

[0961]

[Table 143]

[0962]

[Table 144]

[0963]

[Table 145]

[0964]

[Table 146]

[0965]

[Table 147]

[0966]

[Table 148] Section A. 18 "Time Decoder Operation" A. 18.1 "Data Input" The input data port of the time decoder is a standard token port with a 9-bit wide data word. In most applications this is directly connected to the output token port of the spatial decoder. For more information on the electrical behavior of this interface, see Section A.2. Four
See.

A. 18.2 "Automatic Forming" Parameters for encoded video picture formats are automatically loaded into registers in the temporal decoder by tokens generated by the spatial decoder.

[0968]

[Table 149] A. 18.3 "Manually Create" The user must create the application-dependent element (via the microprocessor interface).

A. 18.3.1 "When to Form" A temporal decoder should be formed only when no data processing is taking place. This is the default state after reset is released. The time decoder can be stopped and recreated by writing a 1 to the chip access register. After the formation is complete, 0 must be written to the chip access.

For details on when to form the DRAM interface, see Section A.6. See 5.3.

A. 18.3.2 "DRAM Interface" The timing of the DRAM interface must be formed before it can decode predictively coded video (eg H.162 or MPEG).
Section A. 5 See DRAM Interface.

[0972]

[Table 150]

[0973]

[Table 151] A. 18.3.3 “Picture Buffer Register Number” The picture buffer pointer (18 bits) and component offset (17 bits) registers specify block (8 × 8 byte) addresses, not byte addresses.

A. 18.3.4 "Picture Buffer Storage Allocation" Predictively coded video (H.261 or MP
To decode EG), the temporal decoder has to process two picture buffers. See Section A.3 for more details on how these buffers are used. 18.4 and A. See 18.4.4.

[0975] The user can store a picture buffer pointer (pic
pure buffer 0 and picture bu
You must make sure that you have sufficient memory on each of the fffer 1). Usually one of the picture buffer pointers is set to 0 (ie the bottom of the storage) and the other one is set to point in the middle of the storage space.

A. 18.3.4.1 "Regular formation for MPEG or H.261" 261 and MPEG both use a ratio of 4: 1: 1 between different color components (ie, there are four times as many luminance pixels as pixels in either chrominance component).

A. Component 0 is the luminance component and components 1 and 2 are the chrominance, as documented in 3.5.1, "Component Specific Number".

An example of forming a component offset register is that component 0 begins at the picture buffer pointer,
Set component offset 0 to 0. Similarly, component offset 1 is set to 4/6 of the picture buffer size, and compo
Nent offset 2 could be set to 5/6 of the picture buffer size.

A. 18.3.5 "Picture Sequence Re-ordering" MPEG uses three different picture types: intra (I), expected (P), and bidirectionally interpolated (B). . B-pictures are based on predictions from two pictures: one from the future and one from the past. The order of the pictures is that before the decoding of B-pictures is required,
It is modified at the encoder so that the I and P pictures are decoded from the encoded data.

The picture sequence must be modified before these pictures are displayed. The temporal decoder can provide this picture reordering (by setting the register MPEG reordering = 1). Alternatively, the user may wish to perform picture reordering as part of his display interface functionality. Forming the temporal decoder to provide picture reordering may reduce the video resolution that can be decoded. A. See 18.5.

A. 18.4 "Predictive formation" The prediction forming requirements for H.261 decoding and MPEG decoding are quite different. The coding standard token is automatically formed by the temporal decoder to accommodate different criteria of prediction requirements.

A. 18.4.1 "JPEG Operations" When formed for JPEG operations, no prediction is made because JPEG does not require temporal decoding.

A. 18.4.2 “H.261 Operation” H.261 At 261, prediction is only done from the decoded pictures. The motion vector is only specified as a precision to integer pixel precision. The encoder can specify that a low pass filter be applied to the prediction result.

As each picture is decoded, it is written to the picture buffer in off-chip DRAM so that it can be used in decoding the next picture. The decoded picture appears at the output of the temporal decoder as it is written to off-chip DRAM.

For details regarding prediction and related arithmetic operations, the reading device is described in H.264. Directed to the 261 standard. The temporal decoder of the present invention is based on H.264. 261
Fully comply with the requirements of.

A. 18.4.3 MPEG operation (without reordering) The operation of the temporal decoder is 3 different MPEG operations.
It changes for each picture type (I, P, B). I-pictures do not require further decoding by the temporal decoder, but must be stored in the picture buffer (frame store) for later use in decoding P and B pictures.

Decoding a P picture requires forming a prediction from a previously decoded P or I picture. The decoded P picture is stored in the picture buffer for use in decoding the P and B pictures. MPEG allows motion vectors to be specified with half-pixel accuracy. On-chip filters provide interpolation to support this half-pixel accuracy.

B-pictures may require prediction from both picture buffers. As with P-pictures, half-pixel motion vector resolution accuracy requires on-chip interpolation of picture information. B-pictures are not stored in the off-chip buffer. They are only temporary.

All pictures appear at the output port of the temporal decoder as their code is decoded. Therefore, the picture sequence is the same as in the encoded MPEG data (see upper part of FIG. 94).

For details on prediction and related arithmetic operations, see the proposed MPEG Standard Draft. The temporal decoder of the present invention satisfies these requirements.

A. 18.4.4 “MPEG operation (with reordering)” MPEG operation with picture reordering (MPE)
G reordering = 1), the predictive formation operation is performed as described in section A. 1
Same as in 8.3.3. However, additional data transmission is performed to rearrange the picture sequence.

B-picture decoding is described in Section A.9. As described in 18.4.3. However, I and P pictures are not output when decoded. Instead, it is written to the off-chip buffer (as described above) and read only when the next I or P picture arrives for decoding. A. 18.4.4.1 "Decoder start-up characteristics" The output of the first I picture is delayed until the decoding of the next P (or I) picture is started. This must be taken into account when estimating the start-up characteristics of the video decoder.

A. 18.4.4.2 Decoder Stop Characteristics The temporal decoder depends on the next P or I picture to flush the previous picture from its off-chip buffer (frame store). This has consequences at the end of the video sequence and at the start of a new video sequence. The spatial decoder provides the convenience of creating a "fake" I / P picture at the end of the video sequence in order to flush the last P (or I) picture. However, this "fake" picture is flushed when the next video sequence begins.

The Spatial Decoder provides the option to suppress this "false" picture. This is useful when it is known that the new video sequence will be provided to the decoder immediately after the completion of the old sequence. The first picture in this new sequence will flush the last picture in the previous sequence.

A. 18.5 "Video Resolution" When decoding MPEG, the video resolution that a temporal decoder can support is limited by the memory bandwidth of its DRAM interface. MP
Two cases need to be considered for the EG: MP
There are cases with and without EG picture reordering.

Section A. 18.5.2 and A. 1
8.5.3 discusses the worst case requirements required by the current draft of the MPEG standard. A subset of MPEG with lower memory bandwidth requirements is contemplated. For example, using only integer resolution motion vectors,
Alternatively, B-pictures are not used, significantly reducing memory bandwidth requirements. No analysis is performed here on such a subset.

A. 18.5.1 DRAM Interface Characteristics The number of cycles taken to transfer data across the DRAM interface depends on a number of factors: -DRAM interface timing shaping to adapt to the DRAM used-Data Bus width (8, 16 or 32 bits) -Data transmission type: -Read or write of 8x8 blocks-For prediction to half pixel precision-For prediction to integer pixel precision For information on the detailed formation of the DRAM interface , Section A. See 5, DRAM Interface.

Table 152 shows how many DRAM interface "cycles" are required for each type of data transmission.

[0999]

[Table 152] Table 154 takes the numbers from Table 152 and rates them for "typical" DRAM. In this example, 27 MH
Assume z clock. It should be appreciated that 27 MHz is used here, but is not limited thereto. 11 ticks (102
ns) is required and 6 ticks (56
ns) is required.

[1000] A. 18.5.2 “M without reordering
PEG resolution "Peak memory bandwidth loading occurs when decoding B-pictures. In the "worst case" scenario, a B-frame is formed from predictions from both picture buffers, all predictions are for half-pixel accuracy.

[1001]

[Table 153] Using the example numbers from Table 153, it can be seen that a DRAM interface 3815ns is required to read the data required (via a 32-bit wide interface) for two accurate half pixel precision predictions. The resolution that the temporal decoder can support is determined by the number of these predictions that can be accomplished in one picture time. In this example, the temporal decoder can process 8737 8 × 8 blocks (eg, for 30 Hz video) in one 33 ms picture period.

[1002] Required video format is 704 × 48
If 0, each picture contains 7920 8x8 blocks (taking into account 4: 2: 0 chroma sampling). This video format is available DRA (before considering other factors such as DRAM refresh)
It consumes about 91% of the M interface bandwidth. Therefore, the temporal decoder can support this video format.

[1003] A. 18.5.3 "MPEG Resolution with Reordering" With MPEG picture reordering, the worst case scenario is encountered while P pictures are being decoded. During this time, three loads are done on the DRAM interface: • Morphological prediction • Write back result • Read previous P or I picture.

[1004] Using the numerical examples from Table 152, one can find the number of times needed for each of these tasks when a 32-bit wide interface is available.
Predictive formation requires 1907 ns / n, while read and write require 991 ns each, for a total of 3889 ns. This is the time decoder 33
Allow 8485 8x8 blocks to be processed during the ms period.

Therefore, processing 704 × 480 video would use almost 93% of the available memory bandwidth (ignoring refresh).

[1006] A. 18.5.4 "H.261" 261 is a CIF (35
It only supports two picture formats, 2x288) and QCIF (172x144). The CIF picture contains 2376 8 × 8 blocks. The only memory operation required is 8x8 block wiring,
Predictive formation with integer precision motion vectors.

Using the numerical example from Table 153 for an 8-bit wide memory interface, 3657 ns are needed to write to each block, while 3963 ns / n is required to predict the formation of one block, It turns out that 7620 ns are required for each block as a whole.
The processing time for one CIF picture is about 18ms, 33m required to support 30Hz video
It is much less than s.

A. 18.5.5 "JPEG" The resolution of JPEG video that can be supported is determined by the capabilities of the inventive spatial decoder or display interface. The temporal decoder does not affect the JPEG resolution.

[1009] A. 18.6 "Events and Errors" A. 18.6.1 "Chip Stop" The present invention requires that writing a 1 to a chip access causes the temporal decoder to stop operation and allow reformation. Once accepted, the normal-time decoder continues to operate until it reaches the end of the current video sequence. After that, the time decoder is stopped.

[1010] When the chip stops, a chip stop event occurs. chip stopped mask = 1
If so, an interrupt occurs.

[1011] A. 18.6.2 "Count Error" The temporal decoder of the present invention comprises an adder that adds a prediction to the error data. If there is a difference between the number of error data bytes and the expected number of data bytes, a count error event occurs.

[1012] When count error mask = 1, an interrupt occurs and prediction formation stops.

[1013] 1 for count error event
Write a to clear the event and let the time decoder advance. This is followed by the DATA token that caused the error. However, the DA that generated the error
The TA token will not be of the correct length (46 bytes). This will easily cause further problems. Thus, count errors should only occur when a significant hardware error occurs.

[1014] Section A. 19 "Connect to Time Decoder Output" The time decoder output is a standard token port with an 8-bit wide data word. See Section A.1 for more information on the electrical behavior of the interface. See 4.

[1015] The token present at the output of the temporal decoder depends on the coding standard used and the MPE
In the case of G, it will depend on whether or not the picture is rearranged. This section identifies which tokens are available at the output of the temporal decoder and which are most useful in designing the circuit to display that output. Other tokens may exist, but they are not discussed here because they do not need to display the output.

[1016] This section focuses on the following points: how the start and end of the sequence can be identified, how the start and end of the picture can be identified, and when the picture is displayed. How to specify: where to place the picture data in the display.

[1017] A. 19.1 "JPEG Output" When decoding JPEG data, the token sequence output by the temporal decoder is the same as that seen at the output of the spatial decoder. Recall that JPEG does not require processing by a temporal decoder. However, the temporal decoder looks at the intra data token for negative values (resulting from the limited arithmetic precision of the IDCT in the spatial decoder),
Replace them with 0.

[1018] For a detailed discussion of the output sequences observed during a JPEG operation, see Section A.1 1
See 6.

[1019] A. 19.2 “H.261 output” 19.2.1 “Start and End of Session” 261 does not signal the start and end of the video stream within the video data. Nevertheless, this is implied by the application. For example, the sequence begins when telecommunication is connected and ends when the line is broken. Thus, the highest layer in the video syntax is the "picture layer".

[1020] The start code detector of the spatial decoder according to the present invention automatically inserts the sequence start and the coding standard token before the first picture start. Section A. 11.7.
3 and A. See 11.7.4.

[1021] At the end of the 261 session (eg, when the line is broken), the user must insert a flash token after the end of the encoded data. This has many effects (see A.31.1): It guarantees that a picture end is emitted to signal the end of the last picture.

[1022] It ensures that the end of the encoded data is pushed through the decoder. A. 19.2.2 “Picture acquisition” Each picture is composed of a hierarchy of elements called a syntax layer. H. When decoding the H.261,
The sequence of tokens at the output of the temporal decoder reflects this structure.

[1023] A. 19.2.1 “Picture Layer” A picture start token is placed before each picture,
The picture end token follows immediately after the picture.
H. 261 naturally does not include the picture end. This token is automatically inserted by the start code detector of the spatial decoder. The picture start token will be followed by a time standard token and a picture type token. The time standard token supports a 10-bit number (of which 5LS is an indicator) when the picture should be displayed.
Only B. 261). H. 261
This point should be investigated by the display system, as the encoder can omit pictures from the sequence (to achieve a low data rate). Picture omissions may be detected by a time reference that increases by one or more between consecutive pictures.

[1024] Next, the picture type token has information about the picture format. The display system can examine this information to detect if the CIF or QCIF picture is decoded.
However, information about the picture format is also available by examining the registers in the Huffman decoder. <Xref: Huffman Decoder Section> A. 19.2.2.2 “Group of block layers” The 261 picture is composed of many “block groups”. Before each of them is placed a slice start token (derived from the group number of H.261 and the group start code). This token carries an 8-bit value that indicates where in the display the group of blocks should be placed. This provides the opportunity for the decoder to have resynchronization again after a data error. Furthermore, it provides the encoder with a mechanism to skip blocks if there is a picture area that does not require additional information to describe the picture. The spatial and temporal decoders each picture contains the correct number of blocks,
By the time the slice start reaches the output of the temporal decoder, the information is effectively redundant, as it already uses this information to ensure that the blocks are in the correct position. Thus, by counting the number of blocks output since the start of the picture, it is possible to calculate where to put the data blocks output by the temporal decoder.

[1025] The number supported by slice start is
H. It is one less than the group of 261 blocks (see the H.261 standard for detailed information). FIG. 103 shows H.264 in CIF and QCIF pictures.
The positioning of the 261 block group is shown.

Note: In the present invention, the block numbering shown is the same as that supported by slice start. This is the H.264 standard for numbering these groups. 261 is different from the standard.

[1027] (Indicates the start of each block group)
There may be other tokens between the slice start and the first macroblock. They do not need to display picture data and can be ignored.

[1028] A. 19.2.2.3 “Macroblock layer” The sequence of macroblocks in each group of blocks is H.264. It is limited by 261. There is no special token information that describes the location of each macroblock. The user must count through the macroblock sequence to determine where to display each piece of information.

[1029] Fig. 105 illustrates a sequence in which macroblocks are arranged in each group of blocks.

[1030] Each macroblock contains 6 data tokens. The sequence of data tokens included in each of the six groups is H.264. It is defined by the macroblock structure of 261. Each data token has one color component of 8x
It should contain exactly 64 data bytes for an 8 pixel area. The color components are contained within a 2-bit number in the Data Token (see Section A.3.5.1).
However, H. The sequence of color components in 261 is limited.

[1031] Each group of data tokens is preceded by a number of tokens that convey information about motion vectors, quantizer scale factors, etc. These tokens do not have to cause the picture to be displayed and can be ignored.

[1032] Each data token contains 64 data bytes for one color component of 8x8. These are raster states.

[1033] A. 19.3 "MPEG Output" MPEG includes many layers in its syntax. These embody concepts such as video sequences and picture groups.

[1034] A. 19.3.1 “MPEG sequence layer” A sequence can have multiple entry points (sequence start), but should have only one exit point (sequence end). When the MPEG sequence header code is decoded, the spatial decoder produces a coding standard token followed by a sequence start token.

[1035] After the sequence start, there will be many tokens of sequence header information describing the video format etc. See the draft MPEG standard for information signaled in the sequence header, and Tables 3-11 for information on how this data is converted into tokens. This information describing the video format is also available in registers within the Huffman decoder.

[1036] This sequence header information may occur several times within an MPEG sequence if the sequence has several entry points.

[1037] A. 19.3.2 “Picture Layer Group” The MPEG group of pictures provides a different type of “entry” point to that provided at the start of the sequence. The sequence header provides information about the picture / video format. Therefore, if the decoder has no knowledge of the video format used in the sequence, it must start with a sequence start. However, once the video format is configured in the decoder, it should be possible to start decoding at any group of picture positions.

[1038] MPEG does not limit the number of pictures in a group. However, for many applications, one group provides a reasonable granularity of random access, which is equivalent to about 0.5 seconds.

[1039] The start of the picture group is indicated by the group start token. The header information provided after the group start has two useful tokens: T
Includes IME CODE and BROKEN CLOSED.

[1040] TIME CODE owns a subset of SMPTE time code information. This may be useful in synchronizing the video decoder to other signals. BROKEN CLOSED is a close of MPEG.
Owns the ed gap and brokenlink bits. For the implications of random access and decoding of the edited video sequence, see Section A.3. 1
See 9.3.8. A. 19.3.3 Picture layer The start of a new picture is indicated by the picture start token. This token is followed by a time standard token and a picture type token. Temporary reference information may be useful if the temporal decoder is not configured to provide picture reordering. Picture type information is especially open to GO
It may be useful if you want to process B pictures at the start of P (see section A.19.3.8).

[1041] Each picture is composed of many slices.

[1042] A. 19.3.4 “Slice Layer” Section A. 19.2.2.2 is H.264. The block groups used in 261 are discussed. MP
Slices in the EG perform a similar function. However, the slice structure is not fixed by the criteria. The 8-bit value owned by the slice start token is MPE
It is one less than the “slice vertical position” transmitted by G. See the MPEG Standard Draft for a description of slice layers.

Since the spatial and temporal decoders have already used this information to ensure that each picture contains the correct number of blocks in the correct position,
By the time the slice start reaches the output of the temporal decoder, that information is effectively redundant. Thus
By counting the number of blocks that have been output since the start of the picture, it is possible to calculate where to put the data blocks output by the temporal decoder.

[1044] For a discussion of the effects of using MPEG picture reordering, see Section A.6. See 19.3.7.

[1045] A. 19.3.5 “Macroblock Layer” Each macroblock contains 6 blocks. These are (MP
Appears at the output of the time decoder in raster state (as specified by the draft EG convention).

[1046] A. 19.3.6 "Block Layer" Each macroblock contains 6 data tokens.
The sequence of data tokens contained in each of the six groups is defined by the MPEG standard draft (which is the same as the macroblock structure of H.261). Each data token should contain exactly 64 data bytes for an 8x8 pixel area of one color component. The color component is contained within the 2-bit number in the data token (A.
See 3.5.1.). However, the sequence of color components within MPEG is limited.

Each group of data tokens is preceded by a number of tokens that convey information about motion vectors, quantizer scale factors, etc. These tokens do not have to cause the picture to be displayed and can be ignored.

[1048] A. 19.3.7 "Effect of MPEG picture reordering" The temporal decoder provides MPEG picture reordering (MPEG r as described in 18.3.5).
eordering = 1). The output of P and I pictures is delayed until the next P / I picture in the data stream is decoded by the temporal decoder. At the output of the temporal decoder, the data tokens of the newly decoded P / I picture are replaced with the data tokens from the old P / I picture.

When reordering P / I pictures, the picture start, time standard, and picture type tokens of the picture are temporarily stored on-chip as the picture is written to the off-chip picture buffer. These stored tokens are retrieved when the picture is read for display. Therefore, the rearranged P / I picture has the correct values for picture start, time standard, and picture type.

All other tokens below the picture layer are not reordered. As the reordered P / I picture is read out for display, it picks up the lower level non-data tokens of the picture just decoded. Thus, these sub-picture layer tokens should be ignored.

[1051] A. 19.3.8 "Random access and edited sequence" Spatial decoder uses edited MPEG video data and M
Provide equipment to help correct video decoding after random access to PEG video data. A. 19.3.8.1 "Open GOP" A picture group (GOP) can start with a B picture predicted from a P picture in the previous GOP. This is called an "open GOP". FIG. 116 illustrates this. Pictures 17 and 18 are the second GO
It is a B picture at the start of P. Once the GOP is "opened", the encoder can use the predictions from P-picture 16 and I-picture 19 to encode these two pictures. Alternatively, the encoder could be limited to using predictions from I-pictures 19 only. In this case, the second GOP is called a "closed GOP".

[1052] When the decoder starts video decoding in the first GOP, that GOP has already decoded the P picture 16, so that GOP
Even if it is open, encountering a second GOP will not cause any problems. However, if the decoder does random access and starts decoding in the second GOP, it cannot decode B17 and B18 if they depend on P16 (ie the GOP is open). .

If the spatial decoder of the present invention encounters an open GOP as the first GOP following a reset, or receives a flush token, it is assumed that a random access to the open GOP has occurred. In this case, the Huffman decoder will consume the data for the B picture in the usual way. However, it is I
Will output a B-picture that is predicted with a (0,0) motion vector away from the picture. As a result, pictures B17 and B18 (in the above example) will be identical to I19.

This action ensures correct maintenance of the MPEG VBV rules. In addition, it ensures that the B picture is present in the output at the position in the output stream expected by the other data channels. For example,
The MPEG system layer provides display time information about the audio data to the video data. The video display time stamp indicates the first display picture in the GOP, that is, the picture with time reference 0. In the above example, the first picture displayed after random access to the second GOP is B17.

[1055] Broken closed token is MPE
Owns the G closed gop bits. Therefore,
At the output of the temporal decoder, it is possible to determine whether the output of the B-picture is authentic or whether the "substitute" was introduced by the spatial decoder. Some applications may wish to take special action when these "substitute" pictures are present.

[1056] A. 19.3.8.2 Edited Video When an application edits an MPEG video sequence, it may break the relationship between two GOPs. If the edited GOP was an open GOP, it would no longer be able to correctly decode the B picture at the beginning of the GOP. MPEG
The application that edits data is the GOP after editing.
The broken link bit can be set in to indicate to the decoder that these B pictures cannot be decoded.

[1058] Spatial decoder has a broken link G
If an OP is encountered, the Huffman Decoder will decode the data for the B picture in the usual way. However, it is far from the I picture (0,
0) Will output B-pictures predicted with motion vectors. As a result, picture B1 (in the example above)
7 and B18 will be identical to I19.

[1058] Broken closed token is MPE
Owns G's broken link bit. Therefore, it is possible to determine in the output of the temporal decoder whether the output of the B-picture is genuine or the "substitute" introduced by the spatial decoder. Some applications may wish to take special action when these "substitute" pictures are present. Section A. 20 "Post-Write DRAM Interface" The interface can be configured in two ways: The detailed teaming of the interface can be configured to accommodate a variety of different DRAM types.

The "width" of the DRAM interface can be configured to provide a cost / performance tradeoff.

[1060]

[Table 154]

[1061]

[Table 155]

[1062]

[Table 156]

[1063]

[Table 157]

[1064]

[Table 158]

[1065]

[Table 159] A. 20.1 "Interface Timing" In the present invention, the DRAM interface timing is a clock that operates at four times the input clock rate of the device (decod).
er clock). This clock is generated by an on-chip PLL.

For the sake of simplicity, the period of this high speed clock is called a tick.

[1067] A. 20.2 "Interface Operation" The interface uses the fast page mode of DRAM. Three different types of access are supported: -Read-Write-Refresh Each read or write access is one DRAM
A burst of 1 to 64 bytes is transmitted at the page address. Read and write transmissions are not mixed in one access. Each successive access is a new DRAM
Treated as a random access to the page.

[1068] A. 20.3 "Access structure" Each access consists of two parts: -Access start-Data transmission Each access is initiated by an access start, followed by one or more data transmission cycles. There are variations of read, write and refresh for both access start and data transfer cycles.

At the end of the last data transmission in one access, the interface goes to the default state,
It will remain in this state until new access is ready to start. When the last access is completed and the new access is ready to start, the new access starts immediately.

[1070] A. 20.3.1 "Access Start" Access Start provides the page address for read or write transmission and sets some initial signal states.
There are three different access starts: -read start-write start-refresh start In each case, the timing of / RAS / and the row address are set in the registers RAS falling and page.
Controlled by the start length. / O
The states of E / and DRAM data [31: 0] are retained from the end of the previous data transmission to / RAS /. The three different types of access start are /
When becoming RAS // OE / and DRAM data
Only the method of driving [31: 0] is different. See FIG. 118.

[1071]

[Table 160] A. 20.3.2 "Data Transmission" There are three different types of data transmission cycles: -Fast page read cycle-Fast page late write cycle-Refresh cycle A refresh start is followed by only one refresh cycle. A read (or write) start is followed by one or more fast page read (or write) cycles.

[1072] At the start of the read cycle, CAS
Is driven high and a new column address is driven.

A late write cycle is used. / W
E / goes low one tick after / CAS /. The output data is driven one tick after the address.

As / CAS / before / RAS /, the refresh cycle is initiated by a refresh cycle start and there is no interface signal activity during the refresh cycle. The purpose of the refresh cycle is to meet the minimum / RAS / low period required by the DRAM.

[1075] A. 20.3.3 "Interface Default State" The interface signal enters the default state at the end of access:-/ RAS /, / CAS /, and / WE / are high-Data and / OE / remain in their previous state The addr remains stable A. 20.4 "Data Bus Width" The 2-bit register DRAM data width allows the width of the data path of the DRAM interface to be configured. This minimizes DRAM cost when working with small picture formats.

[1076]

[Table 161] A. 20.5 “Address Bit” An on-chip 24-bit address is created. If this address is used and how to form the row and column addresses are different by the number of bits selected for the data bus width and the row address. Some configurations do not allow the full use of internal address bits (thus creating "hidden bits").

The row address is derived from the middle part of the address. This maximizes the rate at which the DRAM is naturally refreshed.

[1078] A. 20.5.1 "Low Order Column Address Bits" The least significant 4-6 bit column address is used to provide the address for fast page mode transmission up to 64 bytes. The number of address bits required to control these transfers depends on the width of the data bus. (See A.20.4) 20.5.2 "Row Address Bits" The number of bits taken from the middle section of the 24-bit internal address to provide the row address is determined by register r
It is formed by ow address bits.

[1079]

[Table 162] The width of the row address used depends on the type of DRAM used and whether or not the MSB of the row address is decoded off-chip to access multiple banks of DRAM.

[1080] Note: The raw address is extracted from the middle of the internal address. Bit with row address is DRAM
When decoded to select a bank, all possible values of these "bank select bits" must select a DRAM bank. Otherwise, holes will be left in the address space.

[1081]

[Table 163] A. 20.6 "DRAM interface enabled (ena
ble) ”There are two ways to make all output signals on the DRAM interface high impedance. DRAM
These are the enable register and the DRAM enable signal. The registers and signals must be logic one for the DRAM interface to operate. If either is low, the interface is placed in high impedance and data transmission through the interface is stopped.

The ability to bring the DRAM interface to a high impedance is the ability of other devices to be controlled by the spatial decoder (or temporal decoder) when the spatial decoder (or temporal decoder) is not in use.
It is provided to allow RAM to be tested or used, not to allow other devices to share memory during normal operation.

[1083] A. 20.7 "Refresh" Unless otherwise disabled by writing to register no refresh, the DRAM interface is at an interval determined by register refresh interval / RAS // CA before the refresh cycle.
Use S / to automatically refresh the DRAM.

[1086] The value of refresh interval specifies the interval between refresh cycles in the period of 16 decoder clock cycles. 1
Values in the range of up to 255 can be selected. The value 0 is automatically loaded after reset, forcing the DRAM interface to perform refresh cycles continuously until a valid refresh interval is formed (once enabled). refres
It is recommended that h interval be formed only once after each reset.

[1085] A. 20.8 "Signal strength" The driving force of the output of the DRAM interface is a 3-bit register / CAS / strength, / RAS / s
It is formed by the user using the strength and the addr strength. This 3-bit value MSB selects either a high speed or a low speed edge rate. The two less significant bits form the output for different load capacitances.

The default strength after reset is 6, forming an output that takes approximately 10 ns for the drive signal between GND and VDD when loaded at 12 pF.

[1087]

[Table 164] When the output is formed approximately for a driving load, it will meet the AC electrical characteristics specified in Tables 168-169. If properly formed, each output will fit closely with its load, and thus minimal overshoot will occur after a signal transition.

[1088] A. 20.9 "After Reset" After reset, all DRAM interface formation registers are reset to their default values.
The most important of these default configurations are listed below: -DRAM interface is disabled,
Allowed to go to high impedance.

The refresh interval is formed to a special value 0, which means to carry out successive refresh cycles after the interface is re-enabled.

The DRAM interface is set to its slowest configuration.

Most DRAMs are 100 μs to 500 μs after power is first applied and then after a number of refresh cycles before normal operation is possible.
Request a “pause” for μs.

[1092] Immediately after reset, the DRAM interface outputs the DRAM enable signal and the DRAM en signal.
It is deactivated until both of the enable registers are set. When these are set, the DRAM interface will wait until the DRAM interface is formed (approximately every 400 ns, depending on the clock frequency used).
Will perform a refresh cycle.

[1093] The user may have enough time after enabling the DRAM interface to make sure that after power up and the required number of refresh cycles occur before attempting the data transfer.
Responsible for ensuring the AM "pause".

[1109] While declaring a reset, the DRAM interface cannot refresh the DRAM. However, the reset times required by the decoder chips are short enough that they can be reset and then the DRAM interface re-enabled before the contents of the DRAM are corrupted. This may be needed during debugging.

[1095]

[Table 165]

[1096]

[Table 166]

[1097]

[Table 167] A. 20.10.1 “AC characteristics”

[1098]

[Table 168]

[1099]

[Table 169] Section B. 1 "Start code detector" B. 1.1 “Perspective” As described above in FIG. 18, the start code detector (SCD) is the first block on the spatial decoder.
Its basic purpose is to use MPE in the input data stream.
G, JPEG and H.G. To detect the 261 start codes and replace them with the relevant tokens. It also allows user access to the input data stream via the microprocessor interface, performing pre-formatting and "tidying" of the token data stream. Recall that the SCD can receive either raw byte data or data already assembled in token format.

[1100] Typically, the start code is MPEG,
H. For H.261 and JPEG, there are 24, 16 and 8 bits respectively. The start code detector receives input data in bytes from either the microprocessor interface (upi) or the token / byte port and moves it through three shift registers. The first register is an 8-bit parallel serial out,
The second register is programmable length (16 or 24 bits), where the start code is detected, and the third register is 15 bits wide, to reformat the data into 15-bit tokens. used. In addition, there are two "tag" shift registers (SR) that operate in parallel with the second and third SR. These include tags for indicating whether the associated bits in the data SR are good or bad. SCD rather than part of the data token
Allows input bytes not recognized by to bypass the shift register and is output when all three shift registers have been flushed (empty) and the contents have been successfully output. The recognized non-data token is S
Used to form CDs, spring traps, set flags. Furthermore, they bypass the shift register and are output unchanged.

[1101] B. 1.2 "Main Block" The hardware for the start code detector consists of 10 state machines.

[1102] B. 1.2.1 “Input circuit (scdip
c. sch. iplm. M) ”The input circuit has three modes of operation: token, byte and microprocessor interface.
These modes are as if the data were a raw byte stream (but using a 2-wire interface),
Allowed to be entered as a token stream or by the user via upi. In all cases,
The input circuit always outputs the correct data token by generating the data token header when appropriate. The transition to / from the upi mode is synchronized to the system clock and the upi waits until a safe point in the data stream before gaining access. The byte mode pin determines whether the input circuit is in token mode or byte mode. Furthermore, an initial notification to the system as to which standard is being decrypted (and thus a coding standard token is issued) can be done in any of the three modes. B. 1.2.2 “Token Decoder (scdipne
w. sch, scdipnem. M) ”This block decodes the input token and issues commands to other blocks.

[1103]

[Table 170] Note: Changes in the coding standard are sent to all blocks via the two wire interface after SR is flushed. This ensures that the change from one data stream to another occurs at the correct point throughout the SCD. This principle applies throughout the display so that changes in the coding standard can flow through the entire chip before a new stream.

[1104] B. 1.2.3 “JPEG (scdjp
eg. sch, scdjpegm. M) ”The start codes (markers) in JPEG are sufficiently different, and all JPEGs have their own state machine. In the present invention, this block is J
It handles all PEG marker detection, length counting / checking, and data removal. The detected JPEG marker is flagged as a start code (with v nott, see text below), scdipne
The command from w is overridden and forced to bypass. The operations are best explained in code. switch (state) {case (LOOKING): if (input = 0xff) {state = GETVALUE; / * Found a marker * / remove; / * Marker gets removed * /} a ;; (Input = 0xff) {state = GETVALUE; / * Overlapping markers * / remove; the 0 × ff back /} else {command = BYPASS; / override command / if (lc) / Dose t he marker have a distance count / state = GETLC0; else state = LOOKING; break; case (GETLC0): loadlc0; / Load liet; = DECLC; break; case (DECLC): lcnt = lcnt-2 state = CHECKLCC; break; case (CHECKLC): if (lcnt = 0) state = LOOKING; = LOOKING; / generate Illegal Le gth Error / else state = COUNT; break; case (COUNT): decrement length count until1 if (lc <= 1) state = LOOKING;} B. 1.2.4 “Input shifter (scinshft.
sch, scinshm. M) ”The basic operation of this block is quite simple.
This block takes a data byte from the input circuit,
Load the shift register and shift it. However, it also follows the command from the input decoder,
Handle transitions to / from bypass mode (other SR
When a BYPASS command is received, the associated bytes are not loaded into the shift register. Instead, "rubbish" (tag =
1) is shifted and forces the data held in another shift register to be output. The block then waits for a "flushed" signal indicating that this "rabbish" has appeared on the token generator. The input bytes are then sent directly to the token generator.

[1105] B. 1.2.5 “Start Code Detector (scdetect.sch, scdetm.M)” This block contains 16 or 24 bit start code detection logic and two shift registers programmable to “valid content” detection logic. . The MPEG start code requires full 24 bits, but Two
61 requires only 16 bits.

In the present invention, the first SR is for data, and there is no gap or stall in the SR (in the meaning of the two-wire interface), but the second SR determines whether the bits in the data SR are valid or not. , But the bits they contain are flushed while they may be invalid (rabbish). When a start code is detected, a bit in the tag shift register is set to invalidate the contents of detector SR.

[1107] The start code cannot be detected unless the contents of SR are all valid. A non-byte aligned start code may be detected and flagged. Moreover, when a start code is detected, the unaligned start code cannot be explicitly flagged until the overlapping start codes are examined. To perform this function, the "value" of the detected start code (followed by a byte) is shifted to the right through scinshift and scdetect, and scoshift. If scoshift is reached without detecting another start code,
Removed is the overlapping start code, which is flagged as a valid start code.

[1108] B. 1.2.6 “Output shifter (sco
shift. sch, scoshm. M) ”The basic operation of the output shifter is scdetect
The serial data (and tag) is extracted from the data, packed into a 15-bit word, and output. Other functions are as follows: B. 1.2.6.1 "Data Padding" The output consists of 15-bit words, but the input can consist of any number of bits. Therefore, to flush, it is necessary to add bits to make the last word up to 15 bits. These extra bits are called padding and must be recognized and removed by the Huffman block. Padding is defined as:
After the last data bit, a "zero" is inserted, followed by enough "things" to make up a 15-bit word.

The data word containing the padding is output with the low extension bit to indicate the end of the data token.

[1110] B. 1.2.6.2 "Flash Occurrence" In accordance with the present invention, the occurrence of a "flash" operation involves detecting when all SRs are flushed and signaling it to the input shifter. When the "rabbish" inserted by the input shifter reaches the end of the output shifter and the output shifter completes its padding,
A "flushed" signal is emitted. This "flushed" signal must pass through the token generator before it is safe for the input shifter to enter bypass mode.

[1111] B. 1.2.6.3 “Display of valid start code flag” When scdetect indicates that the start code is found, padding is performed and the current data is output. The start code value (next byte) is shifted through the detector to remove overlapping start codes. If another “start” code is not detected and the “value” reaches the output shifter, it will not be overlapped and that value will be flagged by the flag v not t (ValueNotTok) indicating that it is a start code value.
en) is sent. However, while another start code is detected (by scdetect),
If the output shifter is waiting for that value, an overlapping error will be issued. In this case, the first value is discarded and the system waits for the second value. This value can also be stacked so that the same procedure is repeated until a non-overlapping start code is found.

[1112] B. 1.2.6.4 "tyding up after start code" A new data header is created when (non-Ravish) data begins to arrive after detecting and outputting the appropriate start code.

[1113] B. 1.2.7 “Data stream generator (sctokrec.sch, sctokrem.
M) ”The data stream generator has one scinshift for tokens bypassed and another one for stuffed data and startcodes.
Has a two wire interface input from ft. Switching between the two sources is only allowed after the current token (from either source) has completed (the low extension bits that have arrived).

[1114] B. 1.2.8 “Start value for starting number conversion (scdromhw.sch,
schrom. M) ”The process of converting start value into tokens is done in two stages. This block deals primarily with coding standard dependent issues that reduce the 520 odd position codes to an independent index of 16 coding standards. As mentioned earlier, the start value (including the JPEG value) is identified from all other data by a flag (value not token). If v not t is high, this block gives a 4- or 8-bit value depending on the coding standard, a 4-bit s that is independent of the standard.
Convert to start number and flag the unrecognized start code. The number of starts is as follows:

[1115]

[Table 171]

[1116]

[Table 172]

[1117]

[Table 173] B. 1.2.9 "Start number for token conversion (sconvert.sch, sconverm.
M) ”In the second stage of conversion, the start number (or index) is converted into a token. This block also handles token expansion and search modes that properly discard expansion and user data.

The search mode is a means to enter the data stream at random points. The search mode can be set to one of eight values: 0: Normal operation-find next start code.

1/2: System level search is not performed for spatial decoders.

[1120] 3: Search for sequences or more 4: Search for groups or more 5: Search for pictures or more 6: Search for slices or more 7: Search for the next start code Non-zero search mode is desired Allow data to be discarded until a start code (or higher in syntax) is detected.

[1121] This block further includes PICTURE and S
Add token extension to LICE start token.

[1122] ・ Picture start is PICTURE N
UMBER is extended with a 4-bit count of pictures.

[1123] The slice start is extended by svp (slice vertical position). This is start code minus 1 (MPEG, H.261), and minus OXDO
It is the "value" of (JPEG).

[1124] B. 1.2.10 “Data stream formatting (scinsert.sch, sci
nserx. M) ”In the present invention, data stream formatting relates to conditional insertion of picture end, flash, coding standard, sequence start tokens, and generation of STOP AFTER PICTURE events. Its function is best simplified and explained in software: switch (input data) case (FLUSH) 1. if (in picture) output = PICTURE END 2. output = FLUSH 3. 3. if (in picture & stop after picture) sample error = HIGH in picture = FALSE; in picture = FALSE; break case (SEQUENCE START) 1. if (in picture) output = PICTURE END 2. if (in picture & stop after picture) 2a. output = FLUSH 2b. sap error = HIGH in picture = FALSE 3. output = CODING STANDARD 4. output = standard 5. output = SEQUENCE START 6. in-picture = FALSE; break case (SEQUENCE END) case (GROUP START): if (in picture) output = PICTURE END 2. if (in picture & stop after picture) 2a. output = FLUSH 2b. sap error = HIGH in picture = FALSE 3. output = SEQUENCE END or GROUP START 4. in-picture = FALSE; break case (PICTURE END) output = PICTURE END 2. if (stop after picture) 2a. output = FLUSH 2b. sap error = HIGH 3. in-picture = FALSE break case (PICTURE START) if (in picture) output = PICTURE END 2. if (in picture & stop after picture) 2a. output = FLUSH 2b. sap error = HIGH 3. if (insert sequence start) 3a. output = CODING START 3b. output = standard 3c. 3. output = SEQUENCE START insert sequence start = FALSE 4. output = PICTURE START in picture = TRUE break default: Just pass it through section B. 2 "Huffman Decoder and Parser (p
arser) ”B. 2.1 "General" This section describes the Huffman decoder and parser circuit according to the invention.

FIG. 127 shows a high level block diagram of the Huffman Decoder and Parser. For clarity, many signals and buses have been omitted from the main diagram, and there are several places where data may be sent backwards (within the large loop shown).

In essence, the Huffman Decoder and Parser of the present invention is comprised of many dedicated processing blocks (shown along the bottom of the diagram) controlled by a programmable state machine.

The data is received from the encoded data buffer by the "in shift" block. At this point, there are essentially two types of information you will encounter: encoded data owned by a data token, and a stard code that has already been replaced by each token by the start code detector. All tokens (except data tokens) are processed in the same way, although other tokens may be encountered. The token (start code) is treated as a special case as multiple pieces of data (in H.261, JPEG or MPEG) are being encoded.

In the present invention, all data possessed by the data token is transmitted to the Huffman decoder in serial form (bit by bit). Of course, this data contains many fields that are not Huffman coded, but are coded with a fixed length. Nevertheless, this data is sent serially to the Huffman Decoder. For Huffman encoded data, the Huffman decoder only performs the first stage of decoding, where the actual Huffman code is replaced by the index number. If there are N District Huffman codes in the special code table to be decoded, the range of this "Huffman index" is 0-N-1.
In addition, the Huffman Decoder has a "no op" or "no operation" mode, which allows it to be sent to the next stage along with the data or token information without processing by the Huffman Decoder.

The data unit index portion is a relatively simple circuit block that performs a table lookup operation. It takes its name from the second stage of the Huffman decoding process, where the index number obtained at the Huffman decoder is converted to the actual decrypted data by a simple table lookup. The data unit index part works as one logical unit in cooperation with the Huffman decoder.

[1130] The ALU is the next block and is provided to apply other transformations to the decrypted data. The data unit index part is suitable for relatively arbitrary mappings and the ALU can be used in places where arithmetic is more appropriate. The ALU comprises a register file that can be manipulated to implement various parts of the decoding algorithm. In particular, the registers holding the vector and DC predictions are included in this block. The ALU is based around a simple adder with operand selection logic. In addition, it contains dedicated circuitry for sign expansion type operation. Shift operations are easy to perform, but this is done in a serial fashion and there will be no barrel shifter.

The token formatter according to the invention is the last block in the video parser and is responsible for the final incorporation of the decrypted data into a token that can be sent to the rest of the decoder. At this point, there are many tokens that will be used by the decoder for this particular picture.

[1132] The Parser State Machine, which is 8 bits wide and is used with the 2-wire interface,
Work to prepare the operations of other blocks.
In essence, it is a very simple state machine, producing a very wide range of "macrocode" control words that are sent to other blocks. In FIG. 127, the command word passes from block to block on the data side. It is important to understand that this is the case and the transmission between different blocks is controlled by a two wire interface.

In the present invention, there is a two-wire interface between each block in the video parser. In addition, the Huffman decoder operates with serial, data, and control tokens, and the inshifter inputs data one bit at a time. Therefore, there are two modes of operation. When data is input to the Huffman decoder via the data token, the data passes through the shifter, one bit at a time. In addition, there is a two wire interface between the inshifter and the Huffman decoder. However,
Other tokens are not shifted (serially) one bit at a time, but rather in the token's header. When the data token is entered, the header containing the address information is deleted and the next data at that address is shifted one bit at a time. If not a data token, all tokens, headers and everything are displayed at once in the Huffman Decoder.

In the present invention, it is important to understand that a two wire interface for a video parser is unusual in that it has two valid lines. One line is valid serially and the other line is valid tokenally. Moreover, both line tokens may not be able to assert at the same time. If one line is asserted, or if there is no valid data, then two valid lines may not be able to assert either, and there is only one accept wire in the other direction. It should be recognized. However, this is not a problem. The Huffman decoder can know whether it wants serial data or token information, depending on the current syntax and as needed to be done next. Therefore, a valid accept signal is set accordingly and Accept is sent from the Huffman decoder to the inshifter. The inshifter sends a valid signal if the appropriate data or token is present. For example,
A typical command is to decipher the Huffman code, convert it in the data unit index part, and
It can be modified in the LU and the result is formed into a token word. One microcode command word is created that contains all the information to do this. The commands are sent directly to the Huffman Decoder, which "inshifts" until it has decoded the complete symbol.
Request one data bit at a time from the block. Control tokens are input in parallel. Once this occurs, the decrypted index value is sent to the data unit index part along with the original microcodeword.
The Huffman decoder requires several cycles to perform this operation, in fact the number of cycles is determined by the data to be decoded. The data unit index portion then maps this value using the table specified in the microcode command word.
This value is sent again to the next block, ALU, along with the original microcodeword. When the ALU completes the appropriate operation (the number of cycles may still be data dependent), the ALU sends the appropriate data to the token formatting block along with a microcode word that controls how the token word is formed.

[1135] The ALU has a number of status wires or "condition codes" that are sent back to the Parser State Machine. This allows the state machine to execute conditional jump commands. In fact, all commands are conditional jump commands: one of the selectable conditions is wired to the value "False". By selecting this condition, a "no jump" command can be constructed.

[1136] According to the present invention, the token formatting section has two inputs: a data field from the ALU and / or a constant field coming from the Parser State Machine. In addition, there is a directive telling the token formatting section how many bits to take from one source and fill with the remaining bits from the other source for a total of 8 bits. For example, HORI
ZONTAL SIZE is HORIZONTAL SI
It has an 8-bit field that is an immutable address that identifies it as a ZE token. In this case, 8 bits are obtained from a fixed field and no data is obtained from the ALU. However, if it were a data token, 6 bits could easily be obtained from a given field, the lower 2 bits indicating the color component from the ALU. Therefore, the token formatter gets this information and places it in the token for use by the rest of the system. It should be noted that in the above example, the number of bits from each source is for illustration purposes only, and one skilled in the art will recognize that the number of bits from any source can be changed. .

The ALU contains a counterbank used to count through the structure of the picture. The picture dimensions are programmed into registers associated with counters that appear to the "microprogrammer" as part of the register bank. Some condition codes are output from this counterbank, which allows conditional jumps based on "picture start", "macroblock start", etc.

[1138] Note that the Parser State Machine is also referred to as the "Demultiplex State Machine". Both terms are used in this document.

Input Shifter In the present invention, the input shifter has two pipeline stage data paths “hfidp” and control Zcells “hf”.
It is a very simple circuit composed of i ″.

[1140] In the first pipeline stage, token decoding occurs. Only data tokens are recognized at this stage. The data contained in the data token is shifted one bit at a time into the Huffman decoder. The second pipeline stage is a shift register. In the last word of the data token,
Special coding occurs so that any number of bits can be conveyed through the encoded data buffer. All possible patterns in the last data word are listed below.

[1141]

[Table 174] As the data bits are shifted left by one in the shift register, the bit pattern "all numbers following 0" is sought (padding). This indicates that the remaining bits in the shift register are not valid and will be discarded. Note that this behavior only occurs in the last word of the data token.

As described above, all other tokens are sent in parallel to the Huffman Decoder. They are also loaded into the second pipeline stage, but no shift occurs. Note that the data header is discarded and never sent to Huffman. Two "effective" wires (out
valid and serial valid) are provided. Only one is asserted at a given time, which indicates what type of data will be displayed at that moment.

[1143] B. 2.2 "Huffman Decoder" The Huffman Decoder has many operation modes. Most notably it breaks the Huffman code,
It is to convert them into Huffman index numbers.
In addition, the Huffman Decoder can decode (in bits) a fixed length code of a length determined by the command word. The Huffman decoder can also receive tokens from the inshift block.

The Huffman Decoder is a very small state machine. It is used when decoding block level information. This is because the Parser State Machine takes too long to make a decision (since it must make a decision about the data and wait for the data to flow through the data unit index part and the ALU before issuing a new command. Because it must be). When this state machine is used, the Huffman decoder itself is
Issue a command to U. Since the Huffman Decoder State Machine cannot control all of the microcode command bits, it cannot issue full range commands to other blocks.

[1145] B. 2.2.1 "Theory of Operation" When deciphering a Huffman code, the Huffman decoder of the present invention uses an arithmetic procedure to decipher an input code into a Huffman index number. This number is between 0 (for a code table with N entries) and N-1. The bits are received one by one from the input shifter.

[1146] Many tables are required to control the operation of the machine. These are the codes (1
For each number of bits possible (~ 16 bits), specify how many codes of that length there are. As expected, this information is typically not sufficient to specify the general Huffman code. However, MPE
G, H. In H.261 and JPEG, the Huffman code is selected so that this information alone can specify the Huffman code table. Unfortunately, there is one exception: H.264, also used in MPEG. 261
Is a Tcoefficient table from. This requires the additional tables described elsewhere (exceptions were systematically introduced in H.261 to avoid start code emulation).

It is important to recognize that the table used by this Huffman Decoder is exactly the same as that transmitted in JPEG. This allows these tables to be used directly, while other designs of Huffman decoders may have required the creation of internal tables from those transmitted. This would have required additional storage and processing for conversion.
MPEG and H.264. The table in 261 can be described in the same way (with the exceptions mentioned above), making a multi-standard decoder practical.

The following "C" fragment illustrates the decoding process: int total = 0; int s = 0; int bit = 0; unsigned long code = 0; int index = 0; while (index> = total). {If (bit> = max bits) fall ("huff decode: ran off end of huff table \ n"); code = (code << 1) Inext bit0; index = code-s + total; ]; S = (s + codes per bit [bit]) <<1; bit ++;} In general, there is an advantage in the fact that certain intermediate values can be calculated in the clock phase before they are needed The process is directly mapped into the silicon implementation.

[1149] What can be understood from the code fragment is: EQ1. total _{n + 1} = total _n + cpb _n EQ2. 's _{n + 1} = 2 (' s _n + cpb _n ) EQ3. _{code n + 1 = 2code n +} bit n EQ4. index _{n + 1} = 2 code _n + bit _n + t
In otal _n -'s _n Unfortunately hardware, it was found that using a series of modified equations are used instead of "pre-shift" variable variable "S" is simpler. In this case: However, in hardware it has been found easier to use a series of modified expressions in which the "shifted" variable is used instead of the variable "S". In this case: EQ5. _{shifted n + 1 = 2shifted n +} c
pb _{n The} following is known: EQ6. i _n = 2shifted _n Therefore, substituting this into equation 4 again, the following equation is made: EQ7. index _{n + 1} = 2 (code _n -shift
ed _n ) + total _n + bit _{n In} addition to calculating successive values of the “index”, it is necessary to know when the calculation is complete. ”
From the C "code fragment: EQ8. It can be seen that it completes when index _{n + 1} <total _{n + 1} .

Substituting Equation 7 and Equation 1, EQ9. _{2 (code n -shifted n) +} bit
It can be seen that the process is completed when _n- cpb _n <0.

In the hardware implementation of the present invention, E
Q. 7 and EQ. Common terms in 9, (coden
-Shiftedn) is calculated one phase before the rest of these expressions are evaluated to give the final result and information that the calculation is "completed".

[1152] One warning word. Various "C" codes,
In particular, the action-edited code Huffman decoder, and sm4
In the code project, the "C" fragment is used almost directly, but the variable "S" is effectively referred to as "shifted". Thus, there are two different variables called "shifted". One of the "C" code and the other in the hardware implementation. These two variables depend on two factors.

B. 2.2.1.1 “Inversion of data bits” There is another piece of information necessary to correctly decode the Huffman code. This is the polarity of the encoded data. H. Two
It can be seen that 61 and JPEG use opposite conventions. This is H. The start code for H.261 is zero bits, while the JPEG marker byte is one bit.

[1154] In order to process both regulations, the encoded data is H.264. To decode a 261 style Huffman code, it is necessary to invert the encoded data bits when read into the Huffman decoder. This is done in an explicit way with exclusive OR gates. Note that when decoding fixed length codes, the inversion is done only for Huffman codes, as the data is not inverted.

[1155] MPEG uses a mixture of two conventions. H. In the aspect succeeded from H.261, H.264. 261
Regulations are used. In the aspect inherited from JPEG (DC intra coefficient decoding), the JPEG standard is used.

[1156] B. 2.2.1.2 “Conversion coefficient table” There are some anomalies when using transform coefficient tables in H.261 and MPEG. First, the MPEG table is an H.264 standard. It is a super set of 261 tables. In the hardware implementation of the present invention, there is no distinction drawn between the two standards, which is an H.264 containing code from an extension of the table (ie MPEG code). It means that the 261 stream will be decrypted in a "correct" way. Of course, other aspects of the compression standard are destroyed as well. For example, these extension codes are H.264.
Would result in start code emulation at 261.

[1157] Second, the conversion coefficient table is code per
It has an anomaly meaning that it cannot be explained by the usual method using the bit table. This anomaly occurs with 6-bit long codes. These codewords are systematically replaced by alternate codewords. In the encoder,
Correct results are obtained in the usual way by the first encoding. Then, for all codes of length 6 bits or more, a simple table lookup operation replaces the first 6 bits with another 6 bits. In the decoder according to the invention, the decoding process is interrupted just before the sixth bit is decoded, the codeword is replaced using a table lookup and the decoding is continued.

In this case, there are only 10 possible 6-bit codes, so the look-up table required is very small. In addition, the operations are the top 2 of the code
Helped by the fact that the bits are not modified by the operation. As a result, there is no need to use a true lookup table. Instead, a small group of gates are wired and the appropriate conversions are made. The module that does this is called "hftcfrng".
This type of code replacement is referred to as a "ring" in the text, because each code from a possible code set is replaced by another code from that set (no new code is introduced or the old code is omitted). Stipulated.

In addition, a unique implementation is made for the first coefficient in the block. In this case, it is impossible for the block end code to occur, so that the most commonly occurring symbols can use the code that would otherwise be interpreted as the block end. Is fixed. This saves 1 bit. The decoding arrangement according to the invention is easily accommodated. Simply put, if the "index" is value zero, then for the first bit of the first coefficient,
Decoding is considered finished. Furthermore, after decoding only one bit, there are only two possible values for the "index", zero and one, and only one bit needs to be tested.

[1160] B. 2.2.1.3 “Register and Adder Size” The Huffman decoder of the present invention can process Huffman codes up to 16 bits. However, the decoding machine is 8 bits wide. This is possible because it is known that the Huffman index number to be decrypted is at most 255. In fact, this is an extended JP
It can only occur in the EG, and in current applications the limit is somewhat lower (but larger than 128, so 7 bits is not enough).

[1161] For all legal Huffman codes,
It can be seen that not only the final value of the "index" but also all the intermediate values are in the range 0-255. However, for an attempt to decipher an illegal code, that is, a code that is not in the current code table (probably due to data error), the index value may exceed 255.
Since we are using an 8-bit machine, at the decoding end the final value of the "index" is 255
Does not exceed Because the more significant bits that tell us that an error has occurred are discarded. Thus, whenever an index value during decoding exceeds 255 (ie, a carry from the adder forming the index), an error occurs and the decoding is abandoned.

[1162] 12 bits of "code" are protected. Three
This is not necessary to decode the Huffman code where the bit register is sufficient. These upper bits are needed for fixed length codes that can read up to 12 bits.

[1163] B. 2.2.1.4 "Operations for Fixed Length Codes" Fixed length codes force the "bit by bit" value to be zero. This means that "overall" and "shifted" remain zero throughout the operation, so the "index" is the same as the code.
In fact, adders and the like only allow 8-bit values to be created for the "index". Thus, when decoding a fixed length code, the upper bits of the output word are taken directly from the "code" register. When decoding a Huffman code, these high order bits are forced to zero.

The fact that enough bits have been read from the input is calculated in an explicit way. The comparator compares the desired number of bits with a "bit" counter.

[1165] B. 2.2.2 "Decoding Coefficient Data" The parser state machine according to the invention is generally used only for very high level decoding. 8
Very low level decoding within a bit by bit data block is not directly handled by this state machine. The Parser State Machine commands the Huffman Decoder of the form "decode block". The Huffman Decoder, Data Unit Index, and ALU operate together under the control of a dedicated state machine (basically in the Huffman Decoder).
This arrangement enables high performance decoding of entropy coded coefficient data. In addition there are other feedback paths operated in this mode of operation. For example, in JPEG decoding where VLC is decoded to provide SIZE and RUN information, the SIZE information is sent directly from the output of the data unit index section back to the Huffman decoder to determine how many FLC bits to read. Instruct the decoder. In addition, some accelerators are implemented. For example, using the same example, by considering the Huffman index value before the data index stage, all VLC values that result in a SIZE of zero are explicitly captured. This means that for a non-zero SIZE value, the Huffman Decoder can proceed to read one FLC bit before the actual SIZE value is known. This means that no clock cycle is wasted because the reading of the first FLC bit overlaps the one clock cycle required to perform the table lookup in the data unit index part.

[1166] B. 2.2.2.1 “MPEG and H.264.
AC coefficient data of H.261 ”is shown in FIG. 133. 261 shows the method of decryption. A flowchart detailing the operation of the Huffman Decoder is shown in FIG.

[1167] The process starts by reading the VLC code. In a normal event, the Huffman index is directly mapped to a 6-bit RUN and a value that represents the absolute value of the coefficient. After that, the FLC 1 bit is read and the sign of the coefficient is given. The ALU assembles the absolute value of the coefficient with this sign bit to provide the final value of the coefficient.

[1168] Since the data format at this point is sine magnitude, there is almost no difficulty in this operation. The RUN value is sent to the 6-bit auxiliary bus, while the coefficient value (level) is sent to the normal data bus.

There are two special cases, which are captured by considering the decrypted index value before the data index operation. These are block end (EOB) and escape coded data. In the case of EOB, the fact that this has occurred is sent through the data unit index part and the ALU block so that the token formatting part can correctly close the opened data token.

[1170] Escape-coded data is more complex. The first 6-bit RUN is read and these are sent directly through the data unit index part, AL
Stored in U. Then, FLC 1 bit is read. This is based on MPEG and H.264. Of the 8 bits of escape described in H.261, it is the most significant bit,
Give level sign. The signature is explicitly read in this implementation because it is necessary to send different commands to the ALU for negative value versus positive value. This allows the ALU to convert the two's complement value in the bitstream to sine magnitude. In either case, the remaining FLC 7 bits are then read. If this is the value zero, then 8 more bits must be read.

In the present invention, the Huffman Decoder's internal state machine is responsible for issuing commands for its own control and also for controlling the Data Unit Index, ALU, and Token Formatters. As shown in FIG. 133, the Huffman Decoder commands come from one of three sources: the Parser State Machine, the Huffman State Machine, or a command previously stored in a register previously received from the Parser State Machine. In essence, the primitive command from the Parser State Machine (which causes the Huffman State Machine to take control and read the coefficients) is held in a register, i.e. used each time a new VLC is needed. All other commands for decoding are provided by the Huffman State Machine.

B. 2.2.2.2 “MPEG DC Coefficient Data” This is processed in the same manner as JPEG DC coefficient data.
It is the responsibility of the controlling microprocessor to ensure that the same (loadable) table is used and its contents are correct. The only real difference from the MPEG standard is that the predictor is set to zero (as in JPEG) and the correction is done in the dequantizer.

[1173] B. 2.2.2.3 “JPEG Coefficient Data” FIG. 129 is a block diagram illustrating hardware for decoding JPEG AC coefficients in accordance with the present invention. DC
The diagram works for both AC and DC coefficients because the process for the coefficients is essentially a simplification of the JPEG process. The only thing that actually adds to the previous diagram for MPEG AC coefficients is that the "SSSS" field is sent back and can be used as part of the Huffman Decoder command to specify the number of FLC bits to be read. The remaining commands are provided by the Huffman State Machine.

FIG. 130 depicts a flow chart for Huffman decoding of AC and DC coefficients.

Take the process for AC coefficients first, but start with reading the VLC using the appropriate table (there are two AC tables). The Huffman index is then converted to RUN and SIZE values in the data unit index section. Two values are captured in the Huffman Index stage, these are for EOB and ZRL. These are the only two values where no FLC bits are read. If the decode index is not for these two values, the Huffman Decoder will immediately use FLC1.
While reading the bits, wait for the data unit index portion to complete the lookup operation and determine how many bits are actually needed. EO
In the case of B, no additional processing is performed by the Huffman State Machine in the Huffman Decoder and another command is read from the Parser State Machine. For ZRL, no FLC bits are needed, but the block is not completed. In this case, the Huffman Decoder will use additional VLC (using the same table as before).
Start decoding immediately.

There are special problems in detecting index values associated with ZRL and EOB. This is because the Huffman table can be downloaded (unlike H.261 and MPEG). 2 JPEG A
Two registers are provided for each of the C tables (one for the ZRL and one for the EOB). These are loaded when the table is downloaded. They hold the index value associated with the appropriate symbol.

[1177] The ALU shall convert the SIZE bit FLC code to the appropriate sign magnitude value. These are loaded when the table is downloaded. They hold the index value associated with the appropriate symbol. The ALU must convert the SIZE bit FLC code to the appropriate sign magnitude value. This is done by first sign-extending the value with the wrong sign.
When the sign bit is newly set, then the remaining bits are reversed (1's complement).

For DC coefficients, there is no ZRL field equivalent, so decision making at the Huffman decoding stage is somewhat easier. The only symbols that cause the FLC bit of zero to be read are those that indicate a DC difference of zero. This is again captured in the Huffman Index stage, to hold this index for each (downloadable) JPEG DC table,
Registers are provided.

[1179] The ALU of the present invention (known as prediction)
It does the job of forming the final decoded DC coefficient by keeping a copy of the last DC coefficient value. Four
A total of four predictors are required, one for each of the two active color components. When the DC difference is decoded, ALU
Adds the appropriate predictors to form the decrypted value. This is also stored as a predictor for the next DC difference of that color component. Since the DC coefficients are signed (due to the DC offset), a two's complement to sine magnitude conversion is required. The value is then output with a RUN of zero. In fact, the orders to carry out part of this last stage are not provided by the Huffman State Machine. They are simply executed by the Parser State Machine.

[1180] In a similar manner for the AC coefficient, A
The LU must first generate a DC difference from the FLC SIZE bits. However, in this case, it is required that the two complement values be added to the predictor. This is formed by first sign-extending with the wrong sign, as described above. If the result is negative, then one must be added to form the correct value. Of course, this can be added at the same time as the predictor by packing the carry in the adder.

[1181] B. 2.2.3 “Error processing” Error processing is worth explaining. There are practically four sources of error that are detected: Off the table end.

[1182] When the token is expected, it is serial.

[1187] A token when serial is expected.

[1184] There are too many coefficients in the block.

[1187] The first of these occurs in two situations. When the bit counter reaches 16 (the legal value is 0 to 15), an error occurs because the longest legal Huffman code is 16 bits. If the median value of the “index” exceeds 255, then section B. An error occurs as described in 2.2.1.3.

The second one occurs when a token is expected but serial data is encountered. The third occurs when the opposite situation occurs.

[1187] The last type of error occurs when there are too many coefficients in a block. This is actually detected in the data unit index part.

[1187] When any of these conditions occur, the error is noted in the Huffman error register and the parser state machine is interrupted. It is the responsibility of the Parser State Machine to handle the errors and issue the necessary commands for recovery.

Huffman works with the Parser State Machine on interrupts to ensure correct operation. When the Huffman Decoder interrupts the Parser State Machine, it can wait for new commands to be accepted at the output of the Parser State Machine. The Huffman Decoder will not accept this command for two full cycles after it interrupts the Parser State Machine. This allows the parser state machine to remove the command there (which should not be executed now) and replace it with the appropriate one. After these two cycles, the Huffman Decoder begins normal operation again and accepts a valid command if it is there. Otherwise it will not do anything until the Parser State Machine indicates a valid command.

[1187] If any of these errors occur,
If the "Huffman Error" event bit is set and the mask bit is set, the block will stop and the control microprocessor will be interrupted in the normal manner.

[1191] In some situations, what looks like an error may not actually be an error. The most important situation where this happens is when reading a macroblock address. MPEG, H.264. 261 and JPEG
In this syntax, it is legal for a token to occur instead of the expected macroblock address. If this happens legally, the Huffman Error Register is loaded with zero (meaning no error), but the Parser State Machine is still interrupted. The parser state machine code recognizes the status of this no error and responds accordingly. In this case, the "Huffman Error" event bit will not be set and the block will not stop processing.

[1192] Various situations must be dealt with.
First, the token is generated immediately and the serial bit does not advance. In this case, "token error when serial is expected" would have occurred, but instead, no
The error error occurs in the manner described above.

Second, there are a few serial bits before the token. In this case, a decision is made. If all the bits before the token had a value of 1,
(In H.261 and MPEG, the coded data is inverted, so remember that there are zero bits in the coded data file), and no error occurs. However, if any of them are zero, they are not valid stuffing bits and thus an error occurs, resulting in a "token when serial is expected" error.

[1194] Third, there are many bits before the token. In this case, the same decision is made. If all 16 bits are 1, then they are treated as padding bits and a no error error occurs. If any of them were zero, an "out of Huffman table" error occurs.

[1195] Another place where tokens occur unexpectedly is JPEG. When processing either the Huffman table or the quantizer table, any number of tables can occur in the same marker segment. The Huffman Decoder cannot know how many tables there are. Thus, after each table is completed, the Huffman Decoder reads another FLC4 bit and assumes it to be the new table number.
However, when a new marker segment starts, a token will be encountered instead of the 4-bit FLC. This requirement is not anticipated and therefore Ignore
The Errors command bit has been added.

[1196] B. 2.2.4 "Huffman Commands" The bits used by the Parser State Machine to control the Huffman Decoder blocks and their definitions are shown here. The data unit index part command bits are also included in this table. From the microprogrammer's point of view, the Huffman decoder and the data unit index part act as one condensed logic block.

[1197]

[Table 175] B. 2.2.4.1 "FLC read" In this mode, Ignore Errors, Dow
nload, Alutab, Token, First
Coeff and Special and VLC are all zero. Bypass is set so that data index conversion does not occur.

The binary numbers in Table [3: 0] indicate how many bits should be read. Numbers 0-12 are legal. Value zero actually reads the zero bit (as would be expected), so this command is a Huffman Decoder NOP command. Value 13, 14,
Value 15 is used when the Huffman State Machine is under control and indicates the use of "SSSS" as the number of FLC bits to be read.

[1199] B. 2.2.4.2 “VLC Read” In this mode, Ignore Errors, Dow
nload, Alutab, Token, First
Coeff, and Special are zero, and V
LC is 1. Normally, Bypass is set so that data index conversion occurs.

[1200] In this mode, Token, First
Coefficient and Special are all zero and VLC is one.

The binary numbers in table [3: 0] indicate which table to use, as shown below.

[1202]

[Table 176] For tables held in RAM (ie JPEG tables), bit 1 is not used and table selection is 2
Note that it occurs twice. If a non-baseline JPEG decoder is created, there are 4 DC tables and 4 AC tables and table [1] is required.

[1203] If table [3] is zero, the table is H.264. In order to be read correctly as a 261-style table, the input data is inverted when used. If table [3: 0] = 0, the appropriate Ring modification is also applied.

[1204] B. 2.2.4.3 “NOP command” As described above, the operation of reading the zero-bit FLC is used as the No Operation command. No data (either token or serial) is read from the input port and the Huffman decoder outputs a data value of zero with the command word.

[1205] B. 2.2.4.4 “TCoeffic
ient first coefficient ”. 261 and the MPEG TCoefficient table have a special non-Huffman code used for the first coefficient of the block. To decode the TCoefficient at the block start,
The First Coefficient bit may be set to table zero with the VLC command. First
One of the many effects of the Coefficient bit is to allow this code to be decoded.

[1206] In normal operation, TCoeff
Note that issuing a "simple" command to read the icient VLC is unusual. This is usually S
This is because the control is handed over to the Huffman decoder by setting the special Bit.

[1207] B. 2.2.4.5 "Read Token Word" To read the token word, the Token bit should be set to one. Special and First C
The efficient bit should be zero.
If the table [0] bit is operated correctly, VLC
Bits should also be set.

[1208] In this mode, bittable [1] and table [2] are used to modify the token read action as follows:

[1209]

[Table 177] If Table [0] and Table [1] are zero, the presence of serial data before the token is considered an error and such a signal is emitted.

If table [1] is set, all serial data is discarded until a token word is encountered. No error will occur due to the presence of this serial data.

If table [0] is set, padding bits will be discarded. Of course, it is necessary to know the polarity of the padding bit. This is VLC
It is determined by table [3] in exactly the same way as for the reading of data. If table [3] is zero, the input data is first inverted and then any "1" bits are discarded. If table [3] is set to 1, the input data is not inverted and the "1" bit is discarded. This bit must be set to 1 because the operation of inverting the data depending on the table [3] bit is conditional on the VLC bit. Bits that are not padding bits are encountered (ie, H.2
61 and "1" bit in MPEG)
Errors are reported.

[1212] Note that with these commands, only one token word is read. The state of the extension bit is ignored and it is the demultiplexer's responsibility to test this bit and act accordingly. Directives for reading multiple words are also provided-see the section on special directives.

[1213] B. 2.2.4.6 "ALU Register Specify Table" When the Alutab bit is set, the register in the ALU register file can be used to determine the table number actually used. Which ALU is the table number supplied with the command along with the VLC bit
Determine if a register is used:

[1214]

[Table 178] For fixed length codes, the correct number of bits is read to decode the vector. If r size is zero, a NOP command occurs.

For Huffman codes, the table number that occurs is Table [3] set to 1, and the resulting number refers to one of the JPEG tables. B. 2.2.4.7 "Special commands" All commands (or operation modes) described so far are considered as "simple" commands. For each command received, the appropriate amount of input data (whether serial or token data) is read and the resulting data is output.
Exactly one output will occur per command if no error is detected.

In the present invention, the special command is characterized in that one or more output words can be generated for one command. To perform this function, the internal state machine of the Huffman Decoder will take control and issue itself as needed until the parser determines that the command requested is complete.

For all special commands, the first actual command in the sequence to be executed is issued with the Special bit set to one. This means that every sequence must have a unique first command. The advantage of this scheme is that the first actual command of the sequence is available without the required lookup operation based on the command received from the parser.

There are four recognized special commands: TCoefficient, JPEG DC, JPEG AC, Token The first one, until the block end symbol is read.
H. 261 and MPEG conversion coefficients etc. are read. If the block is a non-intra block, this command will read the entire block. In this case, First Coef
The ficit bit should be set so that the first coefficient trick is applied. If the block is an intra block, the DC term should have already been read, and the First Coefficient bit should be zero.

[1219] In the case of the 261, intra block, D
The C-term is read using the "simple" command to read the FLC 8-bit value. In MPEG, the special command of JPEG DC described below is used. J
The PEG DC command is (FLC indicated by VLC
It is used to read JPEG style DC terms (including SSSS bits of. First Coeffi
The ciient bit must be set in the data unit index part in order for the counter (counting the number of coefficients) to be reset. JPEG AC
The command is used to read the rest of the block after the DC term until either EOB is encountered or the 64th coefficient is read.

[1220] The Token command is used to read all tokens. The token word is read until the extension bit is clear. It is a convenient way to handle unrecognized tokens.

B. 2.2.4.8 “Download Table” In the present invention, the Huffman decoder table is Downlo
It can be downloaded using the ad bit. The first step is to nominate a table to download. This is the Download bit set and F
It is done by issuing a command to read the FLC with the irst Coeff bit set. This is NO
The table number is stored in a register, though it is treated as a P and thus no bits are actually read.
Used to specify which table will be loaded in the next download.

[1222]

[Table 179] As the table above shows, either an AC or a DC table can be loaded, and the table [3] will have it's codes-pe (in Huffman decoder itself).
Determine if it is an r-bit table or a loaded data index table.

[1223] Once the table is nominated, Downl
FLAD with the required number of bits (always 8 bits) with the oad bit set (and the First Coeff bit zero)
The data is downloaded into the table by issuing the command to read. This causes the encoded data to be written into the named table. The address counter is maintained, data is written to the current address, then the address counter is incremented. The address counter is reset to zero whenever the table is named.

When downloading the data index table, the data and address are monitored. Note that while the address is a Huffman index number, the data loaded at that address is the final decoded symbol. This information is used to automatically load the register holding the Huffman index number for the symbol of interest. Therefore, JPEG
When data having a value corresponding to ZRL is recognized in the AC table, the current address is specified by the register CED as indicated by the table number.
H KEY ZRL INDEX 0 or CED H
Written to KEY ZRL INDEX1.

The decrypted data is written to the codes-per-bit table one phase after it is decrypted, so it is not possible to read the data from the table during this phase. Therefore, immediately after the table download command, a command issued to read the VLC will fail. In an actual application (that is, when performing JPEG), there is no reason for such a sequence to occur. However, it is possible to build a simulation test that does this.

B. 2.2.5 "Huffman State Machine" The Huffman State Machine according to the present invention acts to provide the Huffman Decoder commands that are created internally in some cases. All commands that can be produced by the internal state machine can be provided to the Huffman decoder by the demultiplexer.

[1227] The basic structure of the state machine is as follows. When a command is issued to the Huffman Decoder, it is stored in a series of auxiliary latches for later reuse. In addition, the commands are executed by the Huffman Decoder and analyzed by the Huffman State Machine.
When the command is recognized as the first of a known command sequence and the SPECIAL bit is set, the Huffman Decoder State Machine takes over control of the Huffman Decoder from the Parser State Machine. At this point, the command for the Huffman Decoder has three sources: 1) Parser State Machine: This choice is made at the completion of the special command (eg, when the EOB is decoded) and the next demultiplexer command. Is accepted.

[1228] 2) Huffman State Machine. The Huffman State Machine can have arbitrary commands.

[1229] 3) By the Parser State Machine,
A primitive command issued to start a command.

In the case of (2), the table number can be provided by feedback from the data unit index section, which will replace the field in the Huffman State Machine ROM.

In the case of (1), in some cases the table number is provided by the value obtained from the ALU register file (eg, for AC and DC table numbers and F-numbers). These values are stored in the auxiliary command storage and the table number is the one stored when the command is later reused. It is not recovered again from the ALU, since the counter is normally advanced to quote the next block.

[1232] The decision needs to be made towards the end of the cycle, as the choice of the next command used depends on the data being decoded. Therefore, the general architecture is such that all possible commands are prepared in parallel, and the multiplexing in the second half of the cycle determines the actual commands.

In each case, in addition to determining the command that the Huffman Decoder will use in the next cycle, the state machine ROM also passes the current data to the data unit index part and then to the ALU. At times, focus on determining the commands that will be attached to the current data. In exactly the same way, all three of these commands are prepared in parallel and a choice is made later in the cycle.

Once again, there are three choices for this part of the command, which correspond to the three choices for the next Huffman Decoder command as described above.

[1235] 1) A fixed command suitable for the block end.

[1236] 2) Huffman State Machine. The Huffman State Machine can provide arbitrary instructions for the data unit index part.

[1240] 3) A primitive command issued by the parser to start the command.

[1238] B. 2.2.5.1 "EOB Comparator" The output of the EOB comparator essentially forces a selection of constant commands to appear in the data unit index section, and the next Huffman command is the next command from the parser. Will let you do. The exact function of the comparator is controlled by the bits in the Huffman State Machine ROM.

[1239] Behind the EOB comparator, AC and D
There are four registers in the C JPEG table that hold the EOB symbol index. In the case of a DC table, of course there is no block end symbol, but there is a zero sized symbol, which is created by the DC zero difference. This allows the zero bits of the FLC to be read exactly as in the EOB symbol, so they are treated exactly the same.

[1240] In addition to the three index values held in the register, the constant value 1 can also be used. This is H.
261 and the index number of the EOB symbol in MPEG.

B. 2.2.5.2 "ZRL Comparator" In the present invention, this is a general purpose comparator. It lets you choose either the Huffman State Machine command or the original command to be used by I to D.

There are four values behind the ZRL comparator. Two are in registers and hold the ZRL code index in the AC table. The other 2
One value is a constant, one is value zero, and the other one is 12 (ESCAPE index in MPEG and H.261). Constant zero is FL
Used for C. The constant 12 is used whenever the table number is less than 8 (and VLC).
One of the two registers is used if the table number is greater than 7 (and VLC), as determined by the low order bits of the table number.

A bit in the State Machine ROM is provided to enable the comparator and another bit is provided to invert its operation. T in the command
If the OKEN bit is set, the comparator output is ignored and replaced by the extension bit. This continues until the end of the token.

[1244] B. 2.2.5.3 "Huffman State Machine ROM" The command field in the Huffman State Machine is as follows: nxtstate [4: 0] The address to use in the next cycle. This address can be modified. statect1 Enables modification of the next state address. If zero, the state machine address is not modified, otherwise the LSB of the address is replaced with the value of one of the following two comparators:

[1245]

[Table 180] Note: In either case, if the next Huffman command is selected as the "Rerun Primitive Command", the state machine will select location 0, as appropriate for the command.
Will jump to 1, 2 or 3. eobct [1: 0] This controls the selection of the next Huffman command based on the EOB comparator and extn bits as follows:

[1246]

[Table 181] zrlct [1: 0] This controls the selection of the next Huffman command based on the ZRL comparator. If the conditions are met, take the state machine command, otherwise rerun the primitive command. In either case, if the eobctl + condition takes a demultiplexer command, then this (eobctl +) has priority as follows:

[1247]

[Table 182] smtab [3: 0] In the present invention, if the selected command is a state machine command, this is the table number used by the Huffman Decoder. However, in the case of the ZRL comparator machine, the zrltab [3: 0] field is preferentially used.

[1240] If a different table number does not need to be used depending on whether a ZRL match occurs, sm
tab [3: 0] and zrltab [3: 0] will have the same value. However, this is Lsim
Note that can lead to strange simulation problems in. In the case of MPEG, there is no explicit need to load the register indicating the Huffman index number for ZRL (JPEG only configuration). However, these are still selected, and in all cases where the ZRL comparator may be "unknown" (thus it does not matter which one is selected) smtab [3:
0] and zrltab [3: 0] have the same value, the ZRL comparator will be "unknown" and the next state will also be "unknown". zrltab [3: 0] This is the table number that will be used by the Huffman Decoder if the selected instruction is a state machine instruction. However, if the ZRL comparator is matched, the zrltab [3: 0] fields are used in preference.

[1240] If a different table number does not need to be used depending on whether a ZRL match occurs, sm
tab [3: 0] and zrltab [3: 0] will have the same value. However, this is Lsim
Note that can lead to strange simulation problems in. In the case of MPEG, there is no explicit need to load the register indicating the Huffman index number for ZRL (JPEG only configuration). However, these are still selected, and in all cases where the ZRL comparator may be "unknown" (thus it does not matter which one is selected) smtab [3:
0] and zrltab [3: 0] have the same value, the ZRL comparator will be "unknown" and the next state will also be "unknown". zrltab [3: 0] This is the table number that will be used by the Huffman Decoder if the selected instruction is a state machine instruction, to which the ZRL comparator fits. smvlc This is the VLC bit used by the Huffman Decoder if the selected command is a state machine command. aluzrl [1: 0] This field controls the selection of commands sent to the ALU. It can be a command from the parser state machine (stored at the start of the command sequence) or a command from the state machine:

[1250]

[Table 183] blueob This wire controls the modification of commands sent to the ALU based on the EOB comparator. This simply forces the ALU output mode to zinput. This is an arbitrary choice; any output mode away from none can be used. It sends a block end command word to the token formatting section, where it controls the proper formatting of the data tokens:

[1251]

[Table 184] The remaining fields are ALU command fields. These are properly documented in the ALU description.

[1252] B. 2.2.5.4 "Modification of Huffman State Machine" In one aspect of the state machine, the data unit index portion is an escape coded Tcoef.
It is necessary to "know" when the RUN part of the ficent is sent to the data unit index part. While this can be achieved with the appropriate bits in the control ROM,
Another approach has been used to avoid modifying the ROM. In this regard, the address entering the ROM is monitored and the address value 5 is detected. This is RUN
It is the appropriate place to be named in the ROM processing field. Of course, it would be obvious if one wanted to program the ROM to use another selected address value. In addition, the previously described approach of using bits of control ROM could have been utilized.

[1253] B. 2.2.6 "Overview Guided Tour" In the present invention, the Huffman Decoder is called hd, which actually contains the data unit index part (which is required due to the limitation of edited code generation). Therefore hd contains the following major blocks;

[1254]

[Table 185] The following description of the Huffman module is provided by a general description of the various subsystem areas shown in the detailed portion of the drawing, which will be readily apparent to one of ordinary skill in the art.

[1255] B. 2.2.6.1 “Explanation of“ hd ”” The logic for controlling the two-wire interface includes three ports normally controlled by the two-wire interface: data input, data output, and command. In addition to it,
There are two "valid" lines from the input shifter: token valid, which indicates that the token should be displayed in data [7: 0], and serial valid, which indicates that the data should be displayed on serial. is there.

The most significant signal generated is the enable going to the latch. The most important is e1 which is the enable for the ph1 latch. Many of the ph0 latches are not enabled, but two enables are provided for them, e0 associated with serial data and e0t associated with token data.

In the present invention, the done signal (done, n
otdone, and don of ph0 variants thereof
e0 and notdone0) indicate when the primitive Huffman command is complete. If a Huffman state machine command is executed, the done will be qualified at the completion of each source command, including all state machine commands. The signal notnew prevents the acceptance of new commands from the parser state machine until all Huffman state machine commands have been completed.

[1258] Regarding control of the information received from the data unit index part, the control logic for the size field is sent back to the Huffman decoder during JPEG coefficient decoding. This actually happens in two ways. If the size is exactly 1, this is the dedicated signal not
It is sent back to fbone0. Otherwise, it is sent back from the output of the data unit index section. (O
ut_data [3: 0] and signal fbvalid1 indicate that this should occur. The signal muxsize is generated to control the multiplexing of the fedback data into the command register (sheet 10). In addition, there is feedback with exactly 64 coefficients decoded. In JPEG the EOB is not coded in this case, so the signal forceeob is produced. By analogy, as mentioned above, there are actually two ways this can be implemented, with a signal for the feedback size. Normal feedback is given (jpe
geob), as the data is fed back, only latch i-971 is loaded and not cleared until a new Parser State Machine command is accepted. The signal forceeob is not actually produced until the Huffman code is decoded. in this way,
Fixed length codes (ie size bits) are not affected, but the following Huffman coded information is replaced by the forced block end. Size is 1, j
If pegob0 is used, only one bit is read, so i-1255 and i-1256 delay the signal until the correct time. Note that it is impossible for zero size to occur in this situation, since the only symbols with zero size are EOB and ZRL.

[1259] Decoding is tcoeff tab0
(Huffman decoding with Tcoeff table), mba tab0 (Huffman decoding with MBA table), and fairly random decoding of commands to make nop (no operation). There are several reasons to generate a nop. A fixed length code of size zero is one of them,
The forceeob signal is one of them (since no output should be read from the input shifter even if the output is made to signal EOB), and finally the table download nomination is the third reason.

Notfreckero (made by zero size FLC, NOP) ensures that the result is zero when the NOP command is used. Furthermore,
Invert indicates when the serial bit should be inverted before Huffman decoding (see Section B.2.2.1.1). ring indicates when the transform coefficient ring should be applied (see section B.2.2.1.2).

[1261] Decoding is further performed in codes-per.
Achieved with respect to the address of the bit ROM. These are built from a small datapath ROM. The signal is duplicated (eg, csha and csla). The address is a bit counter (bit [3: 0]) or a microprocessor interface address (key addr) depending on the UPI access to the selected block.
[3: 0]).

[1264] Additional decoding relates to UPI reading of registers such as those holding Huffman index values for JPEG tables (EOB, ZRL, etc.). Further included are these registers and co
It is a tri-state driver control for reading UPI of des-per-bit RAM.

[1263] Arithmetic data path decoding is also provided for certain significant number of bits. first bit
Is used in connection with Tcoeff's first coefficient trick, and bit five relates to applying a ring in the Tcoeff table. Note the use of forceeob to simulate the behavior of the EOB comparator matching the decoded index value.

For the extn bit, if the token is read from the input shifter, the associated extn bit is read with it. Otherwise, the last value of extn is saved. This allows the extn bit to be tested by the microcode program any time after the token has been read.

[1265] When zerodat is recognized, the upper 4 bits of the Huffman output data are forced to be zero. Since they only have valid values when decoding fixed length codes, they are zeroed when decoding VLC, tokens, or when the NOP command is implemented for any reason.

[1266] In addition, the circuit detects when each command is complete and issues a done signal. There are essentially two groups of reasons for being done; normal and exceptional. These are each processed by one of two three-way multiplexers.

The Lower Mux (i-1275) handles normal reasons. For FLC, the signal ndnflc
Is used. This is the output of the comparator that compares the bit counter with the table number. For VLC, the signal ndnvlc is used. This is the output from the arithmetic data path and directly reflects Equation 9. NO
For P commands or tokens, only one cycle is needed, so the system is unconditionally done.

In the present invention, the upper multiplexer (i-1
274) handles the exceptional case. If the decoder is expecting size feedback in JPEG decoding (fbexpctd0), the decoder is done because only one bit is needed. If the decoder is processing the first bit of the first coefficient using the Tcoeff table, it is a done if bit zero of the current index is zero (see section B.2.2.1.2). . If neither of these conditions are met, there is no exceptional reason to be a done.

The NOR gate (i-1293) finally transforms the done state. The condition caused by i-570 (ie the data is not valid) forces the done. This may seem a little strange. It is basically in preparation for the first command (done resets all counters, registers, etc.)
Used after reset to bring machine to done state.

The signal notdonex is necessary for detecting an error. If an error is detected, the normal done signal cannot be used because the done is forced anyway.
The use of done will provide a combined feedback loop.

Error detection and processing is performed by the circuit which detects all possible error conditions. These are i-
At 1190, both are 0 Red. In this case i-
1193, i-585 and i-584 form a 3-bit Huffman error register. Note i-1253 and i-1254, which disable the error if there is no "real" error (see Section B.2.2.3).

[1272] In addition, i-580 and i-579
Provides a simple state machine with associated circuitry to control the acceptance of the first command after error detection.

As described above, the control signal is delayed in the data unit index section and the ALU to match the pipeline delay.

[1274] Iotd bypass is the actual bypass signal sent to the data unit index section.
It is modified when the Huffman state machine is controlled to have to bypass whenever a fixed length code is decoded.

[1275] Aluinstr [32] is a bit that causes the ALU to feed back (condition code) to the parser state machine. Furthermore, when the Huffman State Machine is controlled, it is important that the signal be recognized only once (rather than each time one of the primitive commands is completed).

[1276] Aluinstr [36] is (other ALU
This is the bit that causes the ALU to step through the block counter (if the command bit specifies an increment). This too has to be certified only once.

In addition, these bits must be qualified only for ALU commands that output data for token conversion. Otherwise, the counter may be incremented before the first output to token conversion, resulting in an incorrect value of cc in the data token.

In the illustrated aspect of the invention, if the ALU is output to the token formatting unit, arun
Either ode [1] or alunode [0] will be low.

[1279] FIG. 127 shows the Huffman State Machine data path called hdstdp. In addition there is UPI decoding to read the output of the Huffman State Machine ROM.

[1280] Multiplexing is provided to handle the case of location (Section B 2.2.4.
See 6).

[1281] ALuinstr [3: 2] is modified by AL
Process the Uoutsrc command field to non-none (section B.2.2.5.3, al).
See description of ueob).

As for the command register for the Huffman Decoder block (x), each bit of the command has an associated multiplexer that selects between the possible sources of the command. Four control signals control this selection: Selhold causes the register to hold the current state.

[1283] Selnew loads a new command from the Parser State Machine. This also allows loading of registers holding the original Parser State Machine commands for later use.

[1284] Seold causes command loading from the register holding the command of the original parser state machine.

[1283] / selsm causes the loading of commands from the Huffman State Machine ROM.

In the case of the table number, the situation is a little more complicated because the table number is also loaded from the output data of the data unit index part (s
elholdt and muxsize).

[1287] Latch is Huffman State Machine ROM
Keep the current address in. Logic is possible 4
Detect which of the two commands is being executed. These signals are combined in the case of a new command to form the lower 2 bits of the start address.

[1288] The logic also detects when the output of the state machine ROM is meaningless (usually because the command is a "simple" command). Signal notsignor
Erom effectively disables the operation of the state machine, and in particular disables modification of commands sent to the ALU.

The circuit that produces fixstate0 controls the limited jumping capability of this state machine.

Decoding is also provided to drive the signal to the Huffman State Machine ROM. This is a data path style combined ROM.

[1289] The occurrence of escape run is described in Section B2.2.5.4.

[1292] Further, the decoding prepares the register holding the Huffman index number for the symbols such as ZRL and EOB. These registers can be loaded from the UPI or datapath. cent
The decoding at er (es [4: 0] and zs [3: 0] produces a select signal for the multiplexer which selects which register or constant value to compare with the decode Huffman index.

[1293] Regarding control logic for the Huffman State Machine. Here, Huffman State Machine RO
The "command" bit from M tells what to do next, and A
Various conditions are combined to determine how to modify the command word for the LU.

[1294] In the present invention, the signals notnew and not
old is used on sheet 10 to control the operation of the Huffman Decoder Command Register. They are the Huffman index comparators (n
eobmatch and nzrlmatch) together with the state machine ROM (section B.2.2.5.
(Explained in Section 3) in an obvious way.

[1295] Further, the source selection for the command sent to the ALU is performed. The actual multiplexing is done in the Huffman State Machine data path hfstdp. Four control signals are generated.

[1296] If you did not encounter the block end,
Either one of aluseldmx (which selects the Parser state machine command) or aluselsm (which selects the Huffman state machine command) will be made. In addition, the outsrc field of the ALU command is modified to push it into zinput.

[1297] The register holds the designated table number during the table download. Decoding is provided for the codes-per-bit RAM. Additional decoding recognizes when symbols such as EOM and ZRL are downloaded so that the Huffman index number register is automatically loaded.

As for the bit counter, the comparator detects when the correct number of bits are being read when reading the FLC. B. 2.2.6.2 “Explanation of“ hddp ”” The comparator detects the special value of the Huffman index. The register holds the value for the downloadable table. Multiplexer (meob)
[7: 0] and mzr [7: 0]) select the value to be used, and the exclusive-or gate and gating constitute a comparator.

[1299] Adders and registers are in Section B.
The expression described in 2.2.1 is directly evaluated. No further explanation is considered necessary here. The exclusive OR is described in Section B. It is used to invert the data (i-807) described in 2.2.1.1.

[1300] The code register is 12 bits wide.
Multiplexing arrangements are described in Section B.
Implement the ring permutation described in 2.2.1.2.

[1301] Regarding pipeline delay for data and decoding between serial data (index [7: 0]) and token data (ntken0 [7: 0]), the Huffman index value is ZR.
It is determined within the L and EOB symbols.

[1302] Codes-per-bit ROMs and their multiplexing are used to determine the tables to use. This arrangement is used because the table select information arrives late. Then all tables are accessed and the correct table is selected.

[1303] Regarding Codes-per-bit RAM, final multiplexing of code-per-bit ROM and code-per-bit RAM
The output of the occurs inside the block hdcpbram.

[1304] B. 2.2.6.3 “Explanation of“ hdstdp ”” In the present invention, Hdstdp is composed of two modules. hdstdel is concerned with delaying the parser state machine control bits until the appropriate pipeline stage, eg, when they are supplied to the ALU and token transforms. It processes only about half of the command words sent to the ALU, the rest is another module h
It is processed by dstmod.

[1307] Hdstmod includes Huffman State Machine ROM. Several bits of this command are used by the Huffman State Machine control logic. The remaining bits are used to replace the unprocessed portion in hdstdel of the ALU command word (from the Parser State Machine).

[1306] Hdstmod is self-explanatory and needs no explanation. There is only a pipeline delay register.

[1307] Hdstdel is also very simple, and RO
It is processed by a multiplexer which modifies the M and ALU commands. The rest of the circuit is the Huffman State Machine ROM
Relevant for UPI read access to half of the output. Buffers are also used for control signals.

[1308] B. 2.3 "Token Formatting" The Huffman decoder token formatting according to the present invention is at the end of the Huffman block. Its function is
As the name implies, it formats the data from the Huffman decoder into an appropriate token structure. The input data is multiplexed with the data in the micro instruction word under the control of the micro instruction word command field. Block has two modes of operation; data word and data token B. 2.3.1 “Micro Instruction Word”

[1309]

[Table 186] B. 2.3.2 “Operation mode”

[1310]

[Table 187] B. 2.3.2.1 “Data Word” In this mode, the 8 most significant bits of the input are sent to the output. The bottom 8 bits are either the bottom 8 bits of the input, the token field of the microinstruction word, or a mixture of both, depending on the mask field. The mask is out data [16: 8] = in data [1
6: 8] out data {7: 0] = (Token [7: 0]
& (Ff << mask)) represents the number of input bits in the mix of data [7: 0].

When the mask is set to 0x8 or more, the output data is equal to the input data. This mode is used for output words in non-data tokens. If the mask is set to 0, out data [7: 0] becomes the token field of the micro instruction word. This mode is used to output a token header that does not contain any data. If the token header contains data, the number of data bits is given by the mask field.

[1312] When external Extn (Ee) is set, o
ut extn = in extn, otherwise out extn = De. Bt, Eb is do
n't care. B. 2.3.2.2 "Data token" This mode is used for formatting with data tokens, signals, first coefficie.
There are two functions depending on nt. At reset, f
The first coefficient is set. When the first data coefficient arrives with a microinstruction word with cmd set to 1, out data
[16: 2] is set to 0x1 and out data
[1: 0] is Bt of the micro instruction word
Take the value of the field. This is the header of the data token. When this word is accepted, the coefficient with the command is loaded into register RL and the first c
oefientent takes a value of Eb. When the next coefficient arrives, out data [16: 0] takes the previous coefficient stored in RL. This is set to Eb when the block end is encountered and the first coeffic
ient is set, and the next data token, that is, if (first coefficient) {out data [16: 2] = 0x1 out data [1: 0] = Bt [1: 0] RL [16: 0] = in data [ 16: 0]} else {out data [16: 0] = RL [16: 0] RL [16: 0] = in data [16: 0]} out extn = -ensure that Eb is ready .

[1313] B. 2.3.3 "Notes" According to the invention, most of the command bits are supplied by the Parser State Machine in the usual way. However, in practice two fields are provided by the other circuit. The Bt field described above is directly connected to the output of the ALU block. The two bit fields are c
Gives the current value of c or color component. Thus, when the data token header is constructed, the lowest order 2 bits retrieve the color component directly from the ALU counter. Second, whenever the block end symbol id is decoded (or JPEG where 1 is assumed because the last coefficient in the block is coded), the Eb bit is qualified in the Huffman decoder. .

[1314] In extn in the Huffman decoder
The signal is pulled out. It only makes sense for tokens when the extension bits are supplied with the token word in the usual way.

[1315] B. 2.4 "Parser State Machine" The parser state machine of the present invention is actually a very simple piece of circuitry. Complex is the programming of the microcode ROM, which is described in Section B. See Section 2.5.

[1316] In essence, the machine consists of registers that hold the current address. This address is looked up in the microcode ROM to produce a microcode word. In addition, the address is incremented in a simple incrementer, which is one of the two possible addresses used for the next state. The other address is a microcode ROM
It is a field within itself. Thus, each instruction is potentially a jump instruction and can jump to the location specified in the program. If no jump is made, control proceeds to the next location in ROM.

[1317] A series of 16 condition code bits is provided. One of these conditions can be selected (by a field in the microcode ROM) and additionally inverted (also by a bit in the microcode ROM). The resulting signal selects either the incremented address or the jump address in the microcode ROM. One of the conditions is wired to evaluate as False. If this condition is selected, no jump will occur. Alternatively, this condition is selected and inverted so that a jump, an unconditional jump, always occurs.

[1318]

[Table 188] B. 2.4.1 "2-wire interface control" The 2-wire interface control according to the present invention is a little unusual in this block. There is a two wire interface between the Parser State Machine and the Huffman Decoder. It is used to control the progress of the command. The Parser State Machine will wait until a given command is accepted before proceeding to read the next command from ROM. In addition to that, the condition code is sent back through the wire from the ALU.

Each command has a bit in the microcode ROM that allows us to specify that we should wait for feedback. If this occurs, the new command will not be displayed until the feedback wire from the ALU is qualified after the command has been accepted by the Huffman Decoder. This wire, fb valid, indicates that the condition codes currently provided by the ALU are valid in the sense that they reflect the data associated with the command asking for feedback.

According to the present invention, that feature is utilized in constructing a conditional jump command that determines the next state to jump to a particular piece of data as a result of decoding (or processing). . Two-wire control means that it is uncertain when a particular command will reach a given processing block (that is, the ALU in this case), so without this convenience, conditions that depend on the data in the pipeline are Could not be tested.

[1321] Not all commands are sent to the Huffman Decoder. Certain instructions can be executed without the need for a data pipeline. These tend to be jump commands. A bit in the microcode ROM selects whether the command is displayed on the Huffman Decoder. Otherwise, the Huffman Decoder does not need to accept the instruction, so it can continue execution in these situations even if the pipeline is stalled.

[1322] B. 2.4.2 “Event processing” Two event bits are placed in the Parser State Machine. One is called the Huffman event and the other one
One is called a parser event.

The Parser Event is the simplest. The "condition" monitored by this event is only a bit in the microcode ROM. Thus, a command can cause a parser event by setting this bit. The command to do this typically writes an appropriate constant to the rom_control register, allowing the interrupt service routine to determine the cause of the interrupt.

After servicing the parser event (or immediately if the event is masked out), control recovers at the point where it stopped. If the command that caused the event has a jump command (that condition evaluates to real), the jump is done in the normal way. Therefore, it is possible to jump to the error handler after servicing by coding the jump.

[1325] The Huffman event is different. The monitored condition is the OR of the three Huffman error bits.
In reality, this situation is handled for parser events in a very simple way. However, the additional wire from the Huffman decoder, huffintrpt, is qualified whenever an error occurs. This causes control to jump to the error handler in the microcode program.

Therefore, when a Huffman error occurs, the sequence includes interrupt generation and block stop. After servicing, control is transferred to the error handler. ca
There is no ll mechanism and unlike normal interrupts it is not possible to return to a point in the microcode before the error that occurs following error processing.

[1327] huffman error does not occur and huf
You can certify fintrpt. This is Section B. As discussed in 2.2.3, no-e
It occurs in a special case of error error. In this case, no interrupt is generated (on the microprocessor interface) and control is passed to the error handler (in the microcode). Since the Huffman error register is apparent in this case, the microcode error handler can decide to respond in this situation and respond accordingly.

[1328] B. 2.4.3 “Special locations” There are several special locations in the microcode ROM. The first four locations in ROM are entry points to the main program. Control proceeds to one of these four locations at reset. The location to jump to is the ALU register, codings
td. Is determined by the coding standard selected in. Control proceeds to location zero because this location is itself reset to zero by a true reset. However, CED H TRAC
UPI register bit in E CED H TRACE
Only the Parser State Machine can be reset using RST. In this case, coding std
The register is not reset and control proceeds to the appropriate one of the first four locations.

[1329] Second four locations (0x004-
0x007) is used when a Huffman interrupt occurs. The actual error handler is typically located at each of these locations. Again, the choice of location is made as a result of coding standards.

[1330] B. 2.4.4 "Tracing" A tracing mechanism is implemented as a diagnostic aid. This allows the microcode to be single stepped. Bits in register CED H TRACE CED H TRACE EVENT and CED H
TRACE MASK controls this. As their name implies, they operate on regular event bits in a very similar way. However, due to some differences (especially the UPI interrupt does not occur), they are not grouped with other event bits.

[1331] When CED H TRACE MASK is set to 1, the tracing mechanism is turned on. A trace event occurs after each microcode instruction is read from ROM, but before it is displayed in the Huffman decoder. In this case, CED H TRA
CE EVENT becomes 1. It does not cause any interruption, so it must be registered. All microcode words are in register CED H KEY DMX
WORD 0-CED H KEY DMXWORD
Available in 9. The directive can be modified at this point if necessary. CED H TRACE EVE
When 1 is written in NT, the command is executed and CED
Clear H TRACE EVENT. Shortly thereafter, the next microcodeword to be implemented is read from the ROM and a new trace event occurs.

[1332] B. 2.5 "Microcode" Microcode is programmed using the assembler hpp, but it is a very simple tool, most of the sampling is done using the macro preprocessor. The standard C preprocessor cpp is used for this purpose.

The code is indicated as follows.

[1334] Ucode. u is the main file. First, this is tokens. Including h. Next, regfile. h is the ALU register map
Is defined. fields. u defines the various fields in the microcodeword and provides a defined symbol list for each possible bit pattern in the field. Next, the labels used in the code are defined. After this step, inst to define a large number of cppmacros that define the basic directives.
r. u is included. Next, errors. h defines the number that defines the parser event. Then unw
ord. u defines the order in which the fields are placed to build the microcodeword.

[1335] The remaining ucode. u is the microcode program itself.

[1336] B. 2.5.1 “Command” In this section, ucode. The various directives defined in u are described. In many cases, one theme (especially the ALU command) has few variations, so a description of all commands is not provided here.

[1337] B. 2.5.1.1 Huffman and Data Index Command In the present invention, the Huffman decoder uses the H NOP command. It is a no-operation command. Huffman does nothing in the sense that no data is decoded. The data produced by this command is always zero. Therefore, the relevant command is sent to the ALU.

[1338] Next command is token group; H TOK
SRCH, H TOKSKIP PAD, H TOKS
KIP JAPAN, H TOK PASS and H TOK
It is READ. They all read the tokens from the input shifter and send them to the rest of the machine. H T
OKREAD reads one token word. H TO
KPASS can be used to read all tokens up to zero extn bits. The associated command is repeated for each word of the token. HTOKSRCH discards all serial data before the token and reads one token word. The H TOKSKIP PAD skips the padding bits (H.261 and MPEG) and reads one token word. H TOKSKIP JPAD
Does the same for JPEG padding.

H FLC (NB) reads a fixed length code of NB bits.

[1340] H VLC (TBL) is (mnemoni
c, eg, sent as H VLC (tcoeff)), read vic using the indicated table.

[1341] The H FLC IE (NB) is the same as the H FLC, but the Ignore errors bit is set.

[1342] H TEST VLC (TBL) is H F
Similar to LC, but with the bypass bit set so that the Huffman index is sent through the unmodified data unit index portion.

[1343] H FWD R and H BWD R are AL
U-registers r fwd r size and r bwd r
Read the FLC of the size indicated by size.

HDCJ reads the JPEG style DC coefficients, the table number from the ALU.

[1345] HTC / OE / FF and HDCTC /
OE / FF reads the conversion coefficient. HDCTC / OE / FF
At, the first coeff bit is set, non
-For the intra block, while H T
C / OE / FF is intra after the DC term is read
Because of the block.

[1346] H NOMINATE (TBL) nominates a table for the next download.

[1347] H DNL (NB) reads the NB bit,
Download them to the designated table.

[1348] B. 2.5.1.2 “ALU Commands” In reality, there are too many ALU commands to explain in detail. The basic way that Mnemonics is built should be discussed, which should make the directive readable. Furthermore, these should be easily understood by one of ordinary skill in the art.

Many ALU commands relate to moving data from place to place, so the generic "load" command is used. At Mnemonic, A
It can be seen that LDxy loads the contents of y into x, ie the destination is recorded first and the source comes next.

[1350]

[Table 189] As an example, the LDAI loads the A register with data from the data input port of the ALU. If an ALU register file is specified, the nimanic will take an address so that LDAF (RA) will load A with the contents of location RA in the register file.

[1351] The ALU has the ability to modify the data as it is moved from source to destination.
In this case, arithmetic is indicated as part of the source data. Therefore, Mnemonic LDA AADDF
(RA) loads A with the current contents of the A register plus the contents of the indicated location in the register file. Another example is the LDA ISGXR, which takes input data, the signature extends from the bit indicated in the RUN register and stores the result in the A register.

[1352] In many cases, more than one destination is specified for the same result. Again, as an example, LDF LDA ASUBC (RA) loads the result of the A minus constant into both the A register and the register file.

[1353] Other Nimanix exists for special behavior. For example, CLRA is used to clear the A register and RMBC is used to reset the macroblock counter. These are fairly self-explanatory and can be found in instr. It is explained in the comments in u.

The one exception is the use of the suffix O to indicate that the result of the operation is to be output to token formatting in addition to normal operation. Thus, LDFI O (RA) stores the input data and sends it to token formatting.
Alternatively, if desired, this can be LDF
It could also be LDO I (RA). B. 2.
5.1.3 "Token Formatting Command" This is a T NOP "no operation" command. This is a misnomer because it is actually impossible to build a no-operation command. However, this is an ALU
Is not used for token formatting, so it is used when the directive is unimportant.

[1355] TTOK outputs the token word.

[1356] T DAT outputs a data token word (used only with Huffman State Machine commands).

T GENT8 produces a token word based on the 8 bits of the constant field.

[1359] T GENT8E is similar to T GENT8, but the extension bit is 1. T OPD (NB)
Is the NB bits of data from the bottom NB bits of the output, with the rest of the bits coming from the constant field.

[1359] T OPDE (NB) is the same as T OPD, but the extension bit is high.

[1351] T OPD8 is a short hand for T OPD (8).

[1361] T OPD8E is a short hand for T OPDE (8).

[1362] B. 2.5.1.4 “Parser State Machine Command” This command, D NOP is a no operation, that is,
Addresses are normally incremented and the Parser State Machine does nothing special. The rest of the command is sent to the data pipeline. No waiting occurs.

[1363] D WAIT is similar to D NOP, but waits for feedback to occur.

[1364] Simple jump group. D JMP (A
Nimanix, such as DDR) and DJX (ADDR), will jump if the conditions are met. The command is not output to the Huffman decoder.

[1365] External jump group. DXJMP (A
Nimanix such as DDR) and D XJNX (ADDR). These are similar to the simple counterparts above, but
The command is output to the Huffman decoder.

[1366] Jump and wait groups. D WJNZ
Nimanix such as (ADDR). These commands are output to the Huffman Decoder and the parser waits for feedback from the ALU before evaluating the state.

The following Nimanix is used for the state itself.

[1368]

[Table 190] D EVENT causes the event to occur.

[1369] D DELT is for the construction of default commands. This triggers the event, label df
Jump to the location with lt. This command is R
It should not be run as it is used to fill the OM and unused locations are trapped.

[1370] D ERROR triggers an event,
Label src that is assumed to attempt recovery from an error
Jump to h display. Section B. 3 "Huffman Decoder ALU" B. 3.1 "Introduction" In accordance with the present invention, the Huffman Decoder ALU sub-block provides general arithmetic and logical functionality for the Huffman Decoder block. It includes addition and subtraction operations, various types of sine expansion operations, and run-of input data.
It has the ability to format into sign-level triples. Furthermore, it has a flexible structure whose precise operation and organization is specified by the microinstruction words arriving at the ALU synchronously with the input data, ie under two-wire interface control.

In addition to the 36 bit command and 12 bit data input ports, the ALU has a 6 bit run port and an 8 bit constant port (actually above the token bus). All of these drive the bus of each width through the ALU datapath, except for the microinstruction words. Represents an extension bit, 17-bit run
Output with sign level (out data)
There is one bit in the microinstruction word. There is a two wire interface at each end of the ALU datapath and a set of condition codes that are output with their own valid signal, cc valid. ALU
There are register files accessible to other Huffman decoder sub-blocks and microprocessor interfaces via the.

[1372] B. 3.2.2 “Basic structure” The basic structure of the Huffman ALU is as shown in FIG. It contains the following components: Input block Output block Condition code block A register with source multiplexing Run register with source multiplexing (6
Bit) adder / subtractor with source multiplexing sine extension logic register file with source multiplexing (except output blocks) Each of these blocks pushes its output onto the bus running through the datapath, These buses are then used as inputs to the multiplexing for block sources. For example, the adder has its own datapath bus, which is one of the possible inputs to the A register. Similarly, the A register has its own bus which forms one of the possible inputs to the adder. In this respect, there is only a subset of all possibilities, as specified in section 7 on microinstruction words.

[1353] In a single cycle, it is possible to execute either an add base command or a sine extension base command. Moreover, both of them can be performed in a single cycle only if their operations are strictly parallel. In other words, a sequence of addition and sign extension or sign extension and addition is not allowed. The register file is read or written in a single cycle, but not both.

The output data has three fields: Run-6 bits Sign-1 bit Level-10 bits When the data is sent straight through the ALU, the least significant 11 bits of the input data register are the sign and Latched in the level field.

[1359] Limited ALU multi-cycle operation can be programmed. At this point, the number of cycles required is determined by the contents of the register file location, its address is specified in the microinstruction, and the same operation is repeated while the repeat counter is decremented to one.
This facility is typically used to perform a left shift, using an adder to add the A register to itself and store the result back in the A register.

[1376] B. 3.3 "Adder / Subtractor Subblock" This is a 12-bit wide adder and its input2
Has an arbitrary inversion to and an optional setting of carry-in bit. The output is a total of 12 bits, car
ry-out is not used. There are seven modes of operation: ADD: add carry of set to zero: input1 + input2 ADC: add carry of set to 1: input1 + input2 + 1 SBC: invert input2, carry of set To zero: input1-input2-1-SUB: invert input2 and set carry to 1: input1-input2- TCI: use SUB if 0, otherwise use ADD . It is used with input1 set to zero to get the magnitude value from the 2's complement value.

[1377] DCD (DC difference): If input2 <0, ADC is performed. If not, ADD is performed.

[1378] VRA (vector residual add): inpu
If t1 <0, perform ADC, otherwise SB
Perform C.

[1379] B. 3.4 "Sign extension sub-block" This is a 12-bit unit where the sign extends the input data from the size input in various modes. Size is 0
Is a 4-bit value from -11 (0 for the least significant bit, 11 for the most significant bit). The output is a 12-bit modified data value,
It is the "sign" bit.

[1380] In SGXMODE = NORMAL, si
All bits greater than or equal to ze-th bits are size-t
Takes the value of h bits. All of the following remain unchanged. The sign takes a value of size-th bit.
For example: data = 1010 1010 1010 size = 2 output = 0000 0000 0010, s
For ign = 0 SGXMOD = INVERSE, all bits greater than or equal to size-th bits take the reciprocal of size-th bits, while all of the following remain unchanged. The sign takes the reciprocal of size-th bits. For example: data = 1010 1010 1010 size = 0 output = 1111 1111 1111, s
For ign = 1 SGXMODE = DIFMAG, if the size-th bit is zero, all bits below the size-th bit are inverted, while all of the above remain unchanged. If the size-th bit is 1, then all bits remain unchanged. In both cases, the sine takes the reciprocal of size-th bits. This is AC
Used to get the magnitude of the difference value.
For example: data = 0000 1010 1010 size = 2 output = 0000 1010 1101, s
ign = 1 data = 0000 1010 1010 size = 1 output = 0000 1010 1101, s
ign = 0 In SGXMODE = DIFCOMP, size-th
All bits above (but not including size-th) bits take the reciprocal of size-th, while all bits below it remain unchanged. The sign takes the reciprocal of size-th bits. This is DC
Used to take the two's complement value for the difference value. For example: data = 1010 1010 1010 size = 0 output = 1111 1111 1110, s
ign = 1 B.I. 3.5 “Condition code” 2 of the condition codes used by Huffman block
There are two bytes (16 bits), of which specific bits are created by the ALU / register file. These are a sign condition code, a zero condition code, an extended condition code, and a change detection bit. The last two of these codes are not really condition codes because they are not used by the parser in the same way as others.

The sine, zero, and extended condition codes are updated when the parser commands them to update, and for each of these commands the condition code valid signal is pulsed only once high.

The sign condition code is simply a latched sign extended sign output, while the zero condition code is set to 1 when the input to the A register is zero. Extended condition code is OU
Input extension bit that is latched independently of TSRC.

The condition code can be used to evaluate a particular condition type: result equals constant-uses the subtract, zero condition result equals the register value-use the subtract, zero condition register Is equal to a constant-using subtraction, zero condition register bit set-using sign extension, sign condition-result bit set-using sign extension, using sign condition sign extension and sign condition code combination Note that it is possible to evaluate only one particular bit, rather than evaluating multiple bits as in the conventional logical AND case.

In the present invention, the change detect bit is made using the same logic as the zero condition code, but does not have a valid signal associated with it. The bit in the microinstruction is different from the one where the recently written value in the register file already exists (requires two clock cycles, first R
Set EG-MODE to READ, then REGMOD
(Which means setting E to WRITE) indicates that the change detect bit should be updated. Then, when the changed value is detected, the interrupt of the microprocessor is started. The change detection bit is reset by activating Change Detect in the normal way, but REGMODE is set to READ.
Is set to

[1385] The hardwired macroblock counter structure (which forms part of the register file, see below) also produces the following condition code: Mb
Start, Pattern Code, Resta
rt and Pic Start.

[1386] B. 3.6 “Register file” The address map for the register file is shown below. It is common to both ALU datapath and UPI 7
Use bit address space. Many locations are not accessible by the ALU, these are typically counters in the hardwired macroblock structure and registers in the ALU itself. The latter register has dedicated access, but the UPI
Form part of an address map for (0 in the table
Is oversized by one) A multibyte location has one ALU address and multiple U
It has a PI address. Similarly, the register group indexed by the component count, CC (indicated by I in the table), is treated as one location by the ALU. This facilitates microprogramming for initialization and resetting, as well as for block level operations.

All locations are read / write except for the dedicated ALU register (UPI read only) and all counters are reset to zero by a bit in the command word. The pattern code register has the capability of shifting to the right and its least significant bit is Pa
It forms the ttern Code condition bits. All registers in the hardwired macroblock structure are represented by M in the table, and counter (n-bit)
Are annotated as Cn.

In the present invention, a particular location is another part of the Huffman Subsystem Coding Standard,
Two r-size locations, and an ac huff table and dc huff for the Huffman decoder
It has content hard-wired into one location (1 bit word) for each of the tables.

Addresses in bold indicate that the location is accessible by both ALU and UPI, otherwise they have only UPI access. AL
A register group not pointed to by the U through the CC can have one ALU address specified in the command word, and the CC will be the data token that selects the physical location within the group to access.
The ALU address should have been that of the first address in the past, but could be of any register in the group. This is true in practice for either address, but it is also the case for multi-byte locations that are to be accessed using the lower address of the pair. Location 2E and 2
F is the top-level address map (displayed as T)
Note that it is accessible at, not only through the keyhole register. These two locations are also reset to zero.

The register file is physically partitioned into four "banks" to improve access speed, but this does not affect addressing in any way. The main table shows the allocations for MPEG and the two repeated sections are JPEG and H.264. Two
61 variations are provided respectively.

[1391]

[Table 191]

[1392]

[Table 192]

[1393]

[Table 193]

[1394]

[Table 194]

[1395]

[Table 195]

[1396]

[Table 196]

[1397]

[Table 197]

[1398]

[Table 198]

[1399]

[Table 199]

[1400]

[Table 200] B. 3.7 "Microinstruction Words" According to the present invention, the ALU microinstruction words are divided into a number of fields, each controlling a different aspect of the structure. The total number of bits used in the command word is 36 (plus 1 for extended bit input) and minimal encoding across fields is employed to maintain maximum flexibility in hardware configuration. To be done. The command word is partitioned as detailed below. The default field value, that is, the one that does not change the state of the ALU or register file is shown in italics.

[1401]

[Table 201]

[1402]

[Table 202]

[1403]

[Table 203]

[1404]

[Table 204]

[1405]

[Table 205] Section B. 4 “Buffer Manager” B. 4.1 "Introduction" This document describes the purpose, operation and implementation of the buffer manager according to the invention.

[1406] B. 4.2 "Perspective" The buffer manager has 4 for the DRAM interface.
Provide one address. These addresses are DRAM
Is the page address inside. The DRAM interface maintains two FIFOs in DRAM, a coded data buffer and a token data buffer. Therefore, for four addresses, there is a read address and a write address for each buffer.

[1407] B. 4.3 "Interface" The buffer manager is only connected to the DRAM interface and the microprocessor. The microprocessor is B.M. It need only be used to set the "initialization register" shown in 4.4. DRA
Interface with M interface, REQuest / ACKnowledg for each address
There are four 18-bit addresses controlled by the e-protocol. (The buffer manager lacks a 2-wire interface because it is not in the data path.) In addition, the buffer manager turns off the DRAM interface clock generator and turns on the DRAM interface scan chain.

[1408] B. 4.4 "Address calculation" The read address and write address for each buffer are 9
Made from 18 18-bit registers: -initialization register (RW from microprocessor) -BASECB-base address of coded data buffer-LENGTHCB-maximum size (in pages of coded data buffer) -BASETB-token data buffer Base address LENGTHTB-maximum size of token data buffer (in page) LIMIT-size of DRAM (in page) dynamic register (R from microprocessor)
O) -Coded data buffer read pointer for READCB-BASECB-NUMBERCB-Coded data buffer write pointer for READCB-READTB-Token data buffer read pointer for BASETB-NUMBERTB-Token data buffer write pointer for READTB : -Readaddr = (BASE + READ) mod L
IMIT writeaddr = (((READ + NUMBER)
mod LENGTH) + BASE) mod LIM
Since the IT buffer may wrap around the DRAM, m
The term od LIMIT is used.

[1409] B. 4.5 “Block Description” In the present invention, the buffer manager is
It consists of three top-level modules connected in a ring that monitors the interface junction. Module is bmprize (priority), bm
instr (command), bmrecalc (recalculation) is placed in the ring of that order, omsn
oop is placed above the address output.

[1410] Module, Bmprize is REQ /
ACK protocol, FULL / EMPT for buffer
The Y flag is processed and the state of each address, i.e. i
sit a valid address? To maintain. From this information, it tells bminstr which address (if any) should be recomputed.
In addition, it is a BUF indicating a FULL / EMPTY flag.
Buf, which manipulates CSR (status) microprocessor registers and also controls microprocessor write access to buffer manager registers
Access the microprocessor registers.

[1411] Module, Bminstr, is bmpr
When commanded by the size to calculate the address, it issues six commands (one every two cycles) to control bmrecalc to calculate the address.

[1412] Module, Bmrecalc is bmin
The address is recalculated under the command of str. It runs commands every two cycles and includes all of the initialization and dynamic registers and a simple ALU capable of addition, subtraction and modulus. FU it detects
Notify Sbmprize of LL / EMPTY state and when it is done, calculate address.

[1413] B. 4.6 "Block implementation" B. 4.6.1 "Bmprize" At reset, the buf access microprocessor register is set to 1, allowing the initialization register to be set. No address calculation is started while the buf access is reading back 1. Because, without a valid initialization register, those calculations are meaningless.

[1414] Once the buf access is fully qualified (writing a zero to it), its purpose is to enable all four addresses, so mbprti
ze serves to validate all addresses (recalculating them). At this stage, the buffer manager is "starting up" (i.e., all addresses have not been calculated) and thus no requests are qualified. Once all addresses are valid, the startup will be terminated and all requests will be qualified. From this point on, if the address becomes invalid (since it has been used and recognized once), it will be recalculated.

[1415] It is not necessary to prioritize addresses. Because the buffer manager can recalculate the address every 12 cycles, while the DRA
This is because the M interface, in the best case, can use the address every 17 cycles. Therefore, after startup, only one address is invalid at a time. Therefore, bmprize will recalculate invalid addresses that are not currently calculated.

[1416] In the invention, the startup is buf ac
No address is supplied to the DRAM interface during microprocessor access because the cess are re-entered each time it is qualified.

[1417] B. 4.6.2 "Bminstr" Module, Bminstr contains a MOD12 cycle counter (the number of cycles it takes to create an address). Note that even cycles start the command, while odd cycles finish the command. The upper 3 bits, along with whether it is a read or write computation, are decoded into a command for bmrecal as follows: For the read address:

[1418]

[Table 206] For write address:

[1419]

[Table 207] Note: The result of the last operation is always kept in the accumulator.

If there are no addresses to be recomputed, the cycle counter spins at zero thus giving a command to not write to any of the registers. This has no effect.

[1421] B. 4.6.3 The "Bmrecalc" module, Bmrecalc, performs one operation every two clock cycles. It is a bmi on an even counter cycle (start alu cyc)
Latch in command from nstr (and its buffer and io type) and
Latch the operation result on u cyc). The operation result is always stored in the Accum register in addition to the register specified by the command. Furthermore, e
On nd alu cyc, bmrecalc informs bmprize as to whether the use of the just calculated address fills or emptys the buffer, and when the address and full / empty are calculated successfully. addr).

[1422] Full / empty is calculated using the sign bit of the operation result.

[1432] The modulus operation is not a true modulus, but A mod B is implemented as follows: (A> B? (AB): A) However, this is only wrong in the following cases: Yes, A> (2B-1) This never happens.

[1424] B. 4.6.4 "Bmsnoop" module, Bmsnoop consists of four 18-bit super snowpers that monitor the address supplied to the DRAM interface. Snowpa is an external DRA
It must be super (ie, must be accessible by clock running) to allow on-chip testing of M. These snowpas are REQ
Must work on the / ACK system, and therefore
Different from the others on the device.

REQ / ACK is used on this interface as opposed to the 2-wire protocol. Because it is essential to send the information back to the sender (that is, to acknowledge receipt), and accept does not. Therefore, it closely monitors the FIFO pointer.

[1426] B. 4.7 "Register" In order to gain microprocessor write access to the initial register, a 1 should be written to buaccess and access is gained when buf access reads back 1. Conversely, it zeroes out the buf access to relinquish the microprocessor's write access.
Should write to. Access is gained when the buf access reads back zero. buf acces
Note that s is reset to 1.

Although the dynamic and initialization registers of the present invention can be read at any time, write access must be obtained to ensure that the dynamic registers are not changing the microprocessor.

The Initialization Register is intended to be written only once. By rewriting them,
You might get the buffer manipulated incorrectly. However, it is contemplated to increase the length of the buffer on the fly and have the buffer manager use the new length in a timely manner. No checking has been done to date to see if the value in the initialization register, for example, feels that the buffers do not overlap. This is the user's responsibility.

[1429]

[Table 208] In the table, D indicates a register bit and x does not indicate any register bit.

[1430]

[Table 209]

[1431]

[Table 210]

[1432]

[Table 211] B. 4.8 "Verification" Verification was performed in Lsim and in C-code as part of the top-level chip simulation, with small FIFO's placed on the dummy DRAM interface.

[1433] B. 4.9 "Test" Test coverage for bman is through a snowpaer in bmsnoop, a dynamic register (shown in B.4.4) and is done using a scan chain that is part of the DRAM interface scan chain. Section B. 5 "Reverse Modeler" B. 5.1 "Introduction" This document describes the purpose, operation and execution of the inverse modeler and token formatting (hspk) according to the present invention.

Note: hsppk is a hierarchical part of the Huffman decoder, but functionally part of the inverse modeler. Therefore, it is better discussed in this section.

[1435] B. 5.2 "Perspective" The token buffer lies between the model and hspk and can contain large amounts of data all in off-chip DRAM. To ensure efficient use of this memory, the data must be in 16 bit format. Token formatting "packs" the data from the Huffman Decoder into this format for the token buffer. Therefore, the inverse modeler "fetches" (unpacks) data from the token buffer format.

[1436] However, the main function of the reverse modeler is
To extend from a "run / level" code to a run of zero data, and then to levels. In addition, Inverse Modeler ensures that the data token has a factor of at least 64 and provides a "gate" for stopping streams that do not meet the startup criteria.

[1437] B. 5.3 "Interface" B. 5.3.1 "Hsppk" In the present invention, Hsppk has a Huffman decoder as input and a token buffer as output. Both interfaces are of 2-wire type, and input is 17
It is a bit token port and the output is a 16-bit "packed data" plus a FLUSH signal.
In addition, Hsppk is clocked from the Huffman clock generator and is therefore connected to the Huffman scan chain. B. 5.3.2 "Imodel" The Imodel has a token buffer start-up output gate logic (bsogl) as an input and an inverse quantizer as an output. Input from token buffer is 16-bit "packed data" plus block
end signal and the input from bsogl is one w
It is an irestream enable. Output is 11
Bit token port. All interfaces are controlled by a 2-wire protocol. Imode
l has its own clock generator and scan chain.

Both blocks have microprocessor access to the snowpaer only at their outputs.

[1439] B. 5.4 “Explanation of Blocks” B. 5.4.1 “Hsppk” Hsppk receives 17-bit data from Huffman and outputs 16-bit data to the token buffer.
This is accomplished by first truncating or splitting the input data into 12 bit words and then packing these words into a 16 bit format.

[1440] B. 5.4.1.1 "Splitting" Hsppk receives 17-bit data from inverse Huffman. This data is formatted to 17 bits using the following format.

F specifies format; E is extension bit; R is run bit; L is length bit (shown in sign magnitude) or non-data token bit; x is don't care.

[1442] Normal tokens only occupy the bottom 12 bits,
Take the following form: ExxxxxxLLLLLLLLLLL However, the data token has the run and level in each word in the following forms: ERRRRRRLLLLLLLLLLL which is divided into the following format: ERRRRRRLLLLLLLLLLL-> FRRRRRR00000 Format 1, ELLLLLLLLLLLFormat 0a or run is If it is zero format then 0 is used: E000000LLLLLLLLLLLL-> FLLLLLLLLLLL Format 0 In format 0 it is seen that the extension bit is lost and is assumed to be 1. Therefore, if the extension is zero, it cannot be used. In this case, format 1
Can be used unconditionally.

[1443] B. 5.4.1.2 "Packing" After splitting, all data words are 12 bits wide. All four 12-bit words are three 1s
"Packed" into a 6-bit word:

[1444]

[Table 212] B. 5.4.1.3 "Buffer Flushing" The DRAM interface of the present invention collects blocks, 32 16-bit "packed" words, and writes them to the buffer. This means that data can be pasted into the DRAM interface at the end of the stream if the block is only partially completed. Therefore, a flushing mechanism is needed. Therefore ,. Hspk signals the DRAM interface to unconditionally write the current partially completed block.

B. 5.4.2.1 “Imup” The Imup performs three functions: 4) 12 data from its 16-bit format.
Unpack into bitwords.

[1446]

[Table 213] 5) Maintain correct data during flushing token buffer.

By unconditionally writing the current partially completed block, useless data remains in the block when the DRAM interface flashes. Imup has to delete the useless data, ie all the data from the flash token, until the end of the block.

[1468] 6) Hold data until startup criteria are met. The data output from the block is conditional that a "stream enable" is accepted for each different stream from the buffer startup. Therefore, 12-bit data is output to hspk. B. 5.4.2.2 “Imex (Expande
r) ”In the present invention, Imex extends all run-length codes to a run of zeros and subsequent levels.

[1449] B. 5.4.2.3 "Impad (PADder)" Impad ensures that all data token bodies contain 64 (or more) words. It does this by padding the last word of the token with zeros. 64 data tokens in body
Not checked for having more words.

[1450] B. 5.5 "Block execution" B. 5.5.1 "Hsppk" Typically, splitting and packing are performed in one cycle.

[1451] B. 5.5.1.1 “Splitting” First, the format is determined: IF (datatoken) IF (lastformat == 1) use format 0a; ELSE IF (run == 0) use format 0; ELSE use format 1; Then the format bits are determined: format 0 format bit = 0; format 0a format bit = extension bit; format 1 format bit = 1; if format 1 is used, the code level must still be output at the code level. No new data should be accepted in the next cycle. B. 5.5.1.2 “Packing” The packing procedure cycles through every four valid data entries. The output of the 16-bit word is formed from the last valid word held, followed by the word. If this is not valid then the output is not valid either. The procedure is as follows:

[1452]

[Table 214] In the table, x indicates an undefined bit.

During valid cycle 0, no word is output because it is not valid.

[1454] A valid cycle number is maintained by the ring counter. It is incremented by the valid data from the splitter and the accepted output.

[1455] FLUSH (or picture en
d) When the token is received and the token itself is ready for output, a flash signal is output to the DRAM interface, resetting the valid cycles to zero. When the flush token arrives in something other than cycle 3, the flush signal must delay the valid cycle to ensure that the token itself outputs.

[1456] B. 5.5.2 "Imodel" B.I. 5.5.2.1 "Imup" As in the packer case, the last valid input is stored and combined with the next input to allow unpacking.

[1457]

[Table 215] In the table, x indicates an undefined bit.

[1458] A valid cycle number is maintained by the ring counter. The unpacked data includes token data, flash and picture end decoded from it. In addition, the format and extension bits are decoded from the unpacked data.

[1459] format bit is extn =
(Lastformat == 1) 11 databo
dy format = databody && (format
tbit && lastformatbit) For token decoding and to be sent to imex.

[1460] FLUSH (or picture en
d) The token is unpacked and output to imex,
All data is deleted until the block end signal is received from the DRAM interface (Val
id will be lowered).

[1461] B. 5.5.2.2 "Imex (EX
According to the present invention, the image is a four state machine for expanding run / level codes out. The state machine is as follows: State 0: Load runcount from runcode.

[1462] State 1: Decrease the run count,
Outputs zero.

[1463] State 2: Input data and output level; default state.

[1464] State 3: Illegal state.

B. 5.5.2.3 "Impad (PADder)" Impad is signaled by imex regarding the data token header. Then it counts the number of coefficients in the token body. When the token ends, before the 64 coefficient, a zero coefficient is inserted at the token end, completing it to the 64 coefficient. For example, 64 with unextended data headers inserted after them.
Has a zero coefficient of. Data tokens with a coefficient greater than 64 are not affected by impad.

[1466] B. 5.6 "Registers" The imodel and hspk of the present invention do not have microprocessor registers, except for their snowpers.

[1467]

[Table 216] In the table, V = valid bit; A = accept bit; E = extended bit; D = data bit.

[1468] B. 5.7 "Match" The selected stream flows through the Lsim simulation.

[1469] B. 5.8 "Test" The test coverage for the imodel at the input is through the token buffer output snowper, and at the output is through the snowper of the imodel itself. The logic is covered by the scan chain of the imodel itself.

The output of hspk can be accessed through the Huffman Output Snowpa. Logic can be seen through the Huffman scan chain. Section B. 6 “Buffer startup” B. 6.1 "Preface" This section describes the method and implementation of buffer startup according to the present invention.

[1471] B. 6.2 "Perspective" In order to ensure that the picture stream can be displayed smoothly and continuously, a certain amount of data must be collected before decoding can begin. This is called the startup condition. Coding standards can be translated into almost any amount of data that needs to be collected V
Specify the BV delay. It is the purpose of "buffer start-up" to ensure that all streams meet their start-up conditions before their data travels out of the token buffer and allows decoding. At the output of the token buffer (ie, the inverse modeler), it is held in the buffer by a conceptual gate (output gate). This gate is only opened for streams once its startup conditions are met.

[1472] B. 6.3 "Interface" The Bscntbit (buffer startup bit counter) is in the data path and communicates via a two wire interface and is connected to the microprocessor. In addition, it branches to bsogl (buffer start-up output gate logic) on a two wire interface.
Bsogl controls an imup (inverse modeler unpacker) that implements an output gate via a two-wire interface. B. 6.4 "Block Structure" The Bscntbit is in the data path between the start code detector and the coded data buffer. This one cycle block counts the valid data words that leave the block, and this number is the startup condition (or target) that will be loaded from the microprocessor.
Compare with. When the target is filled, bsogl is delivered. The data is unaffected by bscntbit. Bsogl lies between bscntbit and imup (in the reverse modeler). In effect, it is a string of indicators that the streams have met their targets. The column is moved by the stream that exits the buffer (ie, the flush token received in the data stream at the imup) when another "indicator" is accepted by the imup. If the column is empty (that is, if there are no streams in the buffer that filled its startup target), the stream in imup is stalled.

[1473] The sequence only has a finite depth,
It can be expanded indefinitely by breaking a row in bsogl, allowing the microprocessor to monitor that row. The mechanisms of these columns are referred to as the inner and outer columns, respectively.

[1474] B. 6.5 “Block execution” B. 6.5.1 "Bsbitcnt (Buffer Startup Bit Counter)" The Bscntbit counts all valid words input to the buffer startup. Counter (bs
ctr) is a programmable counter 16 to 24 bits wide. In addition, the bsctr comprises a carry look ahead circuit to give it sufficient speed. Bs
The width of ctr is programmed by ced bs prescale. It does this by raising the bit 8 to 16 so that it always carries a carry. Therefore, those bits remain effectively unused. The upper 8 bits of bsctr are used for comparison with the target (ced bs target).

[1475] Comparison (ced bs count ≧ ced
bs target) is performed by bscmp.

The target is derived from the stream when it is in the Huffman decoder and is calculated by the microprocessor. Therefore it will only be set after the stream start. Before startup, targetvalid is set low. Writing to ced bs target is tar
Set get valid high so that the comparison in bscmp is done. Comparison is ced bs co
When unt ≧ ced bs target is shown, the target valid is set low. The target is satisfied.

[1477] When the target is full, the count is reset. Note that it does not reset at the stream end. In addition, counting is disabled after the target is filled if it is before the stream end. The count saturates at 255.

If a stream end (ie flush) is detected at bsbitcnt, then abs
A flush event is created. If the stream ends before the target is filled, an additional event (bs flush before target m
et event) is created. When these events occur, the block is stranded. This allows the user to redo the target search for the next stream, or bs flush bef
For an ore target met event event, one of the following: 1) Write a zero target that will compel the target met, or 2) the target is unfilled and until combined with the last stream reaches the target. Note the advancing next stream. The target for this next stream should be adjusted accordingly.

[1479] B. 6.5.2 "BSOGL (Buffer Startup Output Gate Logic)" As described above, Bsogl is an indicator string indicating that the stream has satisfied its target.
The column type is set by ced bs queue (internal (0) or external (1)). This is a reset to select internal columns. The row depth determines the maximum number of filled streams that can exist in the coded data buffer, Huffman and token buffers. When this number is reached (that is, the row is full), bso
gl stalls the datapath with bsbitcnt.

[1480] The use of internal columns does not require any action from the microprocessor. However, if you need to increase the depth of the row, (ced to be set
ced bs to get access to bsqueue
set access, target met ev
ent and stream end event are enabled and external columns can be set (by abandoning access).

[1485] The outer column (count maintained by the microprocessor) is inserted into the inner column. The outer row has two events: target met event and st.
Maintained by the ream end event. These are just the servicequeue input,
service queue output, and register ced bs enable nxt stream
It is called m. In effect, target met eve
nt is the upstream end of the internal column that supplies the column. Similarly, ced bs enable nxt
stream is the downstream end of the internal column that consumes the column. Similarly, stream end e
Vent is a request for supplying a downstream stream; stream end event is ce.
Reset the dbs enable nxt stream. Two events should be serviced as follows: / * TARGET MET EVENT * / j = micro read (CED BS ENABLE NXT STM): if (j == 0) / * Is next stream enabled? * / (/ * No, enable * / micro write (CED BS ENABLE NXT STM, 1); printf (“enable next stream (queue = 0x% x) \ n”, (context->queeel); / * Yes, increment the queue of "target met" streams * / {queue ++; printf ("stream address ready enabled (queeue = 0x% x) \ n" / ST) * (context)) (context). if (queue> 0) / * are there any "target mets" left? * / (/ * yes, decrement the queue and enable anano ther stream * / queue ---; micro write (CED BS ENABLE NXT STM, 1); queue empty cannot enable next stream (queue = 0x% x) \ n ", queue; micro write (CED EVENT 1,1 << BS STREAM END EVENT) means / * * ar means / * cear; You can change from inside to outside, but the outside column (from "queue" above)
Ced to be reset when empty (from == 0)
ced to get access to bs queue
Set bs access, target get
It is only possible to change from outside to inside by masking the event and the stream end and giving up access.

[1482] On the other hand, the stream start-up condition check is disabled, ced bqueue (external) is set, and target met event and str are set.
mask end end event, ced bs
Set enable nxtstream. In this way all streams are always enabled. B.
6.6 "Microprocessor Register"

[1483]

[Table 217]

[1484]

[Table 218] In the table, D is a register bit, x is a nonexistent register bit, r is a reserved register bit, and, unless it is an interrupt service routine, to gain access to these registers. , Ced bs a
ccess is set to 1 and must be registered until it reads back 1. ced bs access
Access is relinquished by setting to zero.

[1485] Section B. 7. “DRAM Interface” B. 7.1 "Perspective" In the present invention, the spatial decoder, the temporal decoder and the video formatting unit each include a DRAM interface block for its special chip. The function of the DRAM interface in all three devices is
Data transfer from the chip to the external DRAM and data transfer from the external DRAM to the chip via the block address supplied by the address generator.

The DRAM interface typically operates from a clock that is asynchronous to the address generator and to the clocks of the various blocks through which data is sent. However, this asynchrony is easy to handle because the clocks operate at about the same period.

The data is normally transferred between the DRAM interface and the rest of the chips in a block of 64 bytes (the only exception being the prediction data in the temporal decoder). The transmission occurs by a device known as a "swing buffer". This is essentially a pair of DRAMs operated in a double buffered configuration, and DR
The AM interface fills or empties one RAM, while another part of the chip empties and fills another RAM. Another bus carrying the address from the address generator is associated with each swing buffer.

Each chip has four swing buffers, but the functions of these swing buffers differ in each case. For a spatial decoder, one swing buffer was used to transfer the coded data to the DRAM, another swing buffer was used to read the coded data from the DRAM, and a third swing buffer was tokenized. It is used to transfer data to DRAM and a fourth swing buffer is used to read tokenized data from DRAM. In a temporal decoder, one swing buffer is used to write intra or predicted picture data to DRAM, a second buffer is used to read intra or predicted picture data from DRAM, and the other two. One buffer is used to read the prediction data forward and backward. In the video formatting section, one swing buffer DR data
The other three swing buffers are used to read data from the DRAM, each of which is used for transmitting to AM and each of which has luminance (Y), red and blue color difference data (each C.
for r and Cb).

The following section is a DRAM according to the present invention.
The operation of the interface is described, which has one write swing buffer and one read swing buffer, which is essentially the same as the operation of the spatial decoder DRAM interface. This is illustrated in "DRAM Interface" in FIG.

[1490] B. 7.2 “General DRAM Interface” In FIG. 140, the interface for the address generator and the interface for the block that supplies data and takes data are all two-wire interfaces. The address generator can generate an address as a result of receipt of a control token, or simply generate a fixed sequence of addresses. The DRAM interface handles the two-wire interface associated with the address generator in a special way. Instead of holding the accept line high when it is ready to receive an address, the DRAM interface provides that the address generator supplies a valid address and processes it.
Set the accept line high for one clock period. Thus, the request / acknowledge (REQ / A
CK) Run the protocol.

A unique feature of the DRAM interface is the ability to communicate with the block that provides / accepts data completely separate from the address generator and other blocks. For example, the address generator may generate the address associated with the data in the write swing buffer, but the write swing buffer may generate an external DRAM.
It will not take any action unless it signals that there is a data block ready to be written to. However, no action will be taken unless the address is supplied to the appropriate bus from the address generator.
Furthermore, once one of the RAMs in the write swing buffer is filled with data, the other RAMs are also completely filled before the data input is stuck (the 2-wire interface accept signal is set low) and the DRAM
"Swinged" by the interface.

[1494] An important note in understanding the operation of the DRAM interface of the present invention is that, in a properly configured system, at least as much as the sum of all average data rates between the swing buffer and the rest of the chip. Very fast, the DRAM interface can transfer data between the swing buffer and the external DRAM.

Each DRAM interface includes a method to determine which swing buffer to service next. In general, this is either "round-robin" or a priority encoder, which in the case of round-robin is the next available swing buffer whose serviced swing buffer has recently come out of order, In the case of encoders, some swing buffers have higher priority than others. In both cases, additional requests will come from the refresh request generator, which has a higher priority than all other requests. The refresh request is generated from a programmable refresh counter via the microprocessor interface.

[1494] B. 7.2.1 “Swing buffer” FIG. 141 shows a write swing buffer. The operations are as follows: 1) Valid data is displayed at data in. As each piece of data is accepted, it is written to RAM1 and the address is incremented.

2) When RAM1 is full, the input data gives up control and sends a signal to the read side to indicate that RAM1 is ready to be read. This signal passes between two asynchronous clock regimes and therefore three synchronous flip-flops.

3) The next data item to arrive on the input side is written to RAM2, which is still empty.

4) If the round robin or priority encoder indicates that it is the turn to read this swing buffer, the DRAM interface will use the RAM
Read the contents of 1 and write them to the external DRAM. Then, as in (2), a signal is sent back across the asynchronous interface indicating that it is ready to be written to RAM1 again.

5) DRAM interface is RAM1
Empty, and "swing" it before the input side fills RAM2, the data can be continuously accepted by the swing buffer, otherwise RAM1 will be input when RAM2 is full. Until "swinged back" for use by the side,
The swing buffer will set its accept signal low.

6) This process is repeated indefinitely.

[1500] The operation of the read swing buffer is similar, but the input and output data buses are reversed.

[1501] B. 7.2.2 “External DRAM and Swing Buffer Addressing” The DRAM interface is designed to maximize the available memory bandwidth. Therefore, each 8x8 data block is arranged to be stored in the same DRAM page. In this way, the DRAM fast page access mode can be fully used, where one row address is provided, followed by many column addresses.
In addition, there are 8 data buses for external DRAM,
DRAM that can be 16 or 32 bits wide and is used
Convenience is provided to allow the quantity to fit the size and bandwidth requirements of a particular application.

In this example (showing how the DRAM interface on the spatial decoder works), the address generator provides the block address to the DRAM interface for each of the read swing buffer and the write swing buffer. . This address is DRA
Used as row address for M. The 6 bits of the column address are provided by the DRAM interface itself and these bits are also used as the address for the swing buffer RAM. The data bus for the swing buffer is 32 bits wide, so the external D
If the bus width for the RAM is less than 32 bits, before the next word is read from the write swing buffer or before the next word is written to the read swing buffer (read and write are related to the transmit direction to the external DRAM). ) Two to four external DRAM accesses must be made.

[1503] For the temporal decoder and video formatting part, the situation is more complicated. These will be described separately below.

[1504] B. 7.3 "DRAM Interface Timing" In the present invention, the DRAM interface timing block uses a timing chain to place the edges of the DRAM signal exactly 1/4 of the system clock period. Two quadrature clocks from the phase lock loop are used. These are combined to form a conceptual 2x clock. Any one chain is made up of two shift registers in parallel on opposite phases of the 2x clock.

[1505] First, there is one phase for the page start cycle and another for the read / write / refresh cycle. The length of each cycle is programmable via the microprocessor interface, after which the page start chain has a fixed length, the length of the cycle chain changing appropriately during page start.

[1506] When reset, the chain is cleared and a pulse is created. This pulse travels along the chain and is directed by state information from the DRAM interface. The DRAM interface clock is generated by this pulse. Each DRAM interface clock cycle corresponds to one DRAM cycle. Thus, the DRAM interface clocks are not at a constant rate because the DRAM cycles have different lengths.

In addition, the timing chain combines the pulses from the above chains with the information from the DRAM interface to provide output strobes and enables (notca).
s, notras, notwe, notee).

[1508] Section B. 8 "Inverse Quantizer" B. 8.1 "Introduction" This document describes the purpose, operation and implementation of the inverse quantizer (iq) according to the invention.

[1509] B. 8.2 "Perspective" The dequantizer reconstructs the coefficients from the quantized coefficients, the quantization weight, and the step size, all of which are transmitted in the data stream.

[1510] B. 8.3 "Interface" Iq is in the data path between the inverse modeler and the inverse DCT and is connected to the microprocessor. The data path connection is made via a two wire interface. The input data is 10 bits wide and the output data is 11 bits wide.

[1511] B. 8.4 “Inverse Quantizer Operation” B. 8.4.1 "H261 formula"

[1512]

[Equation 1] B. 8.4.2 "JPEG type"

[1513]

[Equation 2] B. 8.4.3 “MPEG type”

[1514]

[Equation 3] B. 8.4.4 "JPEG variation formula"

[1515]

[Equation 4] B. 8.4.5 "All other tokens" All tokens except data tokens must pass through the unquantized iq. Therefore: Floor (a) returns an integer as follows: (a-1) <floor (a) ≦ a a ≧ 0, a ≦ floor (a) <(a + i) a ≦ 0 Qi is a quantization coefficient.

[1516] Ci is the reconstructed coefficient.

[1517] Wi, j is a value in the quantization table matrix.

[1518] i is a coefficient index along the zigzag.

[1519] j is the quantization table matrix number (0≤j≤3).

[1520] B. 8.4.6 "Multiple Standards Combined" All of the above standards and variations thereof (control data not further modified by iq)
Can be mapped to one expression: OUTPUT = (2INPUT + k) (xy) / 16 with the following additional post dequantization features: -Add 1024-Convert sine magnitude to 2's complement notation-Every even number Round to the nearest odd towards zero • Saturate the result to +2047 or -2048.

The variables k, x and y for each variation of the standard, and the functions they use, are shown in Tables 219 and 220.

[1522] B. 8.4.6 “Multiple standards combined”

[1523]

[Table 219]

[1524]

[Table 220] B. 8.5 “Block Structure” Paragraph B. It can be seen from 8.4.6 and Tables 219 and 220 that one structure can be used for the multi-standard dequantizer. Its arithmetic block diagram is shown in Figure 142 "Arithmetic Blocks": The control for an arithmetic block can be functionally divided into two sections: A token for loading into a status register or quantization table. Decoding to the control signal of the status register.

The token is decoded in the next cycle, iqca, which controls the bank of iqcb of registers. It also controls access to the four quantization tables in igram. Arithmetic, the two multipliers and the post function are in iqarith. The complete block diagram for iq is shown in FIG.

[1526] B. 8.6 “Block execution” B. 8.6.1 "Iqca" In the invention, iqca is the token igram and i
It is a state machine used for decoding into control signals for registers in qcb. The state machine is reset with each new token, so it should be described as a state machine for each token. For example: QUANT SCALE (B.8.7.4, QUANT)
(See SCALE) and QUANT TABLE
The code for (see B.8.7.6 QUANT TABLE) is as follows: if (tokenheader == QUANT SCALE) {printf (report, "QUANT SCALE"); reg addr = ADDR IQ QUAN; QUANT. rnotw = WRITE; enable = 1;} if (tokenheader == QUANT TABLE) / * QUANT TABLE Token * / switch (substate) BL {case terpure resp. , (Headerextn? "(Full)": "(empty)")); nextsubstate = 1; insertne t = (headerextn? 0: 1); reg addr = ADDR IQ COMPONENT; rnotw = WRITE; enable = 1; break; case 1: / * quantification table body * sTA * print (QR); headerextn? "(full)": "(empty)")); nextsubstate = 1; insertnext = (headerextn? 0: (qtm addr 63 == 0)); reg addr = USE twen wertn (wr qn; r). READ); enable = 1; break; default: printf (report, ERROR in iq quanti) ation table tokendecoder (substate% x) \n, substate); break; case}} substate is state in the token, QU
The ANT SCALE has only one substate, for example. However, QUANT TABLE has two sub-states, one for the header and one for the token body.

The State Machine is implemented as a PLA. The unrecognized token does not generate any word lines and does not cause the PLA to output default control.

Therefore, iqca supplies the address to the igram from the BodyWord counter and inserts the word into the stream, eg, in the unextended QUANT TABLE (see B.8.7.4). This is accomplished by stalling the input while keeping the output valid. Subsequent block (iqcb or iqarit
The word can be filled with the correct data in h).

[1529] iqca is one cycle in the data path controlled by the two-wire interface.

[1530] B. 8.6.2 “iqcb” In the invention, iqcb holds an iq status register. Under the control of iqca, iqcb loads or unloads them from / to the datapath.

The status register controls the XY multiplier term and the post-quantization function.
To the control wire for (Table 219, Table 2
20).

[1532] The sign bit of the data path is now separated and sent to the post-quantization function. In addition, zero value words on the data path are detected here. Then the arithmetic is ignored and zeros are maxed out in the datapath. This is the simplest way to follow the iq "zero-in;zero-out" spec.

The status register is the register iq a.
ccess is set to 1 and is accessible to the microprocessor only when 1 is read back. In this situation, iqcb stalls the datapath, thus the register has a stable value and no data is diverted in the datapath.

Iqcb is one cycle in the data path controlled by the two wire interface.

[1535] B. 8.6.3 "Iqram" Iqram shall support for four Quantization Table Matrices (QTM), each of 648 bits. Therefore, it is a 2568-bit 6-transistor RAM capable of 1 read or 1 write per cycle. The RAM is surrounded by 2-wire interface logic that receives its control and writes data from iqca. It reads the data to iqarith. Similarly, igram occupies cycles in the same datapath as iqcb.

The RAM is read and written by the microprocessor when iq access reads back 1. RAM is a keyhole register, iq qtm ke
placed behind yhole, iq qtm keyho
Addressed by le addr. iq qtm
Access to keyhole is iq qtm key
It will increment the address it points to, held in the hole addr. Similarly, iq qtm ke
Yhole addr can be written directly.

[1537] B. 8.6.4 "iqarith" Note that iqarith has three functions that are pipelined and split over three cycles. Functions are discussed below (see Figure 140).

[1538] B. 8.6.4.1 "XY Multiplier" This is the 5 (X) x8 supplied to the datapath multiplier.
(Y) bit carry-save unsigned multiplier. The multiplier and multiplicand are selected on the control wire from iqcb. The multiplicative is in the first cycle and the decomposition adder is in the second cycle.

At the input to the multiplier, the data from iqram can be muxed into the datapath to read the QUANT TABLE into the datapath.

[1540] B. 8.6.4.2 “(XY) * Datapath Multiplier” This 13 (XY) × 12 (datapath) bit carry-save unsigned multiplier is split over three cycles of the block. 3 partial products in 1st cycle, 2nd
There are seven in the cycle and the remaining two in the third cycle.

[1541] All outputs from the multiplier are 2047 or less (non coefficient) or saturated to + 2047 / −2048, so the upper 12
The bits do not have to be decomposed. Therefore, the decomposition adder is only 2 bits wide. For the rest of the high order bits, zero detection is sufficient as the saturation signal.

[1542] B. 8.6.4.3 "Post-quantization function" The post-quantization function: -Adds 1024-Converts from sine magnitude to two's complement notation-Rounds all even numbers toward zero to the nearest odd number • Saturate the result to +2047 or -2048 • Set the output to zero (see B.8.6.2).

The first three functions are performed with a 12-bit adder (pipelined to the second and third cycles). From this, we know what each function needs, and they are combined on one adder.

[1544]

[Table 221] As one of ordinary skill in the art will recognize, care must be taken in re-programming these features as they depend on each other when combined.

Saturation value, zero and zero + 1024 are muxed into the data path at the end of the third cycle.

[1546] B. 8.7 "Dequantizer Token" The following description defines the action of the dequantizer for each token tp to which the dequantizer responds. In all cases, the token is sent to the output of the dequantizer. In most cases, tokens are not modified by the dequantizer with the exceptions noted below. All unrecognized tokens are sent unmodified to the output of the dequantizer.

[1547] B. 8.7.1 "Sequence Start" This token is stored in the register iq prediction.
mode [1: 0] and iq mpeg index
Ensure that ction [1: 0] is set to zero.

[1548] B. 8.7.2 "Coding Standard" This token is the current standard (M
PEG, JPEG or H.264. 261), based on iq
Ensure that the appropriate value is loaded into standard [1: 0].

[1549] B. 8.7.3 "Prediction Mode" This token is iq prediction mode.
Load [1: 0]. The prediction mode token possesses more than one bit, but the dequantizer only needs access to the two lowest order bits. These determine if the block is intra-coded.

[1550] B. 8.7.4 "Quantization Scale" This token is iq quant scale [4:
0] is loaded.

[1551] B. 8.7.5 Date In the present invention, this token owns the actual quantized coefficients. The token head contains two bits that identify the color components, which are loaded into iq component [1: 0]. The next 64 token words contain the quantized coefficients. These are modified as a result of the dequantization process and replaced with the reconstructed coefficients.

If exactly 64 expansion words are not present in the token, the dequantizer's behavior is unlimited.

The data token at the input of the dequantizer owns the quantized coefficient. These are represented by 11 bits (10 bits + sign bit) in the sign magnitude format. Value "minus zero"
Should not be used, but is interpreted exactly as zero.

The data token at the input of the dequantizer owns the reconstructed coefficients. These are represented in 12 bits (11 bits + sign bit) in 2's complement format. A data token at the input will have the same number of token expansion words as it had at the input of the dequantizer.

[1555] B. 8.7.6 “Quantization table (QU
ANT TABLE) "This token can be used to load a new quantization table or read the current table. In the inverse quantizer, the token will typically be used to load a new table decoded from the bitstream. The operation of reading the current table is useful in the quantizer in front of the encoder if that table is to be encoded in the bitstream.

The token head contains two bits that specify the table number to be used. These are placed in iq component [1: 0]. Note that this register now contains the "table number", not the color component.

If the extension bit in the token head is 1, then the inverse quantizer expects there to be exactly 64 extension token words. Each is interpreted as a quantized table value and placed in successive locations in the appropriate table starting at location zero. The 9th bit of each extended token word is ignored. The tokens are also sent unmodified to the output of the dequantizer in the usual way.

If the extension bit of the token head is zero, the dequantizer will read consecutive locations in the appropriate table starting at location zero. Each location becomes an extended token word (9th bit becomes zero). At the end of this operation,
The token will contain exactly 64 extended token words.

The operation of the dequantizer answering this token is not limited for all extended word numbers except zero and 64.

[1560] B. 8.7.7 “JPEG Table Select” This token is iq jpeg indirectio
Used to load / unload translations of color components for table numbers to / from n. These translations are used in JPEG and other standards.

The token head contains two bits that identify the color component of current interest. These are iq comp
placed in onent [1: 0].

If the extension bit of the token head is 1, then the token is one extension word, iq jpeg i
ndirection [2iq component
[1: 0] +1: 2 iq component [1:
0]] contains the lowest 2 bits written to the location. The value just read is the token expansion word (the upper 7 bits are zero). At the end of this operation, the token will contain exactly one token expansion word.

[1563]

[Table 222] B. 8.7.8 "MPEG Table Select" This token is used during processing via the MPEG standard to define whether to use the default or user-defined quantization table. The token head contains 2 bits. Bit 0 of the header is iq mpeg direction
Determines which bit is to be written. Bit 1 is written to that location.

[1564] iq mpeg direction
The [1: 0] register is cleared by the sequence start token, so it would only be necessary to use this token if the user-only quantization table was sent in the bitstream.

B. 8.8 "Microprocessor Register" B. 8.8.1 "iq access" To get microprocessor access to any iq register, iqaccess must be set to 1 and must be registered until it reads back 1 (B.8). See 6.2). Failure to do this will result in the register being read still being controlled by the datapath and not stable. igram
, The access is locked out and reads back zero.

Writing a 0 to iq access releases control to the datapath.

[1567] B. 8.8.2 “Iq coding
standard [1: 0] ”This register holds the coding standard implemented by the inverse quantizer.

[1568]

[Table 223] This register is loaded with the Coding Standard Value.

Although this is a 2-bit register, 8 bits are currently allocated in the memory map, and future implementations will be able to handle more than the above standards.

[1570] B. 8.8.3 “Iq mpeg in
direction [1: 0] "This 2-bit register is used during the MPEG decoding operation to record which quantization table is to be used.

[1571] Iq mpeg direction
[0] controls the table used for intra-coded blocks. If it is 0, then quantization table 0 is used and expected to contain the default quantization table. If it is 1, quantization table 2
Is used and is expected to contain a user-specific quantization table.

[1572] This register is an MPEG TABLE S
It is loaded by the ELECT token and set to 0 by the sequence start token.

[1573] B. 8.8.4 “Iq jpeg in
direction [7: 0] "This 8-bit register determines which four quantization tables to use for each of the four possible color components that occur in the JPEG scan.

[1574] Bits [1: 0] hold the table number used for component 0.

[1575] Bits [3: 2] hold the table number used for component 1.

Bits [5: 4] hold the table number used for component 2.

[1577] Bits [7: 6] hold the table number used for component 3.

[1580] This register is a JPEG TABLE S
Affected by the ELECT token.

[1579] B. 8.8.5 “iq quant s
calle [4: 0] ”This register holds the current value of the quantization scale function. This register is loaded by the QUANT SCALE token.

[1580] B. 8.8.6 “iq compone
nt [1: 0] ”This register is a quantization table matrix (QTM)
Holds the value translated into a number. It is loaded by token number.

The data token header causes this register to be loaded with the color component of the block being processed. This information is used only in JPEG and JPEG variations to determine the QTM number, which is the reference to iq j.
Process with peg direction [7: 0]. Other standards include iq component
[1: 0] is ignored. The JPEG table select token causes this register to be loaded with color components. Then it is iq accessed by the token body
Used as an index into jpeg_indirection [7: 0].

The quantizer scale token causes this register to be loaded with the QTM number. This table is then loaded (if an expanded form of token is used) from the token or read from the table to form an appropriately expanded token.

[1583] B. 8.8.7 “iq predict
ion mode [1: 0] ”This 2-bit register holds the prediction mode used for the following blocks. The only use for the inverse quantizer to make use of this information is to determine whether intra coding is used. If both bits of the register are 0, the next block is intra-coded.

[1584] This register is loaded with the Prediction Mode Token. This register is set to 0 by the sequence start token.

[1585] Iq prediction mode
[1: 0] has no effect on operation in JPEG and JPEG variation modes.

[1586] B. 8.8.8 “Iq jpeg in
direction [7: 0] "Iq jpeg direction is used as a lookup table to translate the color components into QTM numbers. Therefore, the iq component is, as shown in Table 222, iq jpeg indirec
It is used as an index for a section.

This register location is directly written by the JPEG table select token if the expanded form of the token is used.

This register location is directly read by the JPEG table select token if the non-extended form of the token is used.

[1589] B. 8.8.9 “Iq quant t
[3: 0] [63: 0] [7: 0]] There are four quantization tables, each having 64 locations. Each location is an 8-bit value.
Value 0 should not be used in any location.

[1590] These registers are Ig of 8.6.3
It is implemented as the RAM described in ram.

[1591] These tables can be loaded using quantization table tokens.

Note that the data in these tables is stored in zigzag scan order. Many documents display the quantized table values as a square 8x8 number array. The DC term is typically on the upper left, with horizontal frequencies increasing from left to right and vertical frequencies increasing from top to bottom. The Karl table must be read along a zigzag scan path as the numbers are placed in a quantization table with consecutive i's.

B. 8.9 "Microprocessor Register Map"

[1594]

[Table 224] B. 8.10 "Test" The test coverage for the dequantizer at the input is through the output quantizer of the dequantizer and at the output through the snowpacker of the dequantizer itself. The logic is covered by the scan chain of the dequantizer itself.

When the ramtest signal is recognized, iq
Access to igram can be obtained regardless of access. Section B. 9 “IDCT” B. 9.1 "Introduction" The purpose of describing the Inverse Discrete Cosine Transform (IDCT) block here is to provide an engineering source for IDCT. It contains the following information: -The purpose and main features of the IDCT-How it was designed and validated-Structure In addition, its purpose is to provide those skilled in the art with sufficient information for such an explanation to facilitate or aid the following tasks: It is to be.

[1596] -Recognizing the IDCT as a "silicon macro function" -Integrating the IDCT into another device-Developing a test program for the IDCT silicon-Modifying, redesigning or maintaining the IDCT-Developing a forward DCT block B. 9.2 “Perspective” The Discrete Cosine Transform / Zig Zag (DCT / ZZ) transforms on blocks of pixels, each block displaying a screen area 8 pixels high by 8 pixels wide. The purpose of the transform is to display the pixel blocks in the frequency domain, which is sorted by frequency. Since the eye is sensitive to the DC components in the picture, but less sensitive to the high frequency components, the frequency data allows each component to reduce magnitude separately according to the eye sensitivity. The magnitude reduction process is known as quantization. The quantization process reduces the information contained in the picture, i.e. the quantization process is lossy. Lossy processes provide overall data compression by removing some information. The frequency data is classified so that high frequencies are easily quantized to 0, and all appear consecutively. The consecutive 0s mean that coding run-length coding is not a lossy process in general, but coding data that is quantized by using a run-length coding scheme results in further data compression. To do.

Takes the frequency data into which the IDCT block (which actually contains the inverse zigzag RAM or IZZ, and the IDCT) is classified and transforms it into spatial data. This inverse classification process is a function of IZZ.

[1598] The picture decompression system of which the IDCT block forms a part specifies pixels as integers. This means that the IDCT block takes an integer value and must produce an integer value. However, since the IDCT function is not based on integers, the internal number representation uses the fractional part to maintain internal precision. Although full floating point arithmetic is preferred, the implementation described here uses fixed point arithmetic. There is a slight loss of precision when using fixed point arithmetic, but the precision for this implementation is H.264. 261 and the precision specified by IEEE is exceeded.

[1599] B. 9.3 "Design Goals" According to the present invention, the main design goal was to design a functionally accurate IDCT block that uses a minimum of silicon area. Furthermore, although the design was required to run at a clock rate of 30 MHz under specified operating conditions, it was thought that it should also have future applicability. Higher clock rates will be needed in the future, and the design's architecture will allow this wherever possible.

[1600] B. 9.4 “Description of IDCT interface” The IDCT block has the following interfaces.

[1601] 12-bit width token data input port-Bit width token data output port-Microprocessor interface port-System service input port-Test interface-Resync signal Both token data ports are the standard 2-wire interface described above It is a type. The width shown refers to the number of bits in the data representation, not the total number of wires in the port.
Additionally associated with the input token data port are the clock and reset signals used for resynchronization to the output of the previous block. In addition, there are two resync clocks associated with the output token data port and used by the blocks that follow.

[1602] The microprocessor interface is standard and uses a 4-bit address. In addition, there are three externally decoded select inputs, which are used to select the address space for events, internal registers, and test registers. This mechanism provides the flexibility to map the IDCT address space to different locations within different chips. In addition, one event output, idcevent
And two i / o signals, n derrd and n serrd
There are event tristate data wires that are externally connected to the appropriate bits of the IDCT and microprocessor non-data buses.

The system service port consists of a standard clock signal and a reset input signal, and a two phase override clock and associated clock override mode select input.

[1606] The test interface includes a JTAG clock signal, a reset signal, scan path data, a control signal, and ramtest and chiptest inputs.

[1605] In normal operation, the microprocessor port is inactive because the IDCT does not require microprocessor access to perform its designated function. Similarly, test interfaces are only active when testing or verification is required.

[1606] B. 9.5 “Arithmetic Basis for Discrete Cosine Transform” In video bandwidth compression, the input data represents the rectangular part of the picture. Therefore, the transformation applied must be two-dimensional. Two-dimensional transforms are difficult to calculate effectively, but two-dimensional DCTs have the property of being separable. Separable transformations can be calculated along each dimension separately from the other dimensions. To do this, a one-dimensional I specially designed for mapping to hardware is used.
It uses the DCT algorithm; it is not suitable for software models. One-dimensional algorithms are applied sequentially to obtain two-dimensional results.

Two-dimensional DC for N × N pixel block
The arithmetic definition of T is as follows:

[1608]

[Equation 5] The above definition is arithmetically matrix equivalent during multiplication and is equivalent to multiplying two N × N matrices twice in succession. One-dimensional DCT is equivalent to multiplying two NN matrices arithmetically. Operationally, in the two-dimensional case: Y = [XC] TC where C is a matrix of cosine terms.

[1609] Thus, the DCT is sometimes described in terms of matrix operations. It should be emphasized that the description of the matrix is convenient because of the arithmetic reduction of the transformation, but this only simplifies the notation. Note that the 2 / N term dominates the DC level. The constants c (j) and c (k) are known as normalization functions.

[1610] B. 9.6 "IDCT Transform Algorithm" The algorithm used to compute the actual IDCT transform is "fast", as will be described in more detail below.
Should be an algorithm. The algorithms used are optimized for efficient hardware structure and implementation. The main features of the algorithm are the use of 〓2 scaling to remove one multiplication, and the transformation of the algorithm designed to produce greater symmetry between the upper and lower sections. This symmetry allows many of the most expensive computing parts to be efficiently reused.

In the diagram showing the algorithm (FIG. 145), the symmetry between the upper and lower halves is apparent in the middle part. The last column of adders and subtractors also has a symmetric part, allowing the adders and subtractors to be combined at a relatively low cost (4 adders / subtractors are 4 adders and 4 adders as shown). Much smaller than a subtractor).

[1612] Note that all outputs of the one-dimensional transform are estimated by 〓2. This means that the last two-dimensional answer is estimated by 2. This can be easily corrected in the final saturation and shifting rounding stage.

The illustrated algorithm was coded in double precision floating point C and the results were compared (using unambiguous matrix multiplication) to the reference IDCT. The next stage is used to encode a bit-accurate integer version of the algorithm in C (no timing information included), which verifies the performance and accuracy of the algorithm when it will be implemented on silicon. Could have been used to An acceptable conversion error is H.264. 261 standard, this method is used to implement a bit-accurate model and measure the accuracy conveyed.

FIG. 146 shows the overall IDCT structure in a way that illustrates the commonality between the upper and lower sections, and shows the points at which intermediate results need to be stored. The circuit is time division multiplexed so that the upper and lower sections are calculated separately.

[1615] B. 9.7 "IDCT Transform Structure" As mentioned above, the IDCT algorithm is optimized for an efficient structure. The main features of the resulting structure are: -significant reuse of expensive arithmetic operations-a small number of multipliers, all of which are constant factors rather than multi-purpose (reducing multiplier size, separate Eliminates the need for coefficient storage of a small number of latches, slightly needed to pipeline the structure, operations are arranged so that only one decomposition operation is required per pipeline stage Can be arranged to produce results in natural order No complex crossbar switching or significant multiplexing (both expensive in final implementation) Two carry save operations (one add, one add, 1
The advantage that is derived from the decomposition result is to remove (one is subtraction) the structure that allows each stage to take 4 clock cycles, thus eliminating the need for very fast (large) arithmetic operations. A structure that would support much faster operation than current 30MHz pixel clock operation by simply changing the decomposition operation from a slow ripple carry to a large, fast carry look ahead version. While decomposing operations require the largest part of the time required in each stage, speeding up only these operations has a significant impact on overall operational speed, while increasing the overall size of the transformation Can be relatively small. In addition, increased speed can also be achieved by increasing the depth of the pipe lining.

[1616] The control of the conversion data flow is very clear and efficient.

[1617] Diagram of one-dimensional conversion fine structure (FIG. 150)
Show how the algorithm can be mapped to a small set of hardware resources to achieve the required performance. Control of this structure is achieved by adapting a "control shift register" to the data flow pipeline. This control is clear to the design and efficient in the silicon layout.

[1618] The control signal (lat) specified in FIG.
ch, sel byp, etc.) are the various enable signals used to control the latches, through which the signals flow. The clock signal for the latch is not shown.

[1619] While the transformation structure minimizes the transformation size,
Some implementation details are important in terms of meeting the required accuracy criteria. Commonly used techniques fall into two main classes.

[1620] -Maintaining the maximum dynamic range with a fixed word width in each intermediate state by controlling the fixed point position individually.

Use the statistical definition of accuracy requirements to achieve accuracy by selecting operations of arithmetic operations (rather than increasing accuracy by simply increasing the word width of the entire conversion).

A clear way to design a transform would consist of simply performing a fixed point with a fixed word width sized large enough to achieve precision. Unfortunately, this approach results in larger word widths and thus large conversions. The approach used in the present invention is to maximize the available dynamic range for a particular intermediate value,
Allow the fixed-point position to change through the transformation to achieve the maximum possible precision.

Since the available results are statistically specified, selective adjustments can be made to any intermediate value to improve the overall accuracy. The adjustment selected is just a manipulation of the LSB calculation, which is almost non-expensive. An alternative to this technique is to increase the word width, but at a significant cost. The adjustment is to effectively weight the results in a given direction, if the final result was previously found to tend to go in the opposite direction. Adjusting the fractional portion of the results can effectively shift the overall average of these results.

[1624] B. 9.8 "Description of IDCT Block Diagram" The block diagram of the IDCT shows all the blocks related to the processing of the token stream. This diagram, FIG. 147, does not detail the clocking, test and microprocessor access, and event mechanisms. The snowpa block used to provide test access is not shown in the diagram.

[1625] B. 9.8.1 “Data Error Checker” The first block is the data error checker and collector, called decheck, which selects / creates a 12-bit wide token stream, analyzes this stream and checks the data tokens. All other tokens are ignored and sent directly. The check performed is for data tokens with an extension number not equal to 64. Possible error is deficient
(<64 extension), i dct too few even
t, and supernumerary (excess) (> 64)
Extension), called idct to many event. The error is signaled by a standard event mechanism, but the block also attempts a simple error recovery by manipulating the token stream. In the case of a missing error, the data token is padded with "0" value extensions (stops accepting inputs and performs inserts), creating the correct 64 extension. In the case of excess error, the extension bit is forced to a "0" at the 64th extension and all extra extensions are removed from the token stream.

[1626] B. 9.8.2 “Inverse Zigzag” The next block of the spatial decoder is the inverse zigzag RAM, iz
z, which also selects / creates a 12-bit wide token stream. As with all other blocks, the stream is parsed but only data tokens are recognized.
All other tokens are sent unchanged. A data token is also sent, but the expansion order is changed. This block relies on the correct data token (ie, 64 extensions only). If this is not true then no operation is specified. The relocation is done according to the standard inverse zigzag pattern, by default to provide the data to be horizontally scanned at the IDCT output. It is also possible to change the ordering to provide more vertically scanned output. Standard IZ
In addition to Z-ordering, this block performs an extra relocation of every 8 word rows. This is done because of the special requirements of the IDCT one-dimensional transform block, (0, 1, 2,
Not the order of 3, 4, 5, 6, 7), but (1,
3,5,7,0,2,4,6) results in rows being output.

B. 9.8.3 “Input Formatting Unit” The next block is the input formatting unit, ip fm.
t, which forms the data input for the first dimension of the IDCT transform. This block has a 12-bit wide token stream input and a 22-bit wide token stream output. Data token is IDCT conversion standard 22
To move the integer part to the correct valid in the bit-width word, it is shifted left and the fractional part is set to zero. This means that there is a 10-bit fraction at this point. All other tokens are not shifted,
The extra unused bits are simply set to 0.

B. 9.8.4 "One-Dimensional Transform-First Dimension" The next block shown is the first one-dimensional IDCT transform block, oned. It inputs / outputs a 22-bit wide token stream, and as usual the stream is parsed and the data tokens are recognized. All other tokens are sent unchanged. The data tokens pass through a pipelined data path that performs a one-dimensional implementation of an 8x8 inverse discrete cosine transform. The first dimension output has a 7-bit fraction in the data word. All other tokens simply pass the data conversion latency and go through a simple shift register datapath that is recombined into a token stream before output.

[1629] B. 9.8.5 "Transpose RAM" The transpose RAMtram is similar in many ways to the inverse zigzag RAM in the way it handles token streams. The width of the token processed (22 bits) and the relocations performed are different, but they work the same in other respects and actually share much of their control logic. Again, the rows are additionally rearranged for the next IDCT dimension requirement, with basic swapping of columns into rows.

[1630] B. 9.8.6 "One-Dimensional Transform-Second Dimension" The next block shown is another example of a one-dimensional IDCT transform block, which is the same as the first dimension in all respects. There are 4 bit fractions in the output of this dimension. B. 9.8.7 "Rounds and Saturations" The round saturation block, ras, selects a 22-bit wide token stream containing a 22-bit fixed-point format data extension and outputs a 9-bit wide token stream. The extension is rounded to an integer (towards + ve infinity), saturated to a 9-bit two's complement representation, and all other tokens are sent directly.

[1631] 9.9 “Block hardware description” B. 9.9.1 “Standard Block Structure” For all blocks that process a token stream, there is a standard conceptual structure as shown in FIG.
This separates the 2-wire interface latch from the section that performs the operations on the token stream. An extra internal block (RA
M cores, etc.) are included. In some of the blocks shown, the structure is not very clear in the schematic (although it still exists) due to the requirement that all "datapath" logic be grouped together, Separate this from all standard cell logic.
In the case of a very simple block such as ras, the latched outaccept is directly input 2 without any logical operation.
Can be placed in a wire latch.

[1632] B. 9.9.2 “Decheck-Data Error Checking / Recovery” The first block in the token stream does data checking and correction as specified in the Block Diagram Perspective section. The detected error is handled by a standard event mechanism, which can be masked to an event and the block can either continue in the recovery procedure or stop depending on the event mask status when the error is detected. Since the IDCT should not see incorrect data tokens, the recovery it attempts is just a fairly straightforward attempt to include what might be a serious problem.

This block has a two stage pipeline depth and is fully executed in zcells. Input 2-wire interface latch is "front"
Type, and when this block (before the IDCT) is on a different power supply than the one before it, all inputs reach the transistor gates for safe operation. It means to do. This block works by parsing the token stream and sending non-data tokens directly. When a data token is found, the count starts with the number of extensions found after the header. When the extension bit is found to be "0" when the count is not 63, an error signal is emitted (it goes to the event logic),
Depending on the state of the mask bit for that event, dec
eck is stopped (i.e. no longer accepts inputs or emits outputs) or initiates error recovery. The recovery mechanism for the defensive error uses a counter to control the insertion of the correct extension number into the token stream (the inserted value is always "0"). Obviously, no input is accepted, but this insertion continues. If the extension bit of the 64th extension is found not to be a "0", a supernumerary error is emitted and the data token is completed by forcing the extension bit to "0" and the extension bit set to "1". All subsequent words with a are removed from the token stream by continuing to accept data but invalidating the output.

The two error signals are not permanent (unless the block is stopped), that is, only the error signal remains active from the point where the error is detected until recovery is complete. This is a minimum of one complete cycle and will last forever in the case of an infinite excess of data tokens.

[1635] B. 9.9.3 "Izz and tram-R
Relocation of AM ”izz (inverse zigzag RAM) and tram (transpose RA)
M) carry out variations of the same function and have more similarities than differences, so they are considered together here. These blocks choose token streams,
Rearrange the data token extension while sending all other tokens unchanged. Although the expanded width and the sequence of relocations processed are different, the large section of control logic for each RAM is the same and is actually organized into the "common control" blocks illustrated in the schematic for each RAM. . Since the difference in width has no effect on this control section, it is only necessary to use a different "sequence address generator" for each RAM with a RAM core and a 2-wire interface block of appropriate width. is there.

The overall behavior of each RAM is essentially a FIF
Same as O's. This is true at the token level and special modifications to the output order are made for the expansion word of the data token. FIFO
Depth is 128 stages. This is necessary to meet the requirement for 30 MHz which can be sustained throughout the system as the output of the FIFO is supported after the start of output of the data token is detected. This is because the relocation sequence feature used requires that the complete 64 extension blocks be collected in the FIFO before relocation output can begin. To be more precise, the minimum number required for inverse zigzag and transpose sequences is different, some less than 64 in both cases. However, the complexity of controlling FIFOs with non-power-of-two lengths means that the small savings in the RAM core will outweigh the additional complexity of the required control logic. 30M RAM core
Performed in a design that allows reading and writing (to the same address or another address) in Hz cycles. This means that the RAM operates effectively with an internal 60 MHz cycle time. Relocation operations range from 0 to 63, but are not in natural order, but by generating a special sequence of read addresses (sequenceaddress g
is performed. The required sequence is specified by the standard zigzag sequence (for 8 horizontal or vertical scannings) or by the sequence required for normal matrix transposition.
These standard sequences are due to the requirement of outputting each row in odd / even format due to the requirement of IDCT transform one-dimensional block (ie (0,1,2,3,4,
(5, 6, 7), not (1, 3, 5, 7, 0, 2, 4,
6)) and then rearranged further.

The transposed address sequence generation is algorithmically quite clear. Direct transpose sequence generation only requires separate generation of row and column addresses, both of which are performed in a counter. The requirement for row relocation simply means that the row address is generated by a simple special state machine rather than a natural counter.

The inverse zigzag sequence occurs algorithmically and is not very clear. Because of this fact
A small ROM is used to hold the full 64-bit value of the address, which is addressed by row and column counters that can be swapped because it changes between horizontal and vertical scan modes. ROM-based generators are designed very quickly and also have the advantage that performing forward zigzag (ROM reprogramming) or adding other alternative sequences in the future is trivial.

[1639] B. 9.9.4 "" Oned "-1D IDCT Transform" This block has a pipeline depth of 20 stages, and the pipeline becomes stiff when stalled. This stiffness should greatly simplify the design and the pipeline depth should not be too great, so it should not unduly affect the overall force, both dimensions take over after the RAM which provides a certain amount of buffering. .

The block follows the standard structure, but internally has a separate path for the data token extension (to be processed) and all other items to be sent unchanged. Note that the schematics are drawn in a special way. Firstly, because of the requirement to group all datapath logic together, and secondly because of the requirement to allow automatically edited code generation (which describes the control logic at the top level).

The tokens are analyzed as usual and then the data extension and other values are each sent through two different parallel paths before being recombined in the multiplexer in front of the output 2-wire interface latch block. . A parallel path is required because the value cannot be sent unchanged through the conversion data path. The latency of the transform data path is matched with a simple shift register to handle the rest of the token stream.

The oned control section needs to analyze the token stream and control token splitting and recombination. The other main section controls the conversion datapath. The main mechanism for controlling this datapath is a control shift register that adapts the datapath pipeline and is tapped to provide the necessary control signals for each stage of the datapath pipeline.

The oned block has the requirement that it can only start an operation on a complete row of data expansion, ie a group of 8. Although it is not possible to handle invalid data (Gaps) in the middle of a row, the IZZ and tram operations virtually guarantee that the complete block of data will be output as an uninterrupted sequence of 64 valid extended values. To do.

[1644] B. 9.9.4.1 "Transformed Data Path" The fine structure of the transformed data path, t dp, was previously illustrated in FIG. Some details (eg clocking,
Note that shifts etc.) are not shown. However, this diagram illustrates how, at any stage of the pipeline, the datapath operates on four values simultaneously. The basic datapath substructure, the three main sections (eg, precommon, common, postcommon) can also be seen as the arithmetic and latch resources are needed. The named control signal is an enable for pipelined latches (and add / subtract selectors) ordered in the decoding of the control shift register states. Note that each pipeline stage is actually four clock cycles in length.

There are many latch stages in the conversion datapath that need to collect inputs, store intermediate results in the pipeline, and order the outputs. Some latches are of the maxsing type, that is, they can be conditionally loaded from more than one source. All latches are enable type, that is, they have separate clock and enable inputs. This means that it is easier to generate the enable signal at the correct timing, rather than having to consider the problem of asymmetry that would occur if the clocking scheme created was applied.

The main arithmetic elements required are as follows:

-Multiple fixed coefficient multipliers (carry save output) -Carry save adder-Carry save subtractor-Decomposition adder-Decomposition adder / subtractor All calculations are performed in 2's complement notation. This is the normal (disassembled) form or carry-save form (ie two numbers whose total represents the actual value).
May be All numbers are broken down before memory, as this is the most expensive operation in terms of time,
Only one decomposition operation is done per pipeline stage. Decomposition operations are performed here and all operations use simple ripple carry.
This means that the decomposer is very small, but relatively slow. Since the decomposition dominates the overall time at each stage, there is clearly an opportunity to use a fast decomposition arithmetic unit to speed up the overall conversion.

[1648] B. 9.9.5 "" Ras "-Rounding and Saturation" In the present invention, the ras block selects 22-bit fixed point numbers from the output of the 2nd dimension oned, which are exactly rounded to the desired saturation. It does the job of turning it into a 9-bit signed integer result. This block also performs the necessary divide-by-four work inherent in the scheme (2 / N term) and also the two required to supplement the pre-scaling of 2 performed in each of the two dimensions.
Carry out the work divided by. This division by 8 is interpreted as the remaining 3 bits beyond what the fixed point position was expected to be, ie the result is a 15-bit integer representation (4
It implies processing as a 7-bit fraction (rather than a bit fraction). The rounding mode performed is "round to positive infinity", that is, adding 1 to a fraction of exactly 0.5. This is the easiest rounding mode to run, so
Basically done. After the rounding (conditional increment of the integer part) is complete, this result is examined to see if the 9-bit signed result needs to be saturated to the maximum or minimum value within this range. . This is done by an incremental check performed with the high order bits of the original integer value.

As usual, the token stream is parsed and round and saturation operations are applied only to the data token extension values. The block has a pipeline with a depth of 2 stages and is completely z
It is executed in cells.

[1650] B. 9.9.6 "Ictctels-
IDCT Register Select Decoder "This block is a simple decoder that decodes four microprocessor interface address lines and sel test inputs into select lines (snooper and RAM) for individual block test access.
The block consists only of zcells combining logic. The decoded selections are shown in table 227.

[1651]

[Table 225]

[1652]

[Table 226] B. 9.9.7 "Idctregs-IDCT Control Registers and Events" This block of the invention provides data under and over errors,
And an example of a standard event logic block processing a single memory mapped bit vscan that can be used to make an izz relocation change such that the IDCT output is scanned vertically. This bit is reset to the value "0", that is, the default mode is horizontally scanned out. The two possible events are ORed together to form the idcevent signal which can be used as an interrupt. For registers and event addresses and bit positions, see Section B. See 9.10.

[1653] B. 9.9.8 “Clock Generator” Two “standard” types (clkge) in IDCT
The clock generator of n) is used. This is done so that there are two separate scan paths. The clock generators are referred to as idctcga and idctcgb. Functionally, the only difference is that idctcgb is notrstl
It is not necessary to generate a signal. The amount of buffering for each of the clock and reset outputs in the two clock generators is individually adjusted to suit the actual load driven by each clock or reset. The adapted load was actually measured from the gate and track capacitance of the final layout.

[1654] IDCT Top-level Block Pl
The advantage when ace and Route (BPR) is implemented is that these tracks carry active currents, so the heavier loaded clocks (ph0 b and ph1 b).
Derived from the ability of the interactive global routing feature to increase the track width of the first section of the clock distribution tree for.

[1655] B. 9.9.9 "JTAG Control Block" Since the IDCT has two separate scan chains and two clock generators, the standard JTAG control block, j
There are two cases for speckle. These connect the test port and the two scan paths.

[1656] B. 9.10 “Event and Control Register” The IDCT can generate two events,
Has one control bit. Two events are d before IDCT when an incorrect data token is detected.
The dct t generated by the check block
oo feevent and idct too many
event. One control bit is vscan which is set when it is necessary to operate the IDCT with vertically scanned output. Therefore, this bit is izz
Control the block. All event logic and mimory map control bits are placed in the block idctregs.

From the IDCT perspective, these registers are located at the following locations: The tristate i / o wire is n errd, and n servd is used to read and write these locations accordingly.

[1658]

[Table 227]

[1659]

[Table 228] B. 9.11. “Execution” B. 9.11.1 "Logic design approach" According to the invention, there is a unified and simple logic design strategy in the design of all IDCT blocks.
It would have meant that it was possible to make a "safe" design in a quick and clear way. Due to the large number of control logic, a simple scheme with only master / slave was adopted. Only the asynchronous set / reset input was connected to the correct system reset. It could have been possible to provide intelligent non-standard circuitry to perform the same function more efficiently,
This scheme has the following advantages.

[1660] ・ Conceptually simple ・ Easy to design ・ Operating speed is fairly clear (latch → logic →
Latch → see logic-style design), follow automatic analysis ・ Glitch is not a problem (see SR Latch) ・ Use only system reset for initialization ・ Allow scan path to work correctly ・ Automatically follow There are a number of places where transparent d-type latches were used which allow C code generation and are described below.

[1661] B. 9.11.1.1 “2-wire interface latch” The standard block structure uses a latch for the input and output 2-wire interface. There is no logic between the output 2-wire latch and the following input 2-wire latch.

B. 9.11.1.2 "ROM Interface" Due to the timing requirements of the ROM circuit, latches are used in the IZZ sequence generator at the output of the ROM.

[1663] B. 9.11.1.3 "Translation Datapath and Control Shift Register" It is possible to implement any pipeline storage stage as a complete master-slave device, but use latches because of the amount of storage required. There are significant savings to be gained by doing. However, this scheme requires users to consider several factors.

The control shift register is used as an enable, so it must produce control signals for both phases (ie the need to use a latch in this shift register). Timing analysis is complicated by the use of latches. Even though t postc will no longer automatically produce edited code because one latch outputs to another in the same phase (due to the timing of the enable, this is not a circuit issue) The area saved by the use of latches makes it worthwhile to accommodate these elements in the present invention.

[1665] B. 9.11.1.4 "Microprocessor Interface" Due to the nature of this interface, there is a requirement for latches (and resynchronizers) in the keyhole logic for event and register blocks idctregs and RAM cores.

B. 9.11.1.5 “JTAG Test Control” These standard blocks utilize latches.

B. 9.11.2 "Circuit design issues" Apart from the work done in the design of the library cells used in the IDCT design (standard cells, datapath libraries, RAM, ROM, etc.)
There are no requirements for transistor level circuit design in IDCT. Circuit simulation (using Hspice) is performed from among several known critical paths in the transform datapath, and Hspice can also provide critical path analysis (CPA) tool results for paths near the maximum allowed length. It was also used to confirm.

The IDCT is completely static in normal operation (ie we can stop the system clock indefinitely) but when the test clock is stopped (or made very slow). Note that there are dynamic nodes in scannable latches that will fail. Due to the non-regenerative nature of some nodes that exhibit Vt drops (eg, max output), the IDCT will not be "micro power" in the static case.

B. 9.11.3 "Layout Approach" The overall approach to layout implementation of the present invention is:
It was to use BPR (some manual interference) to lay out a complete IDCT composed of many zcells and few macroblocks. These macroblocks were manually edited layouts (eg RAM, ROM, clock generator, datapath) or, in the case of oned blocks, additional z
It was built using BPR from the cell and datapath.

The datapath was constructed from kdplib cells. In addition, locally limited layout variations of kdplib cells were defined and used where this was found to provide a valuable size benefit. Each oned block, oned
The datapath used in d is by far the largest element in the design and considerable effort has been made to optimize the size (height) of this datapath.

The organization of the transformed datapath, t dp, is rather deterministic, as the exact ordering of the elements in the datapath affects the way the interconnection is handled. Minimize the number of overs (vertical wires that do not connect to sub-blocks) that occur at the most congested points, as there is a maximum allowed value (ideally 8 and very inconvenient but 10 is possible) is important. The datapath is logically split into three major subsections, thus performing the datapath layout. In each subsection there are actually four parallel data flows (which are combined at various points),
Therefore, there are many ways to organize the data flow (and the location of all elements) within each subsection. Block ordering within each subsection,
And the assignment of logical buses to physical bus pitches was carefully calculated before the layout began, in order to be able to achieve the layout that would be connected correctly.

[1672] B. 9.12. “Matching” IDCT matching was done at many levels, from the algorithm's top-level matching to the final layout check.

The initial work on the transform structure was done in C and full-precision and bit-accurate integer models were developed. H. Various tests have been performed on the bit-accurate model to demonstrate conformance to the 261 accuracy standard and to measure the dynamic range of calculations within the transform structure.

By writing M in most cases, the design has advanced the behavioral explanation of sub-blocks (eg, datapath and RAM control). Before moving on to the design of the general description of the block,
Simulated in sim. In some cases (eg RAM, clock generators) behavioral science was also used for top-level simulations.

The strategy for performing a logic simulation was to simulate a schematic for everything that would properly simulate at that level. Low-level library cell (ie zce
11 and kdplib) were primarily simulated with their behavioral explanations, as this result is a much smaller and faster simulation. In addition, behavioral library cells offer a feature of timing checking that can highlight structural problems in certain circuits. As a confidence check, a simulation was performed using the library cell transistor description. All logic simulations were done in a zero-delay fashion and were therefore intended to verify functional performance. Real-time behavioral matching was done using other techniques.

As a partial collation of timing performance, (R
Lsim switch level simulations (using the C Timing mode) were performed, but provide a check for yet other potential transistor level problems (eg, glitch sensitive circuits).

The main matching technique for checking timing problems is the CPA tool, path for datechk.
Was the use of options. It is used to identify longer signal paths (some were already known), and to match CPA analysis in certain critical cases H
spice was used.

The bulk of the IDCT act is trained by the flow of tokens through the device, so most L
The sim simulation was done with a standard source->block-> sink methodology. Additional simulations are also required to test features that are accessed through the microprocessor interface (configuration, events and test logic) and test features that are accessed via JTAG / scan.

Edited code simulation can be easily accomplished by one of ordinary skill in the art for full IDCT, again using the standard source->block-> sink method and many of the same token streams used in Lsim matching.

[1680] B. 9.13 Testing and Testing Support This section deals with the mechanisms provided for testing and analysis of how each block is tested.

The three mechanisms provided for test access are: microprocessor access to RAM cores microprocessor access to snowpaper blocks scan path access to control and datapath logic. Has two "Snowpa" blocks and one "Super Snowpa" block. FIG. 149 shows the position of the snowpaper block and the test access of other microprocessors.

[1682] Using these and two RAM blocks,
It is possible to isolate each major block for the purpose of testing their behavior on the token flow.
Microprocessor access can be used to control the token input to any block and observe the token port output of that block being isolated. In addition, there are two separate scan paths that pass through (almost) all flip-flops and latches in the control section of each block and through some of the data path latches in the case of the oned conversion data path pipeline. The two scan paths are labeled a and b, the former moving from the decheck block to the ipfmt block, and the latter from the first oned block to ra.
Move to s block.

Access to the Snowpa is possible by accessing the appropriate memory-mapped location in the usual way. (As appropriate, ramt
The same is true for RAM cores (using the est input). The scan campus is accessed through the JTAG port in the usual way.

Each block is discussed with respect to various test questions.

[1685] B. 9.13.1 "Decheck" This block has a standard structure (see Figure 148) in which two latches for the input and output two-wire interface surround the processing block. As always, no scanning is done to the 2-wire latches because these 2-wire latches simply pass by the data whenever they are enabled and do not have the depth of logic to be tested.
In this block, the "control" section consists of a one-stage pipeline of zcells, all above scan path a. The logic in the control section is relatively simple and most of the complicated paths are probably in the generation of data extension counts where a 6 bit incrementer is used.

[1686] B. 9.13.2 "Izz" This block is a variant of the standard structure and contains a RAM core block added to the 2-wire interface latch and control section. The control section is implemented with a zcell and a small ROM used for address sequence generation. All zcells are on scan path a, with access to the ROM address and data via the zcell latch. In addition, there is logic, eg, for the generation of numbers plus the ability to increment or decrement. In addition, there is a 7-bit full adder used for read address generation. The RAM core is accessible via the microprocessor interface through the keyhole register. Table 225,
See Table 226.

[1687] B. 9.13.3 "lp fmt" This block also has a standard structure. The control logic is implemented in a fairly simple zcell logic (all on scan path a), but the logic here is very shallow and simple, so the data latching and shifting / maxing is straightforward. In the data path without any access.

[1688] B. 9.13.4 “Oned” Again, this block follows a standard structure and is divided into a random logic and a data path section. zcell
The logic is relatively clear and all zcells are on scan path a. The control signal for the transform pipeline data path is derived from a long shift register consisting of a zcell latch on the scan path. In addition, some pipeline latches are on the scan path, because the logic between some stages of the pipeline (eg, multiplier and adder) is fairly deep. Non-data tokens are sent along the shift register, implemented as a data path, and there is no test access to any of these stages.

[1689] B. 9.13.5 "Tram '" This block is very similar to the izz block. However, in this case no ROM is used in the address sequence address generation. This is done algorithmically. All zcell control states are on datapath b.

[1690] B. 9.13.6 "Rras'" This block follows a standard structure and is implemented entirely in zcell. The most complex logic function is the 8-bit incrementer used at round up. All other logic is pretty simple. All states are on scan path b.

[1691] B. 9.13.7 “Other top-level blocks” There are several other blocks that appear at the top level of the IDCT. The Snowpa is obviously part of the test access logic as well as the JTAG control block.
In addition, there are two clock generators that do not have special test access (though they support various test features). The block idctsels is a combination z for decoding microprocessor addresses.
cell logic, block idctregs is a microprocessor accessible event, and IDC
Includes T-related control bits.

[1692] Section B. 10 "Introduction" B. 10.1 “Perspective of Time Decoder” FIG. 151 shows the internal structure of the time decoder according to the present invention.

[1693] All data flow between chips and blocks (and many data flows within a block) are controlled by a 2-wire interface (see Technical References and Sections for details) and each arrow in FIG. 151. Represents a two-wire interface. The incoming token stream passes through an input interface that synchronizes the data from the external system clock with the internal clock derived from the phase locked loop (ph0 / ph1). The token stream is then split into two passes via the top fork; one stream goes to the address generator and the other stream goes to the 256 word FIFO. The FIFO buffers the data while the data from the previous I or P frame is extracted from the DRAM and processed in the prediction filter before being added to the input error data from the spatial decoder in the prediction adder (P and B frames). To be done. During MPEG decoding, frame relocation data must be extracted for I and P frames so that the output frames are in the correct order. The rearranged data is inserted into the stream in the read ladder block.

The address generator generates separate addresses for forward and backward prediction, relocation, read and writeback, and the data written back is split from the stream in the write ladder block. Finally, the data is resynchronized to the external clock in the output interface block.

All main blocks in the Time Decoder are connected to the Internal Microprocessor Interface (UPI) Bus. It is derived from the external microprocessor interface (MPI) in the microprocessor interface block. This block has address decodes for the various blocks within its associated chip. Also associated with the microprocessor interface is the event logic.

The rest of the time decoder logic is basically test related. First, IEEE 1149.1 (JT
AG) interface provides an interface to the JTAG boundary scan feature as well as to the internal scan path. Second, during test mode,
A two wire interface stage that allows intrusive access to the data flow through the microprocessor interface is included at a strategic point within the pipeline structure.

[1697] Section B. 11 "Clocking,
Testing and related issues "B. 11.1 "Clock Style" Before considering the individual functional blocks within a chip, it may be useful to be aware of the clock styles within a chip and their relationships.

During normal operation, most blocks of the chip move synchronously with the signal pllsysclk from the Phase Locked Loop (PLL). The exception to this is the DRAM interface, the timing of which is governed by the need to synchronize to the iftime sub-block, which sub-block has DRAM control signals (notwe, notoe, notcas, notra).
s) is generated. The core of this block is 2 phases
Clocked by non-overlapping clocks clk0 and clk1, which are PLL cki0, cki.
1 and clkg0, derived from a quadrature 2 phase clock supplied separately from ckg1.

Clk0, clk1 Since the DRAM interface clock is synchronized to the clock in the rest of the chip, to eliminate the possibility of metastable behavior (as much as practical) at the interface between the DRAM interface and the rest of the chip Countermeasures were taken. Synchronization occurs in two areas: (addrgen / predre) at the output interface of the address generator.
ad / psgsync, addrgen / ip wrt
2 / sync18 and addrgen / iprd2 / s
sync18), and in the block that controls the "swinging" of the swing buffer RAM in the DRAM interface (see section on DRAM interface). In each case, the synchronization process is achieved by three series metastable hard flip-flops. Note that this is clk0 / cl
It means that k1 is used in the output stage of the address generator.

In addition to these fully synchronized clock schemes, there are many separate clock generators that generate two phase non-overlapping clocks (ph0, ph1) from pllsysclk. Address generator,
The predictive filter and the DRAM interface each have their own clock generator; the remaining chips are driven away from the common clock generator. There are two parts to this reason. First, it reduces the capacitive loading on the individual clock generators, allowing smaller clock drivers and reduced clock routing widths.
Second, each scan path is controlled by the clock generator, thus increasing the number of clock generators allows the use of shorter scan paths.

It is necessary to re-synchronize the signals that are moved across these clock style boundaries. This is because an insignificant skew between non-overlapping clocks derived from different clock generators would mean that an underlap has occurred at the interface. Circuitry built into each "Snowpa" block (see Section B.11.4) ensures that this does not occur, and the Snowpa block is an address generator where resynchronization takes place in the token decode block. Was placed at the boundary between all clock formats except before.

[1702] B. 11.2 "Clock Control" Each standard clock generator generates many different clocks that enable operation in normal mode and scan test mode. Controlling the clock in scan test mode is described in detail in other sections, but it should be noted that the clock generators (tph0, tph0, tph0
Some of the clocks issued by ph1, tckm, tcks) are usually not added to the primitive symbols in the schematic.
This is because the scan path is automatically created by the post processor that connects these clocks correctly. From a functional point of view, the fact that the post-processor has connected a different clock than the one shown in the schematic can be ignored; the behavior is the same.

[1703] During normal operation, the master clock can be derived in many different ways. Table 2
Reference numeral 29 indicates that various modes can be selected depending on the state of the pin pllselect and the override.

[1704]

[Table 229] B. 11.3 "2-Wire Interface" The overall functionality of the 2-wire interface is described in detail in the technical reference. However, the 2-wire interface is used for communication between all the blocks in the temporal decoder, most blocks are made up of many pipeline stages, all of which are 2-wire interface stages. Therefore, it is important to understand the internal implementation of the two-wire interface in order to be able to interpret many schematics. In general, these internal pipeline stages are configured as shown in FIG.

[1705] Figure 152 shows a latch-logic-latch display, which is the commonly used arrangement. However, when many stages are placed together, it is also useful to think of "stages" as latch-latch-logic (a mode well known to many engineers). The use of a latch-logic-latch arrangement ensures that all inter-block communications, whether sending or receiving blocks,
It can be a latch-to-latch without interposing logic.

[1706] Referring again to FIG. 152, the logic block is removed and the data and valid signals are directly connected between the latches, and the in valid directly latched into the NOR gate on the input is the same as the out valid and out accept gated. A simple 2-wire interface FIF by connecting the
O stage can be constructed. The data and valid signals propagate when the next corresponding accept signal is high. In valid with out illustrated by the method shown in the figure
By ringing, the data is out accept
Even if t reg is low, it is accepted when in valid is low. This way, whenever a stall (accepting a low signal) occurs, gaps
(Data with low valid bits) is removed from the pipeline.

As shown in FIG. 152, a logic block is inserted, and in accept and out valid can depend on the state of data or block. In the arrangement shown, every state in the block is
It is standard to be held in a master-slave device with a master enabled by 1 and a slave enabled by ph0.

[1708] B. 11.4 "Snowpabloc" Snowpabloc allows access to a data stream through a microprocessor interface at various points within the chip. There are two types of snow pave locks. Normal snowpapers are only accessible in test mode, where the clock can be controlled directly. A "super snowpa" contains circuitry that is accessible while the clock is running and that synchronizes asynchronous data from the microprocessor bus to the internal chip clock. Table 230 lists the location and type of all snowpaers in the temporal decoder.

[1709]

[Table 230] Details on the use of both snowpaers are given in the test section. Details of the operation of the JTAG interface are described in the JTAG document.

[1710] Section B. 12 “Function block” B. 12.1 “Top Fork” According to the invention, the top fork serves two different functions. First, it branches the data stream into two different streams: one for the address generator, the other for the FIFO, and second, the starting means and stopping of the chip so that the chip can be placed. Provide the means.

[1711] The aspects of the fork portion of the component are very simple. The same data is presented to the address generator and FIFO and must be accepted by both blocks before the accept is sent back to the previous stage. Thus, the vali of the two branches of the fork
ds depends on accepts from other branches. If the chip is in a stalled state, vali for both branches
ds is kept low. The chip powers up with inaccept held low until the configuration bit is set high. This guarantees that no data will be accepted until the user has placed the chip. If the user needs to place the chip at other times, the user must set the place bit and wait until the chip has completed the current stream. The stopping process is as follows: 1) If the placement bit is set, no data is accepted after the flush token is detected by the top fork.

2) When the Flash Token reaches the Read Ladder, the chip will have finished processing the stream. This causes the signal seq done to go high.

3) Set the event bit readable by the microprocessor when seq done goes high. The event signal may be masked by the event block.

[1714] B. 12.2 "Address Generator" In the present invention, the address generator (addrgen) is responsible for counting the number of blocks in a frame and generating the correct address sequence for DRAM data transmission. The input of the address generator is the token stream from the token input port (via the top fork) and its output to the DRAM interface consists of the address and other information controlled by the request / acknowledge protocol.

[1715] The basic sections of the address generator are as follows: -Token decoding-Block counting and DRAM block address generation-Motion vector data conversion to address offset-Request and address generator for predictive transmission -Relocation read address generator-Write address generator B. 12.2.1 “Token Decoding (tokde
c)] At the token decoder, the tokens, frames, block information, and motion vectors associated with the coding standard are decoded. The information extracted from the stream is stored in a series of registers that up
It can be accessed via i. Detection of the data token header is signaled to the next block, enabling block counting and address generation. Nothing happens when running JPEG.

[1716] Decoded Token List-Coding Standard-Data-DEFINE MAX SAMPLING-DEFINE SAMPLING-HORIZONTAL MBS-MVD BACKWARDS-MVD FORWARDS-Picture Start-Picture Type-Prediction Mode This block further combines information from the request generator. Control the frame pointer toggling to stall the input stream. When a new frame appears at the input (in the form of a picture start token), the stream is stalled, but the writeback or relocation read associated with the previous frame is incomplete.

[1717] B. 12.2.2 "Macroblock Counter (mblkcntr)" The macroblock counter of the present invention consists of four basic counters that point to the horizontal and vertical position of the macroblock within the frame. At the beginning of time and at the start of each picture, all counters are set to zero. When the data token header arrives, the counter is incremented and reset according to the color header number in the token header and frame structure. This frame structure is explained by the sampling register in the token decoder.

For the given color component, counting proceeds as follows. The horizontal block count is incremented for each new data token of the same component until the width of the macroblock is reached, then reset. The vertical block count is incremented by this reset until it reaches the macroblock height, then reset. When this happens, it waits for the next color component. Therefore, this sequence is repeated for each of the components in the macroblock-probably different for each component, in the horizontal and vertical sizes of the macroblock. Even if less than the expected block is received for one component, the count will proceed to the next component without error.

If the color component of the data token is less than the expected value, then the horizontal macroblock count is incremented. (This will happen because the counter is expecting a higher component index when more blocks than are expected for a given color component appear.) This horizontal count is Reset when the picture width in a macroblock is reached. This reset increments the vertical macroblock count.

[1720] Furthermore, H.264. It is capable of counting macroblocks in the 261 CIF format. In this case, there is an extra level of hierarchy between macroblocks and pictures called block groups. This is 11 macroblocks wide and 3 macroblocks deep, and the picture is always 2 groups wide. The token decoder extracts the CIF bit from the picture type token and sends it to the macroblock counter, instructing it to count the block group. If the number of blocks is too low or too high for each component, the above reaction will be triggered.

[1721] 12.2.3 “Block calculation (bl
kcalc) "block calculation transforms the coordinates of a macroblock and blocks within a macroblock into coordinates for the position of a block within a picture, that is, knocking out hierarchical levels. Of course, this must take into account the sampling rates of the different color components. B. 12.2.4 “Base block address (bsb
lkadr) ”information from blkcalc, along with the color component offset,
Used to calculate the block address in the linear DRAM address space. In essence, for a given color component, the linear block address is the number of vertical blocks x picture width + number of horizontal blocks. This is added to the color component offset to form the base block address.

[1722] B. 12.2.5 "Vector Offset" The motion vector information displayed by the token decoder is in the form of horizontal and vertical pixel offset coordinates. That is, for each of the forward and backward vectors, give the transpose of a half pixel from the block formed into the block in which it is predicted (x,
y). Note that these coordinates can be positive or negative. They are first estimated according to the sampling of each color component and used to form the block and new pixel offset coordinates. In FIG. 154, a block in which a shadow part is being formed is shown. The dotted outline is the block from which it is predicted to come. The large arrows indicate the block offset-horizontal and vertical vectors to the DRAM block containing the origin of the prediction block, in this case (1,4). The small arrow indicates the new pixel offset-the position of the predicted block origin within that DRAM block. 8 DRAM blocks
Since it is × 8 bytes, the pixel offset is (7,
It seems to be 2).

The multiplier array vmarrla then converts the block vector offset into a linear vector offset. The pixel information is (x, y) coordinates (pix inf
o) to the prediction request generator.

[1724] B. 12.2.6 “Prediction Request” The frame pointer, base block address, and vector offset are added to each other, and the DRAM (Inblka)
Form the address derived from d3). If the pixel offset is 0, only one request will be issued. If there is an offset in either the x or y dimension, then two requests-the original block address, and either the one right or just below-
Is emitted. If there is an offset in both x and y,
Four requests are issued. The synchronization between the chip clock mode and the DRAM interface clock mode is the first.
Occurs (Inblkad3) and the state machine issuing the appropriate request. Thus, the state machine (psgstate) is clocked by the DRAM interface clock and its scanned elements form part of the DRAM interface scan chain.

[1725] B. 12.2.7 “Relocation Read Request and Write Request” Since the pixel offset is not included here, each address is formed by adding the base block address to the associated frame pointer. Relocation reads use the same frame store because the predictions and data are written back to another frame store. Each block is equipped with a short FIFO for storing addresses, as the transmission of read and write data tends to delay the expected transmission at the corresponding address. (This is because the read / write data interacts with the stream along the chip data flow rather than the predicted data.) In addition, each block contains synchronization between the chip clock and the DRAM interface clock.

[1726] B. 12.2.8 "Offset" DRAMs are arranged as two frame stores, each containing three color components. The frame store pointer and the color component offset within each frame must be programmed via upi.

[1727] B. 12.2.9 "Snowpas" In the present invention, snowpaas are arranged as follows: Between blkcalc and bsblkadr-this interface provides horizontal and vertical block coordinates, appropriate color component offsets, and (for that component) Picture width within a block).

[1728] After bsblkadr-base block address.

After vec pipe-linear block offset, prediction mode, color components and H.264. Pixel offset within the block, along with information about H.261 operations.

After Inblkad3-As explained in the "Predictive Requests" section, the physical block address supersnooper is in the relocation read and write request generator for use during external DRAM testing. Placed. See the DRAM Interface section for details.

[1731] B. 12.2.10 The "scan" addrgen block has its own scan chain and its clocking is controlled by the block's own clock generator (adclkgen). Note that the request generators at the end of the block are within the DRAM interface clock scheme.

[1732] B. 12.3 “Prediction Filter” The overall structure of the prediction filter according to the present invention is shown in FIG. The forward and backward filters are the same, filtering MPEG forward / backward predictive blocks. H. Only the forward filter is used in H.261 mode (the h261 on input of the backward filter should be permanently low as the H.261 stream does not include backward prediction). The overall prediction filter block consists of a pipeline of 2-wire interface stages.

[1733] B. 12.3.1 "Prediction Filters" Each prediction filter operates completely independently of the other prediction filters and processes the data as soon as valid data appears at its input. As is clear from FIG. 156, the prediction filter consists of four separate blocks, two of which are the same. The operation of these blocks is described in MPEG and H.264. It would be better to describe it independently for the operation of 261. H. 261 is the most complex and will be described first.

[1734] B. 12.3.1.1 "H.261 Operation" The one-dimensional filter equation used is as follows: Fi = (xi + 1 + 2xi + xi-1) / 4 (i≤i≤6) Fi = xi (others I) It is applied by the x prediction filter to each row of the 8x8 block and by the y prediction filter to each column. The mechanism by which this is achieved is illustrated in Figure 157, which basically represents a pfltldd schematic. The filter consists of three 2-wire interface pipeline stages. For the first and last pixel of the row,
Registers A and C are reset, data is in register B,
Pass through D and F unchanged (the contents of B and D are added to 0). The control of B × 2 mux is set so that the output of register b is shifted left by one. This shifting is always shifted at any event 1
It is added to one place. Thus, all values are multiplied by 4 (later than this). For all other pixels, xi + 1 is loaded into register C, xi is loaded into register B, and xi-1 is loaded into register A. As can be seen from FIG. 261 filter expressions are executed. Since vertical filtering is performed in three horizontal groups (see note on dimension buffer below), it is not necessary to process the first and last pixels in a row separately. The control and counting of pixels within a row is performed by the control logic associated with each one-dimensional filter. Note that the result is not divided by 4. After horizontal and vertical filtering, the prediction filter adder (section
B. At the input of 12.4.2), the work of dividing by 16 (shifting to the right by 4) is performed. Register DA, D
D and DF send control information to the pipeline. This is h2
61 on and last byte are included.

Of the other blocks found in the prediction filter, the function of formatting is simply to ensure that the data is displayed in the x-filter in the correct order. As can be seen from the above, this simply requires a three stage shift register, the first stage connected to the input of register C, the second stage to register B and the third stage to register A. Connected.

Between the x and y filters, a dimension buffer buffers the data so that groups of three vertical pixels are displayed in the y-filter. These three groups are still processed horizontally, but no transposition occurs in the prediction filter. With reference to FIG. 158, the sequence in which pixels are output from the dimension buffer is shown in Tables 231 and 232.

[1737]

[Table 231]

[1738]

[Table 232] B. 12.3.1.2 "MPEG Operation" During MPEG operation, the predictive filter performs a simple half-pixel interpolation: Fi = (xi + xi + 1) / 2 (0≤i≤8, half-pixel) Fi = xi (0 ≦ i ≦ 7, integer pixel) h261 on If input is not low, this is the default filter operation. If the signal dim to the one-dimensional filter is low, integer pixel interpolation will be performed. Thus, if h261 on is low and xdim and ydim are low, then all pixels are sent directly without filtering. Dim to one-dimensional filter
It is a clear requirement that the row (or column) be 8 pixels wide (or more) when the signal is high. This is summarized in Table 233.

[1739]

[Table 233] In FIG. 157, “One-dimensional prediction filter”, the operation of the one-dimensional filter is H.264. MPEG as well as for the first and last pixels in the 261 rows
This is because of interpixels. Due to the MPEG half-pixel operation, register A is permanently reset and the output of register C is left shifted by 1 (the output of register B is always left shifted by 1). Thus, after two clocks, register F becomes (2B + 2
C), which is four times the desired result, but is processed at the input of the predictive filter adder where the number passed through the x and y filters is shifted right by 4.

The formatting and dimension buffer functions are simple in MPEG. Formatting must collect the two valid pixels and send them to the x-filter for half-pixel interpolation; the dimension buffer need only buffer one row. It is worth noting that after the data has passed the x-filter, the filtering operation transforms the 9-pixel row into an 8-pixel row so that there are only 8 pixels in the row. "Lost" pixels are replaced by gaps in the data stream. When performing half-pixel interpolation, the x-filter is at the end of each row (8 each).
Insert gaps (after pixel); y-filter inserts 8 gaps at the end of the block. This means that at the end of the block, 8 or 9 gap groups will have a data token header, and a FIFO.
It is important because it aligns with other tokens between data tokens in the stream coming from. This minimizes the worst case through the chip, which occurs when 9x9 blocks are filtered.

[1741] B. 12.3.2 "Prediction Filter Adder" During MPEG operation, a prediction is formed using the early pictures, late pictures, or the average of both. The prediction formed from the early frames is called forward prediction, and the prediction formed from the later frames is called backward prediction. Prediction filter adder (pfa
The function of dd) is to determine which filtered predictor to use (forward, backward or both) and whether to pass forward or backward filtered prediction or an average of both. (Rounded to positive infinity).

The prediction mode can only be changed between blocks, ie at power-up or after the fwd 1st byte and / or bwd 1st byte signals indicating the last byte of the current prediction block become active. If the current block is forward prediction, only the fwd 1st byte is examined.
If it is backward predictive, bwd 1st
Only bytes are examined. If it is a bidirectional prediction, fwd 1st byte and bwd 1
The st byte is examined.

The signals fwd on and bwd on determine which prediction value to use. At any time, both of these signals may be active, or both may not be active. If there is a gap at start-up or when there is no valid data at the block's inputs, the block will enter state when neither signal is active.

Two criteria are used to determine the prediction mode for the next block: signal fwd indicating whether the forward or backward block is part of a bidirectional prediction pair. imat
win and bwd ima twin, and bus fwd
p num [1: 0] and bwd p num [1:
0]. These buses contain a number that increments by 1 for each new prediction block or prediction block pair. These blocks are needed, for example, if there are two forward prediction blocks followed by a bidirectional prediction block, so that the input of the prediction filter adder is reached before the second forward prediction block, This is because the DRAM interface can retrieve backward blocks for bidirectional prediction long ago. Similarly, backward and forward predictions of other sequences can leave the sequence at the input of the prediction filter adder. Thus, the next prediction mode is determined as follows: 1) There is valid forward data and fwd ima
If twin is high, the block stalls until valid backward data arrives with the bwd ima twin set, then passes through the block averaging each predicted value pair.

2) There is valid backward data and b
If wd ima twin is high, the block stalls until valid forward data arrives with the fw ima twin set, then proceeds as described above. If both the forward and backward data are valid, the stall is not performed.

[1746] 3) Effective forward data exists, but
If the fwdima twin is not set, the fwd
p num is examined. This is (pred num
The prediction mode is set to forward if it is equal to the number from the previous prediction + 1 (stored in).

3) If valid backward data exists but bwd ima twin is not set, b
The wd p num is examined. This is (pred n
If equal to the number from the previous prediction + 1 (stored in um), the prediction mode is set to backward.

[1748] e from one stage behind in the pipeline
Note that the early valid signal can be used to set the predictive filter adder mode before the first data from the new block arrives. This ensures that no stalls are introduced into the pipeline.

The ima twin and pred num signals are not sent along with the filtered data along the forward and backward predictive filter pipelines. This is for the following reasons: 1) These signals are fwd 1st byte and /
Alternatively, it is checked only when bwd 1st byte is valid. This saves about 25 3-bit pipeline stages in each prediction filter.

2) Since the signal remains valid throughout the block, it is valid when the fwd 1st byte and / or bwd 1st byte arrive at the prediction filter adder.

[1761] 3) The signal is checked in any case one clock before the arrival of data.

[1752] B. 12.4 "Predictive Adder and FIFO" The predictive adder (padder) forms a predicted frame by adding the data from the prediction filter to the error data. To compensate for the delay from the input through the address generator, DRAM interface and prediction filter, the error data is 256 word FI before reaching the padder.
Pass through FO (sfifo).

[1753] Coding Standard tokens, prediction mode tokens, and data tokens are decoded to determine when a prediction block is formed.
The 8-bit prediction data is added to the 9-bit 2's complement error data in the data token. The result is 0-25
It is limited to the range of 5 and proceeds to the next block. Note that this data limitation also applies to all intra-coded data, including JPEG.

The prediction adder of the present invention further includes a mechanism for detecting inconsistencies in the data arriving from the FIFO and the prediction filter. Theoretically, the amount of data from the filter should correspond exactly to the number of data tokens from the FIFO containing the prediction. In the event of a major malfunction, the padder will attempt recovery.

The end of the data block from the FIFO and filter is marked by the in extn and f1 last inputs, respectively. If the end of the filter data is detected before the end of the data token, the remaining tokens continue to output unchanged. On the other hand, if the filter block is longer than the data token, then all excess filter data is accepted and the input is stalled until discarded.

The chip is configured to send data from the token input port directly to these blocks, and to these outputs directly to the token output port.
There are no snowpaers in either the IFO or the predictive adders.

B. 12.5 "Write and Read Ladder" B. 12.5.1 "Light ladder
r) ”The light ladder sends all tokens coming from the predicted adder to the read ladder. In addition, it is MPEG I or P
All data blocks in the picture and H.264. It sends all the data blocks in 261 and the DRAM interface so that they can be written back to the external frame store under the control of the address generator. The writeback data passes through the snowpaer on its way to the DRAM interface, but all the basic functionality is contained within one 2-wire interface stage.

The Light Ladder decodes the following tokens:

[1759]

[Table 234] After the data token header is detected, all data bytes are output on the DRAM interface. The end of data token is detected by a low in extn, which causes the flash signal to be sent to the DRAM interface swing buffer. In normal operation this would be aligned with the point where the swing buffer would swing anyway, but if the data token does not contain 64 bytes of data,
It provides a recovery mechanism (but the next two
3 output pictures may be incorrect).

[1760] B. 12.5.2 "Lead ladder (rr
The read ladder of the present invention has the following three functions, and the two main functions relate to the picture sequence rearrangement in MPEG: 1) The data read back from the external frame store at the correct position. Insert into the token stream.

[1761] 2) Rearrange the picture header information in the I picture and P picture.

3) Detecting the end of the token stream by detecting a flush token (see Section B.12.1, "Top Fork").

The structure of the lead ladder is shown in FIG. The entire block is made from standard 2-wire interface technology. The tokens in the input interface latch are decoded and these decodes determine the block operation:

[1764]

[Table 235] The relocation function is initiated via the microprocessor interface, but is prohibited unless the coding standard is MPEG, regardless of the register state. The same MPI register controls whether the address generator is generating the relocation address, thus the relocation is the output from this block. To understand how the read ladder works, consider the input and output control logic separately, keeping in mind that the sequence of tokens is as follows: Coding Standards Sequence Start Picture Start Time standard-Picture type-Picture containing data token and other tokens-Picture end-. ． ．・ Picture Start ・． ． ． B. 12.5.2.1 “Input Control Logic” From power-up, all tokens go to FIFO1 (referred to as the current input FIFO) until the first picture type token for an I or P picture is encountered. Then FIFO2 becomes the current input FIFO and I
Or all inputs are directed to it until the next picture type for a P picture is encountered, making FIFO1 the current input FIFO again. In I and P pictures, all tokens between the picture type and the picture end are discarded, except for data tokens. This prevents the motion vector from associating with the wrong picture in the relocated stream, which would have no meaning.

A 3-bit code is inserted in the FIFO along with the token stream to indicate the presence of a particular token header. This eliminates the need to perform token decoding on the output of the FIFO.

B. 12.5.2.2.2 “Output Control Logic” From power-up, until a picture start code is encountered, the token is accepted by FIFO1 (called the current output FIFO), after which FIFO2 is the current F.
Become an IFO. Section B. In 12.5.2.1, at this stage, three picture header tokens, picture start, time standard and picture start are FIFO.
It can be seen that it is held at 1. The current output FIFO is swapped each time it encounters a picture start code in an I or P frame. Therefore, the three picture header tokens will be stored until the next I or P frame, at which time they will become associated with the correctly relocated data. B
The picture is not rearranged and any tokens pass by without being discarded. All tokens in the first picture, including the picture end, are discarded.

During the I and P pictures, the data contained in the data token of the token stream is replaced with the relocated data from the DRAM interface. During the first picture, the "relocated" data is present at the relocated data input. This is because the address generator requests the DRAM interface to bring it out. This is considered a hash,
Be thrown away.

[1768] Section B. 13 "DRAM Interface" B. 13.1 "Perspective" In the present invention, the spatial decoder, temporal decoder and video formatting are each DR for its particular chip.
It has an AM interface block. In all three devices, the function of the DRAM interface is to transfer the data from the chip to the external DRAM and to transfer the data from the external DRAM into the chip via the block address provided by the address generator. Is.

The DRAM interface typically operates from an address generator, and a clock that is asynchronous to the clocks of the various blocks through which data is sent. However, this desynchronization is easily handled because the clocks operate at about the same frequency.

Normally, data is transferred between the DRAM interface and the rest of the chips in a block of 64 bytes (the only exception being prediction data in the temporal decoder). The transmission is done by a device known as a "swing buffer". This is essentially a set of RAM operated in a double buffered arrangement,
Another chip portion empties or stuffs another RAM while the AM interface stuffs or empties one RAM. Another bus carrying the address from the address generator is associated with each swing buffer.

Each chip has four swing buffers, but the functions of these swing buffers differ in each case. DRA coded data in spatial decoder
One swing buffer is used to transfer to M, another swing buffer is used to read the coded data from the DRAM, and the tokenized data is transferred to the DRAM.
3rd swing buffer for transmitting to D
A fourth swing buffer is used to read the tokenized data from RAM. In the time decoder,
DRAM for intra or predicted picture data
One swing buffer is used to write to
A second swing buffer is used to read intra or predicted picture data from DRAM,
The other two are used to read forward and backward prediction data. In video formatting, one swing buffer is used to transfer data to DRAM, and the other three are used to read data from DRAM, which are luminance (Y) and color difference between red and blue, respectively. Data (Cr and Cb, respectively). D
The operation of the general features of the RAM interface is described in the Spatial Decoder document. The next section describes features specific to temporal decoders.

[1772] B. 13.2 “Time Decoder DRAM Interface” Section B. As mentioned in 13.1, the temporal decoder has four swing buffers: two are used to read / write decoded intra and predictive (I and P) picture data, which are as described above. To operate. The other two are used to derive the prediction data.

In general, the prediction data will be offset from the position of the block being processed as specified by the motion vectors in x and y. Thus, the data block to be retrieved is generally the one in which it was encoded (and written to DRAM).
So it would not correspond to a block boundary of data. This is illustrated in FIG. 160, where the shaded areas represent the blocks that are being formed. The dotted outline shows the block from which it is predicted. The address generator translates the address specified by the motion vector to the block offset (total number of blocks), as shown by the large arrow, and as shown by the small arrow,
Translates the address specified by the motion vector to pixel offset.

At the address generator, the frame pointer, base block address, and vector offset are added to form the address of the block to be fetched from DRAM. If the pixel offset is 0, only one request will be issued. If there is an offset in either the x or y dimension, then two requests will be issued-the original block address and either the immediate right or the one directly below. x and y
If both have an offset, four requests are issued. For each block to be fetched, the address generator calculates the start / stop address parameters and sends them to the DRAM interface. The use of these start / stop addresses may be illustrated by example as outlined below.

As shown by the shaded area in FIG. 161,
Consider a pixel offset of (1,1). The address generator makes four requests, labeled A to D in the figure. The problem to be solved is how to quickly provide the required sequence of row addresses. The solution is to use a "start / stop" technique, which will be described.

[1773] Consider block A in FIG. Reading starts at position (1,1) and starts at position (7,
You must finish in 7). For the moment, assume that one byte is being read at a time (ie, an 8-bit DRAM interface). The x-value of the coordinate pair forms the 3 LSBs of the address and the y-value is the 3 MSs.
Form B. The start values of x and y are both 1 and give the address 9. Data is read from this address and the x value is incremented. The process is repeated until the x value reaches its stop value.
At this point, the y value is incremented by 1 and the x start value is reloaded, giving address 17. As each byte of data is read, the x value is incremented again until the stop value is reached. The process is repeated until both the x and y values reach their stop value. Thus, 9, 10, 11, 12, 1
3, 14, 15, 17 ,. ． ． 23, 25 ,. ． ． Three
1, 33 ,. ． ． ,. ． ． , 57 ,. ． ． , 63 address sequences are generated.

Similarly, the start / stop coordinates for block B are: (1,0) and (7,0), and for block C: (0,1) and (0,7). , For block D: (0,0) and (0,0).

The next issue is where to write this data. Obviously, looking at block A, the data read from address 9 should be written to address 0 of the swing buffer, the data read from address 10 should be written to address 15 of the swing buffer, and so on. Similarly, address 8 of block B
The data read from is swing buffer address 15
, And the data read from address 16 should be written to address 15 of the swing buffer. It turns out that this function is a very simple implementation, as outlined below.

[1779] Consider Block A. At the start of reading, the swing buffer address register is loaded with the inverse of the stope value, the y-reverse stop value forms 3 MSBs, and the x-reverse stop value forms 3 LSBs. In this case, DRAM
The swing buffer address is 0 while the interface reads address 9 in the external DRAM. Then, as the external DRAM address register is incremented, the swing buffer address register is incremented, as illustrated in Table 236:

[1780]

[Table 236] The discussion so far has centered around 8-bit DRAM interfaces. For 16- or 32-bit interfaces, some modifications have to be made. First, the pixel offset vector is 16 or 3
Must be "clipped" to point to a 2-bit binary. In the example used so far, for block A, the first DRAM read points to address 0,
The data in addresses 0-3 will be read. Then the unwanted data must be discarded. This is accomplished by writing all the data into the swing buffer (which must be physically larger than what was required in the 8 bit case now) and reading at the offset. When performing MPEG half pixel interpolation, 9 bytes in x and / y must be read from the DRAM interface. in this case,
The address generator provides the appropriate start and stop addresses and some additional logic within the DRAM interface is used, but there is no fundamental change in the way the DRAM interface operates.

[1781] The last point to note about the temporal decoder DRAM interface is that additional information must be provided to the predictive filter to dictate what processing is desired for the data. This includes the following: "Last Byte" signal indicating the last byte of the transmission (of 64, 72 or 81 bytes). 261 flag ・ Bidirectional prediction flag ・ 2 bits that indicate the dimension of the block (8 or 9 bytes in x and y) ・ 2 bit number that indicates the order of the block Generated when read from. The other signals are derived from the address generator and piped through the DRAM interface so that they associate with the correct data block as it is read from the swing buffer by the predictive filter block.

[1782] Section B. 14 "UPI Documentation" B. 14.1 "Introduction" This document is intended to give the reader a proper understanding of the operation of the microprocessor interface according to the invention. The interface is basically the same as for the spatial and temporal decoders, the only difference being the number of address lines.

The logic described herein is purely microprocessor internal logic. The relevant schematics are as follows: UPI UPI101 UPI102 DINLOGIC DINCELL UPIN TDET MONOVRLP WRTGEN READGEN VREFCKT Circuits UPI, UPI101, UPI102, UPI0
All the same except that one has a 7-bit address input and the 8th bit is hardwired to ground while the other two have an 8-bit address input. Input / Output Signals The signals described here are U (defined in terms of UPI).
A list of all inputs and outputs to the PI module, with notes detailing the source or destination of these signals: NOTRSTInputGlobal chip reset,
Active low, from pad input driver.

[1780] E1InputEnable signal 1, active low, from pad input driver (Schmitt).

[1780] E2InputEnable signal 2, active low, from pad input driver (Schmitt).

[1786] RNOTWInputRead not
Write signal from pad input driver (Schmi
tt).

[1787] ADDRIN [7: 0] InputAdd
From the res bus signal, pad input driver (Sc
hmitt).

[1788] NOTDIN [7: 0] InputInp
ut data bus, from bidirectional microprocessor data pin input pad driver (TTLin).

[1789] INT RNOTW OutputThe
Read not Write signal, to internal circuitry accessed by microprocessor interface (see memory map).

[1790] INT ADDR [7: 0] Output
The Internal Address Bus,
To all circuits accessed by the microprocessor interface (see memory map).

[1791] INTDBUS [7: 0] Input / O
outputThe Internal Data bu
s, to all circuits accessed by the microprocessor interface (see memory map) and to microprocessor data output pads. The internal data bus carries data that is the opposite of the data on the pins of the chip.

[1792] READ STROUTAn is an internal timing signal that directs the reading of a location in the device memory map.

[1793] WRITE STROutputAn is an internal signal which instructs writing of a location in the internal memory map.

[1794] TRISTATED PADOutputA
n is an internal signal connected to the microprocessor data output pad which indicates that they should be tri-stated. General comments: The UPI schematic consists of 6 small modules: NONOVRLP, UPIN, DI
NLOGIC, VREFCKT, READGEN, WR
TGEN. It should be noted from the general list of signals that there is no clock signal associated with the microprocessor interface other than the microprocessor bus timing signal which is synchronous with all other timing signals on the chip. Therefore, no timing relationship should be assumed between the operation of the microprocessor and the operation of the rest of the devices other than those that can be forced by external control. For example, stopping the external system clock for access to the microprocessor interface on the test system.

Another close relationship without clocks in UPI is that some internal timing is self-timed. In other words, the delay of a certain signal is UPI
Controlled internally for the block.

The UPI's overall function is to take address, data, enable, and read / write signals from the outside world and format them so that they can operate their internal circuitry correctly. The internal signal that defines access to the memory map is INT
RNOTW INT ADDR [. ． ． ], INTD
BUS [. ． ． ], And READ STR and WRITE
STR. The timing relationships of these signals are shown below for read and write cycles. Although the data sheet definition and the diagram below always show a chip enable cycle, circuit operations are kept enabled low so that addresses can be cycled to perform continuous read or write operations. You should be careful. This function is possible because of the address transition circuit.

[1797] Furthermore, INT RNOTW and READ
The existence of STR and WRITE STR reflects some degree of redundancy. It is a READ ST with separate internal circuits.
Use either R or WRITE STR (and ignore INT RNOTW), or INT R
Strobe signal separate from NOTW (strobe signal is R
(Derived from the OR of EAD STR and WRITE STR) can be used.

The internal data bus is precharged high during the read cycle, which also has a resistive pull-up to default to the 0XFF state for an extended period when the internal data bus is not being driven. This translates to 0x00 on the external pins when they are enabled because the internal data bus is the inverse of the data on the pins. This means that when an external cycle accesses a register or a register bit that is a hole in the memory map, the output data is limited and is Low.

[1799] Circuit Details: UPIN-This circuit is the overall change detection block. It is 1
It includes a sub circuit called TDET which is a bit change detection circuit. UPIN is TDE for each address bit
T module, and rnotw for each enable signal. UPIN further includes some sort of coupling logic to gate the outputs of the change detection circuit together. This gating produces the following signals: TRAN-indicating a transition to one of the input signals, and UPD DONE-indicating that the transition is complete and the cycle can be executed.

[1800] CHIP EN-Indicates that the chip has been selected.

[1801] TDET- This is a 1-bit change detection circuit. Two latches and two
It consists of two dedicated OR gates. The first latch is the signal SA
Clocked by MPLE, the second latch is signal U
Clocked by PDATE. These two non-overlapping signals are stored in the module NONOVRLP.
come from. In general operation, input transition is CHA
This is done to give rise to NGE, which in turn gives rise to SAMPLE. When all input changes are accepted while SAMPLE is high, and when the input changes stop, CHA
NGE is low, SAMPLE is low, which is U
Bring PDATE high, which in turn transfers data to the output latch, indicating UPD DONE.

[1802] NONOVRLP- This circuit basically inputs TRAN and outputs SMAPLE.
Is a non-overlapping clock generator that generates UPDATE. External gating on the UPDATE output is UPD until the write pulse is complete.
Stop ATE going high.

[1803] DINLOGIC-This module is composed of 8 stages of the data input circuit DINCELL and some gating for driving the TRISTATADEP signal. This is because the output data port has Enable1 low, Enable2 low, and Rn
Indicates that it will only drive if otW is high and the internal read str is high.

DINCELL-This circuit consists of a data input latch and a tri-state driver that drives the internal data bus. Signal DATAH
When OLD is high and both Enable1 and Enable2 are low, the data from the input pad is latched.
The tri-state driver uses the internal signal INT RNOTW
Drives the internal data bus whenever is low. The internal data bus precharges the transistors, and bus pullups are also included in this module.

WRTGEN-This module generates the WRITE STR and the latch signal DATAHOLD for data latching. The write strobe is a self-time signal, but the self-time delay is defined by VREFCKT. The output from the timing circuit RESETWRITE is WRI
Used to terminate the TE STR signal. It should be noted that the actual write pulse that writes to the register only occurs after the access cycle is over. This is because the data input to the chip is sampled only on the back edge of the cycle. Therefore, the data is only valid after the normal access cycle has ended.

READGEN-This circuit, as the name implies, generates a READ STR and also a PRECH signal which is used to precharge the internal data bus. PREC
The H signal is also a self-time signal, and VREF is applied during that period.
It depends on the voltage of CKT and the internal data bus. REA
D STR is not self-time, but lasts from the end of the precharge period to the end of the cycle. The precharge circuits use inverters, but their transmission characteristics are biased so that they require a voltage of approximately 75% of the supply voltage before they are out of phase. This circuit is READ
Ensure that the internal bus is properly precharged before the STR begins. If the internal bus is already precharged, the timing circuit will stop the PRECH pulse, which tends to be zero width, by the signal RESET.
Guarantee the minimum width via READ.

VREFCKT- VREFCKT is the only circuit that controls the self-timing of the interface. Both delays, WRITE
1 / Width of STR and 2 / Wid of PRECH
th is controlled by the current through the P-transistor.
The gate of this P-transistor is controlled by the signal VREF and this voltage is set by a diffusion transistor of 25 kΩ.

[1808] Section C. 1 "Prospect" C. 1.1 “Introduction” The structure of the image formatting unit according to the present invention is shown in FIG. Two address generators, one for writing, one for reading, a buffer manager that manages two address generators, provides frame rate conversion, and a data processing pipeline that includes vertical and horizontal ensamplers There is a final control block that regulates the output of the processing pipeline, color space conversion and gamma correction, and.

[1809] C. 1.2 "Buffer Manager" Tokens arriving at the input of the image formatter are buffered in the FIFO and transmitted to the buffer manager. This block detects the arrival of new pictures and determines the availability of buffers to store each picture. If there is a buffer available, it is assigned to the arriving picture and its index is transmitted to the write address generator. If no buffers are available, incoming pictures are stalled until one is available. All tokens are sent to the write address generator.

Each time the read address generator receives a VSYNC signal from the display system, a request is made to the buffer manager for a new display buffer index. If there is a buffer containing the complete picture data, and the picture appears to be ready for display, the index of the buffer is sent to the display address generator. Otherwise, the buffer manager sends the last buffer index to be displayed. At startup, 0 is sent as an index until the first buffer is full. A picture is ready for display if the number of pictures (calculated as each picture is input) is greater than or equal to the number of pictures expected on the display (presentation) given the encoding frame rate. There is. The expected number is determined by counting the picture clock pulses, where the picture clock can be generated locally by the clock distributor or externally. This technique allows frame rate conversion (eg 2-3 pulldown).

The external DRAM is used for the buffer, and the number thereof may be two or three. If the frame rate conversion is to be performed, three are required.

[1812] C. 1.3 "Write Address Generator" The write address generator receives tokens from the buffer manager and detects the arrival of each new data token. As each data token arrives, the address generator has a DRAM to store the arriving block.
Calculate a new address for the interface. The raw data is then sent to the DRAM interface where it is written to the swing buffer. DRAM addresses are block addresses and pictures organized as a raster of pictures or blocks in DRAM. However, since the incoming picture data is actually a sequence of organized microprocessors, the address generation algorithm must account for (intra-block) linewidth offsets for the lower rows of blocks within the microprocessor. .

The arrival buffer index provided by the buffer manager is used as an address offset for the entire stored picture. In addition, the component offsets are also used in the calculations because each component is stored in a separate area within the designated buffer.

[1814] C. 1.4 "Read Address Generator" The read address generator (dispaddr) does not receive or generate tokens, but only generates addresses.
Answer VSYNC, address generator field in
Request the buffer index from the buffer manager according to fo, read start, sync mode, and lsb_invert. Upon receipt of the index, it generates three sets of addresses, one for each component, one for the current picture read in raster order. Different setups allow: interlaced / forward display, and / or vertical unsampling, and field synchronization (for interlaced displays). At the lower level, the read address generator translates the base address into a block address and a sequence of byte counts for each of the three components compatible with the DRAM page structure. The addresses provided to the DRAM interface are page and line addresses along with block start and block end counts.

[1815] C. 1.5 "Output Pipeline" Data from the DRAM interface is provided to the output pipeline. The three component streams are first interpolated vertically and then horizontally. After interpolation, the three components must be in equal proportion (4: 4: 4) and sent to the color space converter and color look-up table / gamma correction. The output interface is HS display
The stream can be held at this point until YSC is reached. The output controller then multiplexes the three components as needed and directs them to one, two or three 8-bit buses.

[1816] C. 1.6 “Timing style” Basically, it is associated with the image formatting section 2
There are one major timing style. First, there is the system clock that provides the timing for the front end of the chip (address generator and buffer manager, plus front end of the DRAM interface). Second, there is a pixel clock that drives all timing for the back end (DRAM interface outputs, and the entire output pipeline).

Each of the two aforementioned clocks drives many on-chip clock generators. The FIFO, buffer manager, and read address generator operate from the same clock (DΦ), and the write address generator uses a similar but separate clock (WΦ). Data is clocked to the DRAM interface on the internal DRAM interface clock (outΦ). DΦ,
WΦ and outΦ are all generated from syscik.

The read and write addresses are clocked into the DRAM interface by the clock of the DRAM interface itself.

The data is read from the DRAM interface on bifRΦ and transmitted to a section of the output pipeline called bushyne (because of its physical location, Northeast) that operates on the clock represented by NEΦ. . Gamma R in front
The pipeline section from the AM is clocked on a separate but similar clock (RΦ). bifR
Φ, NEΦ, RΦ are all pixel clocks, pixin
Drawn from.

[1820] For testing purposes, all major interfaces between blocks have either an attached Snowpa or a Super Snowpa. This depends on the timing modality and the type of access required. Block boundaries between separate but similar timing schemes have associated retiming latches.

[1821] Section C. 2 “Buffer management” C. 2.1 "Introduction" The purpose of the buffer management block according to the invention is to provide the address generator with an index identifying two or three external buffers for writing and reading picture data. The allocation of these indexes is affected by three main elements, each representing one effect of the timing modality on the operation.
These are the rate at which the picture data arrives at the input to the image formatting unit (coded data rate), the rate at which the data is displayed (display data rate), and the frame rate of the encoded video sequence (presentation rate).

[1822] C. 2.2 "Functional Perspective" A three-buffer system, given the timing constraints of the system, ensures that frames are repeated or skipped as necessary to achieve the best possible frame sequence. The presentation rate and the display rate can be different (for example, 2-3 pulldown). If the picture takes longer than the available display time to decode,
Pictures that represent the difficulty in decoding can be accommodated in a similar manner so that the previous frame is repeated while all others "catch up". In a two-buffer system, three timing modalities must be locked-it is the third buffer that provides the flexibility to handle slack.

The buffer manager operates by maintaining specific status information associated with each external buffer. This is a flag that indicates whether the buffer is in use, is full of data, or is ready to be displayed, and a flag that indicates the number of pictures in the picture sequence currently stored in the buffer. including. The number of presentations is also recorded, which is a number that increments each time a picture clock pulse is received and represents the number of pictures currently predicted to be displayed based on the frame rate of the encoded sequence.

The 1818 arrival buffer (the buffer into which incoming data is written) is allocated each time a picture start token is detected at the input. This buffer is then flagged as IN USE.
When the picture end is reached, the arrival buffer is de-allocated (reset to 0) and becomes a buffer that is flagged as FULL or READY, depending on the relationship between the number of pictures and the number of presentations.

The display address generator requests a new display buffer once every vsync via the 2-wire interface. READY
If there is a buffer flagged with, it will be allocated for display by the buffer manager. If there is no READY buffer, the previously displayed buffer is repeated.

[1826] Each time the number of presentations changes, it is detected,
By examining the relationship between its picture number and presentation number, the RE of any buffer containing the complete picture
ADY-ness is tested. The buffers are considered in order. When a buffer appears to be READY, it automatically cancels the READY-ness of the buffer that was previously flagged as READY. The previous buffer is then flagged as EMPTY. In a buffer that will be considered later,
This works because the allocation plan stores the number of later pictures.

[1827] When a skipped picture in the input stream is designated, the H.264 standard is specified. The time standard token in 261 modifies the number of pictures in the buffer. However, such features, although contemplated, are not currently included. Similarly, the time standard within MPEG has no effect.

[1828] All tokens in the flash token are E
Stall the input until it is MPTY or allocated as a display buffer. After that, the number of presentations and the number of pictures are reset and a new sequence can be started.

[1829] C. 2.3 “Architecture C. 2.3.1“ Interface ”C. 2.3.1.1 “Interface to bm front” All data is input to the buffer manager from the input FIFO, bm front. This transmission is done via a 2-wire interface and the data is 8 bits wide plus an extension bit. All data arriving at the buffer manager is guaranteed to be a complete token. This is a requirement for continuous processing of the number of presentations and for display buffer requests when there are valid gaps in the data stream.

[1830] C. 2.3.1.2 "waddrgen
Interface to Token "(8-bit data, 1-bit extension) is transmitted to the write address generator via a 2-wire interface. As soon as the picture start token arrives at waddrgen, the arrival buffer index is also transmitted on the same interface so that the correct index is available for the address generator.

[1831] C. 2.3.1.3 "dispaddr
The interface to the read address generator consists of two separate 2-wire interfaces which are considered to act as "request" and "acknowledge" signals, respectively. However, single wire is not suitable because of the two two wire based state machines at either end.

The sequence of events is usually associated with the dispaddr interface as follows. First,
dis-paddr is drq to the buffer manager
By validating valid input, v from the display device
Invoke a request to answer sync. Then, when the buffer manager reaches the appropriate point in its state machine, it accepts the request and tries to allocate the buffer to be displayed. The disp valid wire is then certified and the buffer index is transmitted, which is typically dis
Immediately accepted by paddr. In addition, this final two-wire interface (rst) indicates that the number of fields associated with the current index must be reset independently of the number of previous fields.
There are additional wires associated with fld).

[1833] C. 2.3.1.4 "Microprocessor Interface" The buffer manager block uses a 4-bit microprocessor address space with an 8-bit data bus and read / write strobe. There are two select signals, one indicating user accessible location and the other indicating a test location which should not require access under normal operating conditions.

[1834] C. 2.3.1.5 "Events" The buffer manager can create two different events: index found and late arrival. The first of these is identified when the picture arrives, and its picture start extension byte (picture index) is BU BM TRG at setup.
Matches the value written to the ET IX register. Second
The number of pictures in is less than the current number of presentations, that is, the processing in the system pipeline up to the buffer manager is not processing to meet the presentation requirements.

[1835] C. 2.3.1.6 "Picture Clock" In the present invention, the picture clock is the clock signal for the presentation counter and is either generated on-chip or derived from an external source (usually a display system). The buffer manager accepts both of these signals and selects one based on the value of pclk ext (bit in the buffer manager's control register). This signal also acts as an enable for pad picoutpad so that if the image formatter is producing its own picture clock, this signal is also available as an output from the chip.

[1836] C. 2.3.2 “Main Blocks” The following section is a buffer manager schematic (bml
The various hardware blocks that create

[1837] C. 2.3.2.1 “Input / Output Block (bm input)” This module is a buffer manager's four 2-wire interface (input data and output data, drq v
alid / accept and disp valid / a
ccept) and all of the hardware associated with it. The input data register is shown with some token decoding hardware attached to it. bm to
The signal vheader at the input to kdec is used to ensure that the token decoder output is qualified only at the point where the header will be valid (ie not in the middle of the token). The rtimd block acts as an output data register adjacent to the dual input data register for the next block in the pipeline. This accounts for timing differences due to different clock generators. Signals go and ngo are data valid, ac
It is based somewhere on the state machine, based on an AND of cept and notstopped, to indicate when something is "wrecked" at the input or output. The display index portion of this module consists of a two wire interface with an equivalent "go" signal for the data. The rst_fld bit also occurs here, which, if set, causes disp_val.
A signal that remains high until id goes high for one cycle. Then it is reset. In addition to that, the flash token can be EMPTY or IN U to all external buffers by the display buffer.
After having the SE flagged, rst_fld is reset. This is the same point where both the number of pictures and the number of presentations are reset.

There are a small amount of additional circuitry associated with the input data registers that appear at the next level in the hierarchy. This circuit produces a signal that indicates that the input data register has a value equal to that written to BU BM TARGIX and that it is used for event generation.

[1839] C. 2.3.2.2 "bm index" The index block mainly consists of 2-bit registers that represent the various strategic buffer indexes.
These are arr buf, the buffer into which the incoming picture data is written, disp buf, the buffer from which the picture data is read for display, and rdy buf, most of the present that can be displayed if the buffer is requested by dispaddr. Is a buffer index including a picture of. In addition, there is a register containing the buf ix, which is used as a general pointer to the buffer. This register is incremented (D input to max) and cycled through the buffers examining their status, or if the status needs to be changed, arr--buf,
One value of disp buf or rdy buf is specified. All of these registers (ph0 version) are accessible to the microprocessor as part of the test address space. old ix is a retimed version of buf ix and is used to enable the buffer status and picture number registers in the im stus block. buf ix
And old ix are both output from this block 3
Decoded into two signals, each of which can hold the values 1-3. Other outputs are buf ix and arr b
indicates whether it has the same value as either uf or disp buf, and also rdy buf and d
Indicate whether any of the isp buf has the value 0. 0 is not the reference code for the buffer.
It is simply the currently assigned arrival / dis
It just indicates that there is no play / ready buffer.

The arr--buf and disp--buf are enabled by respective 2-wire interface output accept registers.

[1187] An additional circuit at the bmlogic level is used to determine if the current buffer index (buf ix) is equal to the maximum index in use, as limited by the value written to the control register during setup. It "1" in the control register is 3
It indicates a buffer system, and "0" indicates a 2-buffer system.

[1842] C. 2.3.2.3 "Buffer status" The main components in the buffer status are the status and the picture number register for each buffer. Each of the three groups is a master-slave arrangement, where the slave is a bank of three registers and the master has its output (using register enables,
One register directed to one of the slaves (switched by old ix). One of the possible inputs to the master is (buf at bmlogic level)
Multiplexed between different slave outputs (indexed by ix). Since the buffer status decoded at the bmlogic level is used in the state machine logic, it can take any of the values shown in Table 237 or it can recycle the previous value. The number of pictures can take the previous value or the previous value incremented by 1 (or 1 + delta, the difference between the actual and predicted temporal reference in the case of H.261). This value is supplied by the 8-bit adder present in the block. The first input to this adder is this pnum, the number of pictures of the data currently being written.

[1843]

[Table 237] This is separate so that any of the three-buffer picture number registers can be easily updated based on the current (or previous) picture number rather than their own previous picture number (which is almost always obsolete). It needs to be stored (in its own master-slave arrangement). this
The pnum is reset to -1, and when the first picture arrives, it is added to the output from the adder, so the input to the first buffer picture number register is zero.

In the current version, the delta is tied to 0 due to the lack of a time reference block to supply its value.

[1845] C. 2.3.2.4 "Presentation Number" The 8-bit presentation number register has an associated presentation flag used in the state machine to indicate that the presentation number has changed since it was last examined. This is necessary because the picture clocks are synchronous in nature and can be active in any state, not because they relate to the number of presentations. The rest of the circuitry in this block detects that a picture clock pulse has occurred and "remembers" this fact.
Related to that. In this way, the number of offers can be updated when it is valid to update. A representative sequence of events is shown in FIG. Signal incr prn
Becomes active one cycle after the retimed picture clock rising edge and lasts until a state comes in, during which the number of presentations can be modified. This is indicated by the signal en prnum. The reason why the number of presentations can be updated only during a specific state is
Standar for it to provide the signal rdyst
It is used to drive a significant amount of logic, including d-cell, not-very-fast 8-bit adders. Therefore, it should only be changed while the next succeeding state is not using the result.

[1846] 2.3.2.5 "Time Reference" The time reference block according to the present invention has been omitted from the current aspect of the image formatting section, but its operation is now described for finishing.

The function of this block is delta, H.264. Calculating the difference between the time reference value received at the token in the 261 data stream and the “predicted” time reference (1 + previous value). As a result, H. At 261 the frame can be skipped. The time reference token is used for all non-H. Ignored in the H.261 stream. The calculated value is used in the status block to calculate the number of pictures for the buffer. bml
The effect of omitting blocks from the H. 261
Even if the stream indicates that some pictures should be skipped, the number of pictures is always continuous in any sequence.

The main components of the block (which can be seen in the schematic bm tref) are the registers for tr, exptr and delta. In the invention, tr is reset to 0 and loaded from the input data register when appropriate. Similarly, exptr is -1
, And incremented by 1 or delta during the sequence of time reference states. In addition to that, the delta is reset to 0 and the difference between the other two registers is loaded. All three registers are reset after the flush token. The adders in this block are used for both delta and exptr calculations, ie for subtraction and addition operations, respectively, and the signal del
controlled by ta calc.

[1849] C. 2.3.2.6 Control register (b
control registers for the buffer manager are in block bm
in urges. These are the access bit registers,
A setup register (which limits the maximum number of external buffers and the internal / external picture clock) and a target index register. The access bits are synchronized as expected. Signal stopp 0, sto
pd 1 and nstopd 1 are access bit O
It is derived from R and the two event stop bits. Upi address decoding for all bmllogic is done by the block bm udec, which, along with the two select signals from the top level address decode of the image formatter, upi.
Take the lower 4 bits of the data bus.

[1850] 2.3.2.7 "Control State Machine" The state machine logic is originally its own block,
Occupies the bm state. However, because of the code generation, it is now flattened and
It is on sheet 2 of the schematic.

The main sections of this logic are the same. This is a flag used to decode, generate logic signals for control of other blogic blocks, and select routes through the state machine, f
Includes new state encodings including rom ps and fromfl. bm stus and bmind
There is a separate block for making the max control signal for ex.

[1852] Signals on state machine hardware have been given simple alphabetic names for ease of typing and reference. All of them are listed in Tables 238-241 along with the logic expressions they represent. Furthermore, they are mblogic behaviors M.
It is also shown as a comment in the commentary (bmllogic.M).

[1853]

[Table 238]

[1854]

[Table 239]

[1855]

[Table 240]

[1856]

[Table 241] C. 2.3.2.8 "Monitoring Operation (bminfo)" In the present invention, a module, bminfo, is included so that buffer status information, index values, and the number of presentations can be observed during simulation. It is written into M and produces an output each time one of its inputs changes.

[1857] C. 2.3.3 “Register Address Map” The buffer manager address space is split into two areas, user accessible and test. Therefore, there are two separate enable wires derived from range decoding at the top level. Table 242 shows the user accessible registers and Table 2
43 shows the contents of the test space.

[1858]

[Table 242]

[1859]

[Table 243] C. 2.4 State Machine Operation As detailed in Table 244, there are 19 states in the buffer manager state machine. These are shown in Figure 1.
66, the behavioral explanation bmllogic. They interact as described in M.

[1860]

[Table 244] C. 2.4.1 "Reset State" The reset state is PRES0 and the flag is set to 0 so that the main loop circulates initially.

[1861] C. 2.4.2 "Main Loop" The main loop of the state machine consists of the states shown in Figure 167 (main diagram-highlighted in Figure 166). State PRES0 and PRES1 are signal pre
Relates to detecting the picture clock via sflg. The two cycles are all rdyst,
C. Since it depends on the value of the adder output signal described in 2.3.2.4, it can be included for testing. If the offer flag is detected, all buffers are examined for possible "readiness", else the state machine goes to state DRQ. PRE
Each cycle around the S0-PRES1 loop checks a different buffer for full and ready conditions. When these are filled, the previous ready buffer (if any) is cleared, a new ready buffer is allocated, and its status is updated. This process examines all buffers (index == ma
x buf) and then the state advances. A buffer is considered ready to be displayed if any of the following is true: (pic num> pres num) && (pic num-pres num)> = 128) or (pic num <pres num) && (pres. num-pic num) <= 128) or pic num == pres num The state DRQ is a request for the display buffer (drq validreg && disp acc).
Check reg). If there is no request, the state proceeds (usually to state TOKEN, as described below). Otherwise, the display buffer index is emitted as follows: If there is no ready buffer, the previous index is reissued, or if there is no previous display buffer,
Zero index (zero) is emitted. If the buffer is ready to be displayed, its index is raised and its state is updated. If necessary, the previous display buffer is cleared. The state machine then proceeds as before.

State TOKEN is a typical option for completing the main loop. If there are valid inputs and the output is not stalled, the token is examined for strategic value (discussed in a later section), otherwise control returns to state PRES0.

Control only branches from the main loop when certain conditions are met. These are described in the sections below.

[1864] C. 2.4.3 "Assigning Ready Buffer Index" If the buffer is determined to be ready during the PRES0-PRES1 loop, the previous ready buffer is emptied. This is because only one buffer can be designated as ready at any given time. State VACATE R
DY clears the old ready buffer by setting its state to VACANT and control goes to PRES0.
When returning to state, the buffer index is reset to 1 so that all buffer readiness is tested. The reason for doing this is that the index now points to the previous ready buffer (for the purpose of clearing it) and there is no record of our new ready buffer index. Therefore, it is necessary to retest all buffers.

[1865] C. 2.4.4 “Display buffer index allocation” Display buffer index allocation is State D
State V to clear old display buffer state directly from RQ (state USERDY)
This is done via ACATE DISP. The selected display buffer is flagged as IN USE, the value of rdy_buf is set to 0, and the index is reset to 1 to return to state DRQ. Furthermore, the necessary index is given to the disp buf, and the two-wire interface wire (disp v
alid and drq acc) are controlled accordingly. Control simply returns to state DRQ only so that a decision between states TOKEN, FLUSH, ALLOC need not be made in state USER RDY.

[1866] C. 2.4.5 "Operation on receipt of picture end" Upon receipt of the picture end token, control transfers from state TOKEN to state picture end, where the index is not already pointing to the current arrival buffer. It can be set to point and its state updated. out acc reg and en
Assuming full is true, the status can be updated as described below. Otherwise,
Controls remain at the state picture end until they fit together. The en full signal is provided by the write address generator to indicate that the swing buffer has swung, that is, the last block has been successfully written and thus it is safe to update the buffer status.

[1867] The just completed buffer is tested for readability and status FULL is returned depending on the test result.
Alternatively, either READY is given. If it is ready, the rdy buf is given the value of that index and the set la ev signal (late arrival event) is set high (indicating that the expected display has advanced forward in time for decoding). To). The new value of arr_buf is now 0, and if the previous ready buffer needed to clear its status, the index is set to point to it and control moves to state VACATE RDY.
If not, the index is reset to 1 and control returns to the start of the main loop.

[1868] C.I. 2.4.6 "Operation when picture start is received (allocation buffer allocation)" When the picture start token arrives during the state TOKEN, the flag fromps is set, and the state ALLOC is patroled instead of the state TOKEN. To change the basic state machine loop. State ALLOC concerns the allocation of an arrival buffer (in which picture data arriving in it can be written) and cycles through it until it finds one whose status is VACANT. Since the buffer is output on the data 2-wire interface, it is allocated only if out acc reg is high. Therefore, the circulation around the loop will continue until this is really the case. If a suitable arrival buffer is found, the index is arrb
assigned to uf and its status is IN USE
Is added. The index is set to 1, the flag fromps is reset, and the state is set to go to NEW EXP TR. The index of the picture (contained in the word following the picture start) is checked to determine if it is the same as targ ix (the target index specified in the setup), and if so, set i
f + ev (index found event) is set high.

[1869] Three statuses NEW EXP TR,
SET ARR IX and NEWPIC NUM set the new predicted time reference and the number of pictures for the incoming data. The intermediate state updates the index arr b so that the correct picture number register is updated (note that the hispnum is also updated).
Set to uf. Control then proceeds to the state OUTPUTTAIL which outputs data (assuming the preferred 2-wire interface signal) until a low extension is encountered. At this point the main loop is restarted. This means that all data blocks (64 items) will be output, during which no testing for presentation flags or display requests is done.

[1870] C. 2.4.7 "Action when flash is received" The flash token in the data stream indicates that the sequence information (presentation number, picture number, rst_fld) should be reset. This can only occur when all the data leading to FLUSH has been processed correctly. Therefore, after receiving FLUSH, all frames have been handed to the display, that is, all but one buffer has the status EMPTY and that one buffer is IN USE (as a display buffer). It is necessary to monitor all the status of the buffer until
At that point, the "new sequence" can be safely used.

When a flush token is detected in the state token, the flag from fl is set, causing the basic state machine loop to change to patrol the state flush instead of the state token. State FLUSH examines the status of each buffer in turn and waits for it to become VACANT or IN USE as a display. The state machine simply cycles around the loop until the condition is met, then increments its index and repeats the process until all buffers have been visited. Assume that the presentation number, the number of pictures, and all temporal reference registers have their reset value rst_fld set to 1 if the last buffer satisfies the condition. The flag fromfl is reset and normal main loop operation is restarted.

[1872] C. 2.4.8 “Operation when a time standard is received” When a time standard token is encountered, H.264 A 261 bit check is performed and set, four states TEM
Patrol P REF0 to TEMP REF3. These perform the following operations.

[1873] TEMP REF0: temp ref =
in data reg; TEMP REF1: delta = temp ref-
exp tr; index = arr buf; TEMP REF2: exp tr = delta + ex
p tr; TEMP REF3: pic num [i] = this
pnum + delta; index = 1. C. 2.4.9 "Other Tokens and Tail" State tokens send control to state OUTPUT TAIL in all cases other than those outlined above. Encounter the last token word (in
Control remains here until the extn reg is low) and the main loop is then reentry.

[1874] C. 2.5 "Application Note" C. 2.5.1 "State Machine Stalling Buffer Manager Input" This requirement repeatedly checks for "synchronous" timing events for picture clocks and display buffer requests. The need to stall the buffer manager input during these checks means that there is a limit to the rate of data through the buffer manager when there is a continuous supply of data at the buffer manager input. Typical state sequence is PRES0, PRE
S1, DRQ, TOKEN, OUTPUT TAIL, each of which lasts 1 cycle except for OUTPUT TAIL. This is for each block of 64 data items while the input is stalled (state PRES
It means that there is an overhead of 3 cycles (between 0, PRES1, DRQ) and thereby slowing down the write speed by 3/64 or almost 5%. This number of overheads can be increased, sometimes up to 13 cycles, when the auxiliary branch of the state machine is executed under worst case conditions. Note that this much overhead can only be applied on a frame-by-frame basis.

[1875] 2.5.2 "Presentation act during access" Bm illustrated by the schematic shown in 2.3.2.4
A special aspect of pres means that the number of offers is free to move during upi access. If the number of offers when revoking access is required to be the same as the number of offers when gaining access, this is done by reading the number of offers after access is granted, and just before the access is revoked. It can be achieved by writing back. Note that this is asynchronous, so it is desirable to repeat the access a few times in order to see more effectiveness.

[1876] 2.5.3 “H261 hour reference number” module bm tref (not shown)
should be included in c. H. 261 Hour Reference Value is bm delta input from bmtref
It is processed correctly by directing it to the stus module. The delta input can be tied to 0 if the frames are always consecutive.

[1877] Section C. 3 “Write address generation” C. 3.1 "Introduction" The function of the write address generation hardware according to the present invention is to create a block address for the data to be written to the buffer. This takes into account the base address of the buffer, the components pointed to in the stream, horizontal and vertical sampling in macroblocks, picture dimensions and coding standards. Data arrives in the form of macroblocks but must be stored so that the line can be easily retrieved for display.

[1878] C. 3.2 "Functional Perspective" Each time a new block arrives in the data stream (indicated by the data token), the write address generator is required to create a new block address. Up to 64 data words can be stored by the DRAM interface (in the swing buffer) before the address is actually needed, so it is not necessary to build the address immediately. This means that the various address components can be added to the running sum in consecutive cycles, thus eliminating the need for any hardware multiplier. The macroblock counter function stores the strategic end value and is accomplished by running a count in the register file, which is the operand for comparison and conditional update after each block address calculation.

Considering the picture format shown in FIG. 170, the predicted address sequences are standard data stream and H.264. Can be derived from both data streams such as H.261. These are shown below. The slices are not wide enough (3 macroblocks rather than 11), but here we use the same "half picture width slice" concept for convenience, and the sequence is "H.261 type". As such, the format is actually H.264. Note that it does not comply with 261 regulations. Data is full macroblock,
In the example shown, it arrives as 4: 2: 0 and each component is stored in its own area of the designated buffer. Standard address sequence: 000,001,00C, 00D, 100,200; 002,003,00E, 00F, 101,201; 004,005,010,011,102,202; 006,007,012,013,103, 203; 008, 009, 014, 015, 104, 105; 00A, 00B, 016, 017, 105, 205; 018, 019, 024, 025, 106, 107; 01A, 01B, 026 ,. ． ． ． ． ． ． ． ． ． 080,081,08C, 08D, 122,222; 082,083,08E, 08F, 123,223; 261 type sequence: 000,001,00C, 00D, 100,200; 002,003,00E, 00F, 101,201; 004,005,010,011,102,202; 018,019,024,025,106, 107; 01A, 01B, 026, 027, 107, 207; 01C, 01D, 028, 029, 108, 208; 030, 031, 03C, 03D, 10C, 20C; 032, 033, 03E, 03F, 10D, 20D; 03, 035, 040, 041, 10E, 20E; 006, 007, 012, 013, 103, 203; 008, 009, 014, 015, 104, 105; 00A, 00B, 016, 017, 105, 205; 01E, 01F, 02A, 02B, 109, 209; 020, 02 , 02C, 02D, 10A, 20A; 022, 023, 02E, 02F, 10B, 20B; 036, 037, 042, 043, 10F, 20F; 038, 039, 044, 045, 110, 210; 03A, 03B, 046. , 047, 111, 211; 048, 049, 054, 055, 112, 212; 04A, 04B, 056 ,. ． ． ． ． ． ． ． ． ． 06A, 06B, 076, 077, 11D, 21D; 07E, 07F, 08A, 08B, 121, 221; 080, 081, 08C, 08D, 122, 222; 082, 083, 08E, 08F, 123, 223; 3.3 “Architecture” C.I. 3.3.1 “Interface” C. 3.3.1.1 “Interface to Buffer Manager” The buffer manager outputs data and buffer index directly to the write address generator. This is done under the control of a two wire interface. In one way,
The write address generator block can be thought of as an extension of the buffer manager. Because the two are very closely connected. However, they operate from two separate (but similar) clock generators.

[1880] C. 3.3.1.2 "Interface to dramif" The write address generator provides the data and address for the DRAM interface. Each of these has its own two wire interface and dramif uses each of them in a different clocking scheme. In particular,
The address is clocked dramif on a clock that is not related to the write address generator clock. Therefore,
It is synchronized at the output.

[1881] C. 3.3.1.3 "Microprocessor Interface" The write address generator uses a 3-bit microprocessor address space with an 8-bit data bus and read / write strobe. There is one select bit for register access.

[1882] C. 3.3.1.4 "Event" The write address generator can generate 5 different events. Two respond to the picture size information appearing in the data stream (hmbs and vmbs) and three respond to the DEFINE SAMPLING token (one event for each component).

[1883] C. 3.3.2 “Basic structure” Schematic diagram of the structure of the write address generator
n. Shown on sch. It consists of a data path, some control logic, and snowpapers and synchronization.

[1884] C. 3.3.2.1 “Data path (bw
datapath is the C. 18-bit adder / subtractor of the type described in Chapter 5,
And a register file (see C.3.3.4) to produce a zero flag (based on the output of the adder) for use in the control logic.

[1885] C. 3.3.2.2 "Control Logic" The control logic of the present invention is configured by hardware to generate all register file load and drive signals, adder control signals, and two-wire interface signals, and a writable control register. including.

[1868] 3.3.2.3 “Snowpa and Sync” Supersnowpa exists on both data and address ports. Snowpas in the datapath are controlled as snowpas from zcells. The address has synchronization between the write address generator clock and the dramif clk format. Syncifs are used in zcells for 2-wire interface signals, and a simplified synchronizer is used in the datapath for addresses.

[1887] C. 3.3.3 “Control logic and state machine” C.I. 3.3.3.1 “Input / output block (wa i
nout) ”This block is a latch for input data (for token decoding) and (for decoding in four ways)
Includes an input and two output 2-wire interface with arrival buffer index.

[1888] C. 3.3.3.2 "Two Cycle Control Blocks (wa fc)" Flag fc (first cycle) is maintained here and is the state machine in the middle of a two cycle operation (ie an operation involving an add)? Instruct me.

[1889] C. 3.3.3.3 “Component count (w
a comp) "Another address is required for the data block within each component, which keeps the current component under consideration based on the type of data header received in the input stream.

[1890] C. 3.3.3.4 "Modulo-3 control (wa mod3)" When generating an address sequence for a H.261 data stream, it is necessary to count three rows of macroblocks up to half along the screen (C.3.
2). This is accomplished by maintaining a modulo-3 counter that is incremented each time a new row of macroblocks is traversed.

[1891] C. 3.3.3.5 “Control register (w
a uregs) ”module wa uregs includes step-up registers and coding standard registers, the latter being loaded from the data stream. The setup register uses 3 bits: QCIF (1sb) and the maximum amount of components expected in the data stream (bits 1 and 2). In addition, the access bits are in this block (synchronized as usual) and the "stopped" bit is extracted as the OR of the access bit and the event stop bit at the next higher level of the logic. Microprocessor address decoding is performed by the block wa dec which takes the lower 2 bits of the read and write strobes, the select wire and the address bus.

[1892] C. 3.3.3.6 "Control state machine" The logic within this block is divided into several distinct areas. State decode, new state encode, origin of "intermediate" logic signals, datapath control signals (drive, drive, load, adder control and select signals), multiplier control, 2-wire interface control, and 5 event signals. .

[1893] C. 3.3.3.7 "Event Occurrence" Five event bits are generated as a result of the particular token reaching the input. What is important is that in each case,
All tokens will be received before the event is fired, as the event service routine will calculate based on the newly received value. Because of this, each bit is delayed by the full cycle before entering the event hardware.

[1894] 3.3.4 “Register Address Map” There are two sets of registers in the write address generator block. These are the top-level setup type registers located in the standard cell section, the keyhole datapath registers. These are listed in Table 245 and Tables 246 to 256, respectively.

[1895]

[Table 245]

[1896]

[Table 246]

[1897]

[Table 247]

[1898]

[Table 248]

[1899]

[Table 249]

[1900]

[Table 250]

[1901]

[Table 251]

[1902]

[Table 252]

[1903]

[Table 253]

[1904]

[Table 254]

[1905]

[Table 255]

[1906]

[Table 256] Keyhole registers are broadly divided into two categories. It contains the picture size parameter that must be loaded before address calculation and the running sum of various (horizontal and vertical) block and macroblock counts. The picture size parameter may be loaded in response to an interrupt generated by the write address generator, that is, when either the picture size or the sampling token appears in the data stream. Alternatively, if the picture size is known before receiving the data stream, they can be written after reset. Setup example Section C. As described in 13, the picture size parameter register is defined in the next section.

[1907] C. 3.4 "Programming the Write Address Generator" The next datapath register must contain the correct picture size information before the address calculation can proceed. They are shown in FIG. 171.

[1908] 1) WADDR HALF WIDTH
IN BLOCKS: This defines the width, in blocks, of half the incoming picture.

2) WADDR MBS WIDE: This defines the width of the incoming picture in macroblocks.

3) WADDR MBS HIGH: This defines the height of the incoming picture in the macroblock.

[1911] 4) WADDR LAST MB IN
ROW: This defines the number of blocks in the upper left block of the last macroblock in one full width row of the macroblock. Block numbering starts at 0 in the upper left corner of the leftmost macroblock, increments for each block across the frame, and continues to the next block row within the macroblock row.

[1912] 5) WADDR LAST MB IN
HALF ROW: This is similar to the previous item, but defines the block number of the block to the left of the last macroblock in the halfwidth row of the macroblock.

[1913] 6) WADDR LAST ROW IN
MB: This defines the block number of the leftmost block in the row of the last block within the row of macroblocks. 7) WADDR BLOCKS PER MB RO
W: This defines the total number of blocks contained in one full width row of the macroblock.

[1914] 8) WADDR LAST MB RO
W: This defines the upper left block address of the leftmost macroblock in the last row of macroblocks in the picture.

919) WADDR HBS: This defines the width of the block of incoming pictures.

[1916] 10) WADDR MAXHB: This is 1
Defines the block number of the rightmost block in a row of blocks within one macroblock.

11) WADDR MAXVB: This defines the height-1 of one macroblock in the block.

In addition to that, the registers that define the organization of the DRAM must be programmed. These registers are 3 buffer based registers, and n component offset registers, where n is the expected number of components in the data stream (it can be defined in the data stream, 1 min and 3 max). Can be the limit).

[1919] Note that many parameters specify the number of blocks or block addresses. This is because the final address is expected to be a block address and the calculation is based on a cumulative algorithm.

The screen configuration illustrated in Figure 171 yields the following register values: 1) WADDR HALF WIDTH IN BLO
CKS = 0x16 2) WADDR MBS WIDE = 0x16 3) WADDR MBS HIGH = 0x12 4) WADDR LAST MB IN ROW = 0x
2A 5) WADDR LAST MB IN HALF R
OW = 0x14 6) WADDR LAST ROW IN MB = 0x
2C 7) WADDR BLOCKS PER MB ROW
= 0x58 8) WADDR LAST MB ROW = 0x5D8 9) WADDR HBS = 0x2C 10) WADDR MAXVB = 1 11) WADDR KAXHB = 1 C.I. 3.5 State Machine Operations There are 19 states in the buffer manager state machine, details of which are given in Tables 257 and 258. These are as shown in FIG. 173.
c. Interact as described in M.

[1921]

[Table 257]

[1922]

[Table 258] C. 3.5.1 Address Calculation The main section of the write address generator state machine is illustrated on the left side of FIG. Upon receipt of the data token, the state machine sends the state ID.
From LE to state ADDR1 and then state A
It moves to DDR5 and outputs an 18-bit block address from there under the control of the 2-wire interface. The calculations performed by states ADDR1 to ADDR5 are as follows: BU WADDR SCRATCH = BU BUFFE
Rn BASE + BU COMPm OFFSET; BU WADDR SCRATCH = BU WADDR
SCRATCH + BU WADDR VMBADD
R; BU WADDR SCRATCH = BU WADDR
-SCRACTH + BU WADDR HMBADD
R; BU WADDR SCRATCH = BU + WADDR
SCRACTH + BU WADDR VBADDR; out addr = BU WADDR SCRACTH
+ BU WADDR HB; 1) BU WADDR VMBADDR: The block address of the leftmost macroblock of the macroblock row (upper left block), which contains the block whose address is calculated.

2192) BU WADDR HMBADDR:
The block address of the upper macroblock (upper left block) of the column of the macroblock, and the block whose address is calculated are included therein.

3) BU WADDR VBADDR: The block address within the macroblock row at the left end of the row of the block, which contains the block whose address is calculated.

4) BU WADDR HB: The number of horizontal blocks in the macroblock of the block whose address is calculated.

[1926] 5) BU WADDR SCRATCH:
A scratch register used for temporary storage of intermediate results.

Considering FIG. 172, for example, taking the calculation of a block whose address is 0x62D, the following calculation sequence occurs: scratch = BUFFERn BASE + COMPm.
OFFSET; (assumed to be 0) Scratch = 0 + 0x5D8; Scratch = 0x5D8 + 0x28; Scratch = 0x600 + 0x2C; Block address = 0x62C + 1 = 0x62D; The contents of various registers are shown in the figure.

[1928] C. 3.5.2 Calculate New Screen Location Parameter When the address is output, the state machine continues to perform calculations to update the various screen location parameters above. State HB and MB0-MB
6 performs the calculation and at some point transfers control to the state data, which outputs the rest of the data token.

[1945] These states progress in pairs, the first pair calculates the difference between the current count and its ending value, and
Generate a flag. The second pair either resets the register or adds a fixed offset (based on the value in the setup register derived from the screen size). In either case, when the count under consideration reaches its terminal value (ie, the 0 flag is set), control continues down the MB sequence of states. Otherwise all counts are considered correct (ready for the next address calculation) and the control transmits state data.

All states, including the use of additions or subtractions, take two cycles (allowing the use of the standard ripple carrier) to complete, which is between 1 and 0 for adder-based states. Flag to be replaced by,
This is achieved by using fc (first cycle).

All of the address calculation and screen location calculation states allow the data to be output assuming the preferred 2-wire interface conditions.

[1932] C. 3.5.2.1 “Calculations for standards
(MPEG-style) sequence ”The sequence of operations is as follows (0 flag is based on the output of the adder): State HB and MB0: scratch = hb-maxhb; if (z) hb = 0; else {hb = hb + 1 new state = DATA;} states MB1 and MB2: scratch = vb addr-last row in mb; if (z) vb addr = 0; else {vb addr = vb addr; width in new blocks; } States MB3 and MB4: scratch = hmb addr-last mb in row; if (z) hmb addr = 0; else {hmb addr = hmb addr + maxhb; new state = DATA } State MB5 and MB6: scratch = vmb addr-last mb row; if (! Z) vmb addr = vmb addr + blocks per mb row; (vmb addr is the picture start from the time when the picture end is inferred from the calculation. Reset after token is detected.) C. 3.5.2.2 "Calculation sequence for H.261" The H.261 calculation sequence branches from the standard sequence in state MB4: states HB and MB0: as described above states HB1 and MB2: as described states HB3 and MB4: scratch = hmb addr-last mb in row; if (z & (mod3). == 2)) / * end of slice on right of screen * / {hmb addr = 0; new state = MB5;} else if (z) / * end of row on rainbow = adh = * in blocks; new state = MB4A;} else {scratch = hmb addr-last mb in half row; new state = MB4B;} state MB4A: v b addr = vmb addr + blocks per mb row; new state = DATA; state (MB4) and MB4B: (scratch = hmb addr-last mb in half row;) if (z & (mod3 = d) == 2) slice on left of screen * / {hmb addr = hmb addr + maxhb; new state = MB4C;} else if (z) / * end of row on left; else {hmb addr = hmb addr + maxhb; new state = DATA;} State MB4C and MB4D: vmb addr = vmb addr-blocks er mb row; vmb addr = vmb addr -blocks per mb row; new state = DATA; state MB5 and MB6: as above C. 3.5.3 "Operation on Picture Start Token" Upon receipt of the picture start token, control proceeds to state PIC ST1 where the vb addr register (BU WADDR VBADDR) is reset. State PIC ST2 and PIC ST3
Each is visited once for each component, hmb a
Reset ddr and vmb addr respectively. Control then passes through state OUTPUT TAIL to IDLE
Return to. C. 3.5.4 Operation on DEFINE SAMPLING Token Upon receipt of a DEFINE SAMPLING token, the component register is loaded with the least significant 2 bits of input data. In addition to that, State HSAM
The maxhb and maxvb registers for that component are loaded via P and VSAMP. Furthermore,
Appropriately defined sampling event bits are triggered (delayed by one cycle to ensure that all tokens are written).

[1933] C. 3.5.5 "HORIZONTAL
Operation on MBS and VERTICAL MBS "HORIZONTAL MBS and VERTICAL
When each MBS arrives, the 14-bit value contained in the token is written to the appropriate register in two cycles. The related event bit is triggered and delayed by one cycle.

[1934] C. 3.5.6 "Other Tokens" Coding standard tokens are detected and the input data is written to the top level BU WADDR COD STD register. This is decoded and the nh261 flag (rather than H261) is hardwired into the buffer manager block. All other tokens are in control of the state OUTPUT TAI
Move L and the state accepts data until the token is complete. However, note that it does not actually output any data.

[1935] Section C. 4 "Read Address Generator" C.I. 4.1 "View" The read address generator of the present invention consists of four state machine / datapath blocks. First, the dline generates line addresses and distributes them to the other three (one for each component) same page / block address generator, dracctls. All blocks are 2
Connected by wire interface. The operation modes include interlaced / forward, first field upper / lower, and all combinations of upper / lower / both frame start. Tables 259 and 260 are dispa
dr control register name, address, and reset state are shown in Section C. In 13 an example of programming for both address generators is given.

[1936] C.I. 4.2 "Line Address Generator (dline)" This block calculates the line address generator for each component. Tables 259 and 260 show 1 in dline.
An 8-bit data path register is shown.

[1937] DISP register name and ADDR register name DISP
The name register is distinguished only in dispaddr, which means that the register is special for the display area read from the DRAM. ADDR
name means that the register describes something about the structure of the external buffer.

Operation [1938] The basic operation of dline, ignoring all modes such as repeat, is: if (vsync start) / * first active cycle of vsync * / {comp = 0 DISP VB CNT COMP [comp] = 0; LINE. [Comp] = BUFFER BASE [comp] +0; LINE [comp] = LINE [comp] + DISP COMP OFFSET [comp]; while (VB CNT COMP [comp] <DISP VBS COMP [comp]) {while (line count) ] <8) {{while (comp <3) {→ OUTPUT LINE [comp] to dramctl [comp] line [comp] = LINE [ comp] + ADDR HBS COMP [comp]; comp = comp + 1;} line count [comp] = line count [comp] +1;} VB CNT COMP [comp] = VB CNT COMP [comp] +1; line count [comp] == 0;}}

[1939]

[Table 259]

[1940]

[Table 260] C. 4.3 "Dline Control Register" The above operation is performed by the dispaddr shown in the table below.
It is modified by the control register.

[1941]

[Table 261] C. 4.3.1 "LINES IN LAST RO
W [component] These three registers determine for each component the number of lines in the last row of the block to be read. Thus, the height of the lead window may be any number of lines. Since the top, left and right edges of the window are on the block boundaries and the output controller can clip (discard) extra lines,
This is a backup feature.

[1942] C. 4.3.2 "DISPDDR ACCESS" This is an access bit for the entire dispaddr. If you write "1" to this location, dis
The paddr is stopped in synchronization with the clock. Once this state is reached, it is safe to perform an asynchronous upi access to all dispaddr registers. Note that upi is actively locked out of the datapath register until the access bit is '1'. To achieve access to dispaddr without disrupting the current display or datapath operations, access is granted and released only under the following circumstances:

[1943] Stopping: Data path is currently 2
Access will only be granted if the cycle operation has been completed (one operation has been performed) and if the "safe" signal from the output controller is high. This signal represents the area on the screen below the display window and is used by the output controller (dispaddr).
Is not programmed). Therefore, dis
Note that it is necessary to program the output controller before trying to gain access to paddr.

[1945] Starting Access will only be released when "safe" or high during vsync. This ensures that the display does not start too close to the active window.

[1945] This scheme is accessed by the control software, registered up to the display end, and dispaddr
Modify, and access can be requested to be released. The software is too slow, vsync
If the access bit is not released until after
paddr will not start until the next safe period. Border color will be displayed during this "lost" picture (rather than junk). C. 4.3.3 “DISPDDR CTLO [7:
0] ”When reading the description below, it is important to understand the distinction between interlaced data and interlaced display.

[1946] There may be two forms of interlace data. The top-level registers support field pictures (each buffer contains one field) and frames (each buffer contains the entire frame, whether interlaced or not).

DISPADDR CTL0 [7: 0] contain the following control bits: SYNC MODE [1: 0] In interlaced display, the vsyncs that refer to the top-to-bottom field are distinguished by the field info pin. In such a situation, fieldinfo
= HIGH means the top field. These two control bits indicate which vsyncs dispaddr
Determines whether to request a new display buffer from the buffer manager, thus synchronizing the fields in the buffer (when data is skipped) with the fields on the display: 0: new display buffer on top field 1: Bottom field 2: Both fields 3: Both fields At startup, dispaddr is any vs
Will request a buffer from the buffer manager on ync. Until the buffer is ready, dispadd
r will receive a 0 (no display) buffer. When finally receiving a good buffer index,
dispaddr has no knowledge of where it is on the display. Therefore, it may be necessary to synchronize the display startup with the correct vsync.

[1948] READ START For interlaced display at startup, this bit determines which vsync display actually starts. Further, after receiving the display buffer index, dispaddr may "leave out" the current vsync to align the fields on the display with the fields in the buffer.

INTERLACED / PROGRESSIVE 0: Progressive 1: Interlaced In progressive mode, all lines are read from the display area of the buffer. In interlaced mode, alternating lines are read. Whether reading starts from the first line or the second line f
It depends on the field info. Note that for (interlaced) field pictures, the setting of this bit will be progressive, as the system wants to read all the lines from each buffer. field
The mapping between info and the first / second line is lsb
invert (so named for historical reasons)
May be reversed by.

[1950] LSB INVERT When set, this bit inverts the field info signal seen by the line counter. In this way, readings may start on the correct line of the frame and be aligned to the display regardless of the convention adopted by the encoder, display, or top-level register.

LINE RPT [2: 0] When set, each bit causes the line of the corresponding component to be read twice (bit 0 affects component 0, etc.).
This forms the first part of the vertical unsampling. It is used for the eight color unsamplings required for the QFIF to 601 conversion.

[1952] COMPOHOLD This bit corresponds to the number of lines of components 1 and 2 and component 0.
Used to program the ratio of the number of lines read (as opposed to what is displayed).

0: The same number of lines, that is, data of 4: 4: 4 in the buffer 1: Double of 0 line of component, that is, 4: 2: 0.

Page / Block Address Generators (dracmtls) When passing line addresses, these blocks generate a series of page / line addresses and blocks to read along the line. A minimum page width of 8 blocks is always assumed and the resulting output is the page address,
3 bit line number, 3 bit block start, and 3
It consists of a bit block stop address. (The number of lines is calculated by dline.
sent through ctls. ) Thus, reading 48 pixels on line 5 forms page 0xaa starting from the third block from the left (any point along any line), and the address sent to the DRAM interface is: Would be: page = 0xaa line = 5 block start = 2 block stop = 7 Each of these three machines has 5 datapath registers. These are shown in Table 259 and Table 260. Each dr
The basic behavior of amct1 is as follows: Block Start = 2 Block Stop = 7 Each of these three machines has five datapath registers. These are shown in Table 259 and Table 260. Each dr
The basic behavior of amct1 is as follows: while (true) {CNT LEFT = 0; GET A NEW LINE ADDRESS from dline; (CNT LEFT> BLOCKS LEFT) {BLOCKS LEFT = 8−BLOCK ADDR; → output PAGE ADDR, start = BLOCK ADDR, stop = 7. PAGE ADDR = PAGE ADDR + 1; BLOCK ADDR = 0; CNT LEFT = CNT LEFT-BLOCKS LEFT:} / * Last Page of line * / CNT LEFT = CNT LEFT + BLOCK ADDR; CNT LEFT = CNT LEFT-1; → output PAGE ADDR, start = BLOCK ADDR, stop = CNT LEFT}

[1955]

[Table 262]

[1956]

[Table 263] Programming The next fifteen dispaddr registers must be programmed before initiating an operation.

BUFFER BASE0,1,2 DISP COMP OFFSET0,1,2 DISP VBS COMP0,1,2 ADDR HBS COMP0,1,2 DISP COMP0,1,2 The use of the reset state of the HBS dispaddr control register is 4: 2n. Interlaced display, line repeats are not synchronized, and the top field (field
d info = HIGH). FIG. 168, “Buffer 0 with SIF (22 × 18 macroblock) picture” shows a buffer setup for a typical SIF picture. (For this example,
C. Details will be given in Chapter 13.) In this example, DI
SP HBS COMPn is ADDR HBS COM
Pn, also a vertical register DISP VBS
Note that COMPn and Equivalent Write Address Generator registers are equal, that is, the area to be read is the entire buffer.

[1958] Window processing using read address generator Disabling only part of the buffer (window) to be read
It is possible to program the paddr. The size of the window is the register DISP HBS, DISP
VBS, COMPONENT OFFSET, and L
Program for each component by INES INLAST ROW. Figure 169, "SIF component 0 with display window," shows how this is achieved (for component 0 only).

In this example, the register settings would be: BUFFER BASE0 = 0x00 DISP COMP OFFSET0 = 0x2D DISP VBS COMP0 = 0x22 ADDR HBS COMP0 = 0x2C DISP HBS COMP0 = 0x2A. You can only start and stop. In this example, LINES IN equal to 7
Remaining LAST ROW (meaning all 8's).

[1960] This example is not practical for data other than 4: 4: 4. To be in harmony, window edges for the other two components would not be able to lie on block boundaries.

The color space conversion will stop if the data it receives is not 4: 4: 4. This means that these lead windows, along with the ensampler, must be programmed to achieve this. Section C. 5 “Data Path for Address Generation” The data paths used in dispaddr and waddrgen have the same structure and width (18 bits), but differ only in the number of registers, some masking, and the flags returned to the state machine. is there. A circuit for one slice is shown in FIG. 174, “Slice of data path”. Registers are only assigned to drive A or B buses, which are optimized in the controller,
Used (specified). All registers can be loaded from the C bus, but not all "load" signals are driven. All operations, including adders, cover the two cycles that allow adders to have normal ripple carry. FIG. 175, "Two Cycle Operation of Data Path", shows the timing for the sum of the two cycles of the two registers loaded back into the A bus register. Various flags are ph0'd in the datapath to allow ccode generation. For the same reason, the structure of the datapath schematic is a bit unusual. The tristates for all registers (on the A and B buses) allow for better ccode generation because they are in one block that removes the coupling paths within the cell. To get upi access to the datapath, the access bit must be set so that the upi would otherwise be locked out. Upi access is different from read and write: • Writing: When the access bit is set, all load signals are disabled and one of a series of 3 byte addressed write strobes is moved to one appropriate byte of the register. . The Upi data bus travels vertically through the data path (copied, 2-8-8 bits) and the 18-bit register is written as three separate byte writes.

Reading: This is done using the A and B buses. Again, the access bit must be set. The address register is moved on the A or B bus and the upi byte select picks a byte from the associated bus and moves it onto the upi bus.

[1963] Double cycle datapath operations require the A and B buses to hold their values,
Before the start of any datapath operation, the access is
It must be given only by controlling the state machine.

All data path registers in both address generators pass through a 9-bit wide keyhole of top level address 0x28 (msb) and 0 for the keyhole.
x29 (lsb) and is addressed through 0x2A for data. The keyhole addresses are shown in Tables 282-300.

[1965] Note: 1) Address generators (dispaddr and waddrg)
All address registers in en) contain blocked addresses. Pixel addresses are never used,
The only register that contains the line address is 3 L
INES INLAST ROW register.

2) Some registers are duplicated between address generators, eg BUFFER BASE0 is d
Occurs in the address space for ispaddr and waddrgen. These are two separate registers, both of which require loading. This allows display window processing (reading only part of the display storage) and facilitates display of formats other than three component video.

[1967] Section C. 6 “DRAM interface” C.I. 6.1 "Perspective" In the present invention, the spatial decoder, the temporal decoder, and the video formatting unit each include a data token block for that particular chip. In all three devices, the function of the DRAM interface is to transfer data from the chip to the external DRAM and from the external DRAM to the chip via the block address provided by the address generator.

The DRAM interface typically operates from a clock that is asynchronous to the address generator and to the clocks of the various blocks through which data is sent. However, this asynchrony is easily handled because the clocks operate at about the same frequency.

Data is normally transferred between the DRAM interface and the rest of the chips in a block of 64 bytes (the only exception being the predictive data in the temporal decoder). The transmission is done by a device known as a "swing buffer". This is essentially a pair of RAMs operated in a double-buffered arrangement, whether the DRAM interface packs or empties one RAM while another part of the chip empties the other RAM. I'm packing. Another bus carrying the address from the address generator is associated with each swing buffer.

[1970] Each chip has four swing buffers, but the functions of these swing buffers differ in each case. In the spatial decoder, one swing buffer is used to transfer the coded data to the DRAM, another swing buffer is used to read the coded data from the DRAM, and the tokenized data is D
The third swing buffer for transmitting to RAM,
A fourth swing buffer is also used to read the tokenized data from DRAM. In the temporal decoder, the intra or predicted picture data is D
One swing buffer is used to write to RAM, a second swing buffer is used to read intra or predicted picture data from DRAM, and the other two are used to read forward and backward prediction data. used. In the video formatting section, one swing buffer is used to transfer data to the DRAM and the other three are used to read data from the DRAM, which are luminance (Y) and red and blue respectively. Color difference data (Cr and Cb respectively)
Is.

The operation of the general features of the DRAM interface is described in the Spatial Decoder Document. The next section is the DR according to the invention.
The characteristics of the AM interface, especially the characteristics unique to the video formatting section will be described.

[1972] C. 6.2 "Video Formatting Unit DRAM Interface" In the video formatting unit, data is written in blocks to the external DRAM, but is read in raster order. The lighting is exactly the same as that already described for the spatial decoder, but the reading is a bit more complicated.

[1973] Video Formatting Unit The data in the external RAM is organized so that at least 8 blocks of data fit on one page. These 8 blocks are 8 consecutive horizontal blocks. 8 when rasterizing
It is necessary to read 8 bytes from each of the contiguous blocks and write it to the swing buffer (ie the same row for each of the 8 blocks).

[1974] Considering the top flow (and assuming a byte wide interface), the x address (3
LSB) is set to 0, and so is the y address (3MSB). The x-address is then incremented as each of the first 8 bytes is read. At this point, the upper part of the address (bit 6 and above-LSB = bit 0) is incremented and the x-address (3 LSB) is reset to zero. This process is repeated until 64 bytes have been read. For a 16 or 32 bit wide interface to external DRAM, the x address is simply incremented by 2 or 4 instead of 1.

The address generator can signal the DRAM interface that multiples of 8 bytes are always read, but no more than 64 bytes should be read (this is required at the beginning or end of the raster line). it can. This is done using start and stop values. The start value is used for the upper part (6 bits or more) of the address, the stop value is compared with this, and a signal is issued indicating that the reading should be stopped.

[1976] Section C. 7. “Vertical Upsampling” C.I. 7.1 "Introduction" Given a raster scan of pixels of one color component at its input, a vertical ensampler according to the invention can provide an output scan of twice the height. With mode selection, the output pixel value can be formed in many ways.

[1977] C.I. 7.2 "Port" Input 2-wire interface: -in valid-in accept-in data [7: 0] -in lastpel-in lastline Output 2-wire interface: -out valid-out accept-out data [9: 0] -out last mode [2: 0] nupdata [7: 0], upaddr, upsel
[3: 0], upstrstr, upwstr ramtest tdin, tdout, tph0, tckm, tcks ph0, ph1, notrst0 C.I. 7.3 “Mode” Selectable according to the input bus mode [2: 0].

[1978] Mode register values 1 and 7 are not used.

In each of the above modes, the output pixel is displayed as a 10-bit value, not as a byte. No rounding or censoring is done in this block. If necessary, the value is shifted left to use the same range.

[1980] C. 7.3.1 The "Mode 0: Fifo" block simply acts as a FIFO storage device. The number of output pixels is exactly the same as the number of input pixels. The value is shifted left by 2.

[1981] C.I. 7.3.2 "Mode 2: Repeat" Every line in the input scan is repeated to make an output scan of twice the height. Again, the pixel value is shifted left by 2.

[1982] A → ABACBDB CCDD C.I. 7.3.3 “Mode 4: Lower” Each input line produces two output lines. In this "lower" mode, the second of these two lines (lower on the display) is the same as the input line. The first of the pair is the average of the current and previous input lines. The input line is repeated for the first input line if the previous line was not used.

[1983] This should select where the chroma samples are located with the sub-luminance samples.

[1984] A → ABAC (A + B) / 2DB (B +
C) / 2C (C + D) / 2D C.I. 7.3.4 "Mode 5: Upper" Similar to "Lower" mode, but in this case the input lines form the upper part of the output pair and the lower part is the average of the adjacent input lines. The last output line is a repeat of the last input line.

[1985] This should select where the chroma samples are placed with the upper luma samples.

[1986] A → AB (A + B) / 2CBD (B + C)
/ 2C (C + D) / 2DD C.I. 7.3.5 "Mode 6: Central" This "center" mode corresponds to the situation where the chroma sample is in the middle of the luma sample. A weighted average is used to form the output line for placing the output chroma pixels with the luma pixels.

[1987] A → AB (3A + B) / 4C (A + 3
B) / 4D (3B + C) / 4 (B + 3C) / 4
(3C + D) / 4 (C + 3D) / 4D C.I. 7.4 "Method of operation" There are two line stores that are imaginarily designated by a and b.
In FIFO and repeat mode, line store a
Only used. Each store can accommodate lines up to 512 pixels (vertical upsampling should be done before horizontal upsampling). FIFO
The mode has no line length restrictions.

[1988] in lastpel and in last
The input signal in the line is used to indicate the end of the input line and the end of the picture. in las
At tpel, the input signal should strongly match the last pixel in each line. in lastline
In, the input signal should strongly coincide with the last pixel of the last line of the picture.

The output signal out last strongly matches the last pixel of each output line. In repeat mode, each line is written to store a. The line is then read twice. When reading the second time, writing of the next line may be started.

[1990] lower, upper, and centra
In l mode, lines are written alternately to stores a and b. The first line of the picture is always written to store a. Two small state machines, one for each store, store what is in each store and which output lines are formed. From these states, read and write requests, and signals that determine when to overwrite the next line with the current data are issued to the line store RAM.

[1991] When in lastpel is high, a register (lastaddr) stores the write address, thereby providing the line length for the formation of the output line. C. 7.5 "UPI" This block contains two 512x8 bit RAM arrays, which can be accessed via the microprocessor interface in a typical manner. No registers have microprocessor access.

[1992] Section C. 8 “Horizontal Up Sampler” C.I. 8.1 "View" In the present invention, the top level register comprises three identical horizontal up-samplers, one for each color component. All three are controlled separately, so only one will be described here. From the user's perspective, the only difference is that each horizontal upsampler is mapped to a different set of addresses in the memory map.

[1993] The horizontal up sampler performs an operation that combines duplication and filtering. In all, there are four modes of operation.

[1994]

[Table 264] C. 8.2 "Use of Horizontal Up-Sampler" The address map for each horizontal up-sampler consists of 25 locations corresponding to 12 13-bit coefficient registers and 1 2-bit mode register. The number written to the mode register determines the operation mode, as outlined in Table 264. Depending on the mode, some or all coefficient registers can be used. The equivalent FIR filter will be described below.

[1995] Input, X, depending on the operation mode
n is held constant for 1, 2, or 4 clock periods. The actual coefficients programmed for each mode are:

[1996]

[Table 265]

[1997]

[Table 266]

[1998]

[Table 267] Coefficients not used in a particular mode do not need to be programmed when operating in that mode.

The 1999 first and last pixels of each line are repeated before filtering to provide balanced filtering. For example, when upsampling by 2, the first and last pixels of each line are 2
Not four times, but four times. Since the residual data in the filter is discarded at the end of each line, the pixel number output is always exactly one, two or four times the number in the input stream.

[2000] Depending on the coefficient values, the output samples are placed either coincident with the input samples or shifted from the input samples. Some examples of coefficient values in a certain sample mode are described below. "-" Indicates that the coefficient value is "don't worry." All values are shown in hexadecimal.

[2001]

[Table 268] C. 8.3 “Description of Horizontal Up Sampler” The data path of the horizontal up sampler is shown in FIG. 177.

The operation takes the case of x4 up sample and is outlined below. In addition to that, × 2
Up-sampling and x1 filtering (mode 2
And 1) are degenerate cases of this, bypassing the entire filter (mode 0) and the data goes directly from the input latch to the output latch via the final mux, as shown.

1) When valid data is latched at the input Latch (L), it is held for 4 clock periods.

2) The coefficient registers (labeled C / OE / FF) each have four (PI)
At the same time that two sets of pipeline registers (labeled PE) are clocked, they are multiplexed onto the multiplier for one clock period. Thus, for the input data Xn, the value c00. X
n, c01. Xn, c02. Xn, c03. Xn will be packed.

3) Similarly, the second multiplier will in turn multiply its coefficients by Xn, and the third multiplier will in turn multiply all its coefficients by Xn.

It will be appreciated that the output will be in the form shown in Table 269.

[2007]

[Table 269] In terms of output, each clock period produces an individual pixel. Since each output pixel depends on the weighted value of 12 input pixels (though there are only 3 different values), this is considered as implementing a 12-tap filter on the x4 up-sampled input pixels. .

For x2 upsampling, the operation is essentially the same, except that the input data is held for only two clock periods. Furthermore, 2
Only one coefficient is used and the PIPE block is shortened by the multiplier shown. For x1 filtering,
The input is held only for one clock period. As expected, one coefficient and one PIPE stage is used.

Next, the peculiarity of the mounting in the present invention will be explained a little.

1) Datapath width and coefficient width (13 bit 2's complement) as designed for the color space converter.
, And the same multiplier can be used. These widths are no more suitable for horizontal up samplers.

2) The multiplexer that multiplexes the coefficients onto the multiplier is shared with the UPI readback. This leads to some complexity in the schematic structure (basically due to the difficulty in CCODE generation), but the actual circuit is smaller.

3) As in the color space converter, a carry-save multiplier is used, the result of which is only decomposed at the end.

The control for the entire horizontal up-sampler can be viewed as a single two-wire interface stage that, when on its input, can produce twice or four times the amount of data at the input. The mode programmed via UPI determines the length of the programmable shift register (bob). The mode selected produces an output pulse every clock period, every two clock periods, or every four clock periods. This in turn controls the main state machine, but that state is further (2
In line, out (for wireline interface)
acceptance, and the signal in last. This signal comes from the vertical up sampler,
High for the last pixel in each line. This makes the first and last pixels of each line duplicated,
Clear the pipeline between lines (immediately after the line is complete, the pipeline contains partially processed redundant data).

[2014] Section C. 9 “Color Space Converter” C.I. 9.1 “Perspective” The color space converter (CSC) in the present invention multiplies incoming 9-bit data by a 3 × 3 matrix and then adds:

[2015]

[Equation 6] When x0-2 is input data, y0-2 is output data, and cnm is a coefficient. In the schematic, names correspond to signal names, so some unconventional naming should be used with caution with respect to matrix coefficients.

Although the CSC can perform conversions between many different color spaces, a limited set of these conversions are used in the top level registers. Design color space conversion is as follows: ER, EG, EB → Y, CR, CB R, G, B → Y, CR, CB Y, CR, CB → ER, EG, EB Y, CR, CB → R, G, B where R, G and B are in the range (0.5.511),
All other quantities are in the range (32 ... 470). Inputs to the top level register CSC are Y, CR, CB.
Therefore, only the third and fourth of these equations are relevant.

[2017] In the CSC design, the coefficient precision is +1 of the value that all outputs are created by full floating point simulation of the algorithm for 9-bit data.
Or chosen to be within -1 bit (this is the best accuracy that can be achieved). This gave a 13-bit two's complement coefficient for cx0-cx3 and a 14-bit two's complement coefficient for cx4. All design conversion coefficients are given below in both decimal and hexadecimal.

[2018]

[Table 270]

[2019]

[Table 271] All these numbers have the following basic formula: Y = 0.299ER + 0.587EG + 0.0114EB and are calculated from the following color difference formulas: CR = ER-Y CB = EB-Y The formulas in R, G and B are these These are drawn after considering the full range of quantities.

[2020] C. 9.2 “Use of color space converter” After reset, c01, c12, and c23 are set to 1,
All other coefficients are set to zero. Thus, y0 = x
0, y1 = x1, y2 = x2, and all data is sent unchanged. To select a color space conversion, simply write the appropriate coefficient (eg, from Table 270, Table 271) to the location specified in the address map.

In the schematic diagram, x0. ． 2 is in da
ta0. ． 2 corresponds to y0. ． 2 is out data
0. ． Corresponds to 2. The user should remember that the input data to the CSC must be upsampled 4: 4: 4. Failure to do this will not only make the color space conversion make no sense, but will also lock the chip.

It should be noted that each output can be formed from a possible combination of coefficients and input plus (or minus) constants. Thus, for a given color space conversion, the output order can be modified by swapping the rows in the conversion matrix (address where the coefficients are written).

For all the conversions in Table 270 and Table 271, CSC is guaranteed to work. With other transformations, the user must remember the following: 1) If the intermediate result of the calculation requires more than 10 bits of precision (excluding sign bits), the hardware will not work. .

2) The CSC output is saturated to 0 and 511. In other words, numbers less than or equal to 0 are replaced by 0,
Numbers 511 and above are replaced with 511. Saturation logic execution assumes that the result is just above 511 or just below 0. It would be a common sign that the output would always (or most often) appear to saturate if the CSC was programmed incorrectly.

[2025] C. 9.3 “Explanation of CSC” The structure of CSC is illustrated in FIG. 178, but due to space limitations, only two of the three “components” are shown. In the figure, "register" or R indicates a master / slave register, and "latch" or L indicates a transparent latch.

[2026] All coefficients are loaded into the read / write UPI registers not explicitly shown in the figure. To understand the operation, the leftmost "component"
For (producing output out data0) consider the following sequence: 1) Data arrives at input x0-2 (in data0-2). It represents a pixel in the input color space. This is latched.

2) Multiply x0 by c01 and latch it in the first pipeline register. x1 and x2 move over one register.

3) Multiply x1 by c02 and multiply it by (x1.
c01) and latched in the next pipeline register. x2 moves over one register.

4) x2 is multiplied by c03, which is added to the result of (3) to obtain (x1.c01 + x2.c02 + x3.
c03) is produced. The result is latched in the next pipeline register.

5) The result of (4) is added to c04. This adder is also used to decompose the data from the multiplier chain because the data is held in carry-save format through the multiplier. The result is latched in the next pipeline register.

6) The final operation is to saturate the data. To do this, the partial result is sent from the decomposition adder to the saturating block.

It can be seen that the result is y0, as specified in the matrix equation at the beginning of this section. Similarly, y1 and y2 are formed in the same way.

[2033] The coefficient is the multiplicand and the data is the multiplier.
Three multipliers were used. This allowed an efficient layout to be achieved, with partial results going down the datapath and the same input data being sent across three parallel parallel datapaths, one at each output. Section C. In order to achieve the reset state described in 9.2, each of the three "components" must be reset differently. To avoid having three sets of schematics and three slightly different layouts, this is accomplished by having inputs to the UPI registers tied high or low at the top level.

The CSC has little control associated with it. Nevertheless, since each pipeline stage is a two-wire interface stage, their associated control (in accept = out accept
There is a chain of valid latches and accept latches with tr + lin validr). Thus, the CSC is a five stage deep, two wire interface that can hold 10 levels of data when stalled.

[2035] The output of the CSC includes a resynchronization latch, as the next function in the output pipe deviates from the different clock generator.

[2036] Section C. 10 “Output controller” C.I. 10.1 "Introduction" The output controller according to the present invention performs the following functions: -Provides data in one of three modes-24 bits 4: 4: 4 16 bits 4: 2: 2 8 bits 4: 2: 2 • Align data to video display window defined by vsync and hsync pulses and by programmed timing registers. • Add border around video window, if desired. 10.2 “Port” Input 2-wire interface: ・ in valid ・ in accept ・ in data [23: 0] Output 2-wire interface: ・ out valid ・ out accept ・ out data [23: 0] ・ out active ・out window-out comp [1: 0] in vsync, in hsync nupdata [7: 0], upaddr [4: 0],
upsel, rstr, wstr, tdin, tdou
t, tph0, tckm, tcks chipses
t, ph0, ph1, notrst0, notrst1 C.I. 10.3 "Out Mode" The output format is selected by writing to the opmode register.

[2037] C. 10.3.1 “Mode 0” This mode is 24-bit 4: 4: 4 RGB or Y.
It is CrCb. Input data goes directly to output.

[2038] C. 10.3.2 "Modes 1 and 2" These modes represent 4: 2: 2 YCrCb. In
Assuming data [23:16] is Y i
n data [15: 8] is Cr, and in data
[7: 0] is Cb.

[2039] C. 10.3.2.1 "Mode 1" In 16-bit YCrCb, Y is out dat
a [15: 8]. Cr and Cb are out
Time division multiplexing is performed on data [7: 0], with Cb first. Out data [23:16] is not used.

[2040] C. 10.3.2.2 “Mode 2” In 8-bit YCrCb, Y, Cr and Cb are C
b, Y, Cr, Y order, out data
Time division multiplexing is performed on [7: 0]. Out data
[23:16] is not used.

[2041] C. 10.3.3 "Output Timing" To put data in the video display window,
The following registers are used.

Vdelay-hsyn following the vsync pulse before the first line of video or border.
c number of pulses-number of clock cycles between hdelay-hsync and the first pixel of the video or border-height of the video window at the height-line-width-width of the video window at the pixel-north, south-line The height of each border above and below the video window at, respectively: the border width for the left and right of the video window at the waist, east-pixels. The minimum vdelay is 0. The first hsync is the first active line. The minimum value that can be programmed into hdelay is 2. However, inhsyn
Note that the actual delay from c to the first active output pixel is hdelay + 1 cycles.

[2043] The border edge may have a value of 0. The color of the border is registered in the register as border r, b
It is selected by writing order g and border b. The color of the area outside the border is displayed in the registers blank r, blank g, blank.
Selected by writing b. Output modes 1 and 2
The multiplexing performed in will also affect the border and blank components. That is, the value of these registers is in data [2
3:16], in data [15: 8], and in
This corresponds to data [7: 0].

[2044] C. 10.4 "Output Flag" -out active indicates that the output data is part of the active window, ie the video data or border.

[2045] -out window indicates that the output data is part of the video window.

Out comp [1: 0] indicates that the color component is present on out data [7: 0] of output modes 1 and 2. In mode 1, 0 =
Cb, 1 = Cr. In mode 2, 0 = Y,
1 = Cr and 2 = Cb. C. 10.5 "2-Wire Mode" The 2-wire mode of the present invention is selected by writing a 1 to the 2-wire register. It is not selected following a reset. In the 2-wire mode, the output timing register and sync signal are ignored and the data flow through the block is controlled by out accept.
Note that in normal operation, out accept should be tied to both.

[2047] C. 10.6 Super Snowpa is on the output of the block that contains access to the "Snowpa" output flag. C. 10.7 "How it works" Two identical down counters remember their current position on the display. Vcount decreases on hsyncs and loads from the appropriate timing register on vsync or at its terminal count. Hc
outt decreases on every pixel and loads on hsync or at its terminal count. Note that in output mode 2, one pixel corresponds to two clock cycles.

[2048] Section C. 11 "Clock divider" C.I. 11.1 “Perspective” One of the top-level registers in the present invention is PIC
One is AUDIO to generate TURE CLK
It includes two identical clock dividers to generate CLK. The clock divider is the same and is controlled separately. Therefore, only one will be discussed here.
From the user's perspective, the only difference is that the divisor register of each clock divider is mapped to a different set of addresses in the memory map.

[2049] The function of the clock divider is 4Xsysclk.
It provides a distributed clock frequency and there is no requirement for an even mark to space ratio.

Since the divisor is required to be in the range of 0 to 16,000,000, it can be expressed using 24 bits with the constraint that the minimum divisor is 16. This is because the clock divider will use the divisor / 2 to approximate equal mark to space ratios (within one sysclk cycle). The maximum clock frequency that can be used is sysclk, so the maximum distribution frequency that can be used is sysclk / 2. Furthermore, since 4 counters are used in cascade, the divisor / 2 is never 8
Must not be less than, otherwise the clock output distributed will be driven to the positive power rail.

[2051] C. 11.2 "Use of Clock Divider" The address map for each clock divider consists of four locations corresponding to three 8-bit divisor registers and one 1-bit access register. The clock divider powers up the inactive one and is activated upon completion of access to its divisor register.

The divisor register may be written in the order according to the address map in Table 272. The clock divider is activated by sensing the synchronization 0-1 transition on its access bits. When the transition is first sensed, the clock divider exits the reset state and generates the distributed clock. Subsequent transitions (assuming the divisor is also changed) will simply cause the clock divider to lock to its new frequency on-the-fly. Once activated, the only way to stop the clock divider is with the chip RESET.

[2053]

[Table 272] Divisor values in the range 16-16, 777, 216 can be used.

[2054] C. 11.3 "Explanation of clock divider" The clock divider is
It is implemented as four 22-bit counters cascaded to activate the next counter in turn. Since the counters count down the divisor / 4 value before the carry, each counter will take it in turn and generate a pulse at the distributed clock frequency.

After the carry, the counter reloads the divisor / 8, which is counted down to produce approximately equal mark to space ratio distribution clocks. As each counter is activated by the previous counter, as it reloads from the divisor register, this allows the distributed clock frequency to be changed on the fly by simply changing the contents of the divisor. To

Each counter is clocked by its own independent clock generator to precisely control the clock skew between the counters and to allow each counter to be clocked by a separate clock set.

[2057] State machine is divisor / 4 and divisor / 8
It controls the value generator and also multiplexes the correct source clock from the PLL to the clock generator. The counter is clocked by different clocks depending on the value of the divisor. This is because different divisor values will produce distributed clocks whose edges are placed using different combinations of clocks provided by the PLL.

[2058] C. 11.4 Clock Divider Testing The clock divider can be tested by powering up the chip with CHIPTEST High. This has the effect of forcing all of the clocking logic in the clock divider clocked by sysclk as opposed to the clock generated by the PLL.

Since the clock divider is designed to have a full scan, it can be subsequently tested using standard JTAG access as long as the chip is run as described above.

[2060] If CHIPTEST is held high while the device is operating in normal operation,
The functionality of the clock divider is not guaranteed.

[2061] Section C. 12 “Address Map” C.I. 12.1 "Top Level Address Map" Note: -1) The registers for the top level address maps described in Tables 273 to 281 are names used in the design. They do not necessarily have to appear in the data sheet.

2) Since this is a full address map, many of the locations listed here include locations for testing only.

[2063]

[Table 273]

[2064]

[Table 274]

[2065]

[Table 275]

[2066]

[Table 276]

[2067]

[Table 277]

[2068]

[Table 278]

[2069]

[Table 279]

[2070]

[Table 280]

[2071]

[Table 281] C. 12.1 “Address Generator Keyhole Space” Note on Address Generator Keyhole Table: 1) All registers in the address generator keyhole are
It takes 4 bytes of address space regardless of their width. Omitted addresses (0x00, 0x04, etc.) are always 0
Read back.

2) The access bit of the associated block (dispaddr or waddrgen) must be set before accessing this keyhole.

[2073]

[Table 282]

[2074]

[Table 283]

[2075]

[Table 284]

[2076]

[Table 285]

[2077]

[Table 286]

[2078]

[Table 287]

[2079]

[Table 288]

[2080]

[Table 289]

[2081]

[Table 290]

[2082]

[Table 291]

[2083]

[Table 292]

[2084]

[Table 293]

[2085]

[Table 294]

[2086]

[Table 295]

[2087]

[Table 296]

[2088]

[Table 297]

[2089]

[Table 298]

[2090]

[Table 299]

[2091]

[Table 300]

[2092]

[Table 301]

[2093]

[Table 302]

[2094]

[Table 303]

[2095]

[Table 304]

[2096]

[Table 305]

[2097]

[Table 306]

[2098]

[Table 307] Section C. 13 “Picture Size Parameter” C.I. 13.1 "Introduction" The following stylized code snippet shows the processing required to answer the picture size interrupt from the write address generator. The picture size parameter is HO
RIZONTAL MBS token, VERTICAL
MBS token, and (for each component)
Note that it can be changed on-the-fly by sending a DEFINE SAMPLING token, resulting in a write address generator interrupt. These tokens may arrive in any order, and generally they should all require a recalculation of the picture size parameter. However, at setup, it would be more efficient to detect the arrival of all events before performing the calculations.

[2099] Since a special value can be written in the picture size parameter register at the time of setup, it is not necessary to rely on interrupt processing for answering a token. Therefore, an appropriate register value for SIF picture is also given.

[2100] C. 13.2 Interrupt handling for picture size parameters There are 5 picture size events, the main one of each response is listed below: In addition, the following calculations are required to maintain a consistent picture size parameter: if (hmbs event || vmbs event || def samp1 event || def samp2 event) {for (I = 0; i <max component; i ++) {hbs [i] = addr hbs [i] = (maxhb [i] +1) mbs wide; half width in blocks [i] = ((maxhb [i] +1) last mb in row [i] = hbs [i]-(maxhb [i] +1); last mb in half row [i] = half width in blocks [i]-(maxhb [i] +1 ); Las rows in mb [i] = hbs [i] maxvb [i]; blocks per mb row [i] = last row in mb [i] + hbs [i]; last mb row [i] = blocks per mb row ] (Mbs high-1); It is strictly necessary to modify the dispaddr register value (such as the display window size) in response to the picture size parameter, which is preferably dependent on the application requirements. .

[2101] C. 13.3 “Register Value for SIF Pictures” After the above interrupt processing for SIF, the values included in all picture size registers and the 4: 2: 0 stream are as follows: 13.3.1 “Primary Value” BU WADDR MBS WIDE = 0x16 BU WADDR MBS HIGH = 0x12 BU WADDR COMP0 MAXHB = 0x01 BU WADDR COMPB0BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * BAD * B * W * A * B * B * A * B * W * A * B * B * W * A * B * B * W * A * B * B * A. 0x00 BU WADDR COMP2 MAXVB = 0x00 C.I. 13.3.2 “Secondary Value-Post-Calculation” BU WADDR COMP0 HBS = 0x2C BU WADDR COMP1 HBS = 0x16 BU WADDR COMP2 HBS = 0x16 BU ADDR COMP0 HBS = 0x2C BU ADDR COMPxHDR = 0 COMP0 HBS = 0 WADDR COMP0 HALF WIDTH
IN BLOCKS = 0x16 BU WADDR COMP1 HALF WIDTH
IN BLOCKS = 0x03 BU WADDR COMP2 HALF WIDTH
IN BLOCKS = 0x03 BU WADDR COMP0 LAST MB IN
ROW = 0x2A BU WADDR COMP1 LAST MB IN
ROW = 0x15 BU WADDR COMP2 LAST MB IN
ROW = 0x15 BU WADDR COMP0 LAST MB IN
HALF ROW = 0x14 BU WADDR COMP1 LAST MB IN
HALF ROW = 0x0A BU WADDR COMP2 LAST MB IN
HALF ROW = 0x0A BU WADDR COMP0 LAST ROW I
N MB = 0x2C BU WADDR COMP1 LAST ROW I
N MB = 0x0 BU WADDR COMP2 LAST ROW I
N MB = 0x0 BU WADDR COMP0 BLOCKS PER
MB ROW = 0x58 BU WADDR COMP1 BLOCKS PER
MB ROW = 0x16 BU WADDR COMP2 BLOCKS PER
MB ROW = 0x16 BU WADDR COMP0 LAST MB RO
W = 0x5D8 BU WADDR COMP1 LAST MB RO
W = 0x176 BU WADDR COMP2 LAST MB RO
W = 0x176 Note that if these values are to be written explicitly at setup time, the multi-byte nature of most locations must be taken into account.

[2102] The pipeline system of the present invention described above is a universal adaptation in the form of multiple processing stages and interactive interfacing tokens for control and / or data functions in the processing stages that have long been desired. It is an improved system with devices that allows the processing stage to be more flexible in configuration and processing.

The improved system comprises a multi-standard video decompressor with multiple stages interconnected by a two-wire interface arranged as a pipeline processing machine. Control tokens and data tokens pass through a single two-wire interface to carry both control and data in token format. The token decode circuit is placed at a particular stage to recognize the particular token as the control token for that stage and send the unrecognized control token along the pipeline. A relocation processing circuit is placed in the selected stage and is responsive to the recognized control token to relocate the identified data token to process the identified data token. To implement the system,
A wide range of unique sub-support system circuits and processing techniques are disclosed.

From the foregoing, although particular forms of the invention have been illustrated and described, it will be obvious that various modifications can be made without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the embodiments shown in the drawings except for the limitation of the appended claims.

[2105]

As described above, according to the present invention,
A pipe with multiple processing stages and a universal adaptation unit in the form of an interactive interface control token that performs control and data functions within the processing stages, the processing stages being provided with increased flexibility in configuration and processing. A line system is provided.

[Brief description of drawings]

FIG. 1 shows 6 cycles of a 6-stage pipeline for different combinations of 2 internal control signals.

FIG. 2 is a diagram illustrating a pipeline in which each stage includes a secondary data store to illustrate how the pipeline stage can compress and decompress in response to delays in the pipeline.

FIG. 3 is a diagram showing a pipeline in which each stage includes a secondary data store to show how the pipeline stage can compress and decompress in response to delays in the pipeline.

FIG. 4 is a diagram showing control of data transfer between stages in a preferred embodiment of a pipeline using a two wire interface and a polyphase clock.

FIG. 5 is a diagram showing control of data transfer between stages in a preferred embodiment of a pipeline using a two-wire interface and a polyphase clock.

FIG. 6 is a diagram showing control of data transfer between stages in a preferred embodiment of a pipeline using a two-wire interface and a polyphase clock.

FIG. 7 illustrates control of data transfer between stages in a preferred embodiment of a pipeline using a two wire interface and a polyphase clock.

FIG. 8 is a block diagram showing a basic embodiment of a pipeline stage incorporating a 2-wire interface and two consecutive pipeline processing stages with 2-wire transfer control.

9 is an example of a timing diagram showing a relationship among timing signals used in the pipeline stage shown in FIG. 8, input / output data, and internal control signals.

10 is an example of a timing diagram showing a relationship among timing signals used in the pipeline stage shown in FIG. 8, input / output data, and internal control signals.

FIG. 11 is a block diagram illustrating an example of a pipeline stage that maintains its state under the control of extension bits.

FIG. 12 is a block diagram of a pipeline stage for decoding stage activation data words.

FIG. 13 is a block diagram illustrating the use of two wire transfer control in an example data duplex pipeline stage.

FIG. 14 is a block diagram illustrating the use of two wire transfer control in an example data duplex pipeline stage.

FIG. 15 is an example of a timing diagram showing a two-phase clock, a two-wire transfer control signal, and other internal data and control signals used in the embodiments illustrated in FIGS. 13 and 14;

16 is an example of a timing diagram showing a two-phase clock, a two-wire transfer control signal, and other internal data and control signals used in the embodiments illustrated in FIGS. 13 and 14;

FIG. 17 is a block diagram of a reconfigurable processing stage.

FIG. 18 is a block diagram of a spatial decoder.

FIG. 19 is a block diagram of a time decoder.

FIG. 20 is a block diagram of a video formatter.

FIG. 21 is a memory map showing a first configuration of a macro block.

FIG. 22 is a memory map showing a second configuration of a macro block.

FIG. 23 is a memory map showing another configuration of a macro block.

FIG. 24 is a Venn diagram of possible table selection values.

FIG. 25 is a diagram showing a variable length of image data used in the present invention.

FIG. 26 is a block diagram of a temporal decoder that includes a prediction filter.

FIG. 27 is a diagram showing a picture of prediction filter processing.

FIG. 28 is a general diagram showing the structure of a macroblock.

FIG. 29 is a general block diagram of a start code detector.

FIG. 30 is a diagram exemplifying a start code in a data stream.

FIG. 31 is a block diagram showing the relationship among a flag generator, a decoding index, a header generator, an additional word generator, and an output latch.

FIG. 32 is a block diagram of a spatial decoder DRAM interface.

FIG. 33 is a block diagram of a write swing buffer.

FIG. 34 is an illustration showing a predicted data offset from the block being processed.

FIG. 35 is a diagram showing (1, 1) predicted data offsets.

FIG. 36 is a block diagram showing a split state machine of a Huffman decoder and a spatial decoder.

FIG. 37 is a block diagram of a prediction filter.

FIG. 38 shows an exemplary decoder system.

FIG. 39 is a diagram showing a JPEG still image decoder.

FIG. 40 is a diagram showing a JPEG video decoder.

FIG. 41 is a diagram showing a multi-standard video decoder.

FIG. 42 is a diagram showing the start and end of a token.

FIG. 43 is a diagram showing a token address and a data field.

FIG. 44 shows a token on an interface wider than 8 bits.

FIG. 45 is a diagram showing a macro block.

FIG. 46 is a diagram showing a two-wire interface protocol.

FIG. 47 is a view showing a position of an external 2-wiring interface.

FIG. 48 is a diagram showing clock propagation.

FIG. 49 is a diagram showing a timing of a two-wiring interface.

FIG. 50 illustrates an access structure.

FIG. 51 is a diagram showing a read transfer cycle.

FIG. 52 is a diagram showing access start timing.

FIG. 53 is a diagram showing an access example of two write transfers.

FIG. 54 is a diagram showing a read transfer cycle.

FIG. 55 is a diagram showing a write transfer cycle.

FIG. 56 is a view showing a refresh cycle.

FIG. 57 is a diagram showing an example of a DRAM (9-bit address) having a 32-bit data bus and a depth of 256 kbits.

FIG. 58 is a diagram showing timing parameters for strobe signals.

FIG. 59 is a diagram showing timing parameters between any two strobe signals.

FIG. 60 is a diagram showing timing parameters between a bus and a strobe signal.

FIG. 61 is a diagram showing timing parameters between a bus and a strobe signal.

FIG. 62 is a diagram showing MPI read timing.

FIG. 63 is a diagram showing MPI write timing.

FIG. 64 is a diagram showing the arrangement of large integers in the memory map.

FIG. 65 shows a typical decoder clock configuration.

FIG. 66 is a diagram showing input clock requirements.

FIG. 67 is a diagram showing a spatial decoder.

FIG. 68 is a diagram showing input / output of an input circuit.

FIG. 69 is a diagram showing an encoding port protocol.

FIG. 70 shows a start code detector.

FIG. 71 is a diagram showing detected start codes and converted tokens.

FIG. 72 shows the start code detector passing a token.

FIG. 73 is a diagram showing the overlap of MPEG start codes (byte alignment).

FIG. 74 is a diagram showing overlap of MPEG start codes (non-byte aligned).

FIG. 75 is a diagram showing a jump between two video sequences.

FIG. 76 is a diagram showing a process of inserting an additional token.

FIG. 77 is a diagram showing decoder activation control.

FIG. 78 shows an enabled stream queued before output.

FIG. 79 is a diagram showing a spatial decoder buffer.

FIG. 80 is a diagram showing a buffer pointer.

FIG. 81 is a diagram showing a video demultiplexer.

FIG. 82 is a diagram showing an image structure.

FIG. 83 is a diagram showing the structure of a 4: 2: 2 macroblock.

FIG. 84 is a diagram showing a method of calculating from the size of a macroblock size pixel.

FIG. 85 is a diagram showing a spatial decoder.

FIG. 86 is a diagram showing an overview of H.261 inverse quantization.

FIG. 87 is a diagram showing an overview of JPEG inverse quantization.

FIG. 88 is a diagram showing an overview of MPEG dequantization.

FIG. 89 is a diagram showing a quantization table memory map.

FIG. 90 is a diagram showing an overview of a JPEG baseline sequence structure.

FIG. 91 shows a tokenized JPEG image.

FIG. 92 shows a time decoder.

FIG. 93 is a diagram showing image buffer specifications.

FIG. 94 is a diagram showing an MPEG image sequence (m = 3).

FIG. 95 is a diagram showing how an “I” image is stored and output.

FIG. 96 is a diagram showing how a “P” image is stored and output.

FIG. 97 is a diagram showing how a “B” image is stored and output.

FIG. 98 is a view showing a picture buffer.

FIG. 99 is a diagram showing an H.261 prediction format.

FIG. 100 is a diagram showing an H.261 series.

101 is a diagram showing a hierarchy of H.261 syntax. FIG.

102 is a diagram showing an H.261 image layer. FIG.

103 is a diagram showing an H.261 arrangement of block groups. FIG.

FIG. 104 shows an H.261 “slice” layer.

FIG. 105 is a diagram showing an H.261 arrangement of macroblocks.

FIG. 106 is a diagram showing an H.261 block sequence.

FIG. 107 is a diagram showing an H.261 macroblock layer.

108 is a diagram showing an H.261 arrangement of pixels. FIG.

Fig. 109 is a diagram showing a hierarchy of MPEG syntax.

FIG. 110 is a diagram showing an MPEG sequence layer.

FIG. 111 is a diagram showing an image layer of an MPEG group.

FIG. 112 is a diagram showing an MPEG image layer.

FIG. 113 shows an MPEG “slice” layer.

FIG. 114 is a diagram showing a block of an MPEG series.

FIG. 115 is a diagram showing an MPEG macroblock layer.

FIG. 116 is a diagram showing an example of “open GOP”.

FIG. 117 is a diagram showing an example of an access structure.

FIG. 118 is a diagram showing access start timing.

FIG. 119 is a diagram showing a fast page read cycle.

FIG. 120 shows a fast page write cycle.

FIG. 121 is a view showing a refresh cycle.

FIG. 122 is a diagram showing how to extract row and column addresses from a chip address.

FIG. 123 is a diagram showing timing parameters for strobe signals.

FIG. 124 is a diagram showing timing parameters between any two strobe signals;

FIG. 125 is a diagram showing timing parameters between a bus and a strobe signal.

FIG. 126 is a diagram showing timing parameters between a bus and a strobe signal.

FIG. 127 is a diagram showing a Huffman decoder and a parser.

FIG. 128 is a diagram showing a flowchart of H.261 and MPEG AC coefficient decoding.

FIG. 129 is a block diagram of JPEG (AC and DC) coefficient decoding.

FIG. 130 shows a flow chart for JPEG (AC and DC) coefficient decoding.

FIG. 131 is a diagram showing an interface of a Huffman token formatter.

FIG. 132 is a block diagram of a token formatter.

133 is a diagram showing H.261 and MPEG AC coefficient decoding; FIG.

FIG. 134 is a diagram showing an interface of a Huffman ALU.

FIG. 135 is a view showing the basic structure of Huffman ALU.

FIG. 136 is a diagram showing a buffer manager.

FIG. 137 is a block diagram of imodel and hspk.

FIG. 138 is a view showing an image state.

FIG. 139 is a diagram showing buffer startup.

FIG. 140 is a view showing a DRAM interface.

FIG. 141 is a diagram showing a write swing buffer.

FIG. 142 is a view showing a calculation block.

FIG. 143 is a block diagram of iq.

FIG. 144 shows an iqca state machine.

FIG. 145 is a diagram showing an IDCT 1-D conversion algorithm.

FIG. 146 is a diagram showing an IDCT 1-D conversion structure.

FIG. 147 is a block diagram of a token stream.

FIG. 148 is a standard block diagram.

FIG. 149 is a block diagram illustrating microprocessor test access.

FIG. 150 is a diagram showing a 1-D conversion microstructure.

FIG. 151 is a block diagram of a time decoder.

FIG. 152 is a diagram showing the structure of a two-wiring interface stage.

FIG. 153 is a block diagram of an address generator.

FIG. 154 is a diagram showing an offset between a block and a pixel.

FIG. 155 is a diagram showing a plurality of prediction filters.

FIG. 156 is a diagram showing a prediction filter.

FIG. 157 is a diagram showing a 1-D prediction filter.

FIG. 158 is a diagram showing pixels in one block.

FIG. 159 is a view showing a structure of a read ladder.

FIG. 160 is a diagram showing block and pixel offsets.

FIG. 161 is a diagram showing a prediction example.

162 is a diagram showing a read cycle. FIG.

FIG. 163 is a view showing a writing cycle.

FIG. 164 is a block diagram of a top-level register group that can refer to timing.

FIG. 165 is a diagram showing a control for increasing the presentation number.

FIG. 166 is a diagram showing a buffer manager state machine.

FIG. 167 is a diagram showing a main loop of a state machine.

FIG. 168 is a diagram showing a buffer 0 including an SIF (22 × 18 macroblock).

FIG. 169 is a diagram showing an SIF component 0 having a display window.

FIG. 170 is a diagram showing an example of an image format showing a storage block address.

FIG. 171 is a diagram showing a buffer 0 including an SIF (22 × 18 macroblock).

FIG. 172 is a diagram showing an example of address calculation.

FIG. 173 is a diagram showing a write address generation state.

FIG. 174 is a diagram showing a slice of a data path.

FIG. 175 is a diagram showing a 2-cycle operation of a data path.

FIG. 176 is a diagram showing filtering processing in mode 1;

FIG. 177 is a view showing a horizontal upsampler data path;

FIG. 178 is a diagram showing a structure of a color-space converter.

[Explanation of symbols]

33 ... Token decoder, 34 ... Input latch, 36 ... Processing unit, 39 ... Action identification section, 41
... output latches, 43, 44 ... registers.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04N 1/41 B 7/015 7/24 H04N 7/13 Z (72) Inventor Martin William Sozaran United Kingdom Nation GL 1 1 6 B D, Grossester Sheer, Dursley, Stench Comb, Wik Lane, The Ridings (no street number) (72) Inventor William Philip Robins England, GL 1 1 5 PEE, Grosester Sheer, Kam , Spring Hill 19

Claims (6)

[Claims] 1. A pipeline system having an input, an output, and a plurality of processing stages between the input and the output, wherein the plurality of processing stages are two wires for transmitting a token along the pipeline. Control tokens, data tokens, interconnected by an expression interface, are a form of universal adaptation unit that interfaces with all processing stages in the pipeline and interacts with selected processing stages in the pipeline. A pipeline system characterized in that the processing stages of the line are given increased flexibility in configuration and processing. 2. Transmitted in a single bitstream,
A plurality of processing stages are interconnected by a two-wire interface for handling a plurality of encoded bitstreams as digital bit serial bitstreams, each pair having an encoded encoded control code and corresponding data. In a pipeline system, a start code detecting means for generating a control token and a data token in response to a single serial bit stream, and providing the two-wire interface with the start code detecting means, the token being provided in a predetermined processing stage, the token being unique to the stage. Token decoding means that recognizes as control tokens and passes unrecognized control tokens along the pipeline, and reconfigures to respond to the recognized control tokens and reconfigure certain stages to handle the identified data tokens. Configuration decoding parser processing means and Pipeline system characterized by comprising. 3. A pipeline system having a plurality of reconfigurable processing stages interconnected by a two-wire interface bus, wherein any one of the processing stages is a spatial decoder means, and the other processing stages are control tokens. And token generating means for generating a data token and passing through the two-wire interface, provided in a predetermined processing stage, and recognizing one of the tokens as a control token unique to the spatial decoder means, The spatial decoder means for spatially decoding the data token following
And a token decoding unit configured in the above decoding format. 4. Transmitted in a single bitstream,
A plurality of processing stages are interconnected by a two-wire interface for handling a plurality of encoded bitstreams as digital bit serial bitstreams, each pair having an encoded encoded control code and corresponding data. In a pipeline system, a start code detecting means for generating a control token and a data token in response to a single serial bit stream, and providing the two-wire interface with the start code detecting means, the token being provided in a predetermined processing stage, the token being unique to the stage. Token decoding means that recognizes as control tokens and passes unrecognized control tokens along the pipeline, and reconfigures the time decoding stage in response to the recognized control tokens to handle the identified data tokens,
The data of 8 × 8 pixel block is 2 in the time decoder.
A pipeline system comprising reconstruction time decoder means for moving along a line interface and addressing means for storing and retrieving blocks along block boundaries. 5. In a pipeline system having an input, an output, and a plurality of processing stages between the input and the output, a universal adaptive unit is defined, and control and data functions are executed in the processing stage. Pipeline system, characterized in that it comprises an interactive interface control token to enable the processing stages to be provided with increased flexibility in the functioning of distributed tasks. 6. In a pipeline system having an input, an output and a plurality of processing stages between the input and the output, a mutual adaptation unit defining a universal adaptation unit and performing a data function in the processing stage. A pipeline system comprising a working metamorphic interface data token, wherein said processing stage is provided with increased flexibility in processing the data.