EP1166566A2 - Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique - Google Patents

Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique

Info

Publication number
EP1166566A2
EP1166566A2 EP00921604A EP00921604A EP1166566A2 EP 1166566 A2 EP1166566 A2 EP 1166566A2 EP 00921604 A EP00921604 A EP 00921604A EP 00921604 A EP00921604 A EP 00921604A EP 1166566 A2 EP1166566 A2 EP 1166566A2
Authority
EP
European Patent Office
Prior art keywords
word
data
zero
buffer
code
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP00921604A
Other languages
German (de)
English (en)
Inventor
Robert M. Wolff
Randy Langer
Ulrich Sigmund
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ravisent Technologies Inc
Original Assignee
Ravisent Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US09/283,947 external-priority patent/US6366970B1/en
Priority claimed from US09/287,535 external-priority patent/US6373898B1/en
Priority claimed from US09/467,552 external-priority patent/US6567557B1/en
Application filed by Ravisent Technologies Inc filed Critical Ravisent Technologies Inc
Priority to EP02006387A priority Critical patent/EP1276331A3/fr
Publication of EP1166566A2 publication Critical patent/EP1166566A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the invention relates to data processing. More particularly, the invention relates to handling and manipulating high-speed streaming data in a computing device.
  • the invention also relates to a multi-byte search, for example to locate a pattern in streaming data. More particularly, the invention relates to a high-speed mechanism for scanning a real-time stream of data to locate a start-code-prefix in an MPEG-2 data stream, and streams that use the exact same start-code paradigm.
  • the same general algorithm can be applied to search patterns of any byte length, and is especially useful in cases where the length of the search pattern is an odd number of bytes.
  • the invention further relates to data compression as used in the coding of studio-quality video for digital TV, high-density CD-ROMs and TV-broadcasting, and relates more particularly to MPEG2 type datastream decompression in which a pre-parser locates data- structure information in a MPEG2 datastream and substantially relieves all subsequent decompression and renderer stages from the tedious job of parsing.
  • the present invention additionally relates to computer and digital data compression, and more specifically to preventing rounding errors that can accumulate in MPEG-2 type decompression.
  • Streaming data are data that are transferred, typically in real time, as a continuous and uninterrupted stream using such formats as MPEG2. While MPEG-2 has been used in static media such as DVD and is well known to the general public, it is less known that MPEG-2 has been used in streaming (real-time) applications such as digital satellite broadcast, e.g. DSS. DVB, Primestar, and Dishnetwork.
  • the data are transmitted at variable rates.
  • This streaming of data presents significant problems to a device that is to receive the data. For example, it is not possible to go back and resample the data as with a DVD because the data are streaming, . e. they are not fixed on a medium that may be reread if an error in data transfer is detected - they have evaporated into the ether. Thus, it is extremely difficult to recover from a failure in the transfer of a data block where the data block is part of an emphemeral data stream.
  • the computing device must be capable of handling the data stream in a worst case scenario.
  • the system has fewer resources available for processing other incoming requests and for general activities. This leaves the computing device with too much to do in heavy loading cases, with errors or data loss being the result.
  • a possible approach to solving the problem of processing streaming data may be provided by a memory manager, such as SmartHeapTM, which uses a recycled memory buffer for the specific purpose of memory management.
  • a memory manager such as SmartHeapTM, which uses a recycled memory buffer for the specific purpose of memory management.
  • these techniques have not been applied to the realm of streaming media. It is also possible to use multiple threads for streamlining control and delineation of duties. Again, such known technology is not specifically targeted to the problems associated with the handling of streaming media. Therefore, these techniques are not specifically made for streaming and are found to be suboptimal due to their lack of specific focus on these streaming issues. It would be advantageous to provide an efficient and reliable method and apparatus for processing streaming data in a computer system.
  • the MPEG-2 standard is often used for the formatting and transmission of audio and video information (for more information on the MPEG-2 standard, see http://www.mpeg.org). Due to the widespread use of the MPEG-2 standard, it has been desirable to implement a high-speed mechanism for scanning a real-time (i.e. no flow control) stream of MPEG-2 data to locate start code prefixes (/. e. byte patterns that delimit the structural components or packets which constitute the data stream). This is necessary because the data in such stream are passed to a system for processing in real time. Accordingly, each elementary unit of the stream must be reliably identified in real time for such unit to be parsed into its constituent parts and processed at a rate commensurate with the incoming data rate. Such start code prefix scanning typically represents a very significant portion of the overall CPU use in MPEG-2 processing programs, in some cases exceeding 50% of CPU use.
  • Neither above cited approach is particularly efficient because a significant amount of processor time is expended not only in looking for a target byte value(s), but in further qualifying the neighboring bytes to determine if a complete start code has been discovered.
  • the first technique cited above can be coded in assembly language for processors having so called string instructions, such as the Intel 80x86/Pentium family, and achieve a performance boost.
  • MPEG1 revolutionized video and audio playback performance on the personal computer, becoming a standardized feature through Microsoft's Windows operating system. This gave users their first taste of the possibilities of MPEG and laid the foundation for greater heights of multimedia.
  • the advent of the MPEG2 standard promises to expand audio and visual experiences beyond the personal computer and into everyday household appliances. By combining the "brains" of a personal computer with the entertainment features of consumer devices, a whole new breed of Entertainment personal computer products become possible.
  • MPEG1 technology delivers full 30 frames-per-second TV-like video and CD quality audio.
  • MPEG1 penetrates numerous markets where VHS quality video is acceptable, achieving acceptance in education and training programs, interactive encyclopedias and action games that enhance learning and playing.
  • MPEG1 has made its way onto corporate intranets, the new private networks that connect employees with information using Internet connectivity. MPEG1 is a great complement to intranet-based training applications, for these clips transmit seamlessly over the bandwidth limitations of these networks. Because MPEG1 provides acceptable video and audio playback, works with standard CD players and has a familiar personal computer interface through Microsoft Windows, today's high powered PCs can achieve acceptable quality using software-only MPEG1.
  • MPEG2 technology can produce video images four times larger than MPEG1 for enhanced resolution that is clearer than standard TV display.
  • MPEG2 includes theater-quality video and AC3 Surround Sound, two components in high demand from home entertainment enthusiasts.
  • DVD Digital Versatile Disk
  • MPEG2 achieves a higher level of performance than MPEG1. This positions MPEG2 technology as the standard for high end multimedia personal computer and consumer appliances. With DVD/MPEG2 functionality, the average personal computer is transformed into a full featured home entertainment theater.
  • MPEG1 audio was capable of only directly representing two channels of sound.
  • MPEG2 introduces a scheme to decorrelate multichannel discrete surround- sound audio.
  • the MPEG video syntax uses a few tokens to represent an entire block of sixty- four samples.
  • MPEG also describes a decoding, or reconstruction process where the coded bits are mapped from a compact representation into the original, raw format of the image sequence. For example, a flag in the coded bitstream signals whether the following bits are to be decoded with a "DCT" algorithm, or with a prediction algorithm.
  • the decoding process algorithms are defined by MPEG.
  • the syntax can exploit common video characteristics such as spatial redundancy, temporal redundancy, uniform motion, spatial masking, etc.
  • MPEG can achieve high quality video with compression ratios exceeding 100:1. But such high ratios often include over- sampling factors in the source video.
  • the coded sample rate an MPEG image sequence is usually not much larger than thirty times the specified bit rate.
  • Sub- sampling pre-compression provides much of the high compression ratios for all video coding methods, including those of the non-MPEG variety.
  • MPEG1 and MPEG2 video syntax are useful over wide ranges of bit rates and sample rates.
  • MPEG1 operates at thirty SIF pictures (352 pixels x 240 lines) per second, and a bit rate less than 1.86 megabits/sec, e.g., "Constrained Parameters Bitstreams", as in the Compact Disc Video (White Book).
  • Display picture size is the same as the coded picture size
  • the display picture size and frame rate may differ from the size (resolution) and frame rate encoded into the bitstream. For example, a regular pattern of pictures in a source image sequence may be dropped (decimated), and then each picture may itself be filtered and sub-sampled prior to encoding. Upon reconstruction, the picture may be interpolated and up-sampled back to the source size and frame rate.
  • the three fundamental phases, Source Rate. Coded Rate, and Display Rate may differ by several parameters.
  • the MPEG syntax can separately describe Coded and Display Rates through sequence_headers, but the Source Rate is known only by the encoder.
  • I. P, B Picture coding types
  • All macroblocks within an I-picture must be coded Intra, like a baseline JPEG picture.
  • macroblocks within a P-picture may either be coded as Intra or Non-intra, temporally predicted from a previously reconstructed picture.
  • Macroblocks within the B-picture can be independently selected as either Intra, Forward predicted, Backward predicted, or both forward and backward (Interpolated) predicted.
  • the macroblock header contains an element, called macroblock_type, which can flip these modes on and off like switches.
  • the macroblockjype is a powerful part of the video syntax. Picture types (I, P, and B) enable macroblock modes by widening the scope of the semantics.
  • the component switches are, Intra or Non-intra, Forward temporally predicted (motion_forward), Backward temporally predicted (motion_backward) (2+3 in combination represent "Interpolated"), conditional replenishment (macroblock_pattern), adaptation in quantization (macroblock_quantizer), and temporally predicted without motion compensation
  • the first five switches are mostly orthogonal.
  • the sixth is derived from the first and second in P-pictures, and does not exist in B-pictures. Some switches are non-applicable in the presence of others. For example, in an Intra macroblock, all six blocks by definition contain DCT data, therefore there is no need to signal either the macroblock_pattern or any of the temporal prediction switches.
  • the macroblock_quantizer signal when there is no coded prediction error information in a Non-intra macroblock, the macroblock_quantizer signal would have no meaning.
  • the sequence structure is fixed to a specific I,P,B frame pattern.
  • a sequence may consist of almost any pattern of I, P, and B-pictures. It is common in industrial practice to have a fixed pattern, e.g. IBBPBBPBBPBBPBB.
  • IBBPBBPBBPBBPBBPBBPBB e.g., more advanced encoders attempt to optimize the placement of the three picture types according to local sequence characteristics. Each picture type carries a penalty when coupled with the statistics of a particular picture, e.g., temporal masking, occlusion, motion activity, etc.
  • Digitized images require a large amount of storage space to store and a large amount of bandwidth to transmit.
  • a single, relatively modest-sized image having 480 by 640 pixels and a full-color resolution of twenty-four bits per pixel (three 8-bit bytes per pixel), occupies nearly a megabyte of data.
  • a 24-bit color screen requires 2.3 megabytes of memory to represent.
  • a 24-bit color picture of an 8.5 inch by 1 1 inch page, at 300 dots per inch, requires as much as twenty-five megabytes to represent.
  • Video images are even more data-intensive, since it is generally accepted that for high-quality consumer applications, images must occur at a rate of at least thirty frames per second.
  • Current proposals for high-definition television (HDTV) call for as many as 1920 by 1080 or more pixels per frame, which translates to a data transmission rate of about 1.5 billion bits per second. This bandwidth requirement can be reduced somewhat if one uses 2:1 interleaving and 4:1 decimation for the "U” and "V chrominance components, but 0.373 billion bits per second are still required.
  • JPEG Joint Photographic Experts Group
  • JPEG Joint Photographic Experts Group
  • the JPEG standard is described in a number of publications, including the following incorporated by reference herein: Wallace, The JPEG Still Picture Compression Standard, IEEE Transactions on Consumer Electronics, Vol. 38. No. 1, pp. xviii-xxxiv (1992); Purcell, The C-Cube CL550 JPEG Image Compression Processor, C-Cube Microsystems, Inc. (1992); and C-Cube Microsystems, JPEG Algorithm Overview (1992).
  • An encoder using the JPEG algorithm has four steps: linear transformation, quantization, run-length encoding (RLE), and Huffman coding.
  • the decoder reverses these steps to reconstitute the image.
  • the image is divided up into 8*8 pixel blocks and a Discrete Cosine Transform is applied in both spatial dimensions for each block.
  • the purpose of dividing the image into blocks is to overcome a deficiency of the discrete cosine transform algorithm, which is that the discrete cosine transform is seriously non-local.
  • the image is divided into blocks in order to overcome this non-locality by confining it to small regions, and doing separate transforms for each block.
  • this compromise has a disadvantage of producing a tiled appearance (blockiness) upon high compression.
  • the quantization step is essential to reduce the amount of information to be transmitted, though it does cause loss of image information.
  • Each transform component is quantized using a value selected from its position in each 8*8 block. This step has the convenient side effect of reducing the abundant small values to zero or other small numbers, which can require much less information to specify.
  • the run-length encoding step codes runs of same values, such as zeros, in items identifying the number of times to repeat a value, and the value to repeat.
  • a single item like "eight zeros" requires less space to represent than a string of eight zeros, for example. This step is justified by the abundance of zeros that usually result from the quantization step.
  • Huffman coding translates each symbol from the run-length encoding step into a variable-length bit string that is chosen depending on how frequently the symbol occurs. That is, frequent symbols are coded with shorter codes than infrequent symbols.
  • the coding can be done either from a preset table or one composed specifically for the image to minimize the total number of bits needed.
  • MPEG Motion Pictures Experts Group
  • the standards are known as MPEG-1 and MPEG-2.
  • MPEG algorithms exploit the common fact of relatively small variations from frame to frame.
  • a compression technique similar to that of the JPEG standard is typically used to compress these "reference" or "intra” frames.
  • For the intermediate frames a predicted frame is calculated and only the difference between the actual frame and the predicted frame is compressed and transmitted.
  • Any of several algorithms can be used to calculate a predicted frame, and the algorithm is chosen on a block-by-block basis depending on which predictor algorithm works best for the particular block.
  • Motion detection can be used in some of the predictor algorithms.
  • MPEG I is described in detail in International Standards Organization (ISO) CD 1 1172.
  • the MPEG technique is one which treats the compression of reference frames substantially independently from the compression of intermediate frames between reference frames.
  • the present invention does not relate to the compression of still images or reference frames, it relates to the decompression of intermediate frames.
  • MPEG-2 video compression standard
  • ISO/IEC 13818-2 Information technology— Generic coding of moving pictures and associated audio information: Video
  • MPEG-2 uses motion compensation on fixed sized rectangular blocks of pixel elements ("macroblocks") to use temporal locality for improved compression efficiency.
  • the location of these "macroblocks" in the reference pictures is given on half pixel boundaries, and so requires an interpolation of pixel elements.
  • case-D requires the execution of twenty-eight instructions, as is listed in the left column of Table I. But if "pavgusb" is used, the sub-routine can be reduced to five instructions , as is listed in the right column of Table I. So the temptation to use the quicker solution for case-D is very great.
  • the rounding error for a specific combination of four pixels can be calculated as:
  • the overall error is only affected by the two least significant bits of each coefficient, so the total average error can be calculated as:
  • the rounding error is less than one least-significant-bit for first-generation compensation, and is not a problem.
  • the MPEG-2 standard allows for multi-generation compensation. The consequence of this is that a predicted picture can be used as a reference picture for successive pictures. When a predicted picture is used as a reference picture, rounding errors can add up to more than one least significant bit, and the accumulated rounding errors can be large enough over several compensation generations to create objectionable visual artifacts.
  • Such visual artifacts or optical effects are called “pumping", and are perceived by the eye as image brightening over a series of predicted pictures that jump back to normal brightness at the next intra-picture.
  • sequences that have an unusually high number of predicted pictures between intra codes pictures can experience serious pumping.
  • field structure encoded sequences or material that lack "bidirectional" predicted pictures will suffer from pumping.
  • the invention provides a method and apparatus that offers optimal handling of high bandwidth streaming data in a computer system.
  • An overall objective in such a system is to minimize computational activities and thereby achieve maximal performance. This result is accomplished in the invention described herein by minimizing the amount of memory copying and also by minimizing the number of allocations and deallocations of objects.
  • Memory copying is a CPU/bandwidth intense operation when there is high speed streaming data on the input.
  • the allocation and deallocation of objects is a system resource intense activity and requires a very significant amount of CPU processing per invocation in a computing device.
  • the invention provides a technique that reduces both the number of memory copies as well as the number of objects which get allocated and deallocated during the course of operating on the streaming media data.
  • a word- wise search is performed.
  • Such strategy requires the same number of clock cycles as a byte-wide search, but involves one-half as many addresses, thereby cutting execution time in half.
  • For every start code either the first or second byte is on a word boundary, so that by searching the data twice, first for the 0x00 0x00 word, then again for the 0x00 0x01 word, every start code in the data can be found (each successful search "hit" requires an examination of the neighboring bytes to insure that it is really a start-code, but the rules are simple and can be coded efficiently).
  • There are normally no wait states in the second search because the second search will be executed out of the machine's data cache (if it has one).
  • the invention provides an algorithm that is more efficient at start code scanning than either of the two above-cited techniques, even when they are coded in assembly language.
  • the CPU time made available by the efficiency of this algorithm allows the processor to handle other tasks, thereby allowing a more complex and rich product to be deployed on machines of given processing power.
  • the invention makes more CPU time available to make it easier to implement such features as software picture-within-picture, visually powerful but CPU-intensive Sarnoff video effects, simultaneous view and record, interactive Web access, background Electronic Program Guide (EPG) processing, HDTV video rendering, and software motion compensation.
  • EPG Electronic Program Guide
  • HDTV video rendering HDTV video rendering
  • software motion compensation software motion compensation.
  • the parallel secondary datastream describes the structure of the MPEG2 datastream in an efficient and easy-to-use manner and helps to eliminate duplication of the parser task in various decoder stages.
  • the pre-parser divides MPEG2 datastream inputs into samples of convenient length. Each such sample is then characterized in the secondary datastream as to type and length. Such characterizations assist in the decoding and rendering of MPEG2 datastreams and eliminate parsing redundancy.
  • the invention also comprises a method wherein two-step motion prediction errors for MPEG-2 interpolation case-D are corrected.
  • An improved MPEG-2 decoder includes a logic gate, multiplexer, and adder. When both the horizontal (ho) and vertical (hi) motion vector components require a half pixel interpolation (case-D), the multiplexer forwards the constant minus three to the adder, otherwise a constant zero is used.
  • Such adder modifies the DC coefficient input to the inverse discrete cosine transformer to include a correction term for the predicted pixels calculated by a two- step predictor.
  • a correction value of -0.375 is evenly distributed over all sixty-four resulting spatial coefficients during the inverse discrete cosine transform. This results statistically in a slightly brighter set of correction terms.
  • Such offsets a slightly darker prediction that are formed by the two-step predictor.
  • the output frames are statistically correct images.
  • Fig. 1 is a block schematic diagram of a system for processing streaming data that uses few allocations of objects and that makes many copies of such objects;
  • Fig. 2 is a block schematic diagram of a system for processing streaming data that uses many allocations of objects and that makes few copies of such objects;
  • Fig. 3 is a block schematic diagram of a system for optimal handling and manipulation of high-speed streaming data in a computing device according to the invention
  • ' Fig. 4 is a block schematic diagram of a transport handler minicore for optimal handling and manipulation of high-speed streaming data in a computing device according to the invention
  • Figs. 5a to 5m provide an example of a start code scanner according to the invention
  • Fig. 6 is a functional block diagram of an MPEG2 system embodiment of the present invention.
  • Fig. 7 A is a diagram of the 16-bit descriptor word used in the system of Fig. 5 for the basic payload type of MPEG2 sample;
  • Fig. 7 B is a diagram of the 16-bit descriptor word used in the system of Fig. 5 for other than the basic payload type of MPEG2 sample;
  • Fig. 7 C is a diagram of the 16-bit descriptor word used in the system of Fig. 5 for notifications;
  • Fig. 8 represents an MPEG2 sample and a corresponding descriptor array.
  • Fig. 9 is a functional block diagram of an MPEG-2 video decoding processor.
  • Fig. 10 is a functional block diagram of an MPEG-2 video decoder embodiment of the invention that includes dual-step half-pixel prediction.
  • Fig. 1 is a block schematic diagram of a system for processing streaming data that uses few allocations of objects and that makes many copies of such objects.
  • a satellite antenna 10 receives streaming data in the form of a broadcast signal. The signal is provided to a driver 12 and thence to a memory 14. The incoming data are allocated and copied to the memory. The data are read from memory and processed by a module 15 that parses and modifies the data as necessary for use in the system. The results of the parse and modification step are copied to a memory output area 16 and thence decoded and rendered by a module 17 for use in the system. Once a data packet is processed, the system is alerted to begin processing the next incoming packet 18 and the process repeats.
  • Fig. 2 is a block schematic diagram of a system for processing streaming data that uses many allocations of objects and that makes few copies of such objects.
  • a satellite antenna 10 receives streaming data in the form of a broadcast signal. The signal is provided to a driver 12 and thence to a memory 24 which maintains a local copy of the incoming data.
  • the system includes a module 26 that allocates and creates an I/O object of the incoming data, which is then parsed and modified 25.
  • the I/O object is handed off 27 to an output consumer (not shown).
  • the consumer uses the output data and the advises the system that it is finished with the I/O object at which point the object is deallocated and destroyed. Thereafter, the system is alerted to begin processing the next incoming packet 28, at which point the process is repeated.
  • the invention provides a method and apparatus that offers optimal handling of high bandwidth streaming data in a computer system.
  • An overall objective in such a system is to minimize computational activities and thereby achieve maximal performance. This result is accomplished in the invention described herein by minimizing the amount of memory copying and also by minimizing the number of allocation and deallocations of objects which occur.
  • Fig. 3 is a block schematic diagram of a system for optimal handling and manipulation of high-speed streaming data in a computing device according to the invention.
  • the techniques that are employed by the system disclosed herein to reduce ' the number of memory copies and the number of objects allocations/deallocations while processing streaming data include:
  • Idle queue 30 which is used to keep a cache of currently unused data block objects. This facilitates recycling objects to minimize the allocation and deallocation of these data block objects;
  • Input queue 31 which is used as a holding area for incoming data blocks until each block can be processed
  • Limbo queue 32 which is used for localized streams which are larger than a single data block that needs to be held for further global processing prior to being placed in the output queue;
  • Output queue 33 which is the queue which holds the resultant data blocks after they are processed from input to limbo and finally placed into the output queue.
  • a data block is taken from the output queue and its resultant data is sent to the decoder or post processing unit, the data block is placed back onto the idle queue for reuse. The data block is not deallocated at this point.
  • Input thread 34 which takes incoming data from the streaming media source and makes a data block by pulling a data block from the idle queue, assigns referenced memory pointers to the incoming data, and then places the new data block onto the input queue;
  • Processing thread 35 which takes data blocks from the input queue and processes the block by parsing and/or modifying the data as necessary to prepare the data block for output. If it is determined 36 that all of the required information for a localized stream of data is not fully contained in a single data block, then the data block is placed in the limbo queue 37, 38 until the localized stream is complete 39. When a localized stream is completely processed, the data blocks which makeup the localized fully processed stream are moved to the output queue 44; and
  • Output thread 40 which is responsible for taking data blocks from the output queue and which provides a module 41 for handing the resultant data to the downstream processing unit, which in the preferred embodiment is usually a digital audio/video decoder. The corresponding data block is then placed back on the idle queue 42, 43 for use again by an incoming data packet/stream.
  • Complexity of the invention is minimized by use of the three separate threads which have separate and specific duties. Memory copies are minimized by only making reference to the incoming streaming data packets rather than copying the incoming data.
  • the data block object is a very small object which describes where the input data is found, how big the input data is, and where the output/resultant data is post- processed, as well as the status of each of these input and output memory references. It is possible to have situations where the input and/or output are at times actually allocated and/or copied. These are the exception rather than the rule, but the responsibility for keeping track of such events lies with the data block object itself. Allocation and deallocation of data blocks is reduced to near-zero by use of the idle queue as an object recycling center. This is a very powerful concept for a streaming application because data are always coming in, being processed, and being output. The constant movement of data makes this concept very valuable.
  • Fig. 4 is a block schematic diagram of a transport handler minicore 50 for optimal handling and manipulation of high-speed streaming data in a computing device according to the invention.
  • the following discussion describes a presently preferred architectural approach for handling ISO 13818-1 (See www.mpeg.org, www.ansi.org (ISO 13818-1 )) transport streams in software and for translating these transport streams into various formats of MPEG streams according to the invention.
  • This embodiment of the invention finds application in the area of satellite and terrestrial broadcast transport.
  • a major function of the minicore is to receive ISO 13818-1 transport stream packets as an input, i.e. the DVB transport stream and translate these packets into other useful MPEG formats at the user's request.
  • the minicore also provides timestamp and clock reference information to the consumer of this information on a side-interface which correlates this clock information to the audio and video data streams on the output interfaces.
  • the transport minicore provides on its output interfaces:
  • Length-enriched PES 52 reconstituted to contain length fields for all PES packets regardless of input payload length and 64k/16bit length field limitations.
  • interface outputs and input of transport data are provided in the preferred embodiment of the invention as a Microsoft Visual C++ ".lib” file, ".h” header file, and ".dll” file.
  • Table B is a source code listing of a C++-specific buffer that is recycled for processing data blocks from an input queue and that puts these data blocks on an output queue after processing.
  • PESHEADERSIZE 32 This is a maximum conservative value.
  • CDataBlock :CDataBlock(CCircularBuff* p_bin, CCircularBuff* p bout, QI_InputStreamType it, QI OutputStreamType ot)
  • p_copy_lasttrans NULL
  • p_copy_curtrans NULL
  • p_copy_nexttrans NULL
  • p_circ_in p_bin
  • p_circ_out p_bout
  • CDataBlock : ⁇ CDataBlock() ⁇ if (fp_dump) fclose(fp_dump);
  • DumpDataBlock (p_data_in, I. num_packets. lastAncillary); DeleteTempTrans();
  • p_copy_lasttrans new unsigned char [TRANSPORT_SIZE] ; assert(p_copy_lasttrans) ; memcpy(p_copy_lasttrans, p_packet- TRANSPORT SIZE, TRANSPORT_SIZE); ⁇
  • p_copy_nexttrans new unsigned char [TRANSPORT SIZE] ; assert(p_copy_nexttrans) ; memcpy(p_copy_nexttrans, p_packet+TRANSPORT_SIZE, TRANSPORT_SIZE);
  • ⁇ p_curout p_data_out; // Ancillary data gets laid down in "array” fashion at the end of the
  • p_ancillary_start p_data_out + len_out
  • p_ancillary_cur p_ancillary_start
  • num_ancillary 0;
  • *p_payload_len TRANSPORT_SIZE; return p_packet; default: assert(FALSE); return NULL;
  • PCR_flag tp[index] & 0xl0;
  • PCRb (tp[index] «25)
  • PCRx ((tp[index+4]&0x01) «8)
  • tp[index+5]; MakeAncillary(aPCRb. index, PCRb): MakeAncillary(aPCRx, index, PCRx); index + 6;
  • PES_extensions_flag tp[index++] & 0x01; // PES header length next.
  • PES_header_length tp[index++]; assert(PES_header_length); // Now save off our current position for later calculations.
  • p_payload_start_actual tp + index + PES_header_length; if (data_alignment_indicator)
  • SYNCJndex index+PES_header_length; MakeAncillary(aVidSEQ, index+PES_header_length,
  • SYNC_index index+PES_header_length; MakeAncillary(aAudFrameStart, index+PES_header_length, Oxfff);
  • PTS ParseTS_40(0x31); if (!PTS)
  • index p_payload_start_actual - tp; return;
  • DTS ParseTS_40(0xl l); if (!DTS)
  • index p_payload_start_actual - tp; return;
  • index is tricky.
  • the rules created for ancillary data dictate that the index // listed is "offset” or “modified” depending on the output type and the ancillary data type.
  • index is an offset into // the transport header or PES header (which follows the transport header) and in
  • BOOL CDataBlock :IsErrBitSet(QI_Ancillary Type atype) ⁇ assert((int)atype > InvalidMinAT && (int)atype ⁇ InvalidMaxAT); assert((int)atype ⁇ sizeof(ErrBits)*8 - 1); // ErrBits is a 32Bit signed value. Cant set any more bits than this. return (BOOL)(ErrBits & l «(int)atype); ⁇
  • Table C is a source code listing of a C++-specific implementation of a definition of recycled buffer processing class in Table A.
  • BOOL IsErrBitSet (QI_AncillaryType atype); int getDiscErrors() ⁇ return disc_errors; ⁇ ; BOOL getNeedsResync(); void PreCCCheck(); unsigned char getCC() ⁇ return prev_cont; ⁇ ; void setLastCC(unsigned char lcc); unsigned long getOutputLength() ⁇ return len data out; ⁇ ; unsigned long getInputLength() ⁇ return len_data_in; j ; unsigned long gefNumAncillary() ⁇ return num_ancillary; ⁇ ;
  • BOOL ge LookingForSyncO return bLookingForSync; ⁇ ; void setSyncToFrameO; void GetBlockInfo(PQI_STREAMOUT p_str); void ParsePESHeader(); void ProcessBlock(); int InputCopy(char* p_in, unsigned long len);
  • CDataBlock (CCircularBuff* p bin, CCircularBuff* p_bout, QI_InputStreamType it, QI OutputStreamType ot); virtual ⁇ CDataBlock(); protected: void SetErrBit(QI_AncillaryType atype); int disc errors;
  • QI_AncillaryType lastAncillary; BOOL bLookingForSync; void MakeAncillary(QI_AncillaryType atype, unsigned int at_index, DWORDLONG at value);
  • QI_ANCILLARY_DATA ancillary_entry unsigned int index; // "global” Index into transport packet.
  • unsigned char* tp // Transport packet being scanned.
  • unsigned char prev_cont // PES header entries unsigned int PES length; unsigned char scrambling_control; unsigned char data_alignment_indicator; unsigned char PTS_DTS_flags; DWORDLONG PTS, DTS; int SYNC_index;
  • Table D below is a source code listing of a C++-specific implementation of a definition of global objects used in the system disclosed herein.
  • InvalidMaxOT ⁇ enum QI_AncillaryType ⁇ InvalidMinAT. aPTS, aDTS, aPCRb, aPCRx, aESCR, aPESHeader, aVidSEQ, aAudFrame Start. aDiscontinuity, aScrambled, aRepeatPacket, aStuffingPacket. aReservedField.
  • Table E below is a source code listing of a C++-specific implementation of a module that oversees data blocks and that keeps track of the data blocks.
  • ⁇ pt (CTransport*)GetAt(pos); assert(pt); return pt; ⁇ ⁇ POSITION CTransportList: :FindPos(int source, unsigned int handle)
  • Table F below is a source code listing of a C++-specific implementation of a definition for a Data block tracking.
  • FILE* fp dump FILE* fp dump Getout; int disc_errors; unsigned char prev_lastCC;
  • Table G below is a source code listing of a C++-specific implementation of a definition of recycled buffer processing .
  • Table G Definition of Recycled Buffer Processing, Source Code Listing
  • CDataBlock (CCircularBuff* p_bin, CCircularBuff* p_bout.
  • QI_InputStreamType it, QI OutputStreamType ot);
  • QI_AncillaryType lastAncillary; BOOL bLookingForSync ; void MakeAncillary(QI_AncillaryType atype, unsigned int at index, DWORDLONG at_value);
  • QI_OutputStreamType out ype unsigned char* p ancillary start; unsigned char* p_ancillary_cur; unsigned long num_ancillary; QI_ANCILLARY_DATA ancillary_entry; unsigned int index; // "global" Index into transport packet, unsigned char* tp; // Transport packet being scanned, unsigned char prev cont; // PES header entries unsigned int PES_length; unsigned char scrambling_control; unsigned char data_alignment_indicator; unsigned char PTS_DTS_flags; DWORDLONG PTS, DTS; int SYNC ndex;
  • the invention provides an algorithm that is a more efficient technique for solving a well-defined and relatively long-existing problem, i.e. that of locating start codes in an MPEG-2 stream. Any extant algorithm that validly accomplishes this can be viewed as prior art.
  • One purpose of the invention is to reduce the amount of CPU time necessary to accomplish this defined goal.
  • the invention while useful for MPEG-2 data streams, is not limited to this standard and may readily be applied to other standards.
  • the search pattern implemented for the preferred embodiment of the invention is hard- wired to MPEG-2
  • the general algorithmic concept can be applied to any problem where finding unique 3 -byte patterns in a buffer is required.
  • the concept is generally extensible to searches of any pattern length, but is particularly useful when the pattern is an odd number of bytes.
  • the presently preferred embodiment of the invention is particularly useful when deployed on CPUs having the following capabilities in their instruction sets (although it is not limited to just these types of processors):
  • test condition is FALSE. It should be noted that in some cases (such as a buffer consisting entirely of zero bytes) that this trimming could wind up consuming the entire buffer; in this special case, the original size of the buffer should be reduced by one or two bytes, depending on whether the last one or two bytes of the buffer are zeros, and return to the caller saying no start codes exist in the buffer.
  • the first two BYTES are zero.
  • the third BYTE is one.
  • step 5 Define a temporary section of the evaluation buffer, from the point where the scan started at step 5 (but not advanced by any failed start-code tests in step 6) to the point where the start-code was detected in step 6 (or the end of buffer, if we got here from step 5). Call this sub-buffer the zero-word reach. Set the "current position" to the beginning of this zero- word reach.
  • step 10 Scan forward, as a WORD scan (see step 5 for definition), bounded inside the zero-word reach, for the next WORD of value 00 01 (this would be expressed as 0x0001 on big-endian machines, or 0x0100 on little-endian machines). If not found, go to step 10.
  • step 9 Check to see if this is a valid start-code, using the rules defined in step 6 (but keeping in mind that the pointer is now one BYTE into the potential start- code). If a start-code is found, append its offset into the list to be returned to the caller. In either case, then loop back to step 8.
  • step 5 If the process got to step 7 from step 6 (rather than step 5), append the offset of the start code discovered in step 6 into the list to be returned back to the caller, set the "current pointer" to the WORD following the start-code discovered in step 6, and loop back to step 2. Otherwise, return to caller.
  • the offset buffer (the buffer to which the start-code offsets are written) can hold six entries.
  • This example source buffer has 14 start codes (offsets expressed in hex). Byte patterns of 0x00 0x00 and 0x00 0x01 that are not actual start codes may be present; to save on detail, assume that such "false start codes" are detected and skipped over, and that it is possible to verify that each start code actually is a start code.
  • the buffer pointer is positioned at zero (see Fig. 5a).
  • a scan of the "odd-aligned" start codes finds one at 0x0141 (the 0x00 0x01 pattern found at 0x0142). so add that to the output table, filling it up to six. Because that was the size of the output table, return to the caller indicating that the output table has six entries.
  • start off with an initial offset of 0x0142 to "skip over" the last-detected start code(see Fig. 5f).
  • the initial offset if given, must always be an even number; if an odd number is passed in, it is incremented to make it even.
  • a re-scan of this zero-word reach finds no "odd-aligned" start codes, so the first entry into the output table is 0x01 C6 (see Fig. 5h). Looking for the next zero-word reach finds it bounded at 0x01E2.
  • a re-scan of this sub-buffer for word-aligned 0x00 0x01 patterns finds one at 0x0348, so offset 0x0347 is added to the output table, bringing up its membership to two entries. Because this is the end of the buffer, there is nothing more to do, so the process returns to the caller, indicating that two more start-codes were found.
  • WORD savFLAGS DWORD savESI. savEDI; // C++ compiler wants these
  • dec dword ptr hCnt // see if we've filled output // list jz exitRoutine // if so. we're done // Initial entry point here.
  • cmp byte ptr isAudio,0 je loop Id xor dl.OCOh cmp dl.020h jae looplb // not an audio start code loopl d: lea ebx,[edi-2] // remember the offset of this // 0x0000 WORD
  • loop2a or ecx,ecx jz loopl // end of input buffer; clean up // and exit mov eax, 1 OOh // pattern to search for repne scasw // do the search jne loopl // none found - go look for next
  • the presently preferred embodiment of the invention is optimized for the 80x86/Pentium architecture. Those skilled in the art will appreciate that the invention may be used with other architectures.
  • the preferred embodiment of the invention locates the start codes in a buffer of memory at very high speed by using a combination of string instructions and other features of the Intel processor's architecture, the processor's 32-bit instruction set, and available addressing modes. Because the invention uses features built into the hosting processor's architecture that allow for fast buffer operations, the algorithm is capable of locating start codes in a memory buffer at very high speeds. Those skilled in the art will appreciate that various known hardware and/or software technique may also be used to perform such operations.
  • inventions of the present invention generate a secondary channel of information that notifies later MPEG2 decoding stages of any transitions that occur between various kinds of structures in an MPEG2 datastream.
  • the information in such secondary channel can be used to simplify decoding of the MPEG2 datastream at every stage of processing. This is especially useful in the software implementation of an MPEG2 decoder.
  • an MPEG2 system 100 includes a high-definition television (HDTV) video source 102 and a surround-sound (SS) source 104.
  • a conventional MPEG2 encoder 106 compresses the HDTV and SS signals into an MPEG2 datastream 108 according to the industry-standard MPEG2 Specification.
  • Such MPEG2 datastream 108 may be stored on an optical disk or transmitted over a limited-bandwidth communication line represented by a device 109.
  • a pre-parser 1 10 basically samples the incoming MPEG2 datastream 108 and outputs it as a series of MPEG2 samples 1 12.
  • the pre-parser 1 10 analyzes the MPEG2 datastream 108 and outputs a secondary channel 1 14 which contains "heads-up" descriptions and notifications. In practice, these descriptions and notifications are maintained in a table in main memory.
  • the pre-parser 1 10 can be implemented in software, and communicate the MPEG2 samples and descriptors with table calls between subroutines.
  • the secondary channel 1 14 describes where various data structure transitions occur in the MPEG2 sample.
  • a software-implemented MPEG2 decoder 115 uses the notifications in the secondary channel 1 14 to decode the MPEG2 samples 1 12.
  • Such MPEG2 decoder 1 15 may be stored in an executable DLL-type disk file.
  • the decoding of the MPEG2 compressed sample 112 can be done without the assistance of the information in the secondary channel 1 14, but the MPEG2 decoder 115 would probably require a rather more expensive implementation in digital hardware.
  • Such notifications in the secondary channel 1 14 constitute pre-parsing data, and specify the locations in the MPEG2 compressed sample 1 12 of various structural features.
  • Such features include ES-start codes, audio frame headers, PES- headers, and the far more common basic payloads.
  • the software-implemented MPEG2 decoder 115 is saved the rather laborious and CPU-intensive job of parsing.
  • the MPEG2 decoder 1 15 is represented in Fig. 6 as a three-stage filter, in which a first stage filter 1 16 passes along an MPEG2 output 118 and pre-parser output 120.
  • the first-stage filter (A) 1 16 and a second-stage filter (B) 122 are spared the job of parsing the MPEG2 sample 1 12 because the pre-parser 1 10 has already done most or all of it for them.
  • the second-stage filter 122 outputs a modified MPEG-2 datastream 124 and an updated secondary channel 126 to a final stage filter, e.g., a renderer 128, that in turn outputs the decompressed and reconstructed HDTV and SS signals 130 and 132.
  • the renderer is also spared the job of parsing the MPEG2 sample 1 12 because it receives the pre-parser information in secondary channel 126.
  • Each filter 1 16 and 122 can modify the MPEG2 datastream, unless it's a rendering filter 128. Such modifications can include adding, removing, or changing some data features, and can thereby change the run lengths.
  • Each filter stage sends its modified MPEG2 datastream on to the next filter which also might do some of its own modifications.
  • MPEG2 datastream parsing is tedious and error-prone, so it's best to avoid duplicating this job especially when it's not necessary.
  • MPEG2 decoder embodiments of the present invention use pre-parsing to do this job only once at the point the MPEG2 datastream enters the receiver or decoding system.
  • a list of "descriptors" is generated by the pre-parser 1 10 for each sample of any reasonable length of the MPEG2 datastream 108. Such descriptor lists then travel in secondary channels 114, 120, and 126, alongside the MPEG2 samples 112, 118, and 124.
  • Each filter 116, 122, and 128, that processes the MPEG2 samples will update the descriptor lists in the secondary channels in a daisy-chain style.
  • the descriptor lists in the secondary channels are easy to maintain, and only add small extra processing overheads to each filter. But very significant amounts of processing horsepower are saved by not requiring each processing step in each filter to go through the parsing effort itself.
  • Pre-parsing adds a parallel datastream of data to a MPEG2 data datastream.
  • This secondary datastream includes an array of 16-bit integers called “descriptors" or “notifications”. Notifications are intended to provide information beyond that which is already embedded in the MPEG2 datastream, e.g., discontinuities and audio frame size changes. Such heads-up information in a secondary channel make processing of MPEG2 datastreams that much easier.
  • a descriptor describes both a type and a length.
  • Four types have initially been defined, e.g., a basic payload, an audio-frame header, an ES start-code, and a PES header.
  • Al 6-bit integer is used to encode the type and the length.
  • the basic payload type can be of variable length and very long.
  • the audio-frame header type is of variable length, but small, usually no more than eight bytes.
  • the ES start-code type is usually fixed and only four bytes long.
  • the PES header type is of variable length, but less than 512 bytes.
  • the basic payload type includes all those structures that do not fit into the other three classifications of audio-frame header, ES start-code, and PES header.
  • the MPEG2 sample is a basic payload type.
  • the remaining fifteen bits 14-0 define the MPEG2 sample length in a sample length field 201.
  • the secondary channel can use multiple-contiguous basic payload descriptors. It is permissible to have basic payload descriptors of all zeroes, which implies a sample-length of zero bytes. Such all-zero descriptors can be used as nulls or padding elements in a descriptor array.
  • bits 14-9 in a type field 204 define the type
  • bits 8-0 in a size field 206 define the MPEG2 sample length.
  • bits 15-9 are all ones, the descriptor is defined to be a notification word 208, and bits 8-0 in a notice code field 210 define the exact notification.
  • each MPEG2 datastream sample has a descriptor array or table initially allocated in main memory by pre-parser 1 10.
  • descriptor array must therefore be large enough to hold all the 16-bit descriptors that could be necessary to fully characterize the entire MPEG2 sample, and including any notifications. It is therefore preferable to allocate descriptor arrays that are larger than is initially needed, and to fill the unused end with zeros. That way, downstream processing stages like filters 1 16, 122, and 128 will have enough space to add their contributions to the descriptor table without having to waste time or CPU-overhead in reallocating to a new, larger table.
  • Notifications are preferably coded with negative numbers. Such codes are obtained by extracting the bottom nine bits in the notice code 210 (Fig. 7C), incrementing it by one, and then changing the sign of the result to make it negative. This results in “notification tokens” ranging from “-1 " to "-512". The first of these two are predefined, with “-1 " indicating a discontinuity, and "-2” indicating something-of- interest happened in the MPEG2 sample. When a "-1 " is received in a notice, the receiving process should reset its state machines and view the features that follow as being completely unrelated to the preceding features. In prototypes that have been constructed, the "-2" notification code is issued when the frame size changes in audio datastreams.
  • the descriptors diagrammed in Figs 7A-7B preferably define a length rather than an offset.
  • the corresponding descriptors are simply added to or deleted from the descriptor array. If any of the constituent MPEG2 sample features are reduced or enlarged in length, only the few corresponding descriptors are updated. The others are unaffected.
  • a single descriptor might have to be split into several descriptors. This can happen if the original feature was less than 32768 bytes in length, and the modified feature is larger than the fifteen binary bits in the sample length field 201 can describe.
  • MPEG2 datastreams are normally processed serially. So an "offset counter" can be included in pre-parser 1 10 that is incremented by the length of each MPEG2 sample feature. Such an offset counter is simple to build and operate.
  • descriptors are simple, their maintenance in tables is also simple. Any major reconstruction of the descriptor table by any intermediate stage are preferably done as a repeat of the pre-parser task in a call to a DLL-type file.
  • Fig. 8 illustrates a typical MPEG2 video datastream sample which includes payload, ES-headers, and PES-headers all head-to-toe in a single datastream. If there were no pre-parsing, each processing stage 1 16, 122, and 128 must locate each MPEG2- feature boundary itself. But with pre-parsing like that provided by pre-parser 1 10, an "offset counter" is used that is initialized to zero at the start of each MPEG2 sample. The offset counter is incremented by the length of each feature as the descriptor table is sequentially scanned and functions as an address register. For example, a PES header will have "0x8400" in the top seven bits of its descriptor.
  • ISO/IEC Specification 13818-1 allows for PES-headers to be inserted within other feature types. Embodiments of the present invention only allow such insertions for basic -payload features. So PES-headers be relocated to one side or the other of any feature they invade. When this happens, the content of the PES-headers must be adjusted slightly to compensate for the change in absolute location within the MPEG2 datastream.
  • Fig. 9 is a functional block diagram of an MPEG-2 video decoding processor 100.
  • An incoming MPEG-2 datastream 101 is received by a header parser 102 for decompression.
  • the local spatial decorrelation methods in MPEG and JPEG are very similar.
  • Picture data is block transform coded with the two-dimensional orthonormal 8*8 DCT.
  • the resulting sixty-three AC transform coefficients are mapped in a zigzag pattern to statistically increase the runs of zeros.
  • Coefficients of the vector are then uniformly scalar quantized, run-length coded, and finally the run-length symbols are variable length coded using a canonical (JPEG) or modified Huffman (MPEG) scheme.
  • JPEG canonical
  • MPEG modified Huffman
  • the header parser 1 102 separates a motion vector information stream 1 103 from a compressed MPEG-2 video bit stream 1104.
  • a motion vector (MV) calculator 1106 and a variable length decoder (VLD) 108 receive the separated streams.
  • the bit stream 1 104 is further processed in the VLD 1 108 to reconstruct the original zero frequency (DC) and up to sixty-three non zero frequency (AC) coefficients.
  • a run length decoder and an inverse quantization included in the VLD 1 108 help produce an Fo stream 1110 containing the DC coefficient and an Fi 6 stream 1112 containing the AC coefficients.
  • the DC and AC coefficients are further processed to yield the correction terms in the frequency domain.
  • An inverse discrete cosine transformer 1114 transforms the discrete cosine in the F 0 , Fi 6 3 stream 1 1 10/1 1 12 from the frequency domain back to the spatial domain.
  • a spatial-domain output 1 1 16 is produced.
  • Motion vector information is used to calculate an effective motion vector output stream 1118 in the MV calculator 1106.
  • a predictor 1120 uses such effective motion vectors to build a prediction that is forwarded to a summer 1 122.
  • a frame store 1124 and 1126 allow reference pictures that have been previously decoded and stored to be used later by the predictor 1120.
  • the sub-routines listed herein for MPEG-2 interpolation cases A-D are typically implemented in the construction of the predictor 1120.
  • a resulting predicted block of pixels is combined in summer 1122 with the result of the inverse discrete cosine transform to yield a final reconstructed macroblock, e.g., and stored in frame stores 1124 and 1126.
  • Fig. 10 represents an MPEG-2 video decoder embodiment of the invention, and is referred to herein by the general reference numeral 1200.
  • the MPEG-2 video decoding method 1200 prevents the accumulation of rounding errors. Such enables the use of the improved dual-step half-pixel prediction scheme for MPEG-2 interpolation case-D.
  • the embodiments of the present invention can also be used to improve the image quality for implementations that already use the dual-step half- pixel prediction scheme, i.e., if the steps prior to the inverse discrete cosine transform are implemented in software.
  • the inverse discre.e cosine transform defined in the MPEG-2 decoding standard has an effective division of eight for the DC coefficient. Taking advantage of this, a correction term of 0.375 can be subtracted for all macroblocks that need case-D processing as integer value three off the DC coefficient just before the inverse discrete cosine transformation. The resulting energy difference of twenty-four is distributed over all sixty-four transformed correction terms. This results in a statistical distribution of -0.375 that zeros-out the 0.375 error that is introduced by dual-step half-pixel prediction.
  • Embodiments of the invention eliminate the need to add a correction term during the averaging operation by using appropriate correction terms for the DC coefficient for all cases (A to D) before the inverse discrete cosine transform process. This could enable the use of a simple alpha blending unit in a state of the art graphics accelerator to perform motion compensation compatible with the MPEG-2 standard, without the need to enhance it with an MPEG-2.
  • a two-step motion prediction for MPEG-2 interpolation case-D yields visual artifacts if not corrected.
  • the invention is placed into the embodiment, as a combination of a logic AND circuit in combination with a multiplexer, and an adder. If both the horizontal (h 0 ) and vertical (hi) motion vector components require a half pixel interpolation (case-D), the multiplexer forwards the constant minus three to the adder, in other cases the constant zero is used. The adder adds the outcome of the multiplexer to the DC coefficient, to form a new DC coefficient, that contains the correction term for the predicted pixels calculated by the two step predictor.
  • a correction value of -0.375 is evenly distributed over all sixty-four resulting spatial coefficients during the inverse discrete cosine transform. This results statistically in a slightly darker set of correction terms. This eliminates the slightly brighter prediction formed by the two step predictor.
  • the output frames are statistically correct images.
  • the decoder 1200 implementation in Fig. 10 receives an MPEG-2 datastream 1202 with a header parser 1204.
  • the header parser 1204 separates a motion vector information stream 1206 from a compressed MPEG-2 video bit stream 1208.
  • a motion vector (MV) calculator 1210 produces a vertical motion vector component hi output 1212, a horizontal motion vector component h 0 output 1214, and an MV output 1216.
  • An AND-gate 1218 causes a multiplexer 1220 to select a minus-three valuel 222 or a zero value 1224. When the ho output 1214 and hi output 1212 are both true, the minus-three value 1222 will be selected and passed to an adder 1226.
  • a processor 1228 includes a variable length decoder (VLD), run length decoder, and an inverse quantization unit. It produces a direct current (DC) coefficient Fo output 1230 and an alternating current (AC) coefficients F ⁇ .63 output 1232. The processor 1228 reconstructs the original DC coefficient and up to sixty-three AC coefficients..
  • An inverse discrete cosine transformer 1234 transforms the discrete cosine in the Fo , F ⁇ . . stream 1230/1232 after correction by adder 1226.
  • a spatial-domain output 1236 is produced.
  • a two-step predictor 1238 can then be used which can take advantage of the "pavgusb" instruction in some commercial microprocessors when the MPEG-2 interpolation case-D is encountered. Specifically, case-D is,

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un procédé et un appareil de gestion optimale de données en continu sur large bande dans un système informatique qui permettent de réduire les activités informatiques pour obtenir une performance maximale. On réalise cette amélioration de performance en réduisant la quantité de reproduction de mémoire, ainsi que le nombre d'affectations et de désaffectations d'objets qui surviennent. Une recherche par mot judicieuse est effectuée sur un flux MPEG-2. On utilise un pré-analyseur pour créer un flux de données secondaire parallèle à un flux de données MPEG-2 lors du décodage et du rendu. Le flux de données secondaire parallèle décrit la structure du flux de données MPEG-2 d'une manière efficace et pratique et permet d'éliminer la duplication de la tâche d'analyse à différents stades de décodage. Une prédiction de mouvement à deux étapes pour le cas D d'interpolation MPEG-2 produira des artéfacts visuels si elle n'est pas corrigée.
EP00921604A 1999-04-01 2000-03-31 Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique Withdrawn EP1166566A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP02006387A EP1276331A3 (fr) 1999-04-01 2000-03-31 Méthode pour empêcher l'accumulation d'erreur de compensation de mouvement sous-pixelique à deux étapes pour séquences MPEG-2 à haut degré de prédiction

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US342527 1994-11-21
US283947 1999-04-01
US09/283,947 US6366970B1 (en) 1999-04-01 1999-04-01 Optimal handling and manipulation of high-speed streaming media in a computing device
US09/287,535 US6373898B1 (en) 1999-04-06 1999-04-06 High-speed start code scanner for MPEG-2 data streams
US287535 1999-04-06
US34252799A 1999-06-29 1999-06-29
US467552 1999-12-10
US09/467,552 US6567557B1 (en) 1999-12-10 1999-12-10 Method for preventing dual-step half-pixel motion compensation accumulation errors in prediction-rich MPEG-2 sequences
PCT/US2000/008771 WO2000064186A2 (fr) 1999-04-01 2000-03-31 Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP02006387A Division EP1276331A3 (fr) 1999-04-01 2000-03-31 Méthode pour empêcher l'accumulation d'erreur de compensation de mouvement sous-pixelique à deux étapes pour séquences MPEG-2 à haut degré de prédiction

Publications (1)

Publication Number Publication Date
EP1166566A2 true EP1166566A2 (fr) 2002-01-02

Family

ID=27501365

Family Applications (2)

Application Number Title Priority Date Filing Date
EP02006387A Withdrawn EP1276331A3 (fr) 1999-04-01 2000-03-31 Méthode pour empêcher l'accumulation d'erreur de compensation de mouvement sous-pixelique à deux étapes pour séquences MPEG-2 à haut degré de prédiction
EP00921604A Withdrawn EP1166566A2 (fr) 1999-04-01 2000-03-31 Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP02006387A Withdrawn EP1276331A3 (fr) 1999-04-01 2000-03-31 Méthode pour empêcher l'accumulation d'erreur de compensation de mouvement sous-pixelique à deux étapes pour séquences MPEG-2 à haut degré de prédiction

Country Status (5)

Country Link
EP (2) EP1276331A3 (fr)
JP (1) JP2002542549A (fr)
AU (1) AU4189700A (fr)
GB (1) GB2363279B (fr)
WO (1) WO2000064186A2 (fr)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002015583A1 (fr) 2000-08-15 2002-02-21 Microsoft Corporation Procedes, systemes et structures de donnees pour codage temporel d'echantillons multimedia
US20020089602A1 (en) 2000-10-18 2002-07-11 Sullivan Gary J. Compressed timing indicators for media samples
DE10140289A1 (de) * 2001-08-16 2003-02-27 Technotrend Ag Verfahren und Vorrichtung zur Bandbreitenreduzierung
DE60310368T2 (de) 2002-01-22 2007-03-29 Microsoft Corp., Redmond Verfahren zur verhinderung von startkode-emulation und stopfdaten
US7149247B2 (en) 2002-01-22 2006-12-12 Microsoft Corporation Methods and systems for encoding and decoding video data to enable random access and splicing
JP4448334B2 (ja) 2002-04-19 2010-04-07 マイクロソフト コーポレーション バイト整列されていない（ｎｏｎ−ｂｙｔｅ−ａｌｉｇｎｅｄｐｏｓｉｔｉｏｎｓ）のポジション、および／またはビット・シフトされたポジション（ｂｉｔ−ｓｉｆｔｅｄｐｏｓｉｔｉｏｎｓ）を含む位置におけるスタート・コード・エミュレーションを防ぐための方法およびシステム
US7839930B2 (en) 2003-11-13 2010-11-23 Microsoft Corporation Signaling valid entry points in a video stream
US7609762B2 (en) 2003-09-07 2009-10-27 Microsoft Corporation Signaling for entry point frames with predicted first field
US7852919B2 (en) 2003-09-07 2010-12-14 Microsoft Corporation Field start code for entry point frames with predicted first field
US7924921B2 (en) 2003-09-07 2011-04-12 Microsoft Corporation Signaling coding and display options in entry point headers
US8121196B2 (en) * 2006-11-02 2012-02-21 Corel Corporation Method and apparatus for multi-threaded video decoding
CN113490134B (zh) * 2010-03-23 2023-06-09 杜比实验室特许公司 音频再现方法和声音再现系统
US10158958B2 (en) 2010-03-23 2018-12-18 Dolby Laboratories Licensing Corporation Techniques for localized perceptual audio
US10271069B2 (en) 2016-08-31 2019-04-23 Microsoft Technology Licensing, Llc Selective use of start code emulation prevention

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW224553B (en) * 1993-03-01 1994-06-01 Sony Co Ltd Method and apparatus for inverse discrete consine transform and coding/decoding of moving picture
DE69535553T2 (de) * 1994-06-17 2007-12-06 Snell & Wilcox Ltd., Havant Videokompression
US5566089A (en) * 1994-10-26 1996-10-15 General Instrument Corporation Of Delaware Syntax parser for a video decompression processor
US5638128A (en) * 1994-11-08 1997-06-10 General Instrument Corporation Of Delaware Pixel interpolation filters for video decompression processor
KR960036641A (ko) * 1995-03-21 1996-10-28 김광호 저속의 비디오비트열을 복호하는 고속용 복호화장치
US5650823A (en) * 1995-03-27 1997-07-22 International Business Machines Corporation Half pel motion estimation method for B pictures
EP0872798A1 (fr) * 1997-03-21 1998-10-21 CANAL+ Société Anonyme Organisation de mémoire d'ordinateur
GB9717715D0 (en) * 1997-08-22 1997-10-29 Philips Electronics Nv Data processor with localised memory reclamation
JPH11136225A (ja) * 1997-10-30 1999-05-21 Matsushita Electric Ind Co Ltd ビットストリームにおけるスタートコードを検出する方法および装置
GB9807208D0 (en) * 1998-04-03 1998-06-03 Nds Ltd Method and apparatus for detecting a sequence in a bitstream

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO0064186A3 *

Also Published As

Publication number Publication date
GB2363279A (en) 2001-12-12
JP2002542549A (ja) 2002-12-10
EP1276331A3 (fr) 2005-06-01
EP1276331A2 (fr) 2003-01-15
WO2000064186A2 (fr) 2000-10-26
GB0123396D0 (en) 2001-11-21
WO2000064186A3 (fr) 2001-03-01
AU4189700A (en) 2000-11-02
GB2363279B (en) 2003-10-22

Similar Documents

Publication Publication Date Title
US10397592B2 (en) Method and apparatus for multi-threaded video decoding
US8416859B2 (en) Signalling and extraction in compressed video of pictures belonging to interdependency tiers
US7158571B2 (en) System and method for balancing video encoding tasks between multiple processors
KR100770704B1 (ko) 픽쳐 스킵 방법 및 장치
AU2007319699B2 (en) Techniques for variable resolution encoding and decoding of digital video
US7023924B1 (en) Method of pausing an MPEG coded video stream
JP5047740B2 (ja) 圧縮ノーマル・プレイ・ビデオ・ビットストリームから、トリック・プレイ・ビデオ・ストリームを作成するシステムおよび方法
RU2573778C2 (ru) Устройство декодирования сигнала изображения, способ декодирования сигнала изображения, устройство кодирования сигнала изображения, способ кодирования сигнала изображения и программа
US7839930B2 (en) Signaling valid entry points in a video stream
KR0134166B1 (ko) 영상신호기록장치
US5739862A (en) Reverse playback of MPEG video
WO2000064186A2 (fr) Gestion et manipulation optimales de support d'emission en continu a grande vitesse dans un dispositif informatique
US6754274B2 (en) Video data recording method and apparatus for high-speed reproduction
JP3852366B2 (ja) 符号化装置および方法、復号装置および方法、並びにプログラム
US6600787B2 (en) MPEG decoding device
KR101057590B1 (ko) 화상들의 그룹 내로의 임의 접근을 제공하도록 화상들의그룹을 재구성하는 방법
US20060109906A1 (en) Methods and apparatus for dynamically adjusting f-codes for a digital picture header
JP4906197B2 (ja) 復号装置および方法、並びに記録媒体
Buchanan et al. Motion Video Compression

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20010928

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20030425

RBV Designated contracting states (corrected)

Designated state(s): DE FR