WO2005034516A1 - Method and apparatus for processing image data - Google Patents
Method and apparatus for processing image data Download PDFInfo
- Publication number
- WO2005034516A1 WO2005034516A1 PCT/IB2004/051944 IB2004051944W WO2005034516A1 WO 2005034516 A1 WO2005034516 A1 WO 2005034516A1 IB 2004051944 W IB2004051944 W IB 2004051944W WO 2005034516 A1 WO2005034516 A1 WO 2005034516A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- memory
- data
- image data
- image
- cache
- Prior art date
Links
- 238000012545 processing Methods 0.000 title claims description 75
- 238000000034 method Methods 0.000 title claims description 55
- 230000015654 memory Effects 0.000 claims abstract description 215
- 230000033001 locomotion Effects 0.000 claims abstract description 56
- 239000013598 vector Substances 0.000 claims description 26
- 238000005259 measurement Methods 0.000 claims description 20
- 238000013500 data storage Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 14
- 230000000694 effects Effects 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims description 2
- 239000000872 buffer Substances 0.000 abstract description 16
- 230000008901 benefit Effects 0.000 abstract description 9
- 238000004458 analytical method Methods 0.000 abstract description 4
- 238000006243 chemical reaction Methods 0.000 description 12
- 238000007906 compression Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 101000979001 Homo sapiens Methionine aminopeptidase 2 Proteins 0.000 description 2
- 101000969087 Homo sapiens Microtubule-associated protein 2 Proteins 0.000 description 2
- 102100021118 Microtubule-associated protein 2 Human genes 0.000 description 2
- 101100131116 Oryza sativa subsp. japonica MPK3 gene Proteins 0.000 description 2
- 101100456045 Schizosaccharomyces pombe (strain 972 / ATCC 24843) map3 gene Proteins 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000005055 memory storage Effects 0.000 description 2
- 108010041420 microbial alkaline proteinase inhibitor Proteins 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/43—Hardware specially adapted for motion estimation or compensation
- H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
Definitions
- the invention relates to method and apparatus for arranging image data within an image processor's memory.
- the invention has particular application in the field in decoding block-based encoded image data, such as MPEG video streams.
- Digital transmission and storage systems generally use block-based compression, as used in the well known JPEG or MPEG formats, to achieve acceptable image quality within the available transmission bandwidth and storage capacity.
- JPEG is a video compression system based upon performing Discrete Cosine Transformation (DCT) on groups, or blocks, of pixel data.
- MPEG is a motion video compression system based upon the same principles, but with additional features to support motion between image frames.
- the DCT coefficients representing each block of pixels are subjected to adaptive quantisation and Variable Length Encoding (VLE).
- Blocks are also grouped together in fours, to form "Macroblocks", and chrominance (colour) components are represented with half the spatial resolution provided for luminance (brightness) component.
- JPEG still images
- MPEG motion video
- JPEG still images
- MPEG motion video
- temporal redundancy between image frames is identified and significantly reduced using motion-compensated inter-frame predictive encoding.
- the terminology used for describing MPEG sequences includes 'frames', which contain a complete image, and 'fields', which are half an image, arranged every other line.
- the unit of decoding is a picture, which can be field or frame structured.
- An image buffer is used to store both frames and/or fields, depending upon the storage format employed.
- Three image picture types known as T, 'B' and 'P' pictures, are used to construct an image sequence for transmitting data across a restricted channel between an encoder and a decoder.
- a "channel” includes bandwidth-limited communication links and storage of imagery on mass storage media such as hard drives, Compact Disks or video tape (where it is desirable to maximise storage efficiency).
- T picture frames are "intra" frames, which are similar in construction to a single JPEG frame and contain a complete, moderately compressed, frame of image data.
- 'P' picture frames are "predictive" frames and are encoded with reference to a previous T or 'P' frame (known as 'key frames') within a video stream.
- 'B' picture frames are "bi-directional interpolated" frames which require both earlier and later reference frames in order to be encoded.
- To support production of an MPEG image sequence data has to be stored in image memory.
- 'B' or 'P' frames are being produced, one or two previous source I or P images must have been stored and are referenced to provide motion prediction data.
- a typical arrangement for an image processor performing MPEG processing comprises modules (in hardware or software) for inputting data, converting the image between analogue and digital domains (in either direction, depending on whether the processor is an encoder or a decoder), storing the data, performing compression or decompression on the data, compensating for motion, and for outputting the data for subsequent use, such as display.
- modules in hardware or software
- it must be designed to accommodate worst-case conditions which as a result of processor overload would result in frame skipping or other image artefacts that degrade image quality.
- memory bandwidth is a crucial issue for both hardware and software implementations.
- processors In order to maximise throughput (and therefore reduce the opportunity for processor overload), processors often employ a cache memory with a very high access speed, allowing the processor to obtain data very rapidly.
- the cache is arranged so that as much data as possible is sourced from the cache, rather than from slower "external” memory.
- cache size and traffic are often crucial performance-determining factors. Worst- case performance can be significantly affected by cache activity, as it is possible for prediction data to be sourced from widely separated parts of input images resulting in significant "cache thrashing" (significant and substantially unnecessary cache use and discarding of content).
- Image memory is often provided as "paged" memory, whereby image data for a picture is stored over many pages, each of which is accessed by the processor by page access.
- Page access is a very rapid way of accessing data from a paged Random Access Memory (RAM), requiring only the provision by the processor of a base address, after which the data is clocked out and passed to the processor without provision of further addressing to the memory.
- RAM paged Random Access Memory
- page crossings are inefficient, as they require termination of the current paged memory transfer and generation of new addressing for the next set of data transfer. It is understood in the field of MPEG image processing that when paged memory is used different storage formats provide different advantages, depending upon the content of the images being processed. Storing image data on a linear basis (for example, following a raster scan) minimises the complexity required to subsequently display the image.
- the present inventors have recognised that, since the content of picture sequences varies greatly in the amount and nature of motion between the frames or fields that make up each picture, it is not possible to select a configuration for the reference data memory that will be the optimum one for all sequences. Accordingly, the US patent provides a range of configurations that may be advantageous if properly selected, but does not define what is the proper configuration. Nor is it possible to define a single configuration that will be optimal in all sequences. The optimal configuration may further depend on what steps are to be performed on the image data after decoding. The need for conversion to another format for a subsequent processing stage may for example negate any saving by adopting a preferred format for a motion estimation stage.
- the invention in a first aspect provides a method of arranging image data representing a motion picture sequence within a memory sub-system in an image data processing system, the method comprising dynamically selecting the arrangement of image data for successive pictures of said sequence in said memory according to at least one of: measured characteristics of said image data, measured characteristics of the performance of said processing system, and known characteristics of subsequent processing of said image data within said image processing system.
- the memory sub-system may include an image data storage memory constructed from paged memory.
- Said memory sub-system may include a processor cache memory in addition to a main image data storage memory.
- the step of selecting the arrangement of image data in storage memory may comprise selecting between a linear format, whereby image data is stored in memory on a line-by-line basis, and at least one kind of tiled format, whereby two-dimensional groups of pixels are grouped in memory.
- said tiled format is preferably defined such that data for one tile corresponds to a whole number of cache blocks.
- the method may comprise measuring one or more of the following as characteristics of the image data to influence the selection of the arrangement of data in memory:- A. variability of motion vectors encoded within the received data. For example, where motion is zero or at least relatively constant across the image being decoded, it may be efficient to store the reference image in linear format. Where motion vectors are highly variable, tiled or other formats will be preferred to a simple linear format; B. picture type. For example, to maximise efficiency of processing a key picture (I or P), a format that is more efficient for other pictures to predict from may be selected, such as a tiled storage format; C.
- D structure of a picture from which the image data is derived (for example, whether an image frame is arranged as a single noninterlaced frame, or interlaced fields);
- E structure employed for storing an image in image memory, such as where fields for an image frame are stored separately;
- F constant characteristics of the stream obtained by decoding the sequence header of the image stream, such as image resolution. For example, an image having low resolution (determined from the sequence header) may be processed more efficiently using a linear organisation for storage, as many display lines can fit within the data cache and the layout is optimised for image reconstruction. However, a high resolution image may benefit from storage in a tiled format, to minimise cache stalls and/or page crossing.
- a measured characteristic such as variability of motion vectors for a P picture
- the method may look ahead in motion picture sequence so as to measure said characteristics of the image data for a given portion of the sequence and select the memory arrangement prior to processing that portion.
- measured characteristics of the image data at one part of the sequence are used effectively to predict characteristics of a subsequent portion of the sequence, and the memory arrangement selected according to measured characteristics of recently processed portions of the sequence.
- the measurement of image data characteristics may be averaged over a period of time. Where the variability of motion vectors is measured, it may be measured separately between vertical and horizontal planes, each having a different effect in the selection of the storage arrangement.
- the method may alternatively or in addition comprise measuring one or more of the following as characteristics of system performance to influence the selection of the arrangement of data in memory:- G. data cache stall rates in the memory sub-system; H. processor utilisation (for example by measuring processor percentage idle time); I. quality of service, or other such qualitative measurements that are perceptible to the end user of the content being processed; J. bandwidth of a link feeding data into or out of said image processor.
- measurement of system performance may include measurement of data cache stall rates during image reconstruction. System performance may be measured at least partly on a test basis using a sample of data, prior to processing the data.
- system performance measured while processing a first part of the sequence may be used to influence the arrangement of memory for a subsequent part of the sequence.
- the method may alternatively or in addition comprise using knowledge of subsequent processing steps to influence the selection of the arrangement of data in memory. This can be done to take account of the need for conversion between formats for different types of processing steps.
- the selection of memory arrangement may be implemented at least partly by changing parameters used by memory-accessing program code. Alternatively the selection of memory arrangement may be done by selecting different versions of code to be executed. These two options may be used together, depending on the number and nature of the different arrangements.
- the invention provides a method of processing image data representing a motion picture sequence within a memory sub-system in an image data processing system, the memory sub-system including processor cache memory in addition to main image data storage memory, the method comprising selectively using cache-handling functions under program control, according to at least one of: measured characteristics of said image data and measured characteristics of the performance of said processing system.
- processor Some types of processor provide for program-controlled cache-handling functions. For example a block allocation function may be provided, or a prefetch operation. .
- the inventors have recognised that, as with the selection of memory arrangement, usage of these will not necessarily be equally advantageous for all image sequences, and could be counter-productive in some circumstances.
- a block allocation function whereby a new cache-block is allocated and overwritten without pre-loading it from the main memory, may be selectively used according to said measured characteristics. For example, it may be possible to use block allocation more often when utilising a tiled format for storage, thus improving processor throughput.
- cache pre-fetching may be activated selectively in accordance with the measured characteristics. Pre-fetch tends to be speculative, and risks wasting bandwidth.
- use of a tiled format means that fewer cache- blocks are being processed, and thus less data that is being pre-fetched is ultimately discarded unused.
- the invention further provides an image data processing system, the processing system including a memory sub-system and means for dynamically selecting the arrangement of image data for successive frames of said sequence in within said memory sub-system according to at least one of: measured characteristics of said image data, measured characteristics of the performance of said processing system, and known characteristics of subsequent processing of said image data within said image processing system.
- the memory sub-system may include an image data storage memory constructed from paged memory.
- Said memory sub-system may include a processor cache memory in addition to a main image data storage memory.
- the means for selecting the arrangement of image data in storage memory may be arranged for selecting between a linear format, whereby image data is stored in memory on a line-by-line basis, and at least one kind of tiled format, whereby two-dimensional groups of pixels are grouped in memory.
- said tiled format may be defined such that data for one tile corresponds to a whole number of cache blocks.
- the selecting means may include means for measuring one or more of the following as characteristics of the image data to influence the selection of the arrangement of data in memory:- A. variability of motion vectors encoded within the received data; B. picture type; C. encoded data size per picture of the sequence; D. structure of a picture from which the image data is derived; E.
- the measuring means may be arranged to look ahead in the motion picture sequence so as to measure said characteristics of the image data for a given portion of the sequence and select the memory arrangement prior to processing that portion.
- the selecting means may be arranged such that measured characteristics of the image data at one part of the sequence are used effectively to predict characteristics of a subsequent portion of the sequence, and the memory arrangement controlled according to measured characteristics of recently processed portions of the sequence.
- the measuring means may include means for averaging measured image data characteristics over a period of time.
- the selecting means may alternatively or in addition comprise means for measuring one or more of the following as characteristics of system performance to influence the selection of the arrangement of data in memory:- G. data cache stall rates in the memory sub-system; H. processor utilisation (for example by measuring processor percentage idle time); I. quality of service, or other such qualitative measurements that are perceptible to the end user of the content being processed; J. bandwidth of a link feeding data into or out of said image processor.
- the means for measuring system performance may be arranged to measure data cache stall rates during image reconstruction.
- the means for measuring system performance may be arranged to do so on a test basis using a sample of data, prior to processing the data.
- the selecting means and means for measuring system performance may be arranged such that system performance measured while processing a first part of the sequence is used to control the arrangement of memory for a subsequent part of the sequence.
- the selecting means may alternatively or in addition be arranged to use knowledge of subsequent processing steps to influence the selection of the arrangement of data in memory.
- the image processing system may be implemented at least in part by program code and a programmable processing unit.
- the selecting means may be implemented by means for changing parameters used by said program code in accessing said memory sub-system. Alternatively the selecting means may be done by selecting different versions of code to be executed.
- the invention further provides an image data processing system including a memory sub-system for storing image data representing a motion picture sequence being processed, the memory sub-system including processor cache memory in addition to main image data storage memory, the system being arranged for using cache- handling functions selectively under program control, according to at least one of: measured characteristics of said image data and measured characteristics of the performance of said processing system.
- the system may be arranged such that a block allocation function, whereby a new cache-block is allocated and overwritten without pre-loading it from the main memory, will be selectively used according to said measured characteristics.
- the system may be arranged such that cache pre-fetching will be activated selectively in accordance with the measured characteristics.
- Figure 1 is a functional block diagram of an MPEG image decoder
- Figure 2 is a block diagram of a processing system implementing the decoder of Figure 1
- Figure 3 illustrates the structure of an MPEG data stream, and the sequence of image frames represented therein, within the decoder of Figures 1 and 2
- Figure 4 illustrates a linear format of image storage in the memory of the decoder of Figures 1 and 2
- Figures 5 and 6 illustrate possible locations of data for a block of pixels in the linear format of image storage
- Figure 7 illustrates a tiled format of image storage in the memory of the decoder of Figures 1 and 2
- Figure 8 illustrates locations of data for a block of pixels in the
- Figure 1 illustrates the various modules of an image processor performing MPEG decoding, and the additional features to support image sequence characteristic analysis and dynamic reconfiguration of image buffers and cache control. Further general considerations and detail of the reconfiguration will be described below under separate headings.
- Input data IN is input to interface block l/F.
- the decoder further comprises configuration block MAPI 90, buffer BUFF 100, Variable Length Decoder (VLD) 110, Inverse Scan (IS) and Inverse Quantiser (IQ) 120, Inverse Discrete Cosine Transform (IDCT) 130, Motion Compensator (MC) 140, Image Memory MEM 150, configuration block MAP2 160, configuration block MAP3 170, Image Memory/Output Interface 180 and statistical processor STATS 190.
- the decoder may be implemented in dedicated hardware, or by a processor as a series of process steps, or a combination of both. The present disclosure assumes that these steps, particularly the VLD and IS/IQ steps are implemented by programming a microprocessor, which may be a general purpose microprocessor with or without media processing extensions.
- the processor is a VLIW processor optimised for image processing, such as Philips Semiconductors' TriMediaTM processor family. The action of the steps shown is as follows:
- Interface, at 80 is for converting the input data to a suitable digital format for MPEG decoding. Additional stages in block l/F 80 but not shown may include analogue to digital conversion, synchronisation signal extraction, gain-control blocks etc, but may also comprise interfaces for converting digital signals, depending upon the format of the input data.
- Input bitstream buffering BUFF is for receiving the video stream and forming a time-flexible queue to allow subsequent process steps to decode the video at differing rates to the input video rate (at this point audio and other data maybe separated and routed to its own decoder).
- Variable Length Decoding is for converting variable-length MPEG tokens to their corresponding values, such as quantised DCT coefficients.
- Inverse Quantisation at 120 obtains actual values for the DCT coefficients according to the values of chosen quantisation and scale factor matrices and an Inverse Scan step re-forms a coefficient matrix from the stream of tokens according to a predetermined sequence, usually a zigzag.
- Inverse Discrete Cosine Transforming step 130 is for recomposing the DCT coefficients into the blocks of pixel values that they represent.
- Configuration blocks MAPI 90, MAP2 160 and MAP3 170 are used for translating address values for modifying the format of data in memory, according to control signals from the STATS block 190. As will be described in more detail below, these blocks can be implemented in software by use of parameterised memory addressing. In this way, the program code defining the processing of the image data can be written independently of the memory format actually being used at execution time, and the format can be changed and changed again simply by changing parameter values.
- Memory block 150 is used by the motion compensator MC 140 to support functions such as prediction, where temporary storage of data is required.
- Image Memory/Output interfacing is performed at 150 to represent the video in a suitable format such as a two dimensions video image in memory or a digital video stream, for example.
- FIG. 2 illustrates a physical embodiment of an image processing system performing MPEG decoding.
- a processor 200 employs a cache 210 between it and the main memory 220.
- Cache 210 and main memory 220 form a memory sub-system.
- a cache system will typically divide the memory space into cache blocks in the same way that paged DRAM divides the available memory into pages. Data is normally transferred to the cache a cache block at a time. Minimising the number of cache blocks touched will often reduce memory system traffic and processor delays.
- CPU 200 may be the main processor of a PC workstation or it may be an auxiliary processor dedicated to MPEG decoding, or to video processing generally, such as the aforementioned TriMediaTM processor.
- the processor and memory may alternatively form part of a dedicated digital video apparatus such as a DVD player, video conference terminal or games console.
- CPU 200 is preferably performing the entire, or at least a substantial part, of the decoding process of Figure 1. To illustrate this, the diagram shows a video bitstream IN 230 entering the processor, although another method for inputting the data would suffice, such as DMA transfers into memory.
- a video output OUT 240 is shown emerging from the memory, although it could in practice require processor interaction to read, format and output the data.
- programs, input data and decoded images generated by CPU are held in main memory 220.
- main memory 220 As is well known, however, often-used data or process steps are loaded into and extracted from the cache 210 (when available), in preference to obtaining the same from the slower memory 220.
- Program code (instructions) for the processor to implement the processes described herein may be pre-installed (firmware) or may be supplied separately on a record carrier such as a CD-ROM or floppy disk 250.
- the use of a cache limited bandwidth of the main memory, provided that memory access operations performed together are generally to neighbouring locations.
- the aim is to dynamically configure the storage of image data in the various memories 100, 150, 180 in the most suitable format to maximise throughput, and particularly to maximise efficiency of the cache.
- the apparatus under stored program control, is capable of dynamically selecting the arrangement of the image data in the memory according to measured characteristics of the image data, measured characteristics of the performance of the processing system (which depends on the characteristics of the image data), and/or characteristics of the subsequent destination of the image data within the image processor.
- the memory being dynamically configured in this embodiment is the main memory 220, as it is used for image data storage for reference pictures. It is possible to also apply the principle other functions of the memory 220, and even the processor cache 210, although the number of products currently available that allow altering of cache geometry is minimal.
- the image data storage memory may be static RAM, but is more likely to be paged DRAM.
- the measured characteristics which may be analysed alone or in various combinations include:- A. Measuring the variability of derived motion vectors. The measurement of variability of motion vectors can be averaged over a period of time, the results being used to determine the most appropriate storage format for the reference pictures to which the motion vectors refer. It may be more useful to measure the variability separately between vertical and horizontal planes. The reason for this is as follows: Considering a linear format, then a horizontal motion vector perturbation of (say) 8 pixels will not be onerous, since the cache block represents a region which is large horizontally (therefore, it is likely that the processor will access data from within the same cache block).
- B Acting on picture type. For example, to maximise efficiency of processing a key picture (I or P), a format that is more efficient for other pictures to predict from can be selected, such as a tiled storage format which will limit data cache stall rates when using the key pictures as reference data for a subsequent P or B picture.
- B pictures might all be stored in a linear format (if that is the desired output format) because those pictures will not be used as reference data in the decoder. If essentially all motion vectors are zero, or constant, then it is possible to have the key pictures stored linearly, however, because the reference data will be needed in a relatively smooth sequence.
- information can be obtained by parsing ahead through the image stream to determine characteristics of subsequent images, such as monitoring their compressed size.
- the compressed size may (either solely, or in combination with other measurements), be assumed to indicate to a large extent, the difficulty of processing the image. This in turn can indicate which storage format would typically be the most appropriate. For example, a relatively small amount of data may indicate an image in a sequence comprising little movement, and this in turn would suggest that linear storage may be the most appropriate.
- G Measuring processor utilisation. At times where processor time is required to perform one or more additional 'background' tasks, the arrangement of image data in the memory should be controlled to minimise processor usage. Another measured characteristic of said processor performance may comprise measuring the percentage of processor idle time.
- H Measuring, or using measurements of, quality of service, or other such qualitative measurements that are perceptible to the end user of the content being processed.
- An example of such would be block-noise caused by (for example) discarding data when processor capacity is inadequate, which produces visible artefacts within a reconstructed image. Block-noise and other such visible artefacts can be reduced by increasing image processor throughput.
- the effects of the dynamic selection of arrangement of the image data in the memory can be monitored over a period of time, the results being used to tailor subsequent storage arrangements and maximise throughput of the image processor and/or quality of the processing.
- a small number of macroblocks can be pre-fetched and processed, with measurements taken to provide performance statistics, the results of which are used to tailor the configuration of the image processor.
- Known microprocessors such as the Philips TriMediaTM, provide the measurements directly by allowing a program to monitor the number of cache misses and failed pre-fetch attempts.
- the arrangement of the image data in the memory can be selected according to the destination of the image in the image processor.
- Figure 3 illustrates the difference between physical and temporal ordering of pictures within a Group-Of-Pictures (GOP) sequence.
- GOP Group-Of-Pictures
- Motion compensation provides an estimate of the content of one picture from the content already decoded for a neighbouring picture or pictures. Therefore a group of pictures (GOP) will typically comprise: an intra-coded "I" picture, which is coded without reference to other pictures; two or three "P" (predictive) coded pictures which are coded using motion vectors based on a preceding I picture; and bi-directional predicted "B" pictures, which are encoded by prediction from I and/or P pictures before and after them in sequence.
- the amount of data required for a B picture is less than that required for a P picture, which in turn is less than that required for an I picture.
- the P and B pictures are encoded only with reference to other pictures, it is only the I pictures which provide an actual entry point for starting playback of a given sequence. Furthermore, it will be noted that the I and P pictures are encoded before the corresponding B pictures, and then re-ordered after decoding so as to achieve the correct presentation order. Decoded image data for the I (and P) pictures must be retained or retrieved into memory for use as reference data in the decoding of P and B pictures. As is well known, information about the structure and coding of the group of pictures is contained within a GOP header 300 at the start of the data, while information about the type and coding of each picture is included in a picture header 302 at the start of the data for each picture. Further discussion now follows, of the different ways in which the memory sub-system is organised controlled to achieve the optimised performance using techniques such as those described above with review.
- Figure 5 shows a block 500 of 17 x 17 pixel data, located within 17 separate cache blocks.
- the position of this 17x17 block of data within paged memory may not be cache-block aligned.
- 17 cache-blocks of data are required in order to have access to all the data. If, as shown in Figure 6, these rows cross a cache-block boundary then 34 cache- blocks are required. It is likely that this cross-block occurrence will happen approximately 25% of the time, therefore the average block count will be around 21.
- a cache-block is fetched, then all 64 bytes of a cache-block will be read into the cache even though only 16 or 17 bytes will actually be needed. If motion vectors do not change between macroblocks, then the data that is unused for that block will be used for succeeding blocks. However in a 'difficult' video sequence where the motion vectors vary widely then this data may well be unused.
- Figures 9 and 10 illustrate a skewed-tiled layout whereby image data is again organised so that each cache-block contains an 8x8 square region ('tile') of luminance data, but each alternate row is skewed by half a tile. This layout can further reduce image memory page crossings.
- Experimental results show, as expected, that the tiled organisation improves performance when motion vectors are highly variable. However, the simple linear organisation works very well where the motion vectors are constant, and overhead in conversion between formats cannot be ignored.
- Cache Control Many processors have instructions which enable operation of the cache to be controlled directly by the programmer. Two examples are described here with reference to the Philips TriMedia PNX1300 processor. Cache control functions are indicated by the line 198 leading from the block STATS 190 to memory subsystem in Figure 1. Other processors may offer equivalent or different options for the programmer to control the cache. Cache Block Allocation When the processor executes a store instruction, the effect is that data in a cache-block will be modified. If the memory data being modified is already in the cache then this happens at full speed. However, if this is not the case, then a new cache-block must be allocated and the modified byte written to that block.
- the program has to be written in such a manner as to be sure that none of the existing data in the memory area covered by the cache-block will be needed again.
- the second (tiled) organisation is far simpler from this point of view.
- Each macroblock decode completely covers four cache-blocks.
- the other organisation partially covers 16 cache-blocks, which means that more computational complexity is required to control the use of "alloc". This may be particularly complex near the start and end of display lines. This complexity will have a computational cost that can diminish or even destroy the gain from using the cache management instruction.
- Pre-fetch This operation requests that the memory system allocates and loads a cache line with data from a specified region of memory. An advantage is that this can be called before the data is required.
- the memory system has enough spare time between the moment that the pre-fetch request is issued and the time that the data is required, than the data will already be pre-fetched and the processor will not be delayed waiting for the data.
- the tile based organisation there are considerably fewer cache-blocks involved, so there is more likely to be time available for the pre-fetches to occur.
- TriMediaTM processor This is a 32 bit processor, therefore it is convenient to handle data groups of four values. Each word in this case represents four adjacent values from one of the images (for example the luminance values for four horizontally adjacent pixels).
- the parameterisation substitutes parameters instead of fixed constants to define the address offset which should be used when moving a certain distance in the required direction. For example to move right four values (one word of data) or to move down one row. It is likely that these parameters can be held in processor registers so that the additional cost of using them instead of constant offsets is small.
- the tiled organisation of Figures 7 and 8 orders the tiles in memory in columns. So the first 64 bytes of the image buffer (the first tile in memory order) describe the top left tile of the image. The second 64 bytes in memory order describe the second tile in the first column of tiles (its first, top left, value is at horizontal pixel position zero and vertical pixel position 8). This continues until the entire column of tiles has been described. The second column of tiles then follows and so on fill then end of the luminance image buffer. With the skewed- tiled format, the tiles are still ordered in columns, but in alternate lines each tile is offset, relative to the others. In this tiled organisation, the memory address change required to move down one row is eight.
- the memory address increment required to move right eight pixels is equal to the size of one column of tiles (72 x 8).
- the result of this scheme of parameterisation is that two very different memory organisations can be handled by the same code given different values for two parameters (address changes to move down one pixel and to move right eight pixels).
- address changes to move down one pixel and to move right eight pixels With skewed- tiled format, clearly an offset to the row address has to be added, depending on which row is being accessed.
- tiles need not be 8X8 square, but could be rectangular, such as 16X4 or 4X16.
- the US patent mentioned in the introduction describes further shapes, including triangular tiles. In some cases alternative versions of relevant code may be needed.
- the parameterised method may not work because is works with blocks of four values.
- An example of this is a 'semi-planar' format in which the chrominance U and V values are interleaved (uO, vO, u1 , v1 , u2, v2, u3, v3).
- uO, vO, u1 , v1 , u2, v2, u3, v3 In this format a set of four generated U values are not written in consecutive bytes so the generic parameterised code cannot work.
- This conversion organisation can be arranged to read and write data efficiently in cache-block order and to use cache optimisation instructions such as "alloc" so that the cost of the conversion can be less than the saving in processing time in the decoder.
- cache optimisation instructions such as "alloc"
- the next processing stage/module in the decoder can be designed to accommodate an unusual structure then there may be no need for this conversion.
- the next step after video decoding might be the blending of image content with subtitle data. This blending process could be adapted to read the video data in tile format with no additional cost.
- I and P pictures are 'key' pictures which means that they can be used as the source for motion prediction.
- B pictures cannot so be used, but are often the most expensive to decode because of the amount of prediction data that is used to construct them.
- the processing cost will be the output image conversion which will happen for I and P pictures which are less of a performance problem.
- B picture processing will gain the benefit of the improved input image buffer format while not suffering the overhead of output image buffer conversion.
- the parameterised addressing structure is useful here to allow the same code to be used in both cases. This reduces code size and code maintenance cost.
- sequences are field structured. This means that each picture coded in the stream is one field of the image (either all the odd lines, or all the even lines).
- the parameterised image buffer format supports analysis of the incoming stream and/or measured decoding performance figures so as to dynamically choose the best image buffer format for the current part of a stream.
- the cache control features can be controlled on the basis of the same measurements or independently, to add to the benefits. The skilled person can readily derive a whole range of algorithms to control the memory subsystem based on the measurements and observations given above.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2006530963A JP2007510320A (en) | 2003-10-04 | 2004-10-01 | Method and apparatus for processing image data |
US10/574,143 US8068545B2 (en) | 2003-10-04 | 2004-10-01 | Method and apparatus for processing image data |
CN2004800289228A CN1864410B (en) | 2003-10-04 | 2004-10-01 | Method and apparatus for processing image data |
EP04770149A EP1673942A1 (en) | 2003-10-04 | 2004-10-01 | Method and apparatus for processing image data |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB0323284.0A GB0323284D0 (en) | 2003-10-04 | 2003-10-04 | Method and apparatus for processing image data |
GB0323284.0 | 2003-10-04 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2005034516A1 true WO2005034516A1 (en) | 2005-04-14 |
Family
ID=29415537
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2004/051944 WO2005034516A1 (en) | 2003-10-04 | 2004-10-01 | Method and apparatus for processing image data |
Country Status (7)
Country | Link |
---|---|
US (1) | US8068545B2 (en) |
EP (1) | EP1673942A1 (en) |
JP (1) | JP2007510320A (en) |
KR (1) | KR20060133966A (en) |
CN (1) | CN1864410B (en) |
GB (1) | GB0323284D0 (en) |
WO (1) | WO2005034516A1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2007035209A1 (en) * | 2005-09-20 | 2007-03-29 | Intel Corporation | Dynamically configuring a video decoder cache for motion compensation |
KR100737741B1 (en) * | 2006-01-05 | 2007-07-10 | 주식회사 텔레칩스 | Memory device using multi-dimensional data prefetch cache, and control method for the same |
JP2008078871A (en) * | 2006-09-20 | 2008-04-03 | Toshiba Corp | Image decoding device, and image decoding method |
WO2008139489A1 (en) * | 2007-05-10 | 2008-11-20 | Allgo Embedded Systems Private Limited | Dynamic motion vector analysis method |
WO2010014696A1 (en) * | 2008-07-29 | 2010-02-04 | Marvell World Trade, Ltd. | Processing rasterized data |
WO2013130864A1 (en) * | 2012-02-28 | 2013-09-06 | Qualcomm Incorporated | Customized buffering at sink device in wireless display system based on application awareness |
EP3051816A1 (en) * | 2015-01-30 | 2016-08-03 | Renesas Electronics Corporation | Image processing device and semiconductor device |
US10986373B2 (en) | 2010-01-19 | 2021-04-20 | Renesas Electronics Corporation | Moving image encoding method, moving image decoding method, moving image encoding device, and moving image decoding device |
Families Citing this family (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7716551B2 (en) * | 2005-12-07 | 2010-05-11 | Microsoft Corporation | Feedback and frame synchronization between media encoders and decoders |
US8068541B2 (en) * | 2006-01-30 | 2011-11-29 | Jan Harding Thomsen | Systems and methods for transcoding bit streams |
US20070177519A1 (en) * | 2006-01-30 | 2007-08-02 | Thomsen Jan H | Systems and methods for transcoding bit streams |
JP4862566B2 (en) * | 2006-09-04 | 2012-01-25 | 富士通株式会社 | Moving image processing apparatus and prefetch control method |
KR100836617B1 (en) | 2006-12-26 | 2008-06-10 | 삼성전자주식회사 | Apparatus and method for remote control in portable communication system |
US20130101023A9 (en) * | 2007-03-12 | 2013-04-25 | Vixs Systems, Inc. | Video encoder with video decoder reuse and method for use therewith |
JP4884290B2 (en) * | 2007-05-07 | 2012-02-29 | パナソニック株式会社 | Moving picture decoding integrated circuit, moving picture decoding method, moving picture decoding apparatus, and moving picture decoding program |
KR101479011B1 (en) * | 2008-12-17 | 2015-01-13 | 삼성전자주식회사 | Method of schedulling multi-band and broadcasting service system using the method |
US8379999B2 (en) * | 2011-01-18 | 2013-02-19 | Chanan Gabay | Methods, circuits, devices, apparatuses and systems for providing image composition rules, analysis and improvement |
US8824569B2 (en) * | 2011-12-07 | 2014-09-02 | International Business Machines Corporation | High bandwidth decompression of variable length encoded data streams |
US8823715B2 (en) | 2012-03-02 | 2014-09-02 | Adobe Systems Incorporated | Efficient writing of pixels to tiled planar pixel arrays |
US8810587B2 (en) | 2012-03-02 | 2014-08-19 | Adobe Systems Incorporated | Conversion of contiguous interleaved image data for CPU readback |
JP6082123B2 (en) * | 2012-11-29 | 2017-02-15 | エルジー エレクトロニクス インコーポレイティド | Video encoding / decoding method supporting multiple layers |
US20150382024A1 (en) * | 2013-05-01 | 2015-12-31 | Lg Electronics Inc. | Apparatus and method of transmitting and receiving signal |
EP2903284A1 (en) | 2013-06-05 | 2015-08-05 | Axis AB | Method for encoding digital video data |
US9374106B2 (en) | 2013-08-28 | 2016-06-21 | International Business Machines Corporation | Efficient context save/restore during hardware decompression of DEFLATE encoded data |
US8933824B1 (en) | 2013-08-28 | 2015-01-13 | International Business Machines Corporation | Hardware decompression of deflate encoded data with multiple blocks |
US9270999B2 (en) | 2013-09-25 | 2016-02-23 | Apple Inc. | Delayed chroma processing in block processing pipelines |
US9305325B2 (en) | 2013-09-25 | 2016-04-05 | Apple Inc. | Neighbor context caching in block processing pipelines |
US9299122B2 (en) | 2013-09-25 | 2016-03-29 | Apple Inc. | Neighbor context processing in block processing pipelines |
US9215472B2 (en) | 2013-09-27 | 2015-12-15 | Apple Inc. | Parallel hardware and software block processing pipelines |
US9218639B2 (en) | 2013-09-27 | 2015-12-22 | Apple Inc. | Processing order in block processing pipelines |
US9571846B2 (en) | 2013-09-27 | 2017-02-14 | Apple Inc. | Data storage and access in block processing pipelines |
US9800640B2 (en) | 2013-10-02 | 2017-10-24 | International Business Machines Corporation | Differential encoder with look-ahead synchronization |
US9807410B2 (en) | 2014-07-02 | 2017-10-31 | Apple Inc. | Late-stage mode conversions in pipelined video encoders |
US9396409B2 (en) | 2014-09-29 | 2016-07-19 | At&T Intellectual Property I, L.P. | Object based image processing |
US9779471B2 (en) * | 2014-10-01 | 2017-10-03 | Qualcomm Incorporated | Transparent pixel format converter |
CN104408055B (en) * | 2014-10-29 | 2018-03-13 | 中国石油天然气股份有限公司 | Storage method and device for laser radar point cloud data |
US10771399B2 (en) * | 2018-07-30 | 2020-09-08 | Intel Corporation | Quality of service-aware processing of decoding tasks |
CN110782389B (en) * | 2019-09-23 | 2023-09-15 | 五八有限公司 | Image data byte alignment method and terminal |
US11367271B2 (en) * | 2020-06-19 | 2022-06-21 | Adobe Inc. | Similarity propagation for one-shot and few-shot image segmentation |
CN114528031A (en) * | 2020-10-30 | 2022-05-24 | 武汉斗鱼鱼乐网络科技有限公司 | Picture display method and related equipment |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5874995A (en) * | 1994-10-28 | 1999-02-23 | Matsuhita Electric Corporation Of America | MPEG video decoder having a high bandwidth memory for use in decoding interlaced and progressive signals |
US5912676A (en) * | 1996-06-14 | 1999-06-15 | Lsi Logic Corporation | MPEG decoder frame memory interface which is reconfigurable for different frame store architectures |
US6023295A (en) * | 1996-09-12 | 2000-02-08 | Sgs-Thomson Microelectronics S.R.L. | ADPCM recompression and decompression of a data stream of a video image and differential variance estimator |
EP1091591A1 (en) * | 1999-09-29 | 2001-04-11 | Matsushita Electric Industrial Co., Ltd. | Allocation of storage sub-areas for frame buffer in an MPEG decoder |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5719624A (en) * | 1989-07-18 | 1998-02-17 | Canon Kabushiki Kaisha | Image recording apparatus with arithmetic processing |
US5420703A (en) * | 1990-02-01 | 1995-05-30 | Canon Kabushiki Kaisha | Color image processing system having multi-image processing capabilities |
JP3302113B2 (en) * | 1993-07-23 | 2002-07-15 | キヤノン株式会社 | Image forming apparatus and output characteristic setting method of image forming apparatus |
US5920352A (en) * | 1994-10-28 | 1999-07-06 | Matsushita Electric Industrial Co., Ltd. | Image memory storage system and method for a block oriented image processing system |
US5909224A (en) * | 1996-10-18 | 1999-06-01 | Samsung Electronics Company, Ltd. | Apparatus and method for managing a frame buffer for MPEG video decoding in a PC environment |
US6005624A (en) | 1996-12-20 | 1999-12-21 | Lsi Logic Corporation | System and method for performing motion compensation using a skewed tile storage format for improved efficiency |
WO1999016252A1 (en) | 1997-09-19 | 1999-04-01 | Sony Electronics Inc. | Motion compensated digital video decoding with buffered picture storage memory map |
US6449390B1 (en) * | 1997-09-24 | 2002-09-10 | Canon Kabushiki Kaisha | Image processing apparatus and method therefor |
EP0908847B1 (en) * | 1997-10-06 | 2006-01-25 | Canon Kabushiki Kaisha | Image synthesis apparatus and image synthesis method |
US6046778A (en) * | 1997-10-29 | 2000-04-04 | Matsushita Electric Industrial Co., Ltd. | Apparatus for generating sub-picture units for subtitles and storage medium storing sub-picture unit generation program |
US6104416A (en) | 1997-11-18 | 2000-08-15 | Stmicroelectronics, Inc. | Tiling in picture memory mapping to minimize memory bandwidth in compression and decompression of data sequences |
JP3595745B2 (en) * | 1999-01-29 | 2004-12-02 | キヤノン株式会社 | Image processing device |
US7012623B1 (en) * | 1999-03-31 | 2006-03-14 | Canon Kabushiki Kaisha | Image processing method and apparatus |
JP3639464B2 (en) * | 1999-07-05 | 2005-04-20 | 株式会社ルネサステクノロジ | Information processing system |
JP2004518343A (en) | 2001-01-12 | 2004-06-17 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Apparatus and method for memory address translation and image processing apparatus including such an apparatus |
JP3962607B2 (en) * | 2002-02-28 | 2007-08-22 | キヤノン株式会社 | Image processing apparatus and method, program, and storage medium |
JP2004096500A (en) * | 2002-08-30 | 2004-03-25 | Konica Minolta Holdings Inc | Image pickup apparatus, image processing apparatus, and image recording apparatus |
-
2003
- 2003-10-04 GB GBGB0323284.0A patent/GB0323284D0/en not_active Ceased
-
2004
- 2004-10-01 EP EP04770149A patent/EP1673942A1/en not_active Withdrawn
- 2004-10-01 KR KR1020067006535A patent/KR20060133966A/en not_active Application Discontinuation
- 2004-10-01 WO PCT/IB2004/051944 patent/WO2005034516A1/en active Application Filing
- 2004-10-01 JP JP2006530963A patent/JP2007510320A/en active Pending
- 2004-10-01 CN CN2004800289228A patent/CN1864410B/en not_active Expired - Fee Related
- 2004-10-01 US US10/574,143 patent/US8068545B2/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5874995A (en) * | 1994-10-28 | 1999-02-23 | Matsuhita Electric Corporation Of America | MPEG video decoder having a high bandwidth memory for use in decoding interlaced and progressive signals |
US5912676A (en) * | 1996-06-14 | 1999-06-15 | Lsi Logic Corporation | MPEG decoder frame memory interface which is reconfigurable for different frame store architectures |
US6023295A (en) * | 1996-09-12 | 2000-02-08 | Sgs-Thomson Microelectronics S.R.L. | ADPCM recompression and decompression of a data stream of a video image and differential variance estimator |
EP1091591A1 (en) * | 1999-09-29 | 2001-04-11 | Matsushita Electric Industrial Co., Ltd. | Allocation of storage sub-areas for frame buffer in an MPEG decoder |
Non-Patent Citations (2)
Title |
---|
FENG W-C ET AL: "IMPROVING DATA CACHING FOR SOFTWARE MPEG VIDEO DECOMPRESSION", PROCEEDINGS OF THE SPIE, SPIE, BELLINGHAM, VA, US, vol. 2668, 31 January 1996 (1996-01-31), pages 94 - 104, XP000617098, ISSN: 0277-786X * |
SODERQUIST P ET AL: "OPTIMIZING THE DATA CACHE PERFORMANCE OF A SOFTWARE MPEG-2 VIDEO DECODER", PROCEEDINGS ACM MULTIMEDIA 97. SEATTLE, NOV. 9 - 13, 1997, READING, ADDISON WESLEY, US, vol. CONF. 5, 9 November 1997 (1997-11-09), pages 291 - 301, XP000765787, ISBN: 0-201-32232-3 * |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2007035209A1 (en) * | 2005-09-20 | 2007-03-29 | Intel Corporation | Dynamically configuring a video decoder cache for motion compensation |
US8867609B2 (en) | 2005-09-20 | 2014-10-21 | Intel Corporation | Dynamically configuring a video decoder cache for motion compensation |
US7924914B2 (en) | 2005-09-20 | 2011-04-12 | Intel Corporation | Dynamically configuring a video decoder cache for motion compensation |
US8208539B2 (en) | 2005-09-20 | 2012-06-26 | Intel Corporation | Dynamically configuring a video decoder cache for motion compensation |
CN102970538A (en) * | 2005-09-20 | 2013-03-13 | 英特尔公司 | Dynamically configuring a video decoder cache for motion compensation |
KR100737741B1 (en) * | 2006-01-05 | 2007-07-10 | 주식회사 텔레칩스 | Memory device using multi-dimensional data prefetch cache, and control method for the same |
JP2008078871A (en) * | 2006-09-20 | 2008-04-03 | Toshiba Corp | Image decoding device, and image decoding method |
US8155204B2 (en) | 2006-09-20 | 2012-04-10 | Kabushiki Kaisha Toshiba | Image decoding apparatus and image decoding method |
WO2008139489A1 (en) * | 2007-05-10 | 2008-11-20 | Allgo Embedded Systems Private Limited | Dynamic motion vector analysis method |
US8300697B2 (en) | 2007-05-10 | 2012-10-30 | Allgo Embedded Systems Private Limited. | Dynamic motion vector analysis method |
US8477146B2 (en) | 2008-07-29 | 2013-07-02 | Marvell World Trade Ltd. | Processing rasterized data |
WO2010014696A1 (en) * | 2008-07-29 | 2010-02-04 | Marvell World Trade, Ltd. | Processing rasterized data |
US9055296B2 (en) | 2008-07-29 | 2015-06-09 | Marvell World Trade Ltd. | Processing rasterized data |
US10986373B2 (en) | 2010-01-19 | 2021-04-20 | Renesas Electronics Corporation | Moving image encoding method, moving image decoding method, moving image encoding device, and moving image decoding device |
WO2013130864A1 (en) * | 2012-02-28 | 2013-09-06 | Qualcomm Incorporated | Customized buffering at sink device in wireless display system based on application awareness |
US8996762B2 (en) | 2012-02-28 | 2015-03-31 | Qualcomm Incorporated | Customized buffering at sink device in wireless display system based on application awareness |
US9167296B2 (en) | 2012-02-28 | 2015-10-20 | Qualcomm Incorporated | Customized playback at sink device in wireless display system |
US9491505B2 (en) | 2012-02-28 | 2016-11-08 | Qualcomm Incorporated | Frame capture and buffering at source device in wireless display system |
EP3051816A1 (en) * | 2015-01-30 | 2016-08-03 | Renesas Electronics Corporation | Image processing device and semiconductor device |
CN105847819A (en) * | 2015-01-30 | 2016-08-10 | 瑞萨电子株式会社 | Image processing device and semiconductor device |
US9906805B2 (en) | 2015-01-30 | 2018-02-27 | Renesas Electronics Corporation | Image processing device and semiconductor device |
CN105847819B (en) * | 2015-01-30 | 2020-03-06 | 瑞萨电子株式会社 | Image processing apparatus and semiconductor apparatus |
Also Published As
Publication number | Publication date |
---|---|
KR20060133966A (en) | 2006-12-27 |
CN1864410B (en) | 2011-04-06 |
US20060291560A1 (en) | 2006-12-28 |
CN1864410A (en) | 2006-11-15 |
US8068545B2 (en) | 2011-11-29 |
JP2007510320A (en) | 2007-04-19 |
GB0323284D0 (en) | 2003-11-05 |
EP1673942A1 (en) | 2006-06-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8068545B2 (en) | Method and apparatus for processing image data | |
US7403564B2 (en) | System and method for multiple channel video transcoding | |
KR100232992B1 (en) | Moving picture decoding circuit | |
US5912676A (en) | MPEG decoder frame memory interface which is reconfigurable for different frame store architectures | |
EP1025692B1 (en) | Computational resource allocation in an information stream decoder | |
KR100881539B1 (en) | Image data structure for direct memory access | |
US20070171979A1 (en) | Method of video decoding | |
US20170019679A1 (en) | Hybrid video decoding apparatus for performing hardware entropy decoding and subsequent software decoding and associated hybrid video decoding method | |
KR100298397B1 (en) | Video decoding system | |
US20030160893A1 (en) | Programmable output control of compressed data from encoder | |
US9363523B2 (en) | Method and apparatus for multi-core video decoder | |
WO2008037113A1 (en) | Apparatus and method for processing video data | |
US6456746B2 (en) | Method of memory utilization in a predictive video decoder | |
US7675972B1 (en) | System and method for multiple channel video transcoding | |
KR101602871B1 (en) | Method and apparatus for data encoding, method and apparatus for data decoding | |
US8335256B2 (en) | Motion compensation in video coding | |
Ling et al. | Real-time video decoding scheme for HDTV set-top boxes | |
KR102171119B1 (en) | Enhanced data processing apparatus using multiple-block based pipeline and operation method thereof | |
KR20030057690A (en) | Apparatus for video decoding | |
Eckart | High performance software MPEG video player for PCs | |
JPH1056641A (en) | Mpeg decoder | |
KR100556341B1 (en) | Vedeo decoder system having reduced memory bandwidth | |
WO1996036178A1 (en) | Multiple sequence mpeg decoder and process for controlling same | |
JP4214554B2 (en) | Video decoding device | |
Jaspers et al. | Embedded compression for memory resource reduction in MPEG systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200480028922.8 Country of ref document: CN |
|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2004770149 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006530963 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1162/CHENP/2006 Country of ref document: IN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006291560 Country of ref document: US Ref document number: 10574143 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020067006535 Country of ref document: KR |
|
WWP | Wipo information: published in national office |
Ref document number: 2004770149 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 1020067006535 Country of ref document: KR |
|
WWP | Wipo information: published in national office |
Ref document number: 10574143 Country of ref document: US |