US20040258147A1 - Memory and array processor structure for multiple-dimensional signal processing - Google Patents
Memory and array processor structure for multiple-dimensional signal processing Download PDFInfo
- Publication number
- US20040258147A1 US20040258147A1 US10/867,027 US86702704A US2004258147A1 US 20040258147 A1 US20040258147 A1 US 20040258147A1 US 86702704 A US86702704 A US 86702704A US 2004258147 A1 US2004258147 A1 US 2004258147A1
- Authority
- US
- United States
- Prior art keywords
- dimensional
- data
- ndos
- signal
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000015654 memory Effects 0.000 title claims abstract description 68
- 238000012545 processing Methods 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000003860 storage Methods 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims abstract description 7
- XYIPZQCDLAFBSX-CMPLNLGQSA-N (4ar,9br)-8-methyl-2,3,4,4a,5,9b-hexahydro-1h-pyrido[4,3-b]indole Chemical compound C1CNC[C@H]2C3=CC(C)=CC=C3N[C@@H]21 XYIPZQCDLAFBSX-CMPLNLGQSA-N 0.000 claims description 9
- 238000012546 transfer Methods 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims 1
- 230000008520 organization Effects 0.000 description 7
- 239000013598 vector Substances 0.000 description 7
- 108010076504 Protein Sorting Signals Proteins 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 102100032062 Neurogenic differentiation factor 2 Human genes 0.000 description 3
- 108050000625 Neurogenic differentiation factor 2 Proteins 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013144 data compression Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000007906 compression Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/557—Motion estimation characterised by stopping computation or iteration based on certain criteria, e.g. error magnitude being too large or early exit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
- G06T7/238—Analysis of motion using block-matching using non-full search, e.g. three-step search
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/129—Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/43—Hardware specially adapted for motion estimation or compensation
- H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/53—Multi-resolution motion estimation; Hierarchical motion estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
Definitions
- This disclosure relates generally to data encoding, storage, distribution, and decoding, and more particularly to a memory and array signal processing structure to process the n-dimensional signal sequence.
- FIG. 1A is a simplified block diagram describing a prior art system for compressing image sequence data with an encoder 1 .
- An image frame 4 is provided as input to the encoder 1 and the data is encoded.
- the encoded frame is then transmitted, and a copy of the encoded frame is decoded at the decoder 10 and stored as a reference frame 6 in the frame buffer 3 .
- the encoder 1 searches the reference frame 6 for a closest match point in the reference frame 6 for each block of a plurality of blocks that make up the current frame.
- the best match location is then represented as a “motion vector” 7 , specifying the two-dimensional location displacement of the block in the reference frame 6 relative to the corresponding block in the current frame.
- the difference (residue) 8 between the best match block in the reference frame 6 and the current image input frame 4 is determined and is typically called the “Residue” or “Block Prediction Difference” (BPD).
- BPD Block Prediction Difference
- a very significant issue is the amount of computational power that is required by an encoder in accomplishing the task of finding the best match for each block in the current frame, i.e., determining the displacement vector (motion vector) such that the displacement block in the reference frame is most “similar” to the current block.
- One prior art method of performing this task involves searching every possible location for block matching within a pre-defined search area. This method requires astronomical computing power and is not feasible for practical real time implementation. There are many simplified methods to search a fraction of the large and complete search space to reduce the computation cost. However, even with the reduced computing cycles, the data access is still the major bottleneck for system performance throughput. This is especially true for multi-dimensional data (2D for images, and 3D for video with multiple frame considerations) that need to be rapidly accessed in a selected pattern.
- the present invention is a memory organization and processing array structure to enable efficient processing of n-dimensional signal frames, including: an n-dimensional object store capable of rapidly storing and accessing of n-dimensional objects, the n-dimensional object store could optionally include a multi-level mass memory structure to store massive amount of data economically, and a signal processor array to process the data in the n-dimensional object store.
- Embodiments of the invention can be used to encode general n-dimensional signal sequences, such as one-dimensional, two-dimensional, and three-dimensional signals.
- One important application of this method is in video encoding for transmission and storage purposes. Because of this, in many of the descriptions below the two-dimensional video signal sequence compression is illustrated. However, the method and apparatus taught here can be extended to compress a general sequence of n-dimensional signals, where n is a positive integer.
- a first aspect of the invention is directed to an objected oriented memory organization and processing array structure to enable efficient processing of n-dimensional signal frames.
- the system includes: an n-dimensional object store capable of rapidly storing and accessing blocks of n-dimensional signals, a multi-level mass memory structure to store a large amount of data before the transfer to the n-dimensional memory, and a signal processor array to process the data in the n-dimensional memory.
- a second aspect of the invention is directed to an n-dimensional signal processing array to process n-dimensional data inputs.
- the processing array includes: an array of signal processing units; a group of data registers, to store the data for the signal processing units; and means for controlling the processing array to allow one data element to be used by more than one processor in the array.
- a third aspect of the invention is directed to a method to operate a n-dimensional memory system for storing and retrieving n-dimensional data in an n-dimensional frame.
- a fourth aspect of the invention is directed to a method to operate a two-dimensional memory system for storing and retrieving two-dimensional data in a two-dimensional frame.
- FIG. 1A illustrates a data compression system, in accordance with the prior art.
- FIG. 1B illustrates a preferred embodiment of a system of the present invention for storage and processing of multi-dimensional signal data.
- FIG. 1C illustrates a 2-dimensional application of the invention to video data arranged in a 2-dimensional frame, in accordance with one embodiment of the present invention.
- FIG. 2 illustrates a block diagram of 2-level memory for n-dimensional storage, in accordance with one embodiment of the invention.
- FIG. 3 illustrates allocation of data for a two-dimensional 3 ⁇ 4 block from a frame, in accordance with one embodiment of the invention.
- FIG. 4 illustrates allocation of data for a two-dimensional 3 ⁇ 4 block from a frame, in accordance with an alternative embodiment of the invention.
- FIG. 5 illustrates a frame buffer data allocation pattern in an SDRAM, in accordance with one embodiment of the invention.
- FIG. 6 illustrates the order of arrangement of pixel data within one block, in accordance with one embodiment of the invention.
- FIG. 7 illustrates a 2-dimensional array implementation of the n dimensional processing engine, in accordance with one embodiment of the invention.
- FIG. 8 shows a spiral search with a step size of 4 pixels, in accordance with one embodiment of the invention.
- FIG. 9 shows a parallel spiral pattern with P search points in parallel, in accordance with one embodiment of the invention.
- FIG. 10 illustrates the method of data sharing with 3 ⁇ 3 array of processing units, in accordance with one embodiment of the invention.
- FIG. 11 illustrates a memory access example, in accordance with one embodiment of the invention.
- N-Dimensional Object Store (NDOS)
- the method taught in that patent enabled the placement of multi-million gate designs with data throughput more than 100 times faster than prior methods without a special memory structure.
- the present invention is a different application of similar concepts taught in that IC placement patent, to solve the n-dimensional encoding and decoding signal processing problems.
- FIG. 1B shows a preferred embodiment of this invention.
- the invention includes applying an “object oriented” principle as used in software design to organize the data and processor structure to solve the encoding and decoding signal processing problems.
- the n dimensional encoding decoding object store (NDOS) 108 organizes and stores the n dimensional signal data in the encoding and decoding space to support high throughput processing for encoding and decoding data processing. Since there is large volume of data to be stored in the encoding and decoding signal data store, 108 can in alternative embodiments be organized according to a hierarchical structure.
- NDMS general mass storage memory
- N-D n dimensional objected oriented n dimensional
- NDFS signal storage memory
- the NDFS 102 functions as a cache to provide data with high throughput to serve encoding and decoding data requests from the encoding and decoding signal processing engine 101 .
- the encoding and decoding data can be pre-loaded into the NDFS 102 from the NDMS 103 through the mass storage bus 109 with some amount of pipelining control (for example, the spiral searching pattern for ME search and the neighborhood preserving scanning pattern described in the co-pending U.S. utility patent application, filed on Jun. 11, 2004, entitled “Method and Apparatus for Adaptive Multiple-Dimensional Signal Sequences Encoding/Decoding,” are two example space data access cases.)
- the n dimensional (N-D) signal processing engine (NDPE) 101 requests the data through the encoding and decoding (N-D) object access bus 105 to request N-D “objects” from the NDOS 108 .
- the requested N-D objects are “higher level” meaningful items to the n dimensional encoding decoding signal processing systems compared to the low level bits and bytes in the traditional data access from traditional memory.
- the NDOS 108 “understands” the data processing applications and some semantics of the data.
- the N-D objects requested from the NDOS 108 will be stored in a data register in the NDRF 104 associated with the NDPE 101 .
- NDRF 104 contains registers (may include data, control, and system status) to be used by the NDPE 101 directly.
- the data transfer between the NDOS 108 , NDPE 101 , and NDRF 104 is through a very high speed N-D data bus (NDDB) 106 .
- NDDB N-D data bus
- FIG. 1C A 2-dimensional case in FIG. 1C is illustrated as an example, in the ME algorithm.
- video data is typically arranged in a 2-dimensional “frame” 131 which shows a picture at any instance on the TV screen. Inside the frame 131 , the data is typically organized in a smaller 2-dimensional blocks 133 . These blocks 133 are usually have a size of 16 ⁇ 16 or 8 ⁇ 8 pixels, but can have other configurations. These blocks 133 are formed with a fixed grid pattern 132 on each frame.
- Video algorithms need to access these blocks in a very efficient way, e.g. get all pixels in a block in one single cycle or one single burst of cycles. Furthermore, video algorithms need to access a 2-dimsional block at any random location not aligned to the fixed grid 133 , as shown in FIG. 1C.
- FIG. 2 shows a specific embodiment of the general N-D object oriented memory structure shown in FIG. 1B.
- the n-dimensional object memory 102 is separated into L slices.
- Each of the memory slices is a traditional 1-dimensional memory (e.g., SRAM).
- the data width of each slice is the minimal element size of the object. In video, this size is a pixel (e.g., 8 bits). In other applications, the bus width of the memory slice can be any size.
- the goal of the L-slice organization is to allow the access of an n-dimensional block in one cycle (if the data block has L elements), or in a burst of multiple access cycles with L elements each. To achieve this, the major issue is how the n-dimensional block data allocated into the L slices. There are two criteria for data allocated to each slice:
- the data elements belonging to the same block should be evenly allocated into L-slice such that the L data elements in the block can be accessed simultaneously without conflict.
- the M data elements are put in a contiguous range on a slice to enable a single burst of block access.
- FIG. 3 One application of the invention is illustrated in FIG. 3.
- a memory slice ID 302 , a row access 304 , and a random block access 306 are shown. In this way, any 3 ⁇ 4 block in the frame can be accessed in a single cycle.
- a memory slice ID 302 , a row access 304 , and a random block access 306 are shown.
- any 3 ⁇ 4 block consists of two elements with the same memory slice ID 302 . That is, the 3 ⁇ 4 block can be accessed in two clock cycles.
- any L pixels in a row access 304 can be accessed in one clock cycle, because there is no slice memory duplication in the set of row pixels.
- the address translation and data multiplexing control in FIG. 2 is designed to reflect the allocation pattern.
- the number of dimension n, the number of block sizes in each dimension, the number of memory slices L can all be parameterized to fit any specific application.
- the video ME algorithm has the following unique set of requirements that differentiates itself from a non-real-time CPU system.
- the second and third requirements can be solved by the memory mechanism described previously.
- the large capacity and low cost solution is not met if the n-dimensional storage mechanism is used alone.
- a large slice number L provides large access bandwidth while increasing the cost at the same time.
- a conventional multi-level cache memory hierarchy can be applied to the n-dimensional memory very well. Note that the high speed and cost of n-dimensional store make a multi-level cache memory hierarchy most suitable for the innermost level of memory closest to the processing engine.
- a 2-level memory embodiment for the n-dimensional store was previously shown in FIG. 2.
- the data is organized such that the data is first read from the second level memory 103 (in this embodiment, a SDRAM is used) and stored in the on-chip n-dimensional store. Once the data is in the n-dimensional store, the data can be accessed flexibly and reused many time. In this way, the demand on the external SDRAM 103 bandwidth and the access pattern flexibility is reduced.
- SDRAM When a SDRAM is used as the second level of memory in 2-level n-dimensional store, some elaboration on the use of SDRAM is needed to support the n-dimensional data structure and overcome the SDRAM architecture limitations. Due to the architecture of a SDRAM design, there are overhead associated with the SDRAM access. Typically, a SDRAM access involves the following steps, each with various delays which incur overhead between bursts:
- the frame buffer data allocation pattern is fixed in the SDRAM as illustrated in FIG. 5.
- a frame buffer 501 is first pre-partitioned into block of a fixed size (16 ⁇ 16, 8 ⁇ 8, or other fixed size) with each block allocated into one bank of SDRAM memory.
- the example in FIG. 5 shows 8 ⁇ 8 blocks 504 and a horizontal row of pixels 502 .
- the blocks 504 are aligned to the fixed grid pattern as explained in 102 of FIG. 1B.
- These blocks 504 are arranged sequentially into the sequential bank ID 508 as shown in FIG. 5.
- the 8 ⁇ 8 pixel data 610 are arranged in a zigzag order 608 shown in FIG. 6.
- Block Burst The whole block is arranged continuously within a bank. Therefore the access of the whole block is done with one single burst.
- Sequential Blocks Burst Multiple blocks burst access in the raster scan order (as shown in FIG. 5) are achieved with multiple bursts. Since each block is allocated into a different bank, these bursts commands are pipelined such that there is no overhead.
- Row Access A row of pixels in the same line can be accessed with multiple bursts. Again, the multiple bursts belongs to different bank, therefore pipelining across burst is possible. Whether there is zero overhead depends on how long is the burst within one block, and depends on CAS and RAS delay of the SDRAM.
- the multi-level N-dimensional store using the SDRAM as the second or higher level of memory allows flexible access to the data, once the data is read from the SDRAM to the n-dimensional store.
- FIG. 7 shows an array processor implementation of an NDPE in FIG. 1B, in accordance with one embodiment of the invention.
- the array structure reduces the reference bandwidth need in a ME algorithm by using a parallel spiral search pattern and array-processors. This approach allows multiple processors to share the same data output from the reference buffer.
- the N-D object data (in this case, a Macro Block—MB) is fetched into the current MB register 701 .
- the 9 processors, J( 0 , 0 )-J( 2 , 2 ) also receive inputs through temp registers 704 , 706 , and 708 , and can process the search objective function evaluation concurrently with the reference frame data coming from the high speed data bus (NDDB) 106 .
- the outputs of the 9 processors, J( 0 , 0 )-J( 2 , 2 ) are received by a motion vector decision making block 710 , which provide motion vectors 712 to the high speed data bus (NDDB) 106 .
- This embodiment of the invention exploits the fixed search/access pattern nature in the ME algorithm.
- One way to share the 2-level memory output is to pre-specify the search pattern in the ME algorithm such that multiple search points are done in parallel.
- the ME algorithm uses various algorithms.
- One embodiment uses a spiral search that follows a pre-specified search trace until it eventually finds the best search point.
- FIG. 8 illustrates a spiral search with a step size of 4 pixels, in accordance with one embodiment of the invention.
- this embodiment of the invention uses a search pattern 804 called a “Parallel Spiral Search” to search pixels 802 .
- the P search points processing in parallel in a fixed pattern, e.g., a 3 ⁇ 3 grid pattern, the input data can be further analyzed to enhance the sharing and reduce the memory bandwidth usage.
- FIG. 10 One embodiment of this concept is shown in FIG. 10.
- Each of the search points in FIG. 10 specifies the location where a cost function evaluation is to be performed.
- the cost function is assumed to be based on a 16 ⁇ 16 size block.
- the search-points 1 , 2 and 3 share 16 pixels out of the 24 pixels input in each row of pixels 802 .
- the data is shared by search-points 1 , 2 , 3 , 4 , 5 , and 6 .
- the ninth row the data is shared by all nine search-points, 1 - 9 .
- the access pattern for reference buffer is fixed and easily designed to reuse the data when it is read out from the buffer. Note that in this array processing architecture based on the parallel spiral search pattern, the search pattern step-size, and the array size in x and y dimensions are all parameters that can be set to any specific value.
- the PSP array processor can also go in a column of data, or a block of data (e.g., 4 ⁇ 4) if a n-dimensional memory is used with the parallel spiral array processor.
- FIG. 11 An embodiment of this combination is shown in FIG. 11.
- the search-points 1 , 2 and 3 share 16 pixels out of the 24 pixels input in each row of pixels 802 .
- the data is shared by search-points: 1 , 2 , 3 , 4 , 5 , and 6 .
- the data is shared by all nine search-points: 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , and 9 .
- the array processor architecture can be used alone, or with the n-dimensional memory as taught.
- the usage of the “Parallel Spiral Pattern with Array processor” with the “2-level Memory” enables a more efficient implementation of ME algorithm to search many more points as compared with traditional single spiral point search pattern, and therefore achieve much higher compression performance.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Discrete Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
An object oriented n-dimensional signal object store and processing array structure to enable efficient processing of n-dimensional signal data, including: a fast n-dimensional signal storage memory capable of rapidly storing and accessing n-dimensional signal objects, a multi-level mass memory structure to store massive amounts of data before transferring to the fast n-dimensional signal storage memory, and an n-dimensional signal processor array to process the n-dimensional signal object data in the n-dimensional singal object store.
Description
- This application claims priority from a U.S. provisional patent application, Ser. No. 60/480,985, filed on Jun. 23, 2003, entitled “Method and Apparatus for Adaptive Multiple-Dimensional Signal Sequences Encoding/Decoding,” which is hereby incorporated by reference. This application is related to a co-pending U.S. utility patent application, filed on Jun. 14, 2004, entitled “Method and Apparatus for Adaptive Multiple-Dimensional Signal Sequences Encoding/Decoding,” which is hereby incorporated by reference.
- 1. Field of the Invention
- This disclosure relates generally to data encoding, storage, distribution, and decoding, and more particularly to a memory and array signal processing structure to process the n-dimensional signal sequence.
- 2. Description of the Prior Art
- Digital data systems are frequently challenged to handle large quantities of data quickly enough to meet practical needs. Compact disc music requires about 1500 kilobits per second, and video needs over 200,000 kilobits per second. Both transmission and storage of data is costly, and in some cases impossible. For example, a current telephone line modem can only carry maximum bit rate at 56 kilobits per second with a perfect line condition. Although video image frames need only be handled at approximately 30 cycles per second in order to allow an observer to have the impression of continual image transmission, the data content of each image frame is very large.
- Solutions to the problem of quickly handling large quantities of data have been developed by using methods of data compression, i.e., methods of reducing the quantity of bits required. Data compression has made possible technological developments including digital television, DVD movie, streaming Internet video, home digital photography and video conferencing. Compressing coders and decoders (CODECs) are used to encode (at the capturing/production side) and decode (at the receiving/reproduction side) data containing statistical redundancy.
- FIG. 1A is a simplified block diagram describing a prior art system for compressing image sequence data with an
encoder 1. Animage frame 4 is provided as input to theencoder 1 and the data is encoded. The encoded frame is then transmitted, and a copy of the encoded frame is decoded at thedecoder 10 and stored as areference frame 6 in theframe buffer 3. With the currentinput image frame 4 as input, theencoder 1 then searches thereference frame 6 for a closest match point in thereference frame 6 for each block of a plurality of blocks that make up the current frame. This includes the use of a motion estimator (ME) 5, and requires a calculation of what is called an “energy difference” measure, such as sum of square error or sum of absolute error between the current frame block and corresponding reference frame block located at each search point in the reference frame. The best match location is then represented as a “motion vector” 7, specifying the two-dimensional location displacement of the block in thereference frame 6 relative to the corresponding block in the current frame. Also, the difference (residue) 8 between the best match block in thereference frame 6 and the currentimage input frame 4 is determined and is typically called the “Residue” or “Block Prediction Difference” (BPD). Both themotion vector 7 and theresidue 8 are then encoded and transmitted. Theencoder 1 will then decode themotion vector 7 andresidue 8 and reconstruct the current frame in the same way that a decoder receiving the same data would reconstruct the frame, and then store this frame as a reference frame. - A very significant issue is the amount of computational power that is required by an encoder in accomplishing the task of finding the best match for each block in the current frame, i.e., determining the displacement vector (motion vector) such that the displacement block in the reference frame is most “similar” to the current block. One prior art method of performing this task involves searching every possible location for block matching within a pre-defined search area. This method requires astronomical computing power and is not feasible for practical real time implementation. There are many simplified methods to search a fraction of the large and complete search space to reduce the computation cost. However, even with the reduced computing cycles, the data access is still the major bottleneck for system performance throughput. This is especially true for multi-dimensional data (2D for images, and 3D for video with multiple frame considerations) that need to be rapidly accessed in a selected pattern.
- The present invention is a memory organization and processing array structure to enable efficient processing of n-dimensional signal frames, including: an n-dimensional object store capable of rapidly storing and accessing of n-dimensional objects, the n-dimensional object store could optionally include a multi-level mass memory structure to store massive amount of data economically, and a signal processor array to process the data in the n-dimensional object store. Embodiments of the invention can be used to encode general n-dimensional signal sequences, such as one-dimensional, two-dimensional, and three-dimensional signals. One important application of this method is in video encoding for transmission and storage purposes. Because of this, in many of the descriptions below the two-dimensional video signal sequence compression is illustrated. However, the method and apparatus taught here can be extended to compress a general sequence of n-dimensional signals, where n is a positive integer.
- A first aspect of the invention is directed to an objected oriented memory organization and processing array structure to enable efficient processing of n-dimensional signal frames. The system includes: an n-dimensional object store capable of rapidly storing and accessing blocks of n-dimensional signals, a multi-level mass memory structure to store a large amount of data before the transfer to the n-dimensional memory, and a signal processor array to process the data in the n-dimensional memory.
- A second aspect of the invention is directed to an n-dimensional signal processing array to process n-dimensional data inputs. The processing array includes: an array of signal processing units; a group of data registers, to store the data for the signal processing units; and means for controlling the processing array to allow one data element to be used by more than one processor in the array.
- A third aspect of the invention is directed to a method to operate a n-dimensional memory system for storing and retrieving n-dimensional data in an n-dimensional frame. The method includes: storing one data item into each slice of L memory slices, where L is a positive integer; organizing the n-dimensional data to allow all the data in a given cube, which can be located anywhere in the frame, to be accessed in M=B/L cycles, where B is the total number of points inside the cube; accessing the data from the L memory slices based on n-dimensional address inputs from an addressing translation module; and providing data flow from the L-slices through a data multiplexer and data de-multiplexer to outside processing modules using the n-dimensional data.
- A fourth aspect of the invention is directed to a method to operate a two-dimensional memory system for storing and retrieving two-dimensional data in a two-dimensional frame. The method includes: storing one data item into each slice of L memory slices, where L is a positive integer; organizing the two-dimensional data to allow all the data in a given cube, which can be located anywhere in the frame, to be accessed in M=B/L cycles, where B is the total number of points inside the cube; accessing the data from the L memory slices based on two-dimensional address inputs from an addressing translation module; and providing data flow from the L-slices through a data multiplexer and data de-multiplexer to outside processing modules using the two-dimensional data.
- FIG. 1A illustrates a data compression system, in accordance with the prior art.
- FIG. 1B illustrates a preferred embodiment of a system of the present invention for storage and processing of multi-dimensional signal data.
- FIG. 1C illustrates a 2-dimensional application of the invention to video data arranged in a 2-dimensional frame, in accordance with one embodiment of the present invention.
- FIG. 2 illustrates a block diagram of 2-level memory for n-dimensional storage, in accordance with one embodiment of the invention.
- FIG. 3 illustrates allocation of data for a two-dimensional 3×4 block from a frame, in accordance with one embodiment of the invention.
- FIG. 4 illustrates allocation of data for a two-dimensional 3×4 block from a frame, in accordance with an alternative embodiment of the invention.
- FIG. 5 illustrates a frame buffer data allocation pattern in an SDRAM, in accordance with one embodiment of the invention.
- FIG. 6 illustrates the order of arrangement of pixel data within one block, in accordance with one embodiment of the invention.
- FIG. 7 illustrates a 2-dimensional array implementation of the n dimensional processing engine, in accordance with one embodiment of the invention.
- FIG. 8 shows a spiral search with a step size of 4 pixels, in accordance with one embodiment of the invention.
- FIG. 9 shows a parallel spiral pattern with P search points in parallel, in accordance with one embodiment of the invention.
- FIG. 10 illustrates the method of data sharing with 3×3 array of processing units, in accordance with one embodiment of the invention.
- FIG. 11 illustrates a memory access example, in accordance with one embodiment of the invention.
- N-Dimensional Object Store (NDOS)
- The implementation of block matching and motion estimation typically has a bottleneck at accessing the blocks from any location in the reference frames. With the video streams typically organized in 2-dimensional or 3-dimensional objects in video algorithms, the use of 1-dimensional linear addressing based memory does not provide efficient results.
- To address this problem, what we need is some kind of specially organized memory data organization and processor structure to handle some special data access and processing requirements in the n-dimensional signal space efficiently with high throughput. Special memory organization and processing methods to address data access throughput issues for some space oriented data requests are known. For example, U.S. Pat. No. 5,818,726, entitledSystem and Method for Determining Acceptable Logic Cell Locations and Generating a Legal Location Structure, by Tsu-Chang Lee, issued on Oct. 6th, 1998, demonstrates the solution of the layout placement location searching problem with a special organized cell array structure and processing method to achieve very rapid IC placement data throughput. The method taught in that patent enabled the placement of multi-million gate designs with data throughput more than 100 times faster than prior methods without a special memory structure. The present invention is a different application of similar concepts taught in that IC placement patent, to solve the n-dimensional encoding and decoding signal processing problems.
- FIG. 1B shows a preferred embodiment of this invention. The invention includes applying an “object oriented” principle as used in software design to organize the data and processor structure to solve the encoding and decoding signal processing problems. There are three main components in the encoding and decoding memory/processing system shown in FIG. 1B. The n dimensional encoding decoding object store (NDOS)108, organizes and stores the n dimensional signal data in the encoding and decoding space to support high throughput processing for encoding and decoding data processing. Since there is large volume of data to be stored in the encoding and decoding signal data store, 108 can in alternative embodiments be organized according to a hierarchical structure. Inside
NDOS 108, the encoding and decoding (n dimensional: N-D) signal data is stored in a general mass storage memory (NDMS) 103. A chunk of locally used encoding and decoding signal data is decomposed and organized into a fast n dimensional objected oriented n dimensional (N-D) signal storage memory (NDFS) 102. TheNDFS 102 functions as a cache to provide data with high throughput to serve encoding and decoding data requests from the encoding and decodingsignal processing engine 101. Since most of the encoding and decoding signal processing follows some kind of pre-defined “navigation” pattern through the encoding and decoding space, the encoding and decoding data can be pre-loaded into theNDFS 102 from theNDMS 103 through themass storage bus 109 with some amount of pipelining control (for example, the spiral searching pattern for ME search and the neighborhood preserving scanning pattern described in the co-pending U.S. utility patent application, filed on Jun. 11, 2004, entitled “Method and Apparatus for Adaptive Multiple-Dimensional Signal Sequences Encoding/Decoding,” are two example space data access cases.) - The n dimensional (N-D) signal processing engine (NDPE)101 requests the data through the encoding and decoding (N-D) object
access bus 105 to request N-D “objects” from theNDOS 108. Note here that the requested N-D objects are “higher level” meaningful items to the n dimensional encoding decoding signal processing systems compared to the low level bits and bytes in the traditional data access from traditional memory. In other words, theNDOS 108 “understands” the data processing applications and some semantics of the data. The N-D objects requested from theNDOS 108 will be stored in a data register in theNDRF 104 associated with theNDPE 101.NDRF 104 contains registers (may include data, control, and system status) to be used by theNDPE 101 directly. The data transfer between theNDOS 108,NDPE 101, andNDRF 104 is through a very high speed N-D data bus (NDDB) 106. - NDOS Implementation
- As a specific implementation of the preferred embodiment specified above, details follow below for constructing an object oriented n-dimensional memory store, based on a traditional 1-dimensional addressing based memory to optimize the memory access efficiency and access pattern flexibility for ME algorithm frame buffer accessing. However, the use of this structure is not limited to the ME algorithm. Any n-dimensional data processing can use this mechanism for the flexibility and efficiency advantages.
- This memory access problem is illustrated in FIG. 1C. A 2-dimensional case in FIG. 1C is illustrated as an example, in the ME algorithm. In a video application, video data is typically arranged in a 2-dimensional “frame”131 which shows a picture at any instance on the TV screen. Inside the
frame 131, the data is typically organized in a smaller 2-dimensional blocks 133. Theseblocks 133 are usually have a size of 16×16 or 8×8 pixels, but can have other configurations. Theseblocks 133 are formed with a fixedgrid pattern 132 on each frame. - Video algorithms need to access these blocks in a very efficient way, e.g. get all pixels in a block in one single cycle or one single burst of cycles. Furthermore, video algorithms need to access a 2-dimsional block at any random location not aligned to the fixed
grid 133, as shown in FIG. 1C. - Currently, electronic memories (SDRAM, SRAM, etc.) are organized in a 1-dimensional based addressing mechanism that allows at best a simultaneous access/burst of pixels in a linear way, i.e., a row of pixels. With some pre-arrangement of the pixel data allocation in the memory, it is possible to burst access a block aligned to the fixed grid pattern in the frame. However, it is not possible to allow access in one cycle/burst of a random located block. One embodiment of the invention provides a structure to solve this problem.
- FIG. 2 shows a specific embodiment of the general N-D object oriented memory structure shown in FIG. 1B. In this block diagram, the n-
dimensional object memory 102 is separated into L slices. Each of the memory slices is a traditional 1-dimensional memory (e.g., SRAM). The data width of each slice is the minimal element size of the object. In video, this size is a pixel (e.g., 8 bits). In other applications, the bus width of the memory slice can be any size. The goal of the L-slice organization is to allow the access of an n-dimensional block in one cycle (if the data block has L elements), or in a burst of multiple access cycles with L elements each. To achieve this, the major issue is how the n-dimensional block data allocated into the L slices. There are two criteria for data allocated to each slice: - The data elements belonging to the same block should be evenly allocated into L-slice such that the L data elements in the block can be accessed simultaneously without conflict.
- If the number of slice L is less then the number of data element in a block, say B=L*M, where B is the number of elements in a block, then there are multiple elements (M) of a block residing in the same slice. In one embodiment, the M data elements are put in a contiguous range on a slice to enable a single burst of block access.
- One application of the invention is illustrated in FIG. 3. In FIG. 3, a 2-dimensional block of 3×4 with L=12 is shown to show the allocation of data. A
memory slice ID 302, arow access 304, and arandom block access 306 are shown. In this way, any 3×4 block in the frame can be accessed in a single cycle. - Another application of the invention with L=6 and M=2 is illustrated in FIG. 4. A
memory slice ID 302, arow access 304, and arandom block access 306 are shown. In this case, any 3×4 block consists of two elements with the samememory slice ID 302. That is, the 3×4 block can be accessed in two clock cycles. In addition, as observed in FIG. 3 and FIG. 4, any L pixels in arow access 304 can be accessed in one clock cycle, because there is no slice memory duplication in the set of row pixels. - Once the data allocation is done properly, the address translation and data multiplexing control in FIG. 2 is designed to reflect the allocation pattern. Note that in one embodiment of the invention, the number of dimension n, the number of block sizes in each dimension, the number of memory slices L can all be parameterized to fit any specific application.
- Multi-Level N-dimensional Signal Storage Memory
- The video ME algorithm has the following unique set of requirements that differentiates itself from a non-real-time CPU system.
- 1. Large Capacity
- 2. Large Bandwidth
- 3. Random Access of 2-dimensional data elements
- 4. Low Cost
- Among these requirements, the second and third requirements can be solved by the memory mechanism described previously. However, the large capacity and low cost solution is not met if the n-dimensional storage mechanism is used alone. Furthermore, a large slice number L provides large access bandwidth while increasing the cost at the same time.
- A conventional multi-level cache memory hierarchy can be applied to the n-dimensional memory very well. Note that the high speed and cost of n-dimensional store make a multi-level cache memory hierarchy most suitable for the innermost level of memory closest to the processing engine.
- A 2-level memory embodiment for the n-dimensional store was previously shown in FIG. 2. In this mechanism, the data is organized such that the data is first read from the second level memory103 (in this embodiment, a SDRAM is used) and stored in the on-chip n-dimensional store. Once the data is in the n-dimensional store, the data can be accessed flexibly and reused many time. In this way, the demand on the
external SDRAM 103 bandwidth and the access pattern flexibility is reduced. - When a SDRAM is used as the second level of memory in 2-level n-dimensional store, some elaboration on the use of SDRAM is needed to support the n-dimensional data structure and overcome the SDRAM architecture limitations. Due to the architecture of a SDRAM design, there are overhead associated with the SDRAM access. Typically, a SDRAM access involves the following steps, each with various delays which incur overhead between bursts:
- Pre-Charge of a previously accessed memory bank
- Sending a RAS command.
- Sending a CAS command
- Without a proper arrangement of the pixel data, the overhead between burst accesses can be very high. On the other hand, the SDRAM provides memory organization of multiple banks allows command issuing and pre-charge independently. With a proper organization of pixel data of a frame, the SDRAM access overhead can be minimized. To do this, the frame buffer data allocation pattern is fixed in the SDRAM as illustrated in FIG. 5. A
frame buffer 501 is first pre-partitioned into block of a fixed size (16×16, 8×8, or other fixed size) with each block allocated into one bank of SDRAM memory. The example in FIG. 5shows 8×8blocks 504 and a horizontal row ofpixels 502. Theblocks 504 are aligned to the fixed grid pattern as explained in 102 of FIG. 1B. Theseblocks 504 are arranged sequentially into thesequential bank ID 508 as shown in FIG. 5. Within one block, the 8×8 pixel data 610 are arranged in azigzag order 608 shown in FIG. 6. - With this, the access patterns to the SDRAM listed in the following are done with zero-overhead:
- Block Burst—The whole block is arranged continuously within a bank. Therefore the access of the whole block is done with one single burst.
- Sequential Blocks Burst—Multiple blocks burst access in the raster scan order (as shown in FIG. 5) are achieved with multiple bursts. Since each block is allocated into a different bank, these bursts commands are pipelined such that there is no overhead.
- Row Access—A row of pixels in the same line can be accessed with multiple bursts. Again, the multiple bursts belongs to different bank, therefore pipelining across burst is possible. Whether there is zero overhead depends on how long is the burst within one block, and depends on CAS and RAS delay of the SDRAM.
- Even though the access to the external SDRAM has very limited access pattern, the multi-level N-dimensional store using the SDRAM as the second or higher level of memory allows flexible access to the data, once the data is read from the SDRAM to the n-dimensional store.
- Parallel Spiral Pattern (PSP) Array Processors for ME Search
- FIG. 7 shows an array processor implementation of an NDPE in FIG. 1B, in accordance with one embodiment of the invention. The array structure reduces the reference bandwidth need in a ME algorithm by using a parallel spiral search pattern and array-processors. This approach allows multiple processors to share the same data output from the reference buffer. Here the N-D object data (in this case, a Macro Block—MB) is fetched into the
current MB register 701. The 9 processors, J(0,0)-J(2,2), also receive inputs throughtemp registers decision making block 710, which providemotion vectors 712 to the high speed data bus (NDDB) 106. - This embodiment of the invention exploits the fixed search/access pattern nature in the ME algorithm. One way to share the 2-level memory output is to pre-specify the search pattern in the ME algorithm such that multiple search points are done in parallel. Traditionally, the ME algorithm uses various algorithms. One embodiment uses a spiral search that follows a pre-specified search trace until it eventually finds the best search point.
- FIG. 8 illustrates a spiral search with a step size of 4 pixels, in accordance with one embodiment of the invention. In order to allow the parallelism of search with fixed access memory access pattern, this embodiment of the invention uses a
search pattern 804 called a “Parallel Spiral Search” to searchpixels 802. - FIG. 9 shows an example of the parallel spiral pattern902 with P search points in parallel, with P=9 in this example to search
pixels 802. With the P search points processing in parallel in a fixed pattern, e.g., a 3×3 grid pattern, the input data can be further analyzed to enhance the sharing and reduce the memory bandwidth usage. - One embodiment of this concept is shown in FIG. 10. Each of the search points in FIG. 10 specifies the location where a cost function evaluation is to be performed. In this case, the cost function is assumed to be based on a 16×16 size block. The search-
points share 16 pixels out of the 24 pixels input in each row ofpixels 802. In this way, when the first row is read from the reference buffer, it is shared by all three search-points row 5, the data is shared by search-points - PSP Array Processors with N-dimensional Memory Storage for ME Search
- Alternately, the PSP array processor can also go in a column of data, or a block of data (e.g., 4×4) if a n-dimensional memory is used with the parallel spiral array processor. An embodiment of this combination is shown in FIG. 11. Once again, the search-
points share 16 pixels out of the 24 pixels input in each row ofpixels 802. In this way, when the first row is read from the reference buffer, it is shared by all three search-points: 1, 2, and 3. Starting fromrow 5, the data is shared by search-points: 1, 2, 3, 4, 5, and 6. Starting from the ninth row, the data is shared by all nine search-points: 1, 2, 3, 4, 5, 6, 7, 8, and 9. - The use of parallel spiral array processor with the n-dimensional store provides a better performance. Without the n-dimensional store, only a row or column of data is read and shared by the array processor. Assuming that the reference buffer has a data width of 16 pixels providing input data of 16 pixels at a time, consider the case in FIG. 11. If there is no n-dimensional store available, only a row or a column of 16 pixels are read at a time. To access the total of 24 rows of 24 pixels each, 48 cycles is needed and is shared by 9 processors. In this way, the number of cycles per processor is 48/9=5.33.
- If an n-dimensional store is available to allow access of a 4×4 block in one cycle, a total of 36 cycles is needed. The number of cycle per processor in this case is 36/9=4. Note that without the PSP and array processor, the number of cycle is 16 cycles per processor. The performance improves from 16 to 5.33 for PSP processor alone, and to 4 for PSP with n-dimensional store.
- In summary, the array processor architecture can be used alone, or with the n-dimensional memory as taught. The usage of the “Parallel Spiral Pattern with Array processor” with the “2-level Memory” enables a more efficient implementation of ME algorithm to search many more points as compared with traditional single spiral point search pattern, and therefore achieve much higher compression performance.
- In the description herein, numerous specific details are provided, such as the description of system components and methods, to provide a thorough understanding of embodiments of the invention. One skilled in relevant arts will recognize, however, that the invention can be practiced without one or more of the specific details, or with other systems, methods, components, materials, parts, and the like. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
- Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Claims (15)
1. An object oriented n-dimensional memory system (n is a positive integer) for storing and retrieving n-dimensional signal object in an n-dimensional signal space, comprising:
an n-dimensional object oriented signal object data store (NDOS), which contains at least one fast object oriented encoding and decoding signal data storage memory (NDFS) for high throughput encoding and decoding (N-D) signal object data transfer; and
means for organizing the n-dimensional data into the NDOS, to allow access for certain n-dimensional signal object data operations.
2. The NDOS of claim 1 , further comprising:
at least one level of general mass storage memory (NDMS); and
means for data loading at least one NDFS from at least one NDMS to support a high throughput N-D signal object data operation.
3. The system in claim 2 , wherein the at least one NDMS includes at least one SDRAM.
4. The system of claim 1 , wherein the memory system is adapted for use in storing two-dimensional images.
5. The system of claim 1 , wherein the memory system is adapted for use in storing three-dimensional video sequences.
6. An n-dimensional signal processing system to process n-dimensional data inputs, comprising:
an n-dimensional encoding decoding signal processing engine (NDPE);
a plurality of data registers, to store data for the NDPE; and
means for controlling the NDPE to allow at least one n-dimensional signal object to be used efficiently in NDPE.
7. The system of claim 6 , further comprising: a motion estimator to explore a plurality of reference frames through a parallel spiral search.
8. The system in claim 6 , further comprising: an NDOS to output n-dimensional signal object.
9. The system in claim 8 , wherein the NDOS has more than one level.
10. The system in claim 7 , wherein the motion estimator evaluates at least one system objective on multiple points in the reference frame(s) using at least one NDPE structure.
11. The system of claim 6 , wherein the system is adapted to encoding two-dimensional video sequences.
12. The system of claim 10 , wherein the system is adapted to encoding two-dimensional video sequences.
13. The system in claim 10 , wherein the n-dimensional encoding decoding signal processing engine (NDPE) is composed of a 3×3 array of processors.
14. A method to operate an n-dimensional object oriented signal object data store (NDOS) for storing and retrieving n-dimensional signal data in an n-dimensional frame sequence, comprising:
storing an n-dimensional signal object data into the NDOS;
organizing the n-dimensional signal data in the NDOS to allow high throughput access;
accessing the data from the NDOS, based on an n-dimensional object request from at least one processor; and
providing the data from the NDOS to at least one outside processing module requesting the n-dimensional object data.
15. A method to operate a two-dimensional memory system for storing and retrieving two-dimensional data in a two-dimensional frame, comprising:
storing two-dimensional signal object data into an n-dimensional object oriented signal data object store (NDOS);
organizing the two-dimensional data in the NDOS to allow high throughput access;
accessing the two-dimensional data from the NDOS based on a two-dimensional object request from at least one processor; and
providing data from the NDOS to at least one outside processing module requesting the two-dimensional object data.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/867,027 US20040258147A1 (en) | 2003-06-23 | 2004-06-14 | Memory and array processor structure for multiple-dimensional signal processing |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US48098503P | 2003-06-23 | 2003-06-23 | |
US10/867,027 US20040258147A1 (en) | 2003-06-23 | 2004-06-14 | Memory and array processor structure for multiple-dimensional signal processing |
Publications (1)
Publication Number | Publication Date |
---|---|
US20040258147A1 true US20040258147A1 (en) | 2004-12-23 |
Family
ID=33551961
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/867,447 Expired - Fee Related US7471724B2 (en) | 2003-06-23 | 2004-06-14 | Method and apparatus for adaptive multiple-dimensional signal sequences encoding/decoding |
US10/867,027 Abandoned US20040258147A1 (en) | 2003-06-23 | 2004-06-14 | Memory and array processor structure for multiple-dimensional signal processing |
US10/875,258 Expired - Fee Related US7499491B2 (en) | 2003-06-23 | 2004-06-23 | Apparatus for adaptive multiple-dimentional signal sequences encoding/decoding |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/867,447 Expired - Fee Related US7471724B2 (en) | 2003-06-23 | 2004-06-14 | Method and apparatus for adaptive multiple-dimensional signal sequences encoding/decoding |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/875,258 Expired - Fee Related US7499491B2 (en) | 2003-06-23 | 2004-06-23 | Apparatus for adaptive multiple-dimentional signal sequences encoding/decoding |
Country Status (5)
Country | Link |
---|---|
US (3) | US7471724B2 (en) |
EP (1) | EP1644896A2 (en) |
JP (1) | JP2007525078A (en) |
CN (1) | CN1809839A (en) |
WO (2) | WO2005001773A2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100332547A1 (en) * | 2009-06-24 | 2010-12-30 | Nokia Corporation | Method and apparatus for retrieving nearby data |
US20160055120A1 (en) * | 2002-02-05 | 2016-02-25 | Pact Xpp Technologies Ag | Integrated data processing core and array data processor and method for processing algorithms |
US20220121573A1 (en) * | 2019-12-10 | 2022-04-21 | Pony Ai Inc. | Memory architecture for efficient spatial-temporal data storage and access |
US20220318171A1 (en) * | 2021-04-01 | 2022-10-06 | quadric.io, Inc. | Systems and methods for intelligently implementing concurrent transfers of data within a machine perception and dense algorithm integrated circuit |
Families Citing this family (96)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7180947B2 (en) * | 2003-03-31 | 2007-02-20 | Planning Systems Incorporated | Method and apparatus for a dynamic data correction appliance |
US9330060B1 (en) | 2003-04-15 | 2016-05-03 | Nvidia Corporation | Method and device for encoding and decoding video image data |
US8660182B2 (en) | 2003-06-09 | 2014-02-25 | Nvidia Corporation | MPEG motion estimation based on dual start points |
DE602004017180D1 (en) * | 2003-06-27 | 2008-11-27 | Nxp Bv | CODING METHOD FOR PORTABLE DEVICES |
EP1538844A3 (en) * | 2003-11-26 | 2006-05-31 | Samsung Electronics Co., Ltd. | Color image residue transformation and encoding method |
JP4031455B2 (en) * | 2004-03-29 | 2008-01-09 | 株式会社東芝 | Image encoding device |
US8665955B2 (en) * | 2004-06-11 | 2014-03-04 | Nxp, B.V. | Method of storing pictures in a memory using compression coding and cost function including power consumption |
JP4175296B2 (en) * | 2004-06-25 | 2008-11-05 | キャタピラージャパン株式会社 | Construction machine data processing apparatus and construction machine data processing method |
SE0401812D0 (en) * | 2004-07-08 | 2004-07-08 | Anoto Ab | Method in creating a symbol pattern, symbol pattern obtained thereby method and system for finding a position in such symbol pattern and computer program product for performing the method |
KR20060043120A (en) * | 2004-09-23 | 2006-05-15 | 엘지전자 주식회사 | Method for encoding and decoding video signal |
KR100694059B1 (en) * | 2004-09-30 | 2007-03-12 | 삼성전자주식회사 | Method and apparatus for encoding and decoding in inter mode based on multi time scan |
KR100647294B1 (en) | 2004-11-09 | 2006-11-23 | 삼성전자주식회사 | Method and apparatus for encoding and decoding image data |
EP1689192A3 (en) * | 2005-02-04 | 2011-10-26 | Thomson Licensing | Method and device for scanning in a data frame |
US7536487B1 (en) * | 2005-03-11 | 2009-05-19 | Ambarella, Inc. | Low power memory hierarchy for high performance video processor |
JP2008536414A (en) * | 2005-04-13 | 2008-09-04 | ゴットフリート・ヴィルヘルム・ライプニッツ・ウニヴェルジテート・ハノーヴァー | Video extended encoding method and apparatus |
US7822278B1 (en) * | 2005-09-20 | 2010-10-26 | Teradici Corporation | Methods and apparatus for encoding a digital video signal |
US10585942B2 (en) * | 2005-09-14 | 2020-03-10 | Millennial Media Llc | Presentation of search results to mobile devices based on viewing history |
US10911894B2 (en) | 2005-09-14 | 2021-02-02 | Verizon Media Inc. | Use of dynamic content generation parameters based on previous performance of those parameters |
US10038756B2 (en) | 2005-09-14 | 2018-07-31 | Millenial Media LLC | Managing sponsored content based on device characteristics |
US9703892B2 (en) | 2005-09-14 | 2017-07-11 | Millennial Media Llc | Predictive text completion for a mobile communication facility |
US10592930B2 (en) | 2005-09-14 | 2020-03-17 | Millenial Media, LLC | Syndication of a behavioral profile using a monetization platform |
US8688671B2 (en) * | 2005-09-14 | 2014-04-01 | Millennial Media | Managing sponsored content based on geographic region |
US20110313853A1 (en) | 2005-09-14 | 2011-12-22 | Jorey Ramer | System for targeting advertising content to a plurality of mobile communication facilities |
US8463249B2 (en) | 2005-09-14 | 2013-06-11 | Jumptap, Inc. | System for targeting advertising content to a plurality of mobile communication facilities |
US7676394B2 (en) | 2005-09-14 | 2010-03-09 | Jumptap, Inc. | Dynamic bidding and expected value |
US7885335B2 (en) * | 2005-09-16 | 2011-02-08 | Sont Corporation | Variable shape motion estimation in video sequence |
SG130962A1 (en) * | 2005-09-16 | 2007-04-26 | St Microelectronics Asia | A method and system for adaptive pre-filtering for digital video signals |
JP5013040B2 (en) * | 2005-09-29 | 2012-08-29 | 株式会社メガチップス | Motion search method |
WO2007040361A1 (en) * | 2005-10-05 | 2007-04-12 | Lg Electronics Inc. | Method and apparatus for signal processing and encoding and decoding method, and apparatus therefor |
US8731071B1 (en) | 2005-12-15 | 2014-05-20 | Nvidia Corporation | System for performing finite input response (FIR) filtering in motion estimation |
US7693219B2 (en) * | 2006-01-04 | 2010-04-06 | Freescale Semiconductor, Inc. | System and method for fast motion estimation |
US8724702B1 (en) | 2006-03-29 | 2014-05-13 | Nvidia Corporation | Methods and systems for motion estimation used in video coding |
US8437396B2 (en) * | 2006-08-10 | 2013-05-07 | Vixs Systems, Inc. | Motion search module with field and frame processing and methods for use therewith |
US8660380B2 (en) | 2006-08-25 | 2014-02-25 | Nvidia Corporation | Method and system for performing two-dimensional transform on data value array with reduced power consumption |
JP2008109632A (en) * | 2006-09-28 | 2008-05-08 | Toshiba Corp | Motion vector detector and its method |
US8069321B2 (en) | 2006-11-13 | 2011-11-29 | I365 Inc. | Secondary pools |
KR101356735B1 (en) * | 2007-01-03 | 2014-02-03 | 삼성전자주식회사 | Mothod of estimating motion vector using global motion vector, apparatus, encoder, decoder and decoding method |
TW200838321A (en) * | 2007-03-02 | 2008-09-16 | X8 Technology Inc | Motion estimation method and system with dual search windows for high resolution video coding |
US9253491B2 (en) * | 2007-04-02 | 2016-02-02 | Vestel Elektronik Sanayi Ve | Method and apparatus for transcoding a video signal |
US7929652B2 (en) * | 2007-04-10 | 2011-04-19 | Qualcomm Incorporated | Adaptive pilot and data symbol estimation |
US8756482B2 (en) | 2007-05-25 | 2014-06-17 | Nvidia Corporation | Efficient encoding/decoding of a sequence of data frames |
US9118927B2 (en) * | 2007-06-13 | 2015-08-25 | Nvidia Corporation | Sub-pixel interpolation and its application in motion compensated encoding of a video signal |
US8873625B2 (en) | 2007-07-18 | 2014-10-28 | Nvidia Corporation | Enhanced compression in representing non-frame-edge blocks of image frames |
US8731048B2 (en) * | 2007-08-17 | 2014-05-20 | Tsai Sheng Group Llc | Efficient temporal search range control for video encoding processes |
US8799342B2 (en) * | 2007-08-28 | 2014-08-05 | Honda Motor Co., Ltd. | Signal processing device |
KR101385956B1 (en) | 2007-08-31 | 2014-04-17 | 삼성전자주식회사 | A Method for encoding/decoding a media signal and apparatus thereof |
US8127233B2 (en) * | 2007-09-24 | 2012-02-28 | Microsoft Corporation | Remote user interface updates using difference and motion encoding |
US8619877B2 (en) * | 2007-10-11 | 2013-12-31 | Microsoft Corporation | Optimized key frame caching for remote interface rendering |
KR102037328B1 (en) * | 2007-10-16 | 2019-10-28 | 엘지전자 주식회사 | A method and an apparatus for processing a video signal |
CN101919251B (en) * | 2007-11-28 | 2013-07-10 | 法国电信 | Motion encoding and decoding |
WO2009086181A1 (en) * | 2007-12-19 | 2009-07-09 | Dolby Laboratories Licensing Corporation | Adaptive motion estimation |
US8432975B2 (en) * | 2008-01-18 | 2013-04-30 | Mediatek Inc. | Apparatus and method for processing a picture frame |
CN101933328B (en) * | 2008-01-22 | 2014-11-19 | 杜比实验室特许公司 | Adaptive motion information cost estimation with dynamic look-up table updating |
KR100939917B1 (en) | 2008-03-07 | 2010-02-03 | 에스케이 텔레콤주식회사 | Encoding system using motion estimation and encoding method using motion estimation |
US9967590B2 (en) * | 2008-04-10 | 2018-05-08 | Qualcomm Incorporated | Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter |
US8831086B2 (en) * | 2008-04-10 | 2014-09-09 | Qualcomm Incorporated | Prediction techniques for interpolation in video coding |
US8055025B2 (en) * | 2008-06-14 | 2011-11-08 | City University Of Hong Kong | Motion estimation method |
EP2439952A3 (en) * | 2008-06-20 | 2013-11-27 | Dolby Laboratories Licensing Corporation | Video compression under multiple distortion constraints |
JP5361896B2 (en) * | 2008-09-29 | 2013-12-04 | パナソニック株式会社 | Moving picture coding method and moving picture coding apparatus |
US8363727B2 (en) * | 2008-09-30 | 2013-01-29 | Microsoft Corporation | Techniques to perform fast motion estimation |
AU2008240343B2 (en) * | 2008-10-31 | 2012-04-19 | Canon Kabushiki Kaisha | Rate-distortion control in DVC with no feedback |
US9723330B2 (en) * | 2008-11-25 | 2017-08-01 | Thomson Licensing Dtv | Method and apparatus for sparsity-based de-artifact filtering for video encoding and decoding |
US8666181B2 (en) | 2008-12-10 | 2014-03-04 | Nvidia Corporation | Adaptive multiple engine image motion detection system and method |
CN101924933A (en) * | 2009-04-10 | 2010-12-22 | 特克特朗尼克国际销售有限责任公司 | Method for tracing interested area in video frame sequence |
GB2474250B (en) * | 2009-10-07 | 2015-05-06 | Advanced Risc Mach Ltd | Video reference frame retrieval |
CN102316317B (en) * | 2010-07-10 | 2013-04-24 | 华为技术有限公司 | Method and device for generating predicted value of picture |
US8712930B1 (en) | 2010-08-09 | 2014-04-29 | Google Inc. | Encoding digital content based on models for predicting similarity between exemplars |
CN102413326B (en) * | 2010-09-26 | 2014-04-30 | 华为技术有限公司 | Video coding and decoding method and device |
US9319716B2 (en) | 2011-01-27 | 2016-04-19 | Qualcomm Incorporated | Performing motion vector prediction for video coding |
US20120236940A1 (en) * | 2011-03-16 | 2012-09-20 | Texas Instruments Incorporated | Method for Efficient Parallel Processing for Real-Time Video Coding |
US8515193B1 (en) * | 2011-04-08 | 2013-08-20 | Google Inc. | Image compression using exemplar dictionary based on hierarchical clustering |
CN106170092B (en) * | 2011-06-30 | 2020-02-14 | 华为技术有限公司 | Fast coding method for lossless coding |
US9232233B2 (en) * | 2011-07-01 | 2016-01-05 | Apple Inc. | Adaptive configuration of reference frame buffer based on camera and background motion |
US20130031497A1 (en) * | 2011-07-29 | 2013-01-31 | Nokia Corporation | Method and apparatus for enabling multi-parameter discovery and input |
US9179156B2 (en) * | 2011-11-10 | 2015-11-03 | Intel Corporation | Memory controller for video analytics and encoding |
US9490850B1 (en) | 2011-11-28 | 2016-11-08 | Google Inc. | Method and apparatus for decoding packetized data |
CN104285377A (en) * | 2012-03-15 | 2015-01-14 | 诺基亚公司 | Encoding and decoding of data |
US9185429B1 (en) * | 2012-04-30 | 2015-11-10 | Google Inc. | Video encoding and decoding using un-equal error protection |
US9344727B2 (en) * | 2012-05-14 | 2016-05-17 | Ati Technologies Ulc | Method of using a reduced number of macroblock coding modes in a first view based on total coding mode complexity values of macroblocks in second view |
US10034023B1 (en) | 2012-07-30 | 2018-07-24 | Google Llc | Extended protection of digital video streams |
US9979960B2 (en) | 2012-10-01 | 2018-05-22 | Microsoft Technology Licensing, Llc | Frame packing and unpacking between frames of chroma sampling formats with different chroma resolutions |
WO2014193630A1 (en) * | 2013-05-30 | 2014-12-04 | Intel Corporation | Quantization offset and cost factor modification for video encoding |
WO2015015681A1 (en) * | 2013-07-31 | 2015-02-05 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | Image coding method, and image coding device |
CN104159003B (en) * | 2014-08-21 | 2017-08-25 | 武汉大学 | A kind of cooperateed with based on 3D filters the video denoising method rebuild with low-rank matrix and system |
US10091511B2 (en) * | 2015-01-05 | 2018-10-02 | Getgo, Inc. | Efficient video block matching |
KR102600756B1 (en) * | 2015-03-06 | 2023-11-10 | 한국과학기술원 | Video encoding and decoding method based on low-complexity transformation and device using the same |
CN104767997B (en) * | 2015-03-25 | 2017-10-27 | 北京大学 | A kind of visual signature coding method and device towards video |
CN104767998B (en) * | 2015-03-25 | 2017-12-08 | 北京大学 | A kind of visual signature coding method and device towards video |
CN105338354B (en) * | 2015-09-29 | 2019-04-05 | 北京奇艺世纪科技有限公司 | A kind of motion vector estimation method and apparatus |
WO2017075804A1 (en) * | 2015-11-06 | 2017-05-11 | Microsoft Technology Licensing, Llc | Flexible reference picture management for video encoding and decoding |
US10368080B2 (en) | 2016-10-21 | 2019-07-30 | Microsoft Technology Licensing, Llc | Selective upsampling or refresh of chroma sample values |
CN107194961B (en) * | 2017-05-19 | 2020-09-22 | 西安电子科技大学 | Method for determining multiple reference images in group image coding |
US11070813B2 (en) * | 2018-06-29 | 2021-07-20 | Intel Corporation | Global motion estimation and modeling for accurate global motion compensation for efficient video processing or coding |
US10765944B2 (en) * | 2018-09-07 | 2020-09-08 | Electronic Arts Inc. | Machine learning models for implementing animation actions |
US11443137B2 (en) * | 2019-07-31 | 2022-09-13 | Rohde & Schwarz Gmbh & Co. Kg | Method and apparatus for detecting signal features |
CN110996300A (en) * | 2019-11-12 | 2020-04-10 | 北京航空航天大学 | Vehicle-mounted terminal information safety risk control method based on traffic scene safety |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6317819B1 (en) * | 1996-01-11 | 2001-11-13 | Steven G. Morton | Digital signal processor containing scalar processor and a plurality of vector processors operating from a single instruction |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3308990B2 (en) | 1992-07-03 | 2002-07-29 | 松下電器産業株式会社 | Method and apparatus for predictive encoding of moving image |
EP0779011B1 (en) * | 1994-09-02 | 2001-11-28 | Sarnoff Corporation | Method and apparatus for global-to-local block motion estimation |
US5847776A (en) * | 1996-06-24 | 1998-12-08 | Vdonet Corporation Ltd. | Method for entropy constrained motion estimation and coding of motion vectors with increased search range |
GB9712651D0 (en) * | 1997-06-18 | 1997-08-20 | Nds Ltd | Improvements in or relating to encoding digital signals |
GB9721662D0 (en) * | 1997-10-14 | 1997-12-10 | Philips Electronics Nv | Encoded video signal formatting |
US6130911A (en) * | 1997-11-21 | 2000-10-10 | Sharp Laboratories Of America, Inc. | Method and apparatus for compressing reference frames in an interframe video codec |
US6983018B1 (en) * | 1998-11-30 | 2006-01-03 | Microsoft Corporation | Efficient motion vector coding for video compression |
CN1193620C (en) * | 2000-01-21 | 2005-03-16 | 诺基亚有限公司 | Motion estimation method and system for video coder |
US20020122491A1 (en) * | 2001-01-03 | 2002-09-05 | Marta Karczewicz | Video decoder architecture and method for using same |
US7630566B2 (en) * | 2001-09-25 | 2009-12-08 | Broadcom Corporation | Method and apparatus for improved estimation and compensation in digital video compression and decompression |
US8824553B2 (en) * | 2003-05-12 | 2014-09-02 | Google Inc. | Video compression method |
-
2004
- 2004-06-14 US US10/867,447 patent/US7471724B2/en not_active Expired - Fee Related
- 2004-06-14 US US10/867,027 patent/US20040258147A1/en not_active Abandoned
- 2004-06-17 EP EP04755512A patent/EP1644896A2/en not_active Withdrawn
- 2004-06-17 WO PCT/US2004/019387 patent/WO2005001773A2/en active Application Filing
- 2004-06-23 JP JP2006517628A patent/JP2007525078A/en active Pending
- 2004-06-23 WO PCT/US2004/020317 patent/WO2005002235A2/en active Application Filing
- 2004-06-23 US US10/875,258 patent/US7499491B2/en not_active Expired - Fee Related
- 2004-06-23 CN CNA2004800176253A patent/CN1809839A/en active Pending
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6317819B1 (en) * | 1996-01-11 | 2001-11-13 | Steven G. Morton | Digital signal processor containing scalar processor and a plurality of vector processors operating from a single instruction |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160055120A1 (en) * | 2002-02-05 | 2016-02-25 | Pact Xpp Technologies Ag | Integrated data processing core and array data processor and method for processing algorithms |
US10579584B2 (en) * | 2002-03-21 | 2020-03-03 | Pact Xpp Schweiz Ag | Integrated data processing core and array data processor and method for processing algorithms |
US20100332547A1 (en) * | 2009-06-24 | 2010-12-30 | Nokia Corporation | Method and apparatus for retrieving nearby data |
US8290952B2 (en) | 2009-06-24 | 2012-10-16 | Nokia Corporation | Method and apparatus for retrieving nearby data |
US20220121573A1 (en) * | 2019-12-10 | 2022-04-21 | Pony Ai Inc. | Memory architecture for efficient spatial-temporal data storage and access |
US11809318B2 (en) * | 2019-12-10 | 2023-11-07 | Pony Ai Inc. | Memory architecture for efficient spatial-temporal data storage and access |
US20220318171A1 (en) * | 2021-04-01 | 2022-10-06 | quadric.io, Inc. | Systems and methods for intelligently implementing concurrent transfers of data within a machine perception and dense algorithm integrated circuit |
US11531633B2 (en) * | 2021-04-01 | 2022-12-20 | quadric.io, Inc. | Systems and methods for intelligently implementing concurrent transfers of data within a machine perception and dense algorithm integrated circuit |
US11907146B2 (en) | 2021-04-01 | 2024-02-20 | quadric.io, Inc. | Systems and methods for intelligently implementing concurrent transfers of data within a machine perception and dense algorithm integrated circuit |
Also Published As
Publication number | Publication date |
---|---|
CN1809839A (en) | 2006-07-26 |
WO2005002235A3 (en) | 2005-05-19 |
EP1644896A2 (en) | 2006-04-12 |
WO2005001773A2 (en) | 2005-01-06 |
US7499491B2 (en) | 2009-03-03 |
US20050013369A1 (en) | 2005-01-20 |
US7471724B2 (en) | 2008-12-30 |
WO2005001773A3 (en) | 2005-10-13 |
WO2005002235A2 (en) | 2005-01-06 |
US20050111548A1 (en) | 2005-05-26 |
JP2007525078A (en) | 2007-08-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20040258147A1 (en) | Memory and array processor structure for multiple-dimensional signal processing | |
US6122320A (en) | Circuit for motion estimation in digitized video sequence encoders | |
US7072395B2 (en) | Memory control apparatus and efficient search pattern for block-matching motion estimation | |
US7496736B2 (en) | Method of efficient digital processing of multi-dimensional data | |
US5583803A (en) | Two-dimensional orthogonal transform processor | |
WO2007024413A2 (en) | Transpose buffering for video processing | |
Kim et al. | A high-throughput hardware design of a one-dimensional SPIHT algorithm | |
JP2011120244A (en) | System for processing images | |
Wang et al. | VLSI implementation of HEVC motion compensation with distance biased direct cache mapping for 8K UHDTV applications | |
CN100547567C (en) | Data handling system with high-speed cache of the data streams of being optimized to application | |
US7777751B2 (en) | Tiled memory array for full search motion estimation | |
De Cea-Dominguez et al. | GPU-oriented architecture for an end-to-end image/video codec based on JPEG2000 | |
CN101443808B (en) | Memory organizational scheme and controller architecture for image and video processing | |
US5825420A (en) | Processor for performing two-dimensional inverse discrete cosine transform | |
Jackson et al. | A hardware architecture for real-time image compression using a searchless fractal image coding method | |
de Cea-Dominguez et al. | Real-time 16K video coding on a GPU with complexity scalable BPC-PaCo | |
Chang et al. | VLSI architecture design of MPEG-4 shape coding | |
Pratapa et al. | HMLFC: Hierarchical Motion‐Compensated Light Field Compression for Interactive Rendering | |
Woo et al. | A cache-aware motion estimation organization for a hardware-based H. 264 encoder | |
Jahnke et al. | 4D-DCT hardware architecture for JPEG pleno light field coding | |
Hsia | Efficient memory IP design for HDTV coding applications | |
Yan et al. | A high throughput and energy efficient Lepton hardware encoder with hash-based memory optimization | |
Lee et al. | A cost-effective MPEG-4 shape-adaptive DCT with auto-aligned transpose memory organization | |
Ai et al. | Efficient parallel implementation of motion estimation on the Computational RAM architecture | |
Li | Optimization of queueing performance and design variables in a single-bus shared-memory system with application to MPEG-2 video decoder system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: VICHIP CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LEE, TSU-CHANG;REEL/FRAME:020885/0957 Effective date: 20080204 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |