JP5835879B2 - Method and apparatus for controlling reading of an array of data from memory - Google Patents

Description

  The present invention relates to reading an array of data from a memory for processing. An example of this is the operation of the display controller when processing an image from the frame buffer for display.

  As is known in the art, many electronic devices and systems require processing an array of data such as images. For example, an image to be displayed to the user is usually processed by a so-called “display controller” of the display device for display.

  Typically, a display controller reads an output image to be displayed from a so-called “frame buffer” in a memory that stores images as a data array, and sends image data to a display as appropriate. In the case of a graphics processing system, for example, the output image of the graphics processing system is stored in a frame buffer in memory when ready to display, and then a display controller reads the frame buffer and displays it for display (e.g. , Screen or printer).

  As is known in the art, the frame buffer itself is usually stored in the so-called “main” memory of the system of interest, so it is external to the display device and display controller. Thus, reading data from the frame buffer for display can consume a relatively significant amount of power and memory bandwidth. For example, new image frames need to be read from the frame buffer and displayed at a frame rate of 30 frames per second or more, and each frame is significant, especially for high-resolution displays and high-definition (HD) graphics. May require amount of data.

  Thus, it is known that it is desirable to attempt to reduce the power consumption of frame buffer operations, and various techniques have been proposed to attempt to achieve such a reduction.

  These techniques include on-chip (as opposed to external) frame buffer formation, frame buffer caching (buffering), frame buffer compression and dynamic color depth control. However, each of these techniques has its own drawbacks and disadvantages.

  For example, using an on-chip frame buffer may require a large amount of on-chip resources, especially for high resolution displays. Frame buffer caching or buffering is not practical because frame generation is typically asynchronous to the frame buffer display. Although frame buffer compression can be useful, the required logic is relatively complex and the frame buffer format is changed. Using lossy frame buffer compression degrades image quality. Similarly, the dynamic color depth control is an irreversible method, and therefore the image quality is degraded.

  Other arrangements that may need to read a data array from memory for processing are situations where the CPU may need to read an image generated by a graphics processor and modify it, for example And the case where the graphics processor may need to load an externally generated texture for later use in graphics processing. These arrangements can also consume relatively large memory bandwidth and power when recalling a stored data array for processing.

  Therefore, the applicant is convinced that there is room for improvement to the data array, such as a frame buffer and a read operation.

According to a first aspect of the present invention, a processing device processes an array of data by processing successive blocks of data, each representing a specific area of the data array, A block of data representing is read from the first memory in which the array of data is stored, and the array of data is stored in the memory of the processing device before the block of data is processed by the processing device A method is provided, which is
Determine whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and based on the similarity determination, Processing a block of data already stored in the memory of the processing device for the block or a new block of data from an array of data stored in the first memory.

According to a second aspect of the present invention, a system is realized, the system comprising:
A first memory for storing an array of data to be processed;
A processing device having local memory that processes an array of data stored in a first memory by processing successive blocks of data, each representing a particular region of the array of data;
Read a block of data representing a specific area of the array of data stored in the first memory and store the block of data in the local memory of the processing device before the block of data is processed by the processing device A read controller configured to
Determine whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and based on the similarity determination, For a block, configured to cause the processing device to process a block of data already stored in the memory of the processing device, or a new block of data from an array of data stored in the first memory And a controller.

According to the third aspect of the present invention, a processing device for processing an array of data stored in the first memory is realized, the processing device comprising an array of data, each of which is an array of data. Configured to process by processing successive blocks of data representing a particular area,
Local memory,
Read a block of data representing a specific area of the array of data stored in the first memory and store the block of data in the local memory of the processing device before the block of data is processed by the processing device A read controller configured to
Determine whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and based on the similarity determination, For a block, configured to cause the processing device to process a block of data already stored in the memory of the processing device, or a new block of data from an array of data stored in the first memory And a controller.

  The present invention provides that an array of data to be processed (eg, can be a frame to be displayed, and in one preferred embodiment is a frame to be displayed) represents a particular region of the array of data. Relates to a system that is read from memory for processing by a processing device in the form of a block of data (e.g., may be a display controller, and in a preferred embodiment is a display controller), and Implemented in such a system.

  Thus, in essence, the present invention pertains to a system in which the data array to be processed by the system is read from memory and processed block by block rather than directly as a single output "array". And is intended to be implemented in such a system.

  As described above, this may be the case, for example, for the display of images generated by a tile-based graphics processing system. In this case, the display controller can process each frame for display from the frame buffer on a tile-by-tile basis (however, as will be further explained below, this is not essential and is in fact always preferred) Not necessarily).

  (As is known in the art, in tile-based rendering, a two-dimensional output array or frame (`` renders the target '') of the rendering process (e.g., typically also represents the scene being rendered). Will be displayed or subdivided into multiple smaller areas, usually called “tiles”, for the rendering process. Rendered tiles (partial regions) are then recombined to produce a complete output array (frame) (render target) for display, for example, Form.

  Another term commonly used for "tiling" and "tile based" rendering is "chunking" (partial regions are referred to as "chunks") And "bucket" rendering. The terms “tile” and “tiling” are used herein for convenience, but these terms are intended to encompass all alternative and equivalent terms and techniques. Will be understood.

  In the present invention, each data block (e.g., a rendered tile) is simply read from the memory in which the data array is stored and not processed in sequence, but when the data block is processed (e.g., First) to a data block (e.g. tile) already stored in the (local) memory of the processing device (e.g. display controller) that the block is supposed to process the data array It is determined whether they are similar. It is then determined whether to process an existing data block in local memory or a new data block from a stored data array in memory as a data block to process based on the similarity determination. The

  As described in more detail below, Applicant uses this process to determine the number of data blocks (eg, rendered tiles) that are read from main memory (eg, frame buffer) for processing during use. Can significantly reduce the number of main memory (e.g. frame buffer) read transactions, and thus significantly reduce the power and memory bandwidth consumption associated with main memory (e.g. frame buffer) read operations. Discovered and understood. This may also make it easier to use low power memory systems with lower performance, which may be particularly advantageous, for example, in the context of low power, low cost portable devices.

  For example, if the data block to be processed is found to be the same as a data block that already exists in the processing device's local memory (e.g., a rendered tile), the data block is processed from the stored data array. Can be determined (and preferably determined) that it is not necessary to read into the local memory, thereby eliminating that read “transaction”. Thus, if it is determined that the data block to be processed is similar to a data block already stored in the local memory of the processing device, preferably an (appropriate) existing block in the local memory of the processing device Are processed by the processing device or vice versa.

  In addition, applicants have the same data blocks (e.g. rendered tiles) that are already in local memory of the display controller, e.g. in the case of graphics processing, e.g. There can be relatively common situations that are similar or similar. For example, in the case of graphics processing, there are regions of images that are similar to each other, such as the sky, sea, or other uniform background, which is the majority of user interfaces such as many applications. Thus, by identifying such areas (e.g. tiles) and then making it easier to avoid re-reading such areas (e.g. tiles) from the local memory of the display controller if desired, e.g. Significant savings in read traffic (read transactions) to the display controller's local memory can be achieved.

  Thus, the power consumption and memory bandwidth used for frame buffer and memory read operations is significantly reduced by using the present invention, making it virtually easier to identify and eliminate unnecessary memory (e.g., frame buffer) read transactions. Can be reduced.

  Furthermore, compared to the above-described prior art schemes, the present invention requires relatively little on-chip hardware, can be a reversible process, and does not change the data array (eg, frame buffer) format. . Also, it can be easily used in conjunction with, or complementing, existing output (eg, frame buffer) power reduction schemes, thereby making it easier to save power if desired.

  As will be described further below, the present invention can also be used to avoid first writing data blocks to the initial data array. This elimination of write transactions can further reduce the power and memory bandwidth of memory (eg, frame buffer) transactions (however, reading data arrays can increase compared to writing (updating). Therefore, it seems generally more beneficial to eliminate read transactions).

  As described above, in one particularly preferred embodiment, the processing device determines whether to read a new data block from the data array in main memory into the local memory of the processing device based on the similarity determination.

  Thus, in a particularly preferred embodiment, it has been determined that the (e.g., next) block of data to be processed is considered similar to the block of data already stored in the local memory of the processing device. If the new block of data is not read from the data array in the main memory and is stored in the local memory of the processing device, instead, the existing block of data in the local memory of the processing device Is processed as a (e.g., next) block of data to be processed.

  On the other hand, if it is determined that the (e.g., the next) block of data to be processed is not considered similar to the block of data already stored in the local memory of the processing device, the new data A block is read from a data array in main memory, stored in local memory of the processing device, and then processed as a (eg, next) block of data to be processed by the processing device.

  As described further below, the similarity determination is preferably based on similarity information (metadata) associated with the data block of interest. Such generation of similarity information is another aspect of the present invention. This will be described in more detail below.

  The present invention can be used in a system where data is stored as an array and read and processed by a processing device block by block. Thus, it can be used, for example, in a graphics processor, CPU, video processor, composition engine, display controller, etc.

  In general, the present invention is useful in eliminating read transactions (and write transactions) that are likely to have similar or identical data blocks in the data array to be processed. Thus, using this scheme, for example, read transactions when image data is transferred between any two of the graphics processor (GPU), CPU, video processor, camera controller, and display controller. (And write transactions) can be eliminated.

  For example, to display an image to be displayed in the form of a block of data, potentially and typically, to display the image with the operation of a display controller as described above, but also to use it as a texture by a video processor An image to be transferred to the graphics processor can be generated, in which case the technique of the present invention can be used to eliminate read transactions when reading the image (texture) used by the graphics processor It is. Similarly, a frame generated by the graphics processor can be manipulated by the CPU, in which case the CPU is manipulated in the manner of the present invention so that the CPU reads the frame and manipulates it. The required read transaction can be reduced. This has the added benefit that fewer cache lines can be used by the CPU.

  Similarly, a camera (video or still image) is an image generated by its sensor on a block-by-block basis for delivery to a data processing system such as a computer, display, etc., which is then stored in memory and then processed. Can be processed.

  The memory in which the array of data is stored can include any suitable such memory and can be configured in any suitable and desirable manner. For example, the memory may be an on-chip attached to the processing device, or may be an external memory. In a preferred embodiment, this is an external memory, such as the main memory of the system. This can be a dedicated memory for this purpose, or it can be part of the memory used for other data. In the case of a graphics processing system, in a preferred embodiment, the memory in which the data array is stored is a frame buffer that is the output destination of the graphics processing system.

  The array of data stored in and read from the first (eg, main) memory may be such a suitable and desirable array of data. This may include a suitable and desirable arrangement of data that can be generated using, for example, a graphics processor. In a preferred embodiment, this is, for example, data representing an image to be displayed.

  In one particularly preferred embodiment, this includes an output frame for display, which is a texture that a graphics texture (e.g., a render `` target '' generates using a graphics processor (e.g., Other outputs of the graphics processor, such as in a “render to texture” operation)) or other surface to which the graphics processor system output is written, may alternatively be included. This may include, for example, an image generated by a video processor or CPU, as described above.

  The processing device can be any device that reads and processes the data array (for each block) or changes its contents, eg, for use. Thus, this may be, for example, one of a display controller, CPU, video processor, and graphics processor, and in a preferred embodiment, one of the display controller, CPU, video processor, and graphics processor. One.

  The local memory of the processing device may likewise be such a suitable memory. This is preferably a buffer or cache memory of the processing device or a buffer or cache memory associated with the processing device. The cache may be, for example, a full associative method or a set associative method.

  As mentioned above, in one particularly preferred embodiment, the present invention is implemented with respect to a data array generated by a graphics processing system (graphics processor), in which case the data array to be processed is preferably displayed. The first main memory that is the output frame and in which the data array is stored is preferably a frame buffer of the graphics processing system. Similarly, the processing device that is to process the data array in which the output frame is displayed is preferably a display controller of the display device (eg, screen or printer) or for the display device (eg, screen or printer). In addition, this may be a CPU that is to operate a frame generated by a graphics processor as described above, for example.

  Each block of data to be processed (and compared) can represent any suitable desired area of the entire array of data. As long as the entire array of data is divided or partitioned into multiple identifiable smaller regions each representing a portion of the entire array, and as long as it can be represented as a block of data that can be identified and considered accordingly Subdivision of the array into blocks of data can be performed as needed.

  Each block of data preferably represents a different part (partial region) of the entire data array (these blocks can overlap if desired). Each block will represent an appropriate portion (area) of the data array, such as multiple data positions within the array. The preferred data block size is 8 × 8, 16 × 16, or 32 × 32 data positions in the data array.

  In a particularly preferred embodiment, the array of data is divided into regularly sized and shaped areas (blocks of data), preferably square or rectangular shaped areas. However, this is not essential and other arrangements can be used if desired.

  Similarity determination and result for processing either a block of data already stored in the memory of the processing device or a new block of data from an array of data stored in the first memory The determination can be performed at any desired and preferred point and time point desired and in a suitable manner when the data array is processed.

  For example, similarity determination and resulting data block selection can be performed for each data block when it is time to process that data block (performed in one embodiment). In this case, for example, it is determined whether the next block of data to be processed after the current block of data being processed is similar to a block of data already stored in the memory of the processing device, and then A new or existing block of data is processed accordingly for the next block of data.

  However, in one particularly preferred embodiment, the similarity determination and the resulting data block selection is performed before the data block is actually processed. In this case, similarity determination is used, for example, to control the substantial “prefetching” of the data block into the local memory of the processing device before the data block that is subsequently retrieved from the processing device local memory and processed. Is done. This arrangement works, for example, when a processing device (e.g. a display controller) puts the blocks of data to be processed into queues in its local memory, and then puts those blocks one by one from the queue for display. Suitable for processing. In such an arrangement, similarity determination can be used to control fetching of data blocks to a queue in local memory (i.e., data blocks that are already already in the queue). Whether to repeat or fetch a new block of data from the stored data array into the queue).

  The determination of whether a new data block to be processed is similar to a block already stored in the local memory of the processing device (eg, display controller) may be performed in any suitable and desirable manner. For example, a new block of data to be read from a stored data array can be compared to one or more blocks already stored inside local memory to determine if those blocks are similar It is. Thus, for example, it is possible to compare part of the contents of a new data block with part or all of the contents of one or more data blocks or one or more data blocks already stored in local memory .

  In one particularly preferred embodiment, the information associated with the data array is used to determine whether a given block should be considered similar to each other. Thus, in a particularly preferred embodiment, rather than comparing the contents of the data block itself, the similarity determination process causes the data block to be processed using information associated with the array of data to be in local memory. Determine if it is similar to a block already stored in.

  In other words, the similarity determination process preferably uses “metadata” (information) associated with the data array to resemble blocks where the data block to be processed is already in the local memory of the processing device. Determine whether or not. As described further below, the use of metadata associated with data arrays for this purpose reduces the burden on the processing device, especially to reduce the number of read transactions in use. An effective mechanism can be realized.

  Any suitable form of metadata (information) that can be used by the processing device to determine whether the data blocks should be considered similar can be used (also in the stored array of data). Can be associated as appropriate).

  For example, the metadata can include information that can be used by the processing device itself to assess whether the data blocks should be considered similar to each other, and in a preferred embodiment, such a Contains information.

  In one such preferred embodiment, the information (metadata) that is associated with an array of data and that should be used to determine whether a block of data is similar represents the content of the data block of interest, and Contains information derived from the contents of the data block of interest. In this case, the similarity determination process then preferably represents the content of the new data block and / or information derived from the content of the new data block, the content of the data block already stored in local memory. And / or determine whether each data block is similar by comparing it with information derived from the contents of the data block already stored in local memory.

  Information representing the contents of each data block in these arrangements may be of any suitable form, but is preferably based on or derived from the contents on the data block. Most preferably, it is generated from the content of the data block or takes the form of a “signature” for the data block based on the content of the data block. Such a “signature” of the content of the data block represents, for example and preferably, the content of the data block, such as a checksum, CRC or hash value derived from the data block (generated for the data block) Any suitable set of derivation information that can be considered can be included. Suitable signatures include standard CRCs such as CRC32 or other forms of signatures such as MD5, SHA-1.

  Thus, in a particularly preferred embodiment, a signature indicating or representing the content of the data block and / or derived from the content of the data block is generated for each data block to be compared, and the similarity determination process Compares the signatures of each data block to determine if the blocks are similar.

  For example, it is possible to generate a single signature for one, e.g. RGBA, data block (e.g. rendered tile) or generate a separate signature (e.g. CRC) for each color plane It is possible. Similarly, color conversion can be performed and different signatures can be generated for the Y, U, and V planes as needed.

  As those skilled in the art will appreciate, the longer the signature generated for a data block (the more accurately the signature represents the data block), the more false “matches” between the signatures (and thus, for example, new The possibility of an error that the data block is not read) is reduced. So, depending on the accuracy required and generally desired (and as a trade-off for memory and processing resources required for signature generation and processing, for example), use longer or shorter signatures (e.g. CRC) Is possible.

  The signature can also be weighted towards a particular aspect or aspects of the content of the data block so that, for example, a predetermined total length of the signature makes the data block content more effective for the entire output. Weighting the signature on the (data) portion can give good overall results (eg, as perceived by the viewer of the image represented by the data array).

  For example, different length signatures for different applications, etc. may be used depending on the requirements of the application, eg, display. This can further help reduce power consumption. Thus, in a preferred embodiment, the signature length used can be changed during use. Preferably, the signature length can be changed according to the application in use (adapted tuning can be made according to the application being used).

  In a particularly preferred arrangement of these embodiments, data block signatures are “salt” ed when created (ie, other numerical values (salt values) are appended to the generated signature value). The salt value may conveniently be, for example, a numerical value of a data array (eg, a frame) after activation, or a random value. This is useful for making errors caused by inaccuracies in the signature comparison process non-deterministic as known in the art (i.e., for the process to display a movie or television program, for example). Avoid errors always occurring at the same point for iterative viewing of a given sequence of images, such as when used).

  In the above arrangement, the similarity determination process uses the metadata (information) associated with two (or more) data blocks so that the new data block to be processed is in the local memory of the processing device. To determine whether it is similar to a data block already stored in.

  However, in other particularly preferred embodiments, the metadata (information) associated with the data array is similar to directly indicating whether a given data block in the data array is similar to other blocks in the data array. It takes the form of degree information. In this case, the processing device can simply read the metadata and determine if the new data block is considered similar to the data block already stored in the processing device's local memory, There is no need to perform some form of similarity assessment of the block itself using metadata on the device side. This reduces the necessary processing burden on the processing device during execution of the data array processing operation.

  Thus, in a preferred embodiment, the information (metadata) associated with the array of data in the first (main) memory is between each data block (such as the data block “signature” as described above). Includes information that can be used to assess similarity, but in one particularly preferred embodiment, this information (metadata) indicates that each data block is similar to other data blocks in the data array. Contains information that indicates (directly) whether it can be considered.

  If the metadata directly indicates whether the data block is considered similar to other data blocks in the data array, the metadata can take any desirable form suitable for doing so. This can include, for example, a hierarchical quadtree. In a preferred embodiment, it is in the form of a (2D) bitmap.

  In one such particularly preferred embodiment, the metadata (e.g., bitmap) represents a data block to be read from the data array, and each metadata (e.g., bitmap) entry is for that corresponding data block. Indicates whether the data block is similar to other data blocks in the data array. Most preferably, each data block position in the data array is associated with a metadata entry that indicates whether the block is similar to other blocks. In this case, the similarity determination process determines the associated metadata entry for the data block location of interest to determine whether the data block is similar to a data block already stored in the local memory of the processing device. Simply read.

  Thus, in a particularly preferred embodiment, the data array is associated with metadata, such as a bitmap, so that for each data block in the data array, that data block is the other data in the data array. The similarity determination process (processing device) indicates whether the new data block to be processed is similar to a data block already stored in the local memory of the processing device. Determine by using the relevant metadata for.

  In these arrangements, the metadata can be configured and arranged as desired. For example, this can simply indicate whether the data block is similar to the previous data block in the data array, and in a preferred embodiment it simply indicates. In this case, each metadata entry need only contain a single bit, one value (e.g. `` 1 '') indicates that the corresponding block is similar to the previous block, and the other value (For example, “0”) indicates that they are not similar.

  To make this easy, the data blocks should be processed in a specific predefined order (both when writing to and reading from the data array). Preferably, an order in which spatial coherence between blocks can be utilized is used.

  For example, a more sophisticated metadata arrangement can be used when a data block is considered not only with respect to the immediately preceding data block but also with respect to multiple data blocks in the data array. In this case, the metadata (e.g., bitmap entry) associated with each respective block location is not only that the corresponding data block is similar to other data blocks in the data array, but also the data It should also indicate which data blocks in the array are similar. In this case, the metadata (eg, bitmap entry) associated with each data block position is larger than a single bit because more information is conveyed about each block position. The actual size of the metadata entry depends, for example, on the number of data blocks in the data array to which each data block is compared for the purpose of similarity (after which each metadata entry must be able to represent Since it determines the number of possible similar block permutations).

  In these arrangement configurations, each similarity value (metadata entry) is, for example, a relative indicator (for example, “001” indicating which other data block in the data array is similar to the data block of interest. '' Indicates a data block that precedes the current data block), or an absolute indicator that indicates which data block of interest is similar to other data blocks in the data array (e.g., metadata “125” indicates that the block is similar to the 125th data block in the data array of interest).

  Selection of the size of the metadata entry is a read transaction that is eliminated when the metadata is prepared and stored overhead and the metadata can indicate similarity to more other data blocks in the data array Trade-off or optimization between potentially larger numbers representing the number of times. Thus, the selection of the metadata arrangement to use can be made, for example, based on these criteria, and for example, based on the expected or anticipated usage or implementation of the system. (Note that here, using metadata in the manner of the embodiment of the present invention can result in relatively small metadata overhead per data block, so a fairly small data block size (cache line level (Note that it can be easier to use, etc.)

  In these arrangements, it is also possible to include a “likeness” value indicating how similar each data block is with each metadata entry. The similarity determination process then uses this likeness value, for example, to read a new block from the data array or to reuse an already existing similar data block in the local memory of the processing device being used. Can be determined. For example, the similarity determination process may set a likeness threshold and compare the likeness value for a new data block with the threshold to read or not read a new data block accordingly. This in turn can modify the read process and increase or decrease the accuracy of the data array read process, for example, by changing the likeness threshold during use, eg, during use.

  In another particularly preferred embodiment, the metadata (similarity information) associated with the data array instructs the processing device to read the data block into the local memory of the processing device according to their relative similarity. Takes the form of a command list. For example, if block 1 is read into the local memory of the processing device, the block is repeated for the next three blocks, then the fifth data block is read from the data array into local memory and the block is repeated once. It is possible to create a command list by removing the first data block from the local memory, reading the seventh block from the data array, reading the eighth block from the data array, processing, and so on. Such a command list can be generated directly or, for example, a similarity bitmap can be first generated and then analyzed to create a command list that is then stored for the data array. Is possible.

  When similarity metadata (information) is associated with a data array, it is also necessary to generate necessary metadata to be associated with the data array. The present invention, in its preferred embodiment, extends at least to the generation of metadata.

  Metadata can be generated and associated with the data array in any desired and preferred manner. Preferably, it is generated together with the generation of the data array. In a preferred embodiment, the metadata is generated by the device generating the data array (this device is generated by, for example, a graphics processor, video processor, camera controller (camera sensor) as described above. Data can be processed), or it can be a CPU).

  If the metadata includes a “signature” of content for each data block, those signatures are generated when the data block is generated and then stored in association with the data block generated in an appropriate manner. It is possible.

  If the metadata directly indicates whether the data blocks are considered the same as other data blocks such as the “similarity” bitmap described above, the data array generation process preferably generates the blocks of data As well as making comparisons and generating similarity information accordingly, eg, bitmaps.

  In this case, the data block comparison represents, for example, the content of the data block and / or information such as the signature described above derived from the content of the data block, the content of the other data block, and / or other This can be done by comparing the information derived from the contents of the data block and evaluating the similarity of the data block or others.

  However, in a particularly preferred embodiment, the actual contents of the blocks (rather than some representation of their contents) are compared to determine whether the blocks are considered similar. To do so, part or all of the contents of the data blocks of the data array can be compared with part or all of the contents of other data blocks (multiple data blocks) of the data array. Comparing some or all of the actual contents of the data blocks can reduce complexity and errors in the comparison process.

  The comparison process preferably uses some form of limit criteria to determine whether a block should be considered similar to other blocks. For example, preferably the blocks are considered similar if the selected number of bits of the contents of each block match. Preferably, there is some maximum visual deviation between allowed blocks (if the data array represents an image).

  Most preferably, maximum deviations, such as the amount of pixel LSB difference, are allowed before the blocks are considered dissimilar. Preferably, this threshold, eg, maximum content deviation, can be changed (eg, programmed) during use. For example, it can be set for each application based on the ratio of static frame data to dynamic frame data and / or based on the power mode in use (eg, whether it is a low power mode).

  In one particularly preferred embodiment, each block of data considered includes one cache line in the local memory of the processing device, or a 2D partial tile of the data array (such as in the case of a tile-based graphics processing system, etc. If the array consists of separate tiles). These are units of stored data that can be efficiently manipulated by the processing elements of the processing device that is to process the data array and that can be efficiently fetched from memory by the processing device that is to process the data array. Because it uses, it is a particularly effective implementation.

  In a graphics processing system, in a preferred embodiment, each data block corresponds to a rendered tile that the graphics processor generates as its rendering output. This is beneficial in that the graphics processor generates the rendering tile directly and therefore does not require further processing to “generate” the data blocks to be considered and compared.

  In these arrangements, the render target (data array) can be divided into several (rendering) tiles of any size and shape desired and suitable for rendering. The rendered tiles are preferably all the same size and shape as is known in the art, although this is not essential. In a preferred embodiment, each rendered tile is rectangular and preferably takes a sampling position of 8 × 8, 16 × 16, or 32 × 32 with respect to size.

  In other particularly preferred embodiments, data blocks of different sizes and / or shapes can be used, and preferably used, from the tiles that the rendering process manipulates (generates).

  For example, in a preferred embodiment, one or each data block considered and compared is composed of a set of multiple “rendered” tiles and / or includes only a small portion of the rendered tiles. sell. In these cases, there may actually be an intermediate stage that “generates” the desired data block from one or more rendered tiles generated by the graphics processor.

  In a preferred embodiment, the same block (region) configuration (size and shape) is used throughout the array of data. However, in other preferred embodiments, different block configurations (eg, with respect to their size and / or shape) are used for different regions of a given data array. Thus, in a preferred embodiment, different data block sizes can be used for different regions of the same data array.

  In one particularly preferred embodiment, the block configuration (e.g., with respect to the considered block size and / or shape) is changed in use, e.g., on a data array (e.g., output frame) for each data array. Can do. Most preferably, the block configuration can be adapted during use, for example and preferably, depending on the number or percentage of read (and / or write) transactions that are eliminated (avoided). For example, and preferably, a block of data need not be read from main memory if a particular block size is used, but only a low probability that the block does not need to be read from main memory is known. In order to try to increase the probability of doing so, it is possible to change (eg, also preferably reduce) the considered block size for subsequent arrays of data.

  If the data block size is changed in use, the change can be performed as desired, for example, over the entire data array or only over a specific portion of the data array.

  A data block can be compared to one other data block or to multiple other data blocks. Preferably, the comparison is done by storing each block in an on-chip buffer / cache.

  In a preferred embodiment, the data block is compared only with a single stored data block, preferably the previous data block in the data array.

  In other preferred embodiments, the data block is compared to a plurality of other data blocks in the data array. This can help to further reduce the number of data blocks that need to be read from the data array because it can eliminate reading data blocks that are similar to data blocks at other locations in the data array.

  When a data block is compared to multiple other data blocks in the data array, it is possible to compare each data block to all data blocks in the data array, but preferably each data block is It is compared only with a part, not all of the other data blocks of the data array, for example, and preferably only with data blocks within the same area of the data array as the data block of interest (e.g. these data blocks Covers and surrounds the position of the block). This increases the likelihood of detecting a data block match without having to check every data block in the data array. Most preferably, the data blocks are compared with the data blocks that are on the same line in the data array (in the order in which the blocks are generated).

  For example, it is possible to change the number of other data blocks to be compared with each data block during use for each frame. The metadata width can also be changed by changing the search depth of the data block comparison.

  In a preferred embodiment, every data block of the data array is compared with one or more other data blocks. However, this is not essential and, therefore, in other preferred embodiments, the comparison is performed on some but not all of the data blocks of a given data array (eg, output frame).

  In one particularly preferred embodiment, the number of data blocks compared to one or more other data blocks for each data array is, for example, and preferably, per data array (e.g., per frame) or Changes are made based on multiple sequences of data arrays (eg, frames). This is preferably based on (or not based on) the expected correlation between successive data sequences (eg, frames).

  Thus, the metadata generation process is preferably a means for selecting the number of data blocks in a data array to be compared with one or more other data blocks for a given data array or such Selecting a number of data blocks.

  In one particularly preferred embodiment, the number of data blocks to be compared can be different and preferably different for different regions of the data array.

  In a preferred embodiment, a software application (eg, triggering the generation of a data array) can indicate and control for which region of the data array the data block comparison process should be performed. This then allows the comparison process to be “turned off” by the application for regions of the data array that “know” that the application is always different.

  This can be achieved as desired. In a preferred embodiment, a register is provided that enables / disables data block (eg, rendered tile) comparison for the data array region, where a software application sets the register accordingly (eg, a graphics processor driver). Through).

  As described above, the generation of “similarity” metadata for data blocks of an array of data to be processed may be new and useful in itself.

Thus, according to a fourth aspect of the invention, there is provided a method for generating metadata for use when processing an array of data stored in memory, the method comprising:
For each of one or more blocks of data representing a particular region of the array of data to be processed,
Determining whether a block of data should be considered similar to other blocks of data for a data array;
Storing similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array in relation to the array of data.

According to a fifth aspect of the present invention, a data processing system is realized, the system comprising:
A data processor for generating an array of data for processing;
Means for determining, for each of one or more blocks of data representing a particular region of the data array, whether the block of data should be considered similar to other blocks of data for the data array; ,
Means for storing similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array in relation to the array of data.

According to a sixth aspect of the present invention, a data processor is realized, the processor comprising:
Means for generating an array of data for processing;
Means for determining, for each of one or more blocks of data representing a particular region of the data array, whether the block of data should be considered similar to other blocks of data for the data array; ,
Means for storing similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array in relation to the array of data.

  As will be appreciated by those skilled in the art, these aspects and embodiments of the present invention may include, where appropriate, one or more or all of the preferred and optional features of the present invention described herein. And preferably comprises. Thus, for example, the similarity indication information preferably takes the form of a bitmap associated with an array of data. The similarity of data blocks is preferably determined by comparing the data blocks, preferably by directly comparing their contents. The array of data is preferably data representing an image, and the data processor (data array generation processor) is preferably a graphics processor (but may be, for example, a video processor or CPU).

  Preferably, in these aspects and arrangements, the system is a set of indicating which regions (blocks) in the output data array are the same (can be considered similar) as described above. An output data array is generated together with related similarity information (metadata).

  Most preferably, the entire data array is divided into a number of suitable data blocks, and for each data block that divides the data array, whether that data block is similar to other data blocks in the data (and accordingly) The similarity information stored for the data block is determined.

  In one particularly preferred embodiment, the similarity information is generated when the data array is being written to memory (ie, while the data array is being generated). This eliminates the need to process the data array once it has been generated to generate similarity information. In this case, the data array is preferably generated by writing the data into the data array in blocks, and when each new block is generated for writing to the array, preferably the block is the data array. It is determined whether or not it is similar to other blocks already generated and its similarity information (metadata) generated accordingly.

  Thus, in a particularly preferred embodiment, the array of data is stored in memory (e.g., a frame buffer) by writing a block of data representing a particular region of the array of data to the stored array in memory. When a new block of data is generated for a data array, it is determined whether that new block of data should be considered similar to a block of data already generated for the data array, Similarity information is generated that indicates whether the new block is determined to be similar to the block of data already generated for the data array, and stored accordingly in relation to the array of data .

  In these arrangements, the data blocks are preferably buffered or cached in local memory for the similarity information generation process, so that, for example, a data array is stored to generate similarity information. It is not necessary to read a block from the main memory.

  For example, generate a "signature" (as described above) for a block of data when the array is generated, and then use that signature to make further similarities, such as a similarity bitmap for the data array It is possible to generate degree information, or alternatively it is possible to generate such similarity information.

  In the above aspects and embodiments, metadata (information), such as a block similarity bitmap and / or signature for a data block, associated with the data array and to be used when the data array is processed, Must be stored as appropriate. In a preferred embodiment, this is stored with the data array in memory (in the first memory). However, this need not be the case, and the similarity metadata can be stored in a different location than the data arrangement, such as other suitable locations in the system, if desired. In fact, the similarity metadata may be relatively small, so it can be stored, for example, in an on-chip memory or buffer instead of an off-chip memory, if necessary.

  If metadata is used, it can be retrieved as appropriate by the processing device. Metadata, for example, signatures, preferably for one or more data blocks, and preferably for multiple data blocks, eg, on-chip metadata, eg, signatures, in buffers, locally on the processing device, For example, it is cached on the processing device itself. This method can eliminate the need to fetch metadata from external memory each time a block similarity assessment is made, thus helping to reduce the memory bandwidth used to read the metadata.

  Most preferably, the metadata for the data array being processed is retrieved (read) in several parts (corresponding to multiple blocks of the data array) prior to reading and processing the relevant data block. . Therefore, the similarity metadata (information) is preferably prefetched for the reading process. Thereby, it becomes possible to perform similarity determination more rapidly.

  When metadata such as data block signatures is cached locally on the processing device, eg, stored in an on-chip buffer, the data blocks are preferably processed in a suitable order, such as Hilbert order, thereby , The likelihood that the metadata will match the data block (s) cached locally (stored in the on-chip buffer).

  As will be appreciated by those skilled in the art, the generation and storage of metadata for data blocks (e.g., rendered tiles) requires some processing and memory resources, but applicants are more likely to We are confident that the potential savings in terms of power consumption and memory bandwidth that can be achieved by using the data in the manner described above will be better.

  As will be appreciated by those skilled in the art, in a particularly preferred embodiment, the generated data array and metadata is then read and used by the processing device in the manner described above.

Thus, according to another aspect of the invention, there is provided a method for processing an array of data, the method comprising:
Generating an array of data to be processed;
For each of one or more blocks of data representing a particular region of the array of data to be processed,
Determining whether a block of data should be considered similar to other blocks of data in the data array;
Generating similarity information indicating whether a block of data has been determined to be similar to other blocks of data in the data array;
Storing an array of data and its associated generated similarity information in a first memory;
Read a block of data, each representing a specific area of the data array from the first memory, and store the block of data in the memory of the processing device where the data array should be processed before the block of data is processed by the processing device And steps to
Use the similarity information generated for the data array to determine if the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device And steps to
For a block of data to be processed, based on the similarity determination, a block of data already stored in the memory of the processing device or an array of data stored in the first memory Processing a new block.

According to another aspect of the present invention, a data processing system is implemented, the system comprising:
A first memory for storing an array of data to be processed;
A data processor for generating an array of data to be processed;
Means for determining, for each of one or more blocks of data representing a particular region of the data array, whether the block of data should be considered similar to other blocks of data in the data array; ,
Means for generating similarity information indicating whether a block of data has been determined to be similar to another block of data in the data array;
Means for storing an array of data and its associated generated similarity information in a first memory;
A processing device having local memory that processes an array of data stored in a first memory by processing successive blocks of data, each representing a particular region of the array of data;
Read a block of data representing a specific area of the array of data stored in the first memory and store the block of data in the local memory of the processing device before the block of data is processed by the processing device A read controller configured to
Whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device using the similarity information generated for the data array Based on the similarity determination, for the block of data to be processed, the block of data already stored in the memory of the processing device, or the array of data stored in the first memory And a control circuit configured to cause the processing device to process a new block of data from.

  As those skilled in the art will appreciate, these aspects and arrangements may comprise one or more or all of the preferred and optional features of the invention described herein, as appropriate, Preferably provided.

  As explained above, although the techniques of the present invention are particularly concerned with the process of reading data from memory for use, applicants have found that the principles of the techniques of the present invention are first and foremost. It is recognized that it can also be used to improve the process of writing data arrays into memory. For example, in particular, applicants need to store a new data block further in the data array if it is determined that the data block is sufficiently similar to a block already generated for the data array. Recognize that there may not be.

  Thus, in a particularly preferred embodiment, when a data block for a data array is being written to a data array in memory, a completed data block (e.g., a rendered tile) It is not written to the data array in memory if it is determined that it should be considered similar to a data block that has already been generated (ie will already be stored in the data array). Thus, this eliminates the need to write data blocks that are determined to be the same as data blocks already stored in the data array to the data array.

  Thus, in this case, when each data block to be written to the data array is generated, it is compared to another data block or blocks of the data array and then the new data block is Based on the comparison, the data array can be written or not written.

  Thus, in a particularly preferred embodiment, when a data block for a data array is completed, the data block is compared with at least one other data block of the data array, and the completed data block is converted into a data array based on the comparison. There is a step or means for determining whether to write.

  This process preferably uses the same block comparison arrangement as described above to compare the signatures representing the contents of the data blocks, or most preferably by directly comparing the contents of the blocks, etc. Determine whether they are similar.

  In these arrangements, the data block itself may not be written to the data array, but the similarity metadata will nevertheless be processed by the processing device instead of any other block in the data array. Therefore, the block position of interest must be generated and stored.

  In a preferred embodiment of these arrangements, the write exclusion process is performed (by comparing the blocks) on blocks that are being generated only for the same data array (current data array).

  However, the comparison is extended to include data blocks from previous data arrays already stored in memory (e.g., frame buffer), and similar data blocks already exist in memory from previous data arrays. It is possible to prevent data blocks that are similar to those from being rewritten to memory with respect to the data array. This may be particularly useful when a series of similar data sequences (such as frames of a video sequence) are being generated. In this case, it is possible to compare the newly generated data block with one or more blocks of the data array already stored in memory (eg based on its contents or content signature).

  In these arrangements, the system preferably places the newly generated data blocks into a data array in memory, periodically, e.g. once per second, for each predetermined data block (data block position). Is configured to always write. Thus, new data blocks are guaranteed to be written into the data array at least periodically for every data block position, so that, for example, erroneously matching data blocks (e.g. Are kept in the data array for longer than a predetermined, eg, desired or selected period, because the signatures of the data blocks happen to match, even if they differ. This can be done, for example, by simply writing a whole new data array periodically (e.g. once per second), or by writing a new data block to the data array in a rolling pattern in a rolling basis. As time elapses, all data block positions are finally written out and renewed.

  In one particularly preferred embodiment, the present invention provides other one or more power and bandwidth reduction schemes, e.g., and preferably compression schemes for data arrays (e.g., frame buffers) (irreversible or reversible if desired). Can be used in conjunction with

  As described above, although the present technology has been specifically applied to graphics processor operations, Applicants have determined that they can store data in the form of blocks in a manner similar to, for example, a tile-based graphics processing system. It has been recognized that other systems that process and load frame buffers, textures, and / or images, for example, can be similarly applied. Thus, for example, the present invention can be applied to a host processor that operates a frame buffer, a graphics processor that reads a texture, a synthesis engine that reads an image to be synthesized, or a video processor that reads a reference frame for video decoding. Thus, for example, the technology can be used for video processing (since video processing is performed on blocks of data similar to tiles in graphics processing), and composite image processing (again, the composite frame buffer is a different block of data Can be used equally well) These are, for example, when a digital camera is processing data (images) generated by a camera sensor, and for example when processing data (images) generated by a digital camera for display. Can also be used.

  The technique can also be used when there are multiple master devices each writing to the same data array, eg, a frame in a frame buffer. This may be the case, for example, when the host processor generates an “overlay” to be displayed on the image being generated by the graphics processor.

  In this case, each device that writes to the data array can update the similarity metadata accordingly, or, for example, invalidates the metadata for the portion of the data array that the other master writes to Or they can be cleared (all these portions of the data array are read to the processing device). The latter is necessary when a given master device cannot update the similarity metadata. Further, for example, when another master corrects a relatively large portion of the data array (or when the data array is corrected in the first place), the metadata for the entire data array can be invalidated (cleared).

  More specifically, if there is a “third party” device that also reads and / or writes the data array, the third party device reads from the data array if only read exclusion is used. Sometimes it is simply possible to read an array of data, usually without using similarity metadata (or actually without knowing about similarity metadata), or third-party devices Metadata can be used to eliminate read transactions.

  If a third party device is writing to the data array, this can update the metadata associated with the data array or the similarity metadata for the data array. It is possible to invalidate part or all. In the latter case, for example, there may be a metadata invalidation bit of the data array at the very beginning of the metadata.

  If both read and write transaction exclusions are used, the third party device uses similarity metadata to eliminate read transactions when reading from a data array. (Unless only read exclusion is used, so a third-party device reading the data array may or may not be able to use the metadata as desired to eliminate the read, If write placement is enabled, when write exclusion is used, the data array may not be `` complete '' (in the case of a data block whose write to the data array has not been `` excluded ''). (The reading device will have to determine which block to use instead from the metadata), so the third-party device must read and use the metadata when reading from the data array.)

  In this case, when writing to the data array, the third party device can update the metadata when writing the data to the data array, as in the above case where only read exclusion is enabled Or part or all of the metadata can be invalidated.

  The metadata generation process (and data block comparison process, if used) can be performed as desired. In one preferred embodiment, the data array generation processor (e.g., GPU, CPU, etc.) itself executes, while in other preferred embodiments, this is done by the data array generation process and the memory in which the data array is to be stored. There is another block or hardware element (logic) in between (for example, a frame buffer). If the metadata generation “unit” is separate (external) from the data array generation process, it can be placed as a separate logic block or, for example, one of the bus fabric and / or interconnects. Part.

  Thus, in a preferred embodiment, there is a metadata generation hardware element (logic) that is separate to a data array generation processor (e.g., a graphics processor), and in another preferred embodiment, the metadata generation logic is Built into (part of) that processor. Thus, in a preferred embodiment, the metadata generation means, etc. are part of the data generation processor (e.g., graphics processor) itself, while in other preferred embodiments, the system includes a data generation processor, and Provide another “metadata generation” unit or element.

  The present invention also extends to the provision of specific hardware elements for performing comparisons and resulting similarity metadata determinations. As described above, for example, this hardware element (logic) is configured as one integral part of, for example, a graphics processor or, for example, an interface between a graphics processor and an external memory controller, for example. Can be a stand-alone element that can be. This may be a programmable or dedicated hardware element.

Thus, according to another aspect of the invention, a data processing system in which an array of data generated by a data processing system is read from an output buffer by reading from the output buffer a block of data that represents a particular region of the array of data. A metadata generation device is provided for use in
Whether the block of data for the data array should be compared to at least one other block of data for the data array and whether the block of data should be considered similar to other blocks of data for the data array based on this comparison Means for generating information to indicate;
Means for storing similarity information associated with the data array.

  As those skilled in the art will appreciate, these aspects and embodiments of the present invention may comprise one or more or all of the preferred and optional features described herein, preferably Prepare. Thus, for example, the comparison preferably includes comparing some or all of the contents of each data block.

  The similarity determination process (and the resulting data block selection process) can be similarly performed as desired. In a preferred embodiment, the processing device (e.g., display controller, GPU, CPU, etc.) is executed by itself, but in other preferred embodiments, this is done by a data processing device and a memory in which the data array is stored ( For example, there is another block or hardware element (logic) in between the frame buffer). If a “unit”, such as a similarity measure, is separate (external) from the processing device, it can again be placed as a separate logic block or, for example, bus fabric and / or mutual It can be part of the connection.

  Thus, in a preferred embodiment, there is a similarity determination hardware element (logic) that is separate to the data array processing device (e.g., display controller), and in another preferred embodiment, the similarity determination logic is the data Embedded in (part of) the array processing device. Thus, in a preferred embodiment, the similarity determination means etc. (reading controller and system controller) are part of the processing device (e.g. display controller) itself, while in other preferred embodiments the system is: A processing device and another “similarity determination” unit or element (including a read controller and / or controller).

  The invention also extends to the provision of specific hardware elements for performing similarity and resulting data block determination. As described above, for example, this hardware element (logic) is configured as one integral part of, for example, a display controller, or, for example, to interface a display controller and an external memory controller, for example. Can be a stand-alone element. This may be a programmable or dedicated hardware element.

Therefore, according to another aspect of the present invention, a similarity determination device is provided for use when processing an array of data stored in a first memory, the device comprising:
In a local memory of a processing device that reads a block of data that represents a specific area of the array of data stored in the first memory and should process the array of data before the block of data is processed by the processing device A read controller configured to store a block of data in
Determine whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and based on the similarity determination, For a block, configured to cause the processing device to process a block of data already stored in the memory of the processing device, or a new block of data from an array of data stored in the first memory And a controller.

  As those skilled in the art will appreciate, these aspects and embodiments can, and preferably do, comprise one or more or all of the preferred and optional features described herein. Thus, for example, the similarity determination is preferably based on similarity metadata associated with the data array.

  Various other preferred and alternative arrangements are possible. For example, in the case of a stereoscopic display in which a left image and a right image are generated and used, the respective “left” and “right” blocks to be displayed are preferably read (and optionally written) exclusions. Compared for purposes (compare the block for the “left” image of the frame with only the block for the “left” image (rather than comparing the “right” block only with the “right” block)). In other words, preferably the left and right portions of the image are compared with each other, and the blocks within each portion of the image are compared. This helps to further reduce the number of transactions to read, as many of the left and right tiles in the image are the same as each other, as the applicant has recognized. Similar arrangements can be used (preferably used) for display and volume display using more than two images.

  In one particularly preferred embodiment, the determined similarity information is used to manage the storage of data blocks in the local memory of the processing device, and in particular a factor in determining the eviction of data blocks from the local memory. Also used as For example, in a preferred embodiment, the metadata is used to determine one or more data blocks that will be used repeatedly by the processing device (e.g., used in the displayed frame), That data block (s) is then temporarily locked into the processing device's local memory (after being written there) and made available in the local memory when needed in the future. Thus, the metadata is preferably used to attempt to pre-identify data blocks that are convenient (if possible) to be kept in the local memory of the processing device, and then the local memory is reasonable Managed accordingly. This can be done, for example, by counting the number of other data blocks that are described as being similar to a given data block when the metadata is created. This information can then be used to control the storage of data blocks in the local memory of the processing device accordingly.

  Keep a count of the number of times a given block of data in local memory will be used in the near future (eg, based on metadata prefetched for the portion of the data array being processed), and its “use” count It is also possible to drive out a data block from local memory only when becomes zero.

  Thus, in a particularly preferred embodiment, eviction of data blocks from the local memory of the processing device is controlled at least in part according to the similarity metadata associated with the data array of interest.

  The present invention can be implemented in any suitable system, such as a suitably configured microprocessor-based system. In a preferred embodiment, the present invention is implemented in a computer and / or microprocessor based system.

  Various functions of the present invention may be performed in a desired and preferred manner as well. For example, the functionality of the present invention can be implemented in hardware or software if desired. Thus, for example, the various functional elements and “means” of the present invention are appropriately dedicated hardware elements and / or programmable hardware elements that can be programmed to operate in a desired manner, etc. May include one or more suitable processors, one or more controllers, functional units, circuits, processing logic, microprocessor arrangements, etc. operable to perform various functions and the like.

  In a preferred embodiment, the output data array generation processor and / or the metadata generation unit is implemented as a hardware element (eg, ASIC). Accordingly, in other aspects, the invention is directed to an apparatus of one or more aspects of the invention described herein, or one or more aspects of the invention described herein. A hardware element is provided that includes an apparatus that operates according to the method.

  It should also be noted that various functions of the present invention may be replicated on a given processor and / or executed in parallel on a given processor, as will be appreciated by those skilled in the art.

  When used in a graphics processing system, the present invention is applicable to any suitable form or configuration of graphics processor and renderer, such as a processor that takes a “pipelined” rendering arrangement (in this case, The renderer takes the form of a rendering pipeline). This is particularly applicable to tile-based graphics processors and graphics processing systems.

  As will be appreciated from the above, the present invention is not exclusively applicable, but is particularly applicable to 2D and 3D graphics processors and processing devices, and accordingly, the invention described herein. 2D and / or 3D graphics processor and 2D and / or comprising an apparatus of one or more aspects or comprising an apparatus operating according to the method of one or more aspects of the invention described herein. Or extended to 3D graphics processing platforms. Assuming that hardware is required to perform the specific functions described above, such 2D and / or 3D graphics processors may in some other way be 2D and / or 3D graphics processors. Can comprise one or more or all of the normal functional units and the like.

  Those skilled in the art will also understand that all described aspects and embodiments of the present invention may include one or more or all of the preferred and optional features described herein, as appropriate. Will do.

  The method according to the invention can be implemented at least in part using software, for example using a computer program. As such, when viewed from another aspect, the present invention is directed to computer software, data processing means specifically adapted to perform the methods described herein when installed on a data processing means. A computer program element that includes computer software code portions that perform the methods described herein when the program element is executed, and is described herein when the program is executed on a data processing system It can be seen that a computer program is provided that includes code means adapted to perform all the steps of one or more methods. The data processing system may be a microprocessor, a programmable FPGA (Field Programmable Gate Array), or the like.

  The present invention further includes such software that, when used to operate a processor or system comprising data processing means, causes said processor or system to perform the steps of the method of the present invention in conjunction with said data processing means. It is also extended to include computer software carriers. Such a computer software carrier can be a physical storage medium such as a ROM chip, a CD ROM, or a disk, or an electronic signal on a cable, an optical signal, or a satellite or the like It can be a signal such as a radio signal.

  Furthermore, it will be understood that not all steps of the methods of the present invention need be performed by computer software, and thus, from a broader perspective, the present invention is not limited to the methods described herein. Computer software and such software installed on a computer software carrier are provided for performing at least one of the plurality of steps.

  Therefore, the present invention can be appropriately implemented as a computer program product for use with a computer system. Such an implementation may include a series of computer readable instructions fixed on a tangible medium such as a non-transitory computer readable medium, such as a diskette, CD ROM, ROM, or hard disk. This can be done on tangible media including, but not limited to, optical or analog communication lines, or using intangible means of wireless technology including but not limited to microwave, infrared, or other transmission technologies, It may also include a series of computer readable instructions that can be transmitted to a computer system via a modem or other interface device. These series of computer readable instructions implement all or part of the functionality already described herein.

  Those skilled in the art will appreciate that such computer readable instructions can be written in many programming languages for use with many computer architectures or operating systems. Further, such instructions may be stored using current or future memory technologies including, but not limited to, semiconductor, magnetic, or light, or light, infrared, or micro, without limitation. It can be transmitted using current or future communication technologies, including waves. Such computer program products are distributed pre-installed in a computer system on a removable medium, for example a system ROM or fixed disk, attached with a printed or electronic document, eg distributed as shrink wrap software, or on a server or network It is intended that it can be distributed from electronic bulletin boards on the Internet or the World Wide Web, for example.

  Numerous preferred embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.

1 is a schematic diagram of a first embodiment in which the present invention is used in conjunction with a tile-based graphics processor. FIG. FIG. 4 is a diagram illustrating an outline of how related data is stored in a memory in an embodiment of the present invention. FIG. 2 is a diagram showing an outline and details of the display controller of the embodiment shown in FIG. FIG. 2 is a diagram illustrating the operation of the display controller in the embodiment shown in FIG. FIG. 2 is a diagram showing an outline and details of the graphics processor of the embodiment shown in FIG. 1; FIG. 2 illustrates the operation of the graphics processor in the embodiment shown in FIG.

  A number of preferred embodiments of the invention will now be described. Although these embodiments are described exclusively with respect to processing of images generated by a graphics processing system for display by a display controller, as described above, the present invention describes a data array that represents a region of the entire array. The present invention can be applied to other arrangement configurations processed in units of blocks.

  FIG. 1 schematically illustrates an arrangement of systems that can operate in accordance with an embodiment of the present invention.

  The system shows a tile-based graphics processor (GPU) 1 as shown in FIG. This is in this embodiment an element of the system that generates the data array to be processed. The data array can typically be an output frame intended to be displayed on the display device 2, such as a screen or a printer, as is known in the art, for example, May include a “render to texture” output, such as graphics processor 1.

  The graphics processor generates an output data array, such as an output frame, to be processed by generating tiles representing different regions of each output data array, as is known in the art.

  As is known in the art, in such an arrangement, the tiles are generated by the graphics processor 1 and then through the interconnect 5 which is normally connected to the memory controller 6. Are written in an output buffer in the form of a frame buffer 3 in the main memory 4 of the system (this memory may be DDR-SDRAM).

  Sometime later, the data array in the frame buffer 3 is read by the display controller 7 and output to the display 2. (Thus, the display controller 7 is a processing device that should process (display in this case) the data array generated by the graphics processor 1.)

  As part of this process, the display controller reads blocks of data from the frame buffer 3, stores them in the local memory buffer 8 of the display controller 7, and then outputs those blocks of data to the display 2. The display device 2 may be a screen or a printer, for example.

  In an embodiment of the present invention, this process considers that a new block of data to be output (processed) for display is similar to a block of data already stored in the local memory 8 of the display controller 7. It further includes a display controller 7 for determining whether or not it should be. To this end, in the embodiment of the present invention, the display controller 7 uses similarity metadata associated with the output frame in the frame buffer generated by the graphics processor 1 when generating the output frame. . (This process is described in more detail below.)

  In essence and as described in more detail below, the display controller 7 determines whether the data block to be processed should be considered similar to a data block already stored in the local buffer 8. If the data block to be processed is found to be similar to the data block already stored in the local buffer 8 of the display controller 7, the display controller does not read the new data block from the frame buffer 3, Instead, the existing data block in the buffer 8 is supplied to the display 2.

  Thus, in an embodiment of the present invention, the display controller 7 and the frame buffer 3 for blocks of data in the frame buffer 3 that are similar to the blocks of data already stored in the local buffer 8 of the display controller 7. Can be avoided. (In the case of games, for example, this is typically the case for much of the user interface, the sky, etc., and most of the play field when the camera position is stationary.) A significant amount of bandwidth and power consumption can be saved in terms of operation.

  On the other hand, if it is determined that the data block to be processed is not similar to the data block already stored in the local buffer 8 of the display controller 7, the display controller will retrieve the new data block from the frame buffer 3. Read into the local buffer 8 and then supply the new data block to the display 2.

  In the embodiment of the present invention, the data block read from the frame buffer 3 and compared with the data block already stored in the buffer 8 of the display controller 7 includes a cache line, which is This is because the amount of data read by the display controller 7 for each read operation. However, other arrangements are possible. For example, the display controller can operate this process on data blocks corresponding to rendered tiles generated by the graphics processor 1 or 2D “partial tiles” of the rendered tiles.

  FIG. 1 also shows a host CPU 9 that can exchange data with the main memory 4 via the interconnect 5 and can write to the frame buffer 3 in the main memory 4, for example. This possibility is described in more detail below.

  In the embodiment of the present invention, as described above, the display controller 7 is similar to a data block in which a predetermined data block (cache line) to be processed for display is already stored in the local buffer 8. Is determined by evaluating the metadata in the form of bitmaps stored in association with the data blocks that make up the frame of interest.

  Each data block position (cache line) in the stored data array in frame buffer 3 is associated with a single bit in the bitmap corresponding to the frame (each bit in the bitmap is a frame Corresponding to one data block position (in this case, a cache line). The bit in the bitmap for a data block (cache line) is set to `` 1 '' if the data block is considered the same as the previous data block (cache line) to be read (processed) from the frame And is set to “0” when the data block is considered to be different from the previous data block.

  In this way, the display controller can read the bitmap entry associated with the data block that is to be processed, and if that bitmap entry is set to `` 1 '', the data Recognize that the block is considered to be the same as the previous data block read into buffer 8 of display controller 7 (thus instead of loading new data into local memory 8 of display controller 7 You can display that data block that already exists). Alternatively, if the metadata associated with the data block to be processed is “0”, the display controller reads a new data block from the frame buffer 3 into the local buffer 8 and displays the new block on the display 2 Know what should be displayed above.

  FIG. 2 shows an exemplary memory layout and associated metadata (data block similarity information) 10 for a data array in the frame buffer 3. In this case, the data blocks constituting the frame are stored as the frame buffer 3, and the associated data block similarity bitmap 10 is stored in another part of the memory 4. (Other processing methods are of course possible.)

  As shown in FIG. 2, each data block in the data array in the frame buffer 3 has an associated entry in the similarity information bitmap 10. Thus, for example, data block 11 in frame buffer 3 is associated with bitmap entry 13 in bitmap 10, and data block 12 in frame buffer 3 is associated with bitmap entry 14 in similarity bitmap 10. It is done.

  FIG. 2 also shows the nature of the bitmap entry. Thus, the bitmap entry 13 indicates that the data block 11 in the data array in the frame buffer 3 is the previous data block (thus the “new” data block to be read from the frame buffer into the local memory 8 of the display controller 7). It has a value “0” indicating that it is not the same. On the other hand, the bitmap entry 14 for the next data block 12 has an entry “1” indicating that the data block 12 is the same as the data block 11 in the frame buffer 3. As a result, the display controller displays the data block 11 stored in the local memory 8 instead of reading the new data block 12 from the frame buffer 3.

  Other similarity metadata arrangements can be used if desired. For example, each data block can potentially be shown to be similar to multiple data blocks in the data array, in which case each bitmap entry has a bitmap entry in the display controller 7. More bits can be included to indicate to which of the data blocks in the data array the corresponding data block is considered similar. In these arrangement configurations, each similarity value (metadata entry) is, for example, a relative indicator (for example, “001” indicating which other data block in the data array is similar to the data block of interest. '' Indicates a data block that precedes the current data block), or an absolute indicator that indicates which data block of interest is similar to other data blocks in the data array (e.g., metadata “125” indicates that the block is similar to the 125th data block in the data array of interest).

  Along with each metadata entry, it is also possible to include a “likeness” value that indicates how similar each data block is. The similarity determination process then uses this likeness value, for example, to read a new block from the data array or to reuse an already existing similar data block in the local memory of the processing device being used. Can be determined. For example, the similarity determination process may set a likeness threshold and compare the likeness value for a new data block with the threshold to read or not read a new data block accordingly.

  It is also possible to use an arrangement other than a bitmap such as a hierarchical quadtree. The metadata (similarity information) associated with the data array also takes the form of a command list that instructs the processing device to read data blocks into the local memory of the processing device according to their relative similarity. Is possible.

  As described further below, in the above bitmap example, the similarity metadata (bitmap) indicates whether each data block should be considered similar to other data blocks in the data array. As shown directly in FIG. 1, this allows each data block to allow the display controller itself to perform a comparison between the data blocks to determine whether the data blocks should be considered similar. It is also possible to associate some information. For example, instead of storing information representing the contents of each data block, the display controller 7 then compares the information of each content of the data block to determine if they should be considered similar It is possible.

  FIG. 3 shows the structure of the display controller 7 in more detail, and FIG. 4 is a flowchart showing the above operation of the display controller 7.

  As shown in FIG. 3, the display controller 7 includes a bus interface unit 20, a metadata buffer 21, a display formatter and output unit 22, and a state machine controller 23, and a data block from the frame buffer 3 to the main memory 4. In addition to the local buffer 8 for storing before display.

  The state machine controller 23 serves to control the display controller 7 that performs the operations of the above-described embodiments. The metadata buffer 21 is used to store a chunk of the metadata bitmap 10 for the frame of interest (data array), thereby improving the efficiency of off-chip memory access. Other arrangements are possible, such as the display controller always reading the metadata in the main memory 4 directly.

  When a new display frame is displayed, the display controller first reads a corresponding portion of the metadata 10 associated with the frame from the main memory 4 and stores it in the metadata buffer 21. The display controller then reads the blocks of data from the frame buffer 3 in the main memory 4 into the data cache / buffer 8 and displays these blocks of data for display 2 via the display formatter / output unit 22 for display. Supply properly. The display controller operates to prefetch the block of data to be displayed into the local memory 8. This is to ensure that there is always data available to display (buffer / memory underruns can cause glitches in the displayed image). These blocks are then read from local memory 8 from one to the next for display. However, this operation is modified under the control of the state machine 23 to follow the process shown in FIG. 4 (and described above).

  As shown in FIG. 4, to process for display (this may be triggered by the display of a block from local memory 8, for example, which If a new data block (cache line) is to be prefetched into the local memory 8, the state machine controller 23 An appropriate arrangement in the similarity metadata bitmap in the metadata buffer 21 for the data block is read (step 31). Next, it is determined whether or not the bit stored at an appropriate position in the similarity bitmap has the value “1” (step 32).

  If the value in the bitmap layout is determined to be `` 1 '', it indicates that the new data block is the same as the previous data block (thus should already be in the local memory 8 of the display controller). Thus, instead of reading a new data block from the frame buffer 3, the state machine controller 23 causes the display controller to use the previous data block already in the local buffer 8 (at the appropriate time), i.e. the local buffer The previous data block from 8 is supplied to the display 2 (step 33). (Here, if there is a sequence of similar blocks (i.e., blocks whose metadata has the value `` 1 ''), the state machine controller selects the first block in the sequence for each successive similar data block, (It will be understood that the display controller is actually reused (repeated).)

  On the other hand, if the value in the bitmap is "0", it indicates that the data block is not the same as the previous data block, so that the data block is taken from the frame buffer 3 for display. Must be prefetched within 8. In this case, the state machine controller 23 reads the data block from the frame buffer 3 in the main memory 4 (step 34), and stores the data block in the local buffer 8 of the display controller (step 35). Let me do it. A new block is then supplied from the local buffer 8 of the display controller 7 to the display device 2 (at the appropriate time) (step 36).

  The data block is then displayed (block 37).

  This process is then repeated, such as the next block of data to be processed (prefetched into local memory 8), and the next block.

  In an embodiment of the present invention, the metadata used by the display controller 7 to determine whether the new block to be processed is the same as the data block already stored in the local buffer 8 comprises the frame Generated by the graphics processor 1 when a tile to be generated is generated. FIG. 5 shows the architecture of the graphics processor 1 that performs this process, and FIG. 6 is a flowchart showing the steps of the metadata generation process.

  As shown in FIG. 5, the graphics processor 1 follows the tile rendering logic 40 and the appropriate meta-data for associating additional data block generation logic with the data array (frame) in the frame buffer 3. Modified to include block comparison logic used to generate the data.

  Block generation logic 41 operates to generate appropriate data blocks from the tiles generated by tile rendering logic 40. In the embodiment of the present invention, the block generation logic appropriately generates a block corresponding to the cache line in the cache memory 8 of the display controller 7. However, as described above, data blocks of other sizes and formats are possible and can be generated by the block generation logic 41 if desired.

  The block generation logic stores successive blocks to be generated in the buffer 42. The comparison logic 43 then compares each data block stored in the buffer 42 (in this case, comparing the new data block with the previous data block) and based on this comparison, sets the appropriate metadata output bit. Generate. To increase memory efficiency, metadata output bits for multiple blocks are collected, merged into one buffer, and then stored as appropriate in the metadata bitmap 10 in main memory 4 (written to off-chip memory) ) (Other processing methods are of course possible.) These data blocks are also read from the buffer 42 and stored in the frame buffer 3 as appropriate.

  To facilitate this operation, the data blocks that make up the output frame are processed in a specific predefined order (both when writing to and reading from the frame buffer). An order in which spatial coherence between blocks can be utilized is preferably used.

  This process is shown as a flow diagram in FIG.

  As shown in FIG. 6, the block generation logic 41 generates a data block (corresponding to a cache line in this case) from the rendered tile generated by the tile rendering logic 40 (step 51). The data block is then stored in the buffer 42.

  The comparison logic 43 then compares the new data block with the previous data block (which is already stored in the buffer 42) (step 52). In the embodiment of the present invention, the comparison logic 43 compares the contents of the data blocks. Other processing methods may be possible. For example, for each block of interest, the comparison logic generates a signature, such as a 32-bit CRC, that represents the contents of the block, and then compares the block signatures rather than comparing the actual contents of the blocks.

  The comparison logic then determines whether the new block should be considered similar to the previous block (step 53). In an embodiment of the invention, this evaluation is based on how similar the contents of the two blocks being compared are. A threshold is set for a particular amount of difference in the LSB of the pixel, and if the difference in the contents of the two blocks is less than this threshold, the blocks are determined to be similar and vice versa.

  (This threshold can be changed (eg, programmed) during use. For example, based on the ratio of static frame data to dynamic frame data and / or power mode in use (eg, low power mode) (It can be set for each application.)

  If the block is determined to be different (not similar) by the comparison logic in step 53, the comparison logic operates to write the value “0” to the appropriate location in the metadata bitmap 10 (step 54). ). This new data block is itself written from the buffer 42 to the frame buffer 3 in the main memory 4 (step 55).

  On the other hand, if it is determined in step 53 that the blocks should be considered similar, the comparison logic 43 operates to cause “1” to be written to the appropriate location in the metadata bitmap 10. (Step 56).

  Then again, the new block can simply be written to the frame buffer 3 in the main memory 4 as if the blocks were considered different. However, FIG. 6 shows one preferred arrangement in which possible “write-out” operations can be enabled in the graphics processor 1. This write exclusion process operates to avoid writing blocks that are determined to be similar to each other to the data array in the frame buffer 3 as described further below. . Thus, as shown in FIG. 6, if the write exclusion process is enabled (step 57), if the two blocks are considered similar to each other, the new block will not be in the frame buffer. Is not written in the data array (step 58). (On the other hand, if the write exclusion process is not enabled in step 57, a new block is written to the frame buffer as usual (step 55).)

  Therefore, the write exclusion process of step 57 may also determine that the data block is the same as the previous data block (i.e., the same data block that was already stored in frame buffer 3). Similarly, the new data block operates so that it is not written to the frame buffer. In this way, the write exclusion process can avoid write traffic to sections of the data array (frame buffer) that are identical to each other. This can further save bandwidth and power consumption for frame buffer operation. On the other hand, if it is determined that the data blocks are different, the new data block is written to the frame buffer as if there was no write exclusion process.

  In these arrangements, the data block itself may not be written to the data array, but the processing device (in the embodiment of the present invention, the display controller) determines which other blocks should be processed instead. Since there is still a need to use that information to do so, similarity metadata should still be generated and stored for the block location of interest.

  In one particularly preferred arrangement of these embodiments, if the data block comparison is inaccurate (assuming that it is actually a different but erroneously matched block), the system uses the newly generated data block as a frame buffer. At regular intervals, for example, once per second, each predetermined data block (data block position) is always written. Thus, it is ensured that new data blocks are written into the frame buffer at least periodically for every data block position, so that, for example, erroneously matching data blocks are predetermined, eg desired or selected. It is avoided that it is held in the frame buffer for a longer period. This can be done, for example, by simply writing out a new output data array periodically (e.g. once per second) or by writing new data blocks to the frame buffer in a rolling pattern on a rolling basis, so Over time, all data block positions will eventually be written out and renewed.

  Various alternatives and modifications to the above arrangement are possible. For example, the output array of data generated by the graphics processor is a graphics texture (e.g., a render "target" is a texture generated using the graphics processor (e.g., a "render to texture" operation). )) Or other outputs of the graphics processor, such as other surfaces to which the output of the graphics processor system is written, may alternatively be included.

  For example, a more advanced metadata arrangement can be used when a data block is not only compared with the previous data block, but also with multiple data blocks in the output frame (data array). is there. In this case, the metadata (e.g., bitmap entry) associated with each respective block location will not only indicate that the corresponding data block is similar to other data blocks in the output data array, but also the output data. It should also indicate which data blocks in the array are similar.

  Similarly, the current completed data block can be compared to multiple data blocks in the data array. This can eliminate the reading of data blocks similar to data blocks elsewhere in the data array, further reducing the number of data blocks that need to be read from main memory for processing. Can be helpful.

  In a preferred embodiment, a software application (e.g., one that triggers the generation of a data array and / or uses and / or receives a generated output array) which region of the output data array It is possible to indicate and control how it is processed, in particular and preferably, for which region of the output sequence the data block comparison process should be performed. This, in turn, allows the process of the present invention to be “turned off” by the application for regions of the output array that “know” that the application will always be updated.

  This can be achieved as desired. One preferred embodiment comprises a register that enables / disables data block (eg, rendered tile) comparison for the output array region, where the software application sets the register accordingly (eg, a graphics processor driver). Through).

  Although embodiments of the present invention have been described above with particular reference to graphics processor operation, applicants have found that the principles of the present invention are in the form of blocks in a manner similar to, for example, a tile-based graphics processing system. It is recognized that other data processing systems, such as frame buffers or textures, can be applied as well. Thus, for example, the present invention can be applied to a host processor that operates a frame buffer, a graphics processor that reads a texture, a synthesis engine that reads an image to be synthesized, or a video processor that reads a reference frame for video decoding. Thus, the techniques of the embodiments are, for example, video processing (since video processing is performed on blocks of data similar to tiles in graphics processing), and composite image processing (again, the composite frame buffer is different in data It can be used equally well (because it is processed as a block).

  They can also be used, for example, when processing data (images) generated by a (digital) camera (video or still image). In this case, the data from the camera sensor, for example, as described above by the camera controller to generate appropriate metadata for the image data written to memory (and control writing of the image data if desired). Can be processed. The image and metadata thus stored can then be processed in the manner of the present invention, for example, by a display controller that is to display the image from the camera.

  This embodiment can also be used when there are multiple master devices each writing to the same output data array, eg, a frame in a frame buffer. This may be the case, for example, when the host processor 9 generates an “overlay” to be displayed on the image generated by the graphics processor 1.

  In this case, each device that writes to the output data array can update the similarity metadata accordingly, or, for example, invalidate the metadata for the portion of the output array that the other master writes to Or they can be cleared (all these parts of the output array are read to the output device). The latter is necessary when a given master device cannot update the similarity metadata. Further, for example, when another master corrects a relatively large portion of the output array (or when the output array is corrected in the first place), the metadata for the entire output array can be invalidated (cleared).

  Various other preferred and alternative arrangements of this embodiment are possible.

  For example, metadata can also be used to manage the storage of data blocks in the local memory 8 of the display controller 7, and in particular can be used as a factor in determining the eviction of data blocks from the local memory 8. . For example, the metadata is used to determine one or more data blocks that will be used repeatedly, and the data block (s) is then (for the time being) in the local memory of the processing device. Locked) (after being written to) and available in local memory when needed in the future.

  Record a count of the number of times a given block of data in local memory 8 will be used in the near future (e.g., based on metadata prefetched for the portion of the output array being processed) and its `` use '' It is also possible to purge data blocks from local memory only when the count reaches zero.

  From the above, it can be seen that the present invention, in its preferred embodiments, can help, for example, at least reduce power consumption and memory bandwidth of the display controller.

  This is accomplished, at least in the preferred embodiment of the present invention, by eliminating unnecessary “main” memory read transactions. This reduces the amount of data read from the main memory, thus greatly reducing the system power consumption and memory bandwidth consumption. This can be applied to graphics frame buffer, graphics to texture render, video frame buffer, composite frame buffer read transaction, etc.

  The power and bandwidth savings when using the present invention can be relatively large. For example, for game and video content with a standard definition frame buffer using 32 byte linear blocks, if the previous four blocks are parsed (requires a multi-bit bitmap), the applicant will And found that about 17% of read and write transactions can be eliminated. In high definition frame buffers, this rejection rate is even higher. GUI content with a similar configuration can eliminate about 80% of frame buffer read and write transactions.

  Assuming a frame display speed of 60 fps (reading), a frame update speed of 30 fps (writing), and 2.4 nJ per 32-bit off-chip transfer, both reading and writing are possible for HD (1920 x 1080 x 24 bpp) Excluded, this corresponds to about 90 MB / s bandwidth savings and 57 mW power savings for gaming and video content. For GUI content, this saving is 427MB / s and 268mW.

  As far as the additional overhead incurred by the need to store metadata in the present invention is concerned, for systems where only the preceding data block is parsed (i.e. the metadata contains a single bit for each data block position) It was found that high definition frames using data blocks corresponding to 32 byte cache lines resulted in 32 KB of additional control data for HD frames occupying 7.9 MB. When using a data block corresponding to a 64-byte tile line, the control data is 16 KB. For a data block corresponding to a 512-byte half tile, this is 2 KB, and for a data block corresponding to a 1024-byte tile, this is 1 KB.

1 Tile-based graphics processor (GPU)
2 Display device
3 Frame buffer
4 Main memory
5 Interconnection
6 Memory controller
7 Display controller
8 Local memory buffer
9 Host CPU
10 Metadata (data block similarity information)
11 data blocks
12 data blocks
13 Bitmap entry
14 Bitmap entry
20 Bus interface unit
21 Metadata buffer
22 Display formatter and output unit
23 State machine controller
40 tile rendering logic
41 Block generation logic
42 buffers
43 Comparison logic

Claims (28)

A method of processing an array of data, wherein a processing device processes the array of data by processing successive blocks of data, each representing a particular region of the array of data, A block of data representing a particular area is read from a first memory in which the array of data is stored and stored in the memory of the processing device before the block of data is processed by the processing device In the method
Determining whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and the block of data already stored Is a block of data from the same data array as the block of data to be processed and is for a position in the data array that is different from the block of data to be processed, the similarity determination Based on the block of data to be processed, the block of data to be processed is determined to be similar to the block of data already stored in the memory of the processing device. was the case, then processing block of the data that is already stored in the memory of the processing device, Other blocks of the data to be processed, if said is determined in the memory of the processing device is not already similar to a block of the data stored, stored in the first memory Processing a new block of data from said array of data. Determining whether a block of data to be processed for the data array is similar to a block of data already stored in the memory of the processing device, and processing based on the similarity determination Process a block of data already stored in the memory of the processing device or a new block of data from the array of data stored in the first memory for the block of data to be processed The steps to do are
If the block of data to be processed is determined to be considered to be similar to the block of data already stored in the B Karumemori of the processing device, a new block of data, the first memory Instead of reading from the data array stored in, storing in the memory of the processing device, but instead processing the existing block of data in the memory of the processing device by the processing device Processing as a block of data to be performed;
If it is determined that the block of data to be processed is not considered similar to the block of data already stored in the memory of the processing device, the new block of data is designated as the first block. Reading from the data array stored in memory, storing it in the memory of the processing device, and then processing the new block of data as a block of data to be processed by the processing device; The method of claim 1 comprising:   The method according to claim 1 or 2, wherein the processing device is one of a display controller, a CPU, a video processor, and a graphics processor.   The similarity determination process determines whether a data block to be processed using similarity information associated with the array of data is similar to a block already stored in the memory of the processing device. 4. A method according to any one of claims 1 to 3 for determining.   The data array is associated with similarity information indicating whether each data block in the data array is similar to other data blocks in the data array, and the similarity determination The process determines whether a data block to be processed using the associated similarity information for the data block is similar to a data block already stored in the memory of the processing device. 5. The method according to any one of 4 to 4. A method of generating metadata for use when processing an array of data stored in memory,
For each of one or more blocks of data representing a particular region of the array of data to be processed,
Determining whether the block of data should be considered similar to other blocks of data for the data array, the other block of data from the same data array as the block of data And for a position in the data array that is different from the block of data;
Generating similarity information indicating whether the block of data has been determined to be similar to other blocks of data in the data array;
Storing said similarity information indicating whether said block of data has been determined to be similar to other blocks of data for said data array in relation to said array of data.   The step of determining whether the block of data should be considered similar to other blocks of data for the data array comprises comparing part or all of the actual contents of the data block 7. The method of claim 6, comprising determining whether the data blocks are considered similar. A method of processing an array of data,
Generating an array of data to be processed;
For each of one or more blocks of data representing a particular region of the array of data to be processed,
Determining whether the block of data should be considered similar to other blocks of data for the data array, the other block of data from the same data array as the block of data And for a position in the data array that is different from the block of data;
Generating similarity information indicating whether the block of data has been determined to be similar to other blocks of data in the data array;
Storing the array of data and its associated generated similarity information;
Reading a block of data each representing a particular region of the array of data from the stored array of data and processing the data array before the block of data is processed by the processing device Storing a block of data in memory;
Using the similarity information generated for the data array, the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device. Determining whether or not
For the block of data to be processed, based on the similarity determination, the block of data to be processed is similar to the block of data already stored in the memory of the processing device. If it is determined that the block of data already stored in the memory of the processing device is processed , or the block of data to be processed is already stored in the memory of the processing device Processing a new block of data from the array of data stored in the first memory if it is determined that it is not similar to the block of data being .   Not writing data blocks to the data array in memory if it is determined that the data block should be considered similar to other data blocks for the data array The method according to any one of claims 6 to 8, further comprising:   The method according to claim 1, wherein the array of data is data representing an image.   11. A method according to any one of the preceding claims, wherein each block of data considered includes a cache line or a 2D partial tile of the data array. A first memory for storing an array of data to be processed;
A processing device having local memory that processes an array of data stored in the first memory by processing successive blocks of data each representing a particular region of the array of data;
Read a block of data representing a particular area of the array of data stored in the first memory and data in the local memory of the processing device before the block of data is processed by the processing device A read controller configured to store the blocks of
Determining whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and the block of data already stored Is a block of data from the same data array as the block of data to be processed and is for a position in the data array that is different from the block of data to be processed, the similarity determination Based on the block of data to be processed, the block of data to be processed is determined to be similar to the block of data already stored in the memory of the processing device. and when the processing block of the data that is already stored in the memory of the processing device Is processed device, or, if the block of the data to be processed is determined to not similar to the block of the data already stored in the memory of the processing device, the first memory A controller configured to cause the processing device to process a new block of data from the array of data stored therein.   The system of claim 12, wherein the read controller and controller are part of the processing device. An apparatus for use in processing an array of data stored in a first memory,
The processing device that reads a block of data representing a particular area of an array of data stored in the first memory and that processes the array of data before the block of data is processed by a processing device A read controller configured to store said block of data in a local memory of
Determining whether the block of data to be processed for the data array is similar to the block of data already stored in the memory of the processing device, and the block of data already stored Is a block of data from the same data array as the block of data to be processed and is for a position in the data array that is different from the block of data to be processed, the similarity determination Based on the block of data to be processed, the block of data to be processed is determined to be similar to the block of data already stored in the memory of the processing device. and when the processing block of the data that is already stored in the memory of the processing device Is processed device, or, if the block of the data to be processed is determined to not similar to the block of the data already stored in the memory of the processing device, the first memory And a control circuit configured to cause the processing device to process a new block of data from the array of data stored therein. The controller is
If it is determined that the block of data to be processed is similar to the block of data already stored in the local memory of the processing device, the read controller is given a new block of data. Without reading from the data array stored in the first memory, causing the new block of data to be stored in the memory of the processing device and causing the processing device to store data in the memory of the processing device; Processing the existing block of data as a block of data to be processed by the processing device;
If it is determined that the block of data to be processed is not considered similar to the block of data already stored in the memory of the processing device, the read controller is given a new block of data. Data to be read from the data array stored in a first memory, stored in the memory of the processing device, and then the processing device to process the new block of data by the processing device 15. A system or apparatus as claimed in any one of claims 12, 13, or 14 configured to be processed as a block of.   16. The system or apparatus according to any one of claims 12, 13, 14, or 15, wherein the processing device is one of a display controller, a CPU, a video processor, and a graphics processor.   The controller determines whether a data block to be processed using similarity information associated with the array of data is similar to a block already stored in the memory of the processing device. Item 17. The system or apparatus according to any one of Items 12 to 16.   The data array is associated with similarity information indicating for each data block of the data array whether the data block is similar to other data blocks in the data array, and the controller 18. The method of claims 12 to 17, wherein the associated similarity information for the data block is used to determine whether the data block to be processed is similar to a data block already stored in the memory of the processing device. A system or apparatus according to any one of the preceding claims. A data processor for generating and processing an array of data;
For each of one or more blocks of data representing a particular region of the array of data for determining whether the block of data should be considered similar to other blocks of data for the data array Means wherein the other block of data is from the same data array as the block of data and is about a position in the data array different from the block of data;
Means for generating similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array;
Means for storing said similarity information indicating whether a block of data has been determined to be similar to other blocks of data for said data array in relation to said array of data .   For each one or more blocks of data representing a particular region of the array of data, to determine whether the block of data should be considered similar to other blocks of data for the data array The means for generating similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array, and the block of data is the array of data 20. The means for storing the similarity information indicating whether it is determined to be similar to other blocks of data for the data array relative to the data array is part of the data processor. System.   The system according to any one of claims 19 and 20, wherein the data processor is one of a camera controller, a graphics processor, a CPU, and a video processor.   The means for determining whether the block of data should be considered similar to other blocks of data for the data array comprises comparing part or all of the actual contents of the data block 22. A system according to any one of claims 19, 20, or 21 comprising means for determining whether said data blocks are considered similar. An apparatus for use in the data processing system wherein an array of data generated by a data processing system is read from the output buffer by reading from the output buffer a block of data representing a particular region of the array of data ,
Comparing the block of data for the data array with at least one other block of data for the data array, wherein the at least one other block of data is from the same data array as the block of data; and , For a position in the data array that is different from the block of data, and based on the comparison, indicates whether the block of data should be considered similar to other blocks of data for the data array Means for generating information;
Means for storing similarity information associated with the data array. A data processor for generating an array of data to be processed;
For each one or more blocks of data representing a particular region of the array of data, to determine whether the block of data should be considered similar to other blocks of data for the data array Means of
Means for generating similarity information indicating whether the block of data has been determined to be similar to other blocks of data for the data array;
Means for storing said array of data and its associated generated similarity information;
A processing device having local memory that processes the stored array of data by processing successive blocks of data each representing a particular region of the array of data;
Read a block of data representing a particular area of the array of data from the stored array of data and store the data in the local memory of the processing device before the block of data is processed by the processing device A read controller configured to store the block;
Using the similarity information generated for the data array, a block of data to be processed for the data array becomes a block of data already stored in the memory of the processing device. Determine whether they are similar and the block of data already stored is a block of data from the same data array as the block of data to be processed and the block of data to be processed The block of data to be processed in the memory of the processing device with respect to the block of data to be processed based on the similarity determination . If it already has been determined to be similar to the block of the data stored in the main of the processing device A block of the data already stored in the re is processed in the processing device, or block of the data to be processed, the block of the data already stored in the memory of the processing device A data processing system comprising: a controller configured to cause the processing device to process a new block of data from the array of data stored in the first memory if determined to be dissimilar .   To avoid writing a data block to the data array in memory if it is determined that the data block should be considered similar to other data blocks for the data array 25. The system or apparatus according to any one of claims 19 to 24, further comprising:   26. The system or apparatus according to any one of claims 12 to 25, wherein the array of data is data representing an image.   27. A system or apparatus as claimed in any one of claims 12 to 26, wherein each block of data under consideration comprises a cache line or 2D partial tile of the data array. The computer, the computer program comprising code order to perform all the steps of the method according to any one of claims 1 to 12.

Images