TWI839085B - Multi-resolution cache - Google Patents

Abstract

A multi-resolution cache includes a first, second and third cache segments the first segment having a first resolution and the second and third segments having a second resolution, the second resolution less than the first resolution, the first and third cache segments communicatively coupled to an off-chip memory, the first and third cache segments configured to each receive a cache line of data having the first and second resolutions, a fourth and fifth cache segments having the second resolution, a first downscaler communicatively coupled to the first and fourth cache segments configured to reduce the resolution when a first resolution cached data is shifted from the first cache segment to the fourth cache segment, a first upscaler communicatively coupled to the all cache segments that have the second resolution, and is configured to increase the reduced resolution cached data to the first resolution and output it.

Description

Multi-resolution cache

The present invention relates to a multi-resolution cache.

Image processing operations, such as spatial filtering and motion compensated frame interpolation MCFI (also known as motion estimation and motion compensation or MEMC), are operated on local image neighborhoods, e.g., local neighborhoods in the spatial domain. Typically, image data caches are used to reduce memory access bandwidth, which read image data from memory only once and provide access to local neighborhoods from the cached data. Therefore, larger neighborhoods mean larger cache sizes and, therefore, higher cache costs.

In view of this, we, the inventors, have devoted ourselves to further research and development and improvement, hoping to find a better invention to solve the above problems. After continuous testing and modification, the present invention was finally born.

In one aspect, a multi-resolution cache includes a first cache segment (102), a second cache segment (108), and a third cache segment (110) having a first resolution, a second resolution, and a third resolution, respectively; the second resolution is smaller than the first resolution, and the third resolution is smaller than the second resolution; the first cache segment (102), the second cache segment (108), and the third cache segment (110) are communicatively coupled to an off-chip memory (120); the first cache segment (102) ), a second cache segment (108) and a third cache segment (110) are configured to receive (image) data of a cache line having a first resolution, a second resolution and a third resolution from an off-chip memory (i.e., located outside the cache, such as a DDRRAM, a (cached) video stream, etc.); a fourth cache segment (106) and a fifth cache segment (104) have the second resolution and the third resolution, respectively; a first resolution reducer (114) is communicatively coupled to the first cache The invention relates to a first cache segment (102) and a fourth cache segment (106) configured to reduce the resolution of a first resolution cache line when it is shifted from the first cache segment (102) to the fourth cache segment (106); a second resolution reducer (112) communicatively coupled to the fourth cache segment (106) and the fifth cache segment (104) and configured to further reduce the resolution of the cache data line when the reduced resolution is shifted from the fourth cache segment (106) to the fifth cache segment (104). The resolution of the reduced-resolution cache data; a first resolution enhancer (118) and a second resolution enhancer (116) are respectively coupled to the second cache segment (108) and the third cache segment (110) in a communication manner, and are also respectively coupled to the fourth cache segment (106) and the fifth cache segment (104) in a communication manner, and are configured to enhance the reduced-resolution cache data and the further reduced-resolution cache data to the first resolution, and output the cache data with enhanced resolution.

Regarding the technical means of our inventors, several preferred embodiments are described in detail below with reference to the drawings so that you can have a deeper understanding and recognize the present invention.

Several embodiments use a multi-resolution cache, whereby a portion of the cache contains images with a higher (e.g., full) resolution and other portions of the cache contain images with a lower resolution. In this way, the same cache size can provide access to a larger neighborhood area than a single resolution cache, or in other words, a lower cache size is required for the same neighborhood size.

The cache system may generate lower resolution data from the higher resolution data if the higher resolution data is available in the cache, and read the lower resolution data from the memory when the higher resolution data is not available in the cache.

The cache system provides data at the highest available resolution, or it provides a mix of two resolutions where two cache resolution segments overlap or transition.

Image processing functions (e.g., spatial or spatiotemporal) can often benefit from a larger size of the local neighborhood (filter aperture size). Embodiments can achieve better image processing results by increasing the aperture size, thereby providing data for pixels farther from a center pixel not at the resolution of the original image data (located in memory), but at a reduced resolution.

A practical example of an application that can benefit from a larger aperture despite reduced resolution is MCFI. In MCFI, intermediate frames are generated between the original video frames, for example, converting a 24 frames per second (fps) film to a 120fps film. MCFI requires extracting image pixels from neighboring input frames along the motion trajectory, for which a spatial aperture is required at each input frame. The available local neighborhood (aperture) size defines the maximum amount of motion that can be correctly compensated before interpolation. For example, if the vertical available aperture size is 101 lines centered at the current pixel position, objects with input frame to output frame motion of up to +/-50 vertical lines will be correctly interpolated, while objects with larger vertical motion will produce incorrect interpolation results. If we provide pixels with an aperture of more than 101 lines at a lower resolution, MCFI will be able to perform motion compensation correctly, despite the lower output resolution.

Thus, embodiments provide a system for storing image data at multiple resolutions in a cache and providing access to the image data, thereby reducing the cache size for the same access range compared to a single resolution cache mechanism, or in other words, providing a larger access range for the same cache size compared to a single resolution cache mechanism.

The multi-resolution cache system includes:

1. At least 2 cache segments, one of which is used to store images at a high resolution and one or more of which is used to store images at a lower resolution, whereby the different resolution segments can optionally partially or completely overlap each other.

2. A mechanism for filling a cache with (image) data whereby a given amount of old data (e.g. one image line) is removed from the cache, the data in the cache is shifted by a given amount (note that being shifted does not necessarily mean physically, but conceptually, e.g. using data pointers), and the given amount of new data is written to the cache, whereby lower resolution data for a lower resolution cache segment is generated using higher resolution data available in the cache or read from memory (e.g. by reducing the resolution of available adjacent high resolution pixel data), and data is only read from memory for the case where no higher resolution version is available in the cache.

3. A mechanism to access (read) data at a desired (image) location from a cache, whereby:

•      If the data location is available in the high-resolution cache segment but not in the low-resolution segment, the high-resolution data is returned.

•      If no data location is available in the high-resolution cache segment but data exists in the low-resolution cache segment, an approximation of the data is calculated and returned using the low-resolution data, such as by interpolating (upgrading resolution) data from its immediate neighbors in the low-resolution cache segment.

•      If a data location is located in an area where two resolution segments overlap or transition, a “mix” of (1) data from a higher resolution cache segment and (2) data from a lower resolution cache segment may be returned, such that the mix ratio is, for example, proportional to the distance of the data location from the two resolution segments. This mechanism ensures that output data in areas of resolution overlap or resolution transition transition transitions abruptly from one resolution to another, but fades gradually between the two resolutions.

o For example, the "blending" operation can be implemented, for example, by "interpolation", so that as the pixel distance D1 from segment 1 becomes smaller, the returned pixel value Pout will be closer to the pixel value P1, and as the pixel distance D2 from segment 2 becomes smaller, the returned pixel value Pout will be closer to the pixel value P2 from segment 2. For example, when "linear interpolation" is used, the above operation can be implemented as Pout = (P1 * D2 + P2 * D1) / (D1+ D2).

o Alternatively, Blend may return the highest resolution available in the overlapping region, rather than the smooth transition described above.

1 shows a multi-resolution cache 100 according to one embodiment. The multi-resolution cache 100 includes a full-resolution cache segment 102, a half-resolution segment 106 and a half-resolution segment 108, and a quarter-resolution cache segment 104 and a quarter-resolution cache segment 110 arranged vertically symmetrically. The half-resolution cache segment 106 and the half-resolution cache segment 108 are arranged vertically above and below the full-resolution cache segment 102, respectively. The quarter-resolution cache segment 104 and the quarter-resolution cache segment 110 are arranged vertically above and below the half-resolution cache segment 106 and the half-resolution cache segment 108, respectively. The multi-resolution cache 100 also includes a resolution downsizer 114 and a resolution downsizer 112 that respectively downs the resolution from the full-resolution cache segment 102 to the half-resolution cache segment 106 by 50% (e.g., using a bilinear resolution downsizer), and further downs the resolution from the half-resolution cache segment 106 to the quarter-resolution cache segment 104 by 50%. For the outputs from the quarter-resolution cache segments 104 and 110 and the half-resolution cache segments 106 and 108, respectively, a 4X upscaling 116 and a 2X upscaling 118 upscale the resolution from the quarter-resolution and half-resolution to the full resolution (e.g., by using bilinear interpolation or other interpolation algorithms).

In the multi-resolution cache 100, the size of a full-resolution segment is C = H*W (where H represents height and W represents width), the size of a half-resolution segment is C/4 = (H/2)*(W/2), and the size of a quarter-resolution segment is C/16 = (H/4)*(W/4). Therefore, the total cache size is 1.625*C data elements (1.625 = 1+2*(quarter) + 2*(1/16)).

Assuming an image row scan direction from top to bottom, the multi-resolution cache 100 may be filled as follows.

•      At each new full-resolution image row, the data in the full-resolution cache segment is shifted up by one row, and a new full-resolution (image) data row is read from the off-chip memory 120 and written to the empty (shifted) row of the full-resolution cache segment 102.

•      At each new half-resolution image row, the data in the half-resolution cache segment 108 at the bottom of FIG. 1 is shifted up one row, and a new half-resolution (image) data row is read from the off-chip memory 120 and written to the empty (shifted) row of the half-resolution cache segment 108 at the bottom of FIG. 1 .

•      At each new quarter resolution image row, the data in the quarter resolution cache segment 110 at the bottom of FIG. 1 is shifted up one row, and a new quarter resolution (image) data row is read from the off-chip memory 120 and written to the empty (shifted) row of the quarter resolution high cache segment 110 at the bottom of FIG. 1 .

•      At each new half-resolution image row, the data in the half-resolution cache segment 106 at the top of FIG. 1 is shifted by one row, and a new half-resolution (image) data row is generated using data cached in the full-resolution cache segment 102 (e.g., reduced in resolution using the resolution reducer 114) and written to the empty (shifted) row of the half-resolution cache segment 106 at the top of FIG. 1.

•      At each new quarter-resolution image row, the data in the quarter-resolution cache segment 104 at the top of FIG. 1 is shifted by one row, and a new quarter-resolution (image) data row is generated using the data cached in the half-resolution cache segment 106 at the top of FIG. 1 (e.g., reduced in resolution using the resolution reducer 112) and written to the empty (shifted) row of the quarter-resolution cache segment 104 at the top of FIG. 1 .

Therefore, in the multi-resolution cache 100, processing the entire image only requires reading the image data of each resolution once in three resolutions (full resolution, half resolution and quarter resolution), as shown in Figure 1, by "Total memory read: 1.3125*F" (1+quarter+1/16=1.3125), where F identifies the size of the image frame.

In summary, the multi-resolution cache 100 provides access to cached data at three resolutions at a cost of 1.625*C cache units and 1.3125*F of memory bandwidth.

If the scanning direction is not top-to-bottom, but bottom-to-top, left-to-right, or right-to-left, the order of shifting cache data and filling empty data rows (or columns) is adjusted accordingly to achieve an equivalent mechanism to the cache filling mechanism explained above, but in a mirrored or cyclic order.

When accessing data cached in low resolution cache segments 104 and 106, where the data in the segment is stored at a reduced spatial resolution, adjacent low resolution data may be used to generate an approximation of the data at the requested location, for example using 2-dimensional spatial interpolation.

As will be discussed in the following examples, note that:

•      Cache can have 2 or more levels.

•      Stacks can be implemented horizontally or vertically.

•      Data reduction factors (e.g. spatial resolution reduction factor) can be selected dynamically based on the required access range and available cache size.

•      The horizontal and vertical data reduction for each resolution level (e.g., spatial resolution reduction levels) can be equal or can be different from each other.

•      The cache does not need to be positioned symmetrically around the current pixel position, but can be arranged with a (horizontal and/or vertical) offset relative to the current pixel position (so-called tilted positioning of the cache).

•      Cache data allocation for each resolution level can be done dynamically and in an unequal manner. For example, a larger amount of cache size can be allocated to the bottom cache segments compared to the top cache segments when needed.

FIG. 2 illustrates a multi-resolution cache 200 according to one embodiment. The multi-resolution cache 200 includes a full-resolution cache segment 202, a resolution reducer 204, a half-resolution cache segment 206, a half-resolution cache segment 208, a half-resolution cache segment 210, a half-resolution cache segment 212, and a resolution enhancer. An off-chip memory 214 is communicatively coupled to the full-resolution cache segment 202 and the half-resolution cache segment 210. The multi-resolution cache 200 operates in a manner similar to the multi-resolution cache 100, except that there is no quarter-resolution cache segment. Therefore, only one resolution reduction operation is required to generate half-resolution data for filling the cache segment 206 from the full-resolution data available in the cache segment 202. The half-resolution data used to fill the cache segment 210 is read from the memory. As shown, the multi-resolution cache 200 has symmetrical top/bottom resolution segments (in order to generate the half-resolution data from the full-resolution data available in the cache, a resolution reduction operation is required). The required cache size is 2*C and the required memory read bandwidth size is 1.25*F.

In one embodiment, cache segments 210 and 212 below full-resolution cache segment 202 are effectively unified to form a single segment as it were, and cache segments 206 and 208 above full-resolution cache segment 202 are effectively unified to form another single segment as it were. That is, there are only three segments: a full-resolution segment and two half-resolution (or other resolution) segments.

FIG3 illustrates a multi-resolution cache 300 according to one embodiment. The multi-resolution cache 300 operates in a manner similar to the multi-resolution cache 200, except that the full-resolution cache segment 302 completely overlaps the half-resolution cache segment 304, thereby eliminating the need for a resolution reduction operation. Specifically, the multi-resolution cache 300 includes a full-resolution cache segment 302, a half-resolution cache segment 304, a half-resolution cache segment 308, a half-resolution cache segment 310, a half-resolution cache segment 312, a half-resolution cache segment 314, and a resolution upscaling device 316. The off-chip memory 306 is communicatively coupled to the full-resolution cache segment 302 and the half-resolution cache segment 314. The required cache size is 2.25*C, and the required memory read bandwidth is 1.25*F.

4 illustrates a multi-resolution cache 400 according to one embodiment. The multi-resolution cache 400 includes a full-resolution cache segment 402, a half-resolution cache segment 404, a half-resolution cache segment 406, a half-resolution cache segment 408, a half-resolution cache segment 410, a resolution downsizer 412, a resolution downsizer 414, a resolution upsizer 418, and a resolution upsizer 420. An off-chip memory 416 is communicatively coupled to the full-resolution cache segment 402.

Multi-resolution cache 400, like multi-resolution cache 100, has 3 resolutions, but has asymmetric top/bottom resolution segments. In order to generate half-resolution data from the full-resolution data available in full-resolution cache segment 406, and to generate quarter-resolution data from the half-resolution data available in half-resolution cache segment 408, two resolution reduction operations are required. Using 3 full-resolution levels in the bottom half of the cache stack eliminates the necessity to read multiple image resolutions from memory; only a single (full) resolution image needs to be read from memory. The required cache size is 3.3125*C, and the required memory read bandwidth size is 1*F.

FIG5 illustrates a multi-resolution cache 500 according to one embodiment. The multi-resolution cache 500 is coupled to a full-resolution cache segment 502, a half-resolution cache segment 504, a quarter-resolution cache segment 506, a half-resolution cache segment 508, a quarter-resolution cache segment 510, a resolution reducer 512, a resolution reducer 514, a resolution upscaling 518, and a resolution upscaling 520. However, the current pixel row is tilted upward in this example (i.e., the current pixel row need not be centered in the middle of the full-resolution cache segment 502). An off-chip memory 516 is communicatively coupled to the full-resolution cache segment 502 and the quarter-resolution cache segment 506.

FIG6 shows a multi-resolution cache 600 according to an embodiment. In this embodiment, the sizes of all cache segments are dynamically allocated. The multi-resolution cache 600 includes a full-resolution cache segment 602, a half-resolution cache segment 604, a half-resolution cache segment 606, a half-resolution cache segment 608, a quarter-resolution cache segment 610, a resolution reducer 612, a resolution reducer 614, a resolution enhancer 618, and a resolution enhancer 620. The full-resolution cache segment 602, the half-resolution cache segment 604, and the half-resolution cache segment 606 are coupled to an off-chip memory 616 in a communication manner.

Multi-resolution cache 600 operates in a manner similar to multi-resolution cache 500, where the current pixel row is tilted upward in this example (i.e., the current pixel row need not be centered in the middle of full-resolution cache segment 602), except that the cache data is distributed unequally at the top and bottom of full-resolution cache segment 602 (more cache data is allocated to the bottom and less is allocated to the top, which can be performed dynamically).

In its most basic implementation, a multi-resolution cache system comprises two segments, each having a different resolution, whereby the higher resolution cache segment obtains its content from an off-chip memory if the content is not available in the higher resolution segment, and the lower resolution cache segment obtains its content from an off-chip memory if the content is available in the higher resolution segment, or obtains its content by downscaling the content of the higher resolution segment if the content is available in the higher resolution segment.

FIG. 7 illustrates a method 700 according to one embodiment. Any cache described herein may perform method 700. In box 702, a multi-resolution cache reads data of a cache line at full resolution from an off-chip memory. In box 704, the multi-resolution cache stores the read data in a first full-resolution cache segment of the cache. In box 706, as the data is shifted out of the full-resolution cache, the multi-resolution cache reduces the resolution of the read data to a resolution lower than the full resolution. In box 708, the multi-resolution cache shifts the reduced-resolution data to a first low-resolution cache segment of the cache. In box 710, the multi-resolution cache reads data of a second cache line at full resolution from the off-chip memory. In block 712, the multi-resolution cache stores the data of the read second cache line in the first full-resolution cache segment. In block 714, the multi-resolution cache upscales the reduced-resolution data to full resolution. In block 716, the multi-resolution cache outputs the upscaled-resolution data (e.g., for display or other purposes).

The following examples describe various embodiments of the methods, computer-readable media, and systems (eg, machines, devices, or other apparatuses) discussed herein.

1. A multi-resolution cache comprising:

a first cache segment, a second cache segment, and a third cache segment, the first cache segment, the second cache segment, and the third cache segment having a first resolution, a second resolution, and a third resolution, respectively, the second resolution being less than the first resolution and the third resolution being less than the second resolution, the first cache segment, the second cache segment, and the third cache segment being communicatively coupled to an off-chip memory, the first cache segment, the second cache segment, and the third cache segment being configured to receive data of cache lines having the first resolution, the second resolution, and the third resolution;

a fourth cache segment and a fifth cache segment, the fourth cache segment and the fifth cache segment having the second resolution and the third resolution, respectively;

a first resolution reducer communicatively coupled to the first cache segment and a fourth cache segment and configured to reduce the resolution when the resolution cache line is shifted from the first cache segment to the fourth cache segment;

a second resolution reducer communicatively coupled to the fourth cache segment and the fifth cache segment and configured to further reduce a resolution of the reduced-resolution cache data as the reduced-resolution cache data line is shifted from the fourth cache segment to the fifth cache segment;

A first resolution upscaling device and a second resolution upscaling device, wherein the first resolution upscaling device and the second resolution upscaling device are respectively coupled to the second cache segment and the third cache segment in a communication manner and are also respectively coupled to the fourth cache segment and the fifth cache segment in a communication manner, and are configured to upscale the reduced-resolution cache data and the further reduced-resolution cache data to the first resolution and output the cache data with the upscaled resolution.

2. The multi-resolution cache of Example 1, wherein the second resolution is half of the first resolution, and the third resolution is one quarter of the first resolution.

3. The multi-resolution cache of any of the preceding examples, wherein the first resolution is full resolution.

4. The multi-resolution cache of any of the preceding examples, wherein the second cache segment, the third cache segment, the fourth cache segment, and the fifth cache segment are configured in a symmetrical configuration around the first cache segment.

5. The multi-resolution cache of any of the preceding examples, wherein the cache data allocation of the second cache segment is not equal to the cache data allocation of the fourth cache segment.

6. A multi-resolution cache as in any of the preceding examples, wherein cache data size allocations and resolutions of all cache segments are dynamically or unequally allocated, such that a downscaler is adjusted accordingly to generate a reduced resolution target resolution from a data source resolution.

7. The multi-resolution cache of any of the preceding examples, wherein the current row of pixels is not centered within the first cache segment.

8. The multi-resolution cache of any of the preceding examples, further comprising a sixth cache segment having a resolution less than the first resolution and overlapping the first cache segment.

9. A method of operating a multi-resolution cache of any of the preceding examples, comprising:

Reading data of one cache line at full resolution from the off-chip memory;

storing the read data in the first cache segment of the cache;

reducing the resolution of the data of the first cache segment to a resolution lower than the full resolution;

shifting reduced resolution data to the fourth cache segment of the cache;

Reading data of a second cache line at full resolution from the off-chip memory;

storing the read data of the second cache line in the first cache segment;

Upscaling the reduced resolution data to the full resolution; and

Outputs data with increased resolution.

10. A multi-resolution cache comprising:

a first cache segment, a second cache segment, and a third cache segment, the first cache segment having a first resolution and the second cache segment and the third cache segment having a second resolution, the second resolution being less than the first resolution, the first cache segment and the third cache segment being communicatively coupled to an off-chip memory, the first cache segment and the third cache segment being configured to receive data of cache lines having the first resolution and the second resolution;

a fourth cache segment and a fifth cache segment, the fourth cache segment and the fifth cache segment having the second resolution;

a first resolution reducer communicatively coupled to the first cache segment and a fourth cache segment and configured to reduce the resolution when a cache data line at the first resolution is shifted from the first cache segment to the fourth cache segment;

A first resolution upscaling device is communicatively coupled to the fourth cache segment and the fifth cache segment and is configured to upscale the reduced-resolution cache data to the first resolution and output the upscaled-resolution cache data.

11. The multi-resolution cache of any of the preceding examples, wherein the second cache segment is configured to receive cache data having the second resolution from the third cache segment when data is shifted from the third cache segment to the second cache segment.

12. The multi-resolution cache of any of the preceding examples, wherein the second resolution is half the first resolution.

13. The multi-resolution cache of any of the preceding examples, wherein the second cache segment, the third cache segment, the fourth cache segment, and the fifth cache segment are configured in a symmetrical configuration around the first cache segment.

14. The multi-resolution cache of any of the preceding examples, wherein cache data distribution across all cache segments is unequal.

15. The multi-resolution cache of any of the preceding examples, further comprising a sixth cache segment having a resolution less than the first resolution and overlapping the first cache segment.

16. A multi-resolution cache comprising:

a first cache segment, a second cache segment, and a third cache segment, the first cache segment, the second cache segment, and the third cache segment having a first resolution, the first cache segment being communicatively coupled to an off-chip memory, the first cache segment being configured to receive data of a cache line having the first resolution;

a fourth cache segment and a fifth cache segment, the fourth cache segment and the fifth cache segment having the second resolution and a third resolution, respectively, the second resolution being smaller than the first resolution and the third resolution being smaller than the second resolution;

a first resolution reducer communicatively coupled to the fourth cache segment and configured to reduce the resolution of a cache line of a first resolution when the cache line is shifted from the third cache segment to the fourth cache segment;

a second resolution reducer communicatively coupled to the fourth cache segment and the fifth cache segment and configured to further reduce a resolution of the reduced-resolution cache data line as the reduced-resolution cache data line is shifted from the fourth cache segment to the fifth cache segment; and

A first resolution upscaling device and a second resolution upscaling device are respectively coupled to the fourth cache segment and the fifth cache segment in a communication manner, and are configured to upscale the reduced-resolution cache data and the further reduced-rate cache data to the first resolution and output the cache data with the upscaled resolution.

17. The multi-resolution cache of any of the preceding examples, wherein the second resolution is half the first resolution, and the third resolution is one quarter the first resolution.

18. The multi-resolution cache of any of the preceding examples, wherein the current pixel row is not centered within the third cache segment.

19. A method of operating a multi-cache system as in any of the preceding examples, comprising:

Reading data of one cache line at full resolution from the off-chip memory;

storing the read data in the first cache segment of the cache;

reducing the resolution of the data of the first cache segment to a resolution lower than the full resolution;

shifting reduced resolution data to the fourth cache segment of the cache;

Reading data of a second cache line at full resolution from the off-chip memory;

storing the read data of the second cache line in the first cache segment;

Upscaling the reduced resolution data to full resolution; and

Outputs data with increased resolution.

20. A multi-resolution cache comprising:

a first cache segment and a second cache segment, the first segment having a first resolution and the second segment having a second resolution, the second resolution being less than the first resolution, the first cache segment being communicatively coupled to an off-chip memory, the first cache segment being configured to receive data of a cache line having the first resolution;

a first resolution reducer communicatively coupled to the first cache segment and the second cache segment and configured to reduce a resolution of received cache line data at a first resolution as the received cache line data is shifted from the first cache segment to the second cache segment;

A first resolution upscaling device is communicatively coupled to the second cache segment and is configured to upscale the reduced resolution cache data to the first resolution and output the upscaled resolution cache data.

21. A multi-resolution cache as in any of the preceding examples, wherein cache data size allocations and resolutions of all cache segments are dynamically or unequally allocated, such that a downscaler is adjusted accordingly to generate a reduced resolution target resolution from a data source resolution.

22. The multi-resolution cache of any of the preceding examples, wherein the second cache segment is communicatively coupled to the off-chip memory and is further configured to receive the cache line data from the off-chip memory when the cache line data is not available in the first cache segment.

Although the described flow charts may illustrate operations as a sequential process, many operations may be performed in parallel or simultaneously. In addition, the order of the operations may be rearranged. A process is terminated when its operations are completed. A process may correspond to a method, procedure, algorithm, etc. The operations of a method may be performed in whole or in part, may be performed in conjunction with some or all of the operations in other methods, and may be performed by any number of different systems, such as the systems described herein, or any portion thereof, such as a processor included in any of these systems.

In summary, the technical means disclosed in this invention can effectively solve the problems of knowledge and achieve the expected purpose and effect. Moreover, it has not been seen in publications and has not been publicly used before the application, and it has long-term progress. It is indeed an invention as referred to in the Patent Law. Therefore, I have filed an application in accordance with the law and sincerely pray that the Supreme Court will give a detailed review and grant the invention patent. I will be very grateful.

However, the above are only several preferred embodiments of the present invention, and should not be used to limit the scope of implementation of the present invention. In other words, all equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the invention specification should still fall within the scope of the present invention patent.

[The present invention] 100: multi-resolution cache 102, 104, 106, 108, 110: cache segment 114, 112: resolution reducer 116, 118: resolution enhancer 120: off-chip memory 200: multi-resolution cache 202, 206, 208, 210, 212: cache segment 204: resolution reducer 300: multi-resolution cache 306: off-chip memory 302, 304, 308, 310, 312, 314: cache segment 316: resolution enhancer 400: multi-resolution cache 402, 404, 406, 408, 410: cache segment 412, 414: resolution reducer 418, 420: resolution upscaling 416: off-chip memory 500: multi-resolution cache 502: full-resolution cache segment 504: half-resolution cache segment 506, 510: quarter-resolution cache segment 508: half-resolution cache segment 516: off-chip memory 512, 514: resolution reducer 518, 520: resolution upscaling 600: multi-resolution cache 602: full-resolution cache segment 604, 606, 608: half-resolution cache segment 610: quarter-resolution cache segment 612, 614: resolution reducer 618, 620: resolution upscaling 616: off-chip memory 700: method 702, 704, 706, 708, 710, 712, 714, 716: frame

To easily identify the discussion of any specific element or action, one or more of the most significant digits in a reference number refers to the figure number in which the element is first introduced. [Figure 1] shows a multi-resolution cache according to an embodiment; [Figure 2] shows a multi-resolution cache according to an embodiment; [Figure 3] shows a multi-resolution cache according to an embodiment; [Figure 4] shows a multi-resolution cache according to an embodiment; [Figure 5] shows a multi-resolution cache according to an embodiment; [Figure 6] shows a multi-resolution cache according to an embodiment; [Figure 7] shows a method according to an embodiment.

300:Multi-resolution cache

306: Off-chip memory

302, 304, 308, 310, 312, 314: Cache segments

316: Resolution Enhancer

Claims (19)

A multi-resolution cache, characterized in that it includes: a first high-speed cache segment, a second high-speed cache segment and a third high-speed cache segment, wherein the first high-speed cache segment, the second high-speed cache segment and the third high-speed cache segment have a first resolution, a second resolution and a third resolution respectively, the second resolution is smaller than the first resolution and the third resolution is smaller than the second resolution, the first high-speed cache segment, the second high-speed cache segment and the third high-speed cache segment are coupled to an off-chip memory in a communication manner a cache memory, the first cache segment, the second cache segment, and the third cache segment configured to receive data of cache lines having the first resolution, the second resolution, and the third resolution, the current pixel line not being centered in the first cache segment; a fourth cache segment and a fifth cache segment, the fourth cache segment and the fifth cache segment having the second resolution and the third resolution, respectively; a first resolution reducer, the first resolution reducer being communicatively coupled to The first cache segment and the fourth cache segment are configured to reduce the resolution when the resolution cache line is shifted from the first cache segment to the fourth cache segment; a second resolution reducer, the second resolution reducer is communicatively coupled to the fourth cache segment and the fifth cache segment and is configured to further reduce the reduced resolution when the reduced resolution cache data line is shifted from the fourth cache segment to the fifth cache segment. The first and second resolution upscaling devices are respectively coupled to the second and third cache segments in a communication manner and are also respectively coupled to the fourth and fifth cache segments in a communication manner, and are configured to upscale the reduced-resolution cache data and the further reduced-resolution cache data to the first resolution, and output the cache data with the upscaled resolution. A multi-resolution cache as claimed in claim 1, wherein the second resolution is half of the first resolution, and the third resolution is one quarter of the first resolution. A multi-resolution cache as claimed in claim 2, wherein the first resolution is full resolution. A multi-resolution cache as claimed in claim 1, wherein the second cache segment, the third cache segment, the fourth cache segment and the fifth cache segment are configured to be symmetrical around the first cache segment. A multi-resolution cache as claimed in claim 1, wherein the cache data allocation of the second cache segment is not equal to the cache data allocation of the fourth cache segment. A multi-resolution cache as claimed in claim 1, wherein cache data size allocations and resolutions of all cache segments are dynamically or unequally allocated, thereby adjusting the downscaler accordingly to generate a reduced resolution target resolution from a data source resolution. The multi-resolution cache as claimed in claim 1, further comprising a sixth cache segment, wherein the sixth cache segment has a resolution less than the first resolution and overlaps the first cache segment. A method of operating a multi-resolution cache as described in claim 1, characterized in that it includes: reading data of a cache line at full resolution from the off-chip memory; storing the read data in the first cache segment of the cache; reducing the resolution of the data in the first cache segment to a resolution lower than the full resolution; shifting the reduced resolution data to the fourth cache segment of the cache; reading data of a second cache line at full resolution from the off-chip memory; storing the read data of the second cache line in the first cache segment; upscaling the resolution of the downscaling data to the full resolution; and outputting the upscaling data. A multi-resolution cache, characterized in that it includes: a first cache segment, a second cache segment, and a third cache segment, wherein the first cache segment has a first resolution and the second cache segment and the third cache segment have a second resolution, the second resolution being less than the first resolution, the first cache segment and the third cache segment being coupled to an off-chip memory in a communication manner, the first cache segment and the third cache segment being configured to receive data of cache lines having the first resolution and the second resolution; a fourth cache segment and a fifth cache segment, wherein the fourth cache segment and the fifth cache segment are coupled to an off-chip memory in a communication manner, A cache segment has the second resolution; a first resolution reducer, the first resolution reducer is communicatively coupled to the first cache segment and the fourth cache segment and is configured to reduce the resolution when a cache data line of the first resolution is shifted from the first cache segment to the fourth cache segment; a first resolution upscaling device, the first resolution upscaling device is communicatively coupled to the fourth cache segment and the fifth cache segment and is configured to upscale the downscaled cache data to the first resolution and output the upscaled cache data, and the cache data distribution of all cache segments is unequal. A multi-resolution cache as described in claim 9, wherein the second cache segment is configured to receive cache data having the second resolution from the third cache segment when data is shifted from the third cache segment to the second cache segment. A multi-resolution cache as described in claim 9, wherein the second resolution is half of the first resolution. A multi-resolution cache as claimed in claim 9, wherein the second cache segment, the third cache segment, the fourth cache segment and the fifth cache segment are configured to be symmetrical around the first cache segment. The multi-resolution cache as described in claim 9, further comprising a sixth cache segment, wherein the sixth cache segment has a resolution less than the first resolution and overlaps the first cache segment. A multi-resolution cache, characterized in that it includes: a first cache segment, a second cache segment, and a third cache segment, wherein the first cache segment, the second cache segment, and the third cache segment have a first resolution, the first cache segment is coupled to an off-chip memory in a communication manner, the first cache segment is configured to receive data of a cache line having the first resolution, and a current pixel line is not centered in the third cache segment; a fourth cache segment and a fifth cache segment, wherein the fourth cache segment and the fifth cache segment have the second resolution and the third resolution, respectively, the second resolution is less than the first resolution and the third resolution is less than the second resolution; a first resolution reducer, the first resolution reducer is coupled to the fourth cache segment in a communication manner, and A cache line of a first resolution configured to reduce the resolution when the cache line is shifted from the third cache segment to the fourth cache segment; a second resolution reducer, the second resolution reducer is communicatively coupled to the fourth cache segment and the fifth cache segment, and is configured to further reduce the resolution of the cache data of the reduced resolution when the cache data line of the reduced resolution is shifted from the fourth cache segment to the fifth cache segment; and a first resolution upscaling device and a second resolution upscaling device, the first resolution upscaling device and the second resolution upscaling device are communicatively coupled to the fourth cache segment and the fifth cache segment, respectively, and are configured to upscale the cache data of the reduced resolution and the cache data of the further reduced resolution to the first resolution and output the cache data of the upscaled resolution. A multi-resolution cache as claimed in claim 14, wherein the second resolution is half of the first resolution, and the third resolution is one quarter of the first resolution. A method of operating a multi-cache system as described in claim 14, characterized in that it includes: reading data of a cache line at full resolution from the off-chip memory; storing the read data in the first cache segment of the cache; reducing the resolution of the data in the first cache segment to a resolution lower than the full resolution; shifting the reduced resolution data to the fourth cache segment of the cache; reading data of a second cache line at full resolution from the off-chip memory; storing the read data of the second cache line in the first cache segment; upscaling the resolution of the downscaling data to the full resolution; and outputting the upscaling data. A multi-resolution cache, characterized in that it includes: a first cache segment and a second cache segment, the first cache segment having a first resolution and the second segment having a second resolution, the second resolution being less than the first resolution, the first cache segment being communicatively coupled to an off-chip memory, the first cache segment being configured to receive cache line data having the first resolution; a first resolution reducer, the first resolution reducer being communicatively coupled to the first cache segment and the second cache segment, and being configured to reduce the resolution of the received cache line data at the first resolution when the received cache line data is shifted from the first cache segment to the second cache segment; a first resolution upscaling device, the first resolution upscaling device being communicatively coupled to the second cache segment, and being configured to upscale the downscaled cache data to the first resolution and output the upscaled cache data. A multi-resolution cache as claimed in claim 17, wherein cache data size allocations and resolutions of all cache segments are dynamically or unequally allocated, thereby adjusting the downscaler accordingly to generate a reduced resolution target resolution from a data source resolution. A multi-resolution cache as claimed in claim 17, wherein the second cache segment is communicatively coupled to the off-chip memory and is further configured to receive the cache line data from the off-chip memory when the cache line data is not available in the first cache segment.