CN110583018B

CN110583018B - System and method for intelligent data/frame compression in a system-on-chip

Info

Publication number: CN110583018B
Application number: CN201880025343.XA
Authority: CN
Inventors: S·加代尔拉布; C·秋; M·卡恩; K·厄内万; T·隆哥; S·布思; M·瓦里亚; M·阿列克西克
Original assignee: Qualcomm Inc
Current assignee: Qualcomm Inc
Priority date: 2017-04-18
Filing date: 2018-03-29
Publication date: 2022-07-29
Anticipated expiration: 2038-03-29
Also published as: TW201842783A; US20200195977A1; TWI702837B; TW202118302A; US20180302624A1; CN110583018A; US10609418B2; SG11201908232VA; EP3613211A2; WO2018194815A3; WO2018194815A2

Abstract

An exemplary method for intelligent compression defines a threshold value for a temperature reading generated by a temperature sensor. The data blocks received into the compression module are compressed according to a first mode or a second mode, the selection of the first mode or the second mode being based on a comparison of the activity level for the temperature reading to a defined threshold value. The first compression mode may be associated with a lossless compression algorithm, while the second compression mode is associated with a lossy compression algorithm. Alternatively, both the first compression mode and the second compression mode may be associated with a lossless compression algorithm, however, for the first compression mode, the received data block is generated at a default high quality level setting, and for the second compression mode, the received data block is generated at a reduced quality level setting.

Description

System and method for intelligent data/frame compression in a system-on-chip

Technical Field

Portable computing devices ("PCDs") are becoming a necessity of people on a personal and professional level. These devices may include cellular telephones, Portable Digital Assistants (PDAs), portable game consoles, palm top computers, and other portable electronic devices. PCDs typically contain integrated circuits or systems on a chip ("socs") that include a large number of components designed to work together to deliver functionality to a user. For example, a SoC may contain any number of processing engines, such as modems, central processing units ("CPUs") comprised of cores, graphics processing units ("GPUs"), and the like, that read data and instructions from and write data and instructions to memory components on the SoC. Data and instructions are sent between devices via a set of conductors called a bus.

Efficiently sizing bus bandwidth and memory components in a PCD is important to optimize the functional capabilities of the processing components on the SoC and to guarantee a minimum required quality of service (QoS) level. In general, memory capacity and bus bandwidth utilization are further optimized by compressing data such that the data requires less bus bandwidth to transmit and less memory space to store. However, not all data/frames are compressed with the same efficiency, and thus PCD designers are faced with trade-off decisions: compression using lossy compression methods that produce lower quality output at decompression, and in exchange for smaller memory components and bus bandwidth requirements; or alternatively, compressed using a lossless compression method that produces high quality output at decompression, but requires relatively large memory components and bus bandwidth to maintain satisfactory QoS. Either way, the designers must size the bus and memory components in view of the "actual worst case" of compression, or they are at risk of reduced QoS as measured by any number of key performance indicators.

Briefly, current systems and methods for data/frame compression known in the art indicate that PCD designers, in order to ensure delivery of an acceptable QoS level, must utilize memory components and bus bandwidth that are too large for most use cases, regardless of the compression method used. Accordingly, there is a need in the art for systems and methods that enable PCD designers to properly size memory components and bus bandwidths so that optimal compression methods can be used for a given use case. More specifically, there is a need in the art for intelligent compression systems and methods that utilize a mixed-mode compression approach to optimize QoS in view of any one or more key performance indicators.

Disclosure of Invention

Various embodiments of methods and systems for intelligent data compression in a portable computing device ("PCD") are disclosed. An exemplary method begins by: a threshold value is defined for a temperature reading generated by a temperature sensor within the PCD. Depending on the embodiment, the temperature readings may be associated with a faceplate or case temperature of the PCD, a PoP memory device temperature, a die junction temperature, and the like. Next, a first data block is received into the compression module according to the first compression mode. The compression module is operable to switch between a first compression mode and a second compression mode. Where a first data block is received into the compression module, the activity level for the temperature reading is monitored and compared to a previously defined threshold value for the temperature reading. Based on a comparison of the activity level for the temperature reading to a defined threshold value, the compression module may be switched to a second compression mode such that a second data block is received into the compression module according to the second compression mode.

In an alternative embodiment, a threshold value for an average bandwidth reading over a period of time is generated. The bandwidth readings may be associated with a DRAM or other memory device. Where a first data block is received into the compression module, the activity level for the bandwidth reading is monitored and compared to a previously defined threshold value for the bandwidth reading. Based on a comparison of the activity level for the bandwidth reading to a defined threshold value, the compression module may be switched to a second compression mode such that a second data block is received into the compression module according to the second compression mode.

Depending on the embodiment of the solution, the first compression mode may be associated with a lossless compression algorithm, while the second compression mode is associated with a lossy compression algorithm. In other embodiments, both the first compression mode and the second compression mode may be associated with a lossless compression algorithm, however, for the first compression mode, the received data block is generated at a default high quality level setting, and for the second compression mode, the received data block is generated at a reduced quality level setting. Furthermore, it is contemplated that some embodiments of the solution may be configured for application to data blocks in the form of image frames, while other embodiments of the solution are configured for application to data blocks in the form of cells within image frames.

Drawings

In the drawings, like reference numerals refer to like parts throughout the various views unless otherwise indicated. For reference numerals having an alphabetical character designation such as "102A" or "102B," the alphabetical character designation may distinguish two similar parts or elements appearing in the same figure. When it is intended that a reference numeral cover all parts having the same reference numeral in all drawings, the alphabetic character marks of the reference numeral may be omitted.

Fig. 1 illustrates the effect of compressing an image frame composed of a plurality of data sub-units or tiles;

FIG. 2 illustrates a compressed data transaction utilizing a DRAM memory component having a required minimum access length per transaction ("MAL");

FIG. 3 illustrates a series of compressed data transactions associated with an exemplary image frame;

FIG. 4A illustrates an exemplary sequence of frames in a video stream, each frame having a different complexity and being compressed according to a lossless compression algorithm;

FIG. 4B shows the exemplary frame sequence illustrated in FIG. 4A, the frames being compressed according to a smart compression method using mixed-mode compression including lossless and lossy algorithms;

FIG. 5 is a logic flow diagram illustrating a smart compression method utilizing the mixed mode compression method of FIG. 4B in view of any one or more key performance indicators;

FIG. 6 is a logic flow diagram illustrating a smart compression method utilizing the mixed mode compression method of FIG. 4B in view of thermal energy measurements and thresholds;

FIG. 7 is a logic flow diagram illustrating a smart compression method utilizing the mixed mode compression method of FIG. 4B in view of system reliability targets associated with average voltage settings;

FIG. 8 illustrates an exemplary sequence of frames in a video stream, each frame having a different complexity and being compressed according to a lossless compression algorithm after dynamic adjustment of image quality settings;

FIG. 9 is a logic flow diagram illustrating an intelligent compression method utilizing the dynamic input compression method of FIG. 8 in view of any one or more key performance indicators;

FIG. 10 is a logic flow diagram illustrating a smart compression method utilizing the dynamic input compression method of FIG. 8 in view of thermal energy measurements and thresholds;

FIG. 11 is a logic flow diagram illustrating a smart compression method utilizing the dynamic input compression method of FIG. 8 in view of system reliability goals associated with average voltage settings;

FIG. 12 is a functional block diagram illustrating an embodiment of a system on a chip for intelligent compression;

FIG. 13A is a functional block diagram illustrating an embodiment of the image CODEC module of FIG. 12 configured to implement a smart compression method using mixed-mode compression including a lossless algorithm and a lossy algorithm;

FIG. 13B is a functional block diagram illustrating an embodiment of the image CODEC module of FIG. 12 configured to implement an intelligent compression method utilizing a lossless compression algorithm after dynamic adjustment of image quality settings;

FIG. 14 is a functional block diagram illustrating exemplary, non-limiting aspects of a portable computing device ("PCD") in the form of a wireless telephone for implementing the intelligent compression method and system; and

FIG. 15 is a schematic diagram illustrating an exemplary software architecture of the PCD in FIG. 14 for performing the intelligent compression method.

Detailed Description

The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect described herein as "exemplary" is not necessarily to be construed as exclusive, preferred or advantageous over other aspects.

In this description, the term "application" may also include files having executable content such as object code, scripts, byte code, markup language files, and patches. In addition, an "application" as referred to herein may also include files that are not executable in nature (such as documents that may need to be opened or other data files that need to be accessed).

In this description, references to "DRAM" or "DDR" memory components will be understood to contemplate any of the broader classes of volatile random access memory ("RAM"), and will not limit the scope of the solutions disclosed herein to a particular type or generation of RAM. That is, it will be understood that the various embodiments of the systems and methods provide solutions for managing transactions of data that have been compressed according to lossless and/or lossy compression algorithms, and are not necessarily limited to application to compressed data transactions associated with dual data rate memories. Furthermore, it is envisioned that certain embodiments of the solutions disclosed herein may be applicable to DDR, DDR-2, DDR-3, low power DDR ("LPDDR"), or any future generation of DRAMs.

As used in this description, the terms "component," "database," "module," "block," "system," and the like are intended to refer generally to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution, unless specifically limited to a computer-related entity. For example, a component may be, but is not limited to being: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).

In this description, the terms "central processing unit (" CPU ")," "digital signal processor (" DSP ")," "graphics processing unit (" GPU ")," and "chip" are used interchangeably. Further, a CPU, DSP, GPU, or chip may include one or more different processing components, which are generally referred to herein as "cores.

In this description, the terms "engine," "processing component," "generator," and the like are used to refer to any component within a system on a chip ("SoC") that generates and transmits data and/or frames to and from a memory component on a bus. Thus, an engine may refer to, but is not limited to referring to: CPU, DSP, GPU, modem, controller, camera, video recorder, etc.

In this description, the term "bus" refers to the set of conductors over which data is sent from the processing engine to the memory components and other devices located on or off the SoC. It will be understood that the bus comprises two parts: an address bus and a data bus, wherein the data bus transfers the actual data, and the address bus transfers information specifying the location of the data in the memory component (i.e., the address and associated metadata). The term "width" or "bus width" or "bandwidth" refers to the amount of data that can be sent per cycle over a given bus, i.e., the "block size". For example, a 16 byte bus may send 16 bytes of data at a time, while a 32 byte bus may send 32 bytes of data per cycle. Further, "bus speed" refers to the number of times per second a block of data can be sent over a given bus. Similarly, a "bus cycle" or "cycle" refers to the transmission of one block of data over a given bus.

In this description, the term "portable computing device" ("PCD") is used to describe any device that operates on a limited capacity power source, such as a battery. Although battery operated PCDs have been in use for decades, technological advances in rechargeable batteries coupled with the advent of third generation ("3G") and fourth generation ("4G") wireless technology have enabled a large number of PCDs with multiple capabilities. Thus, a PCD may be a cellular telephone, a satellite telephone, a pager, a PDA, a smart phone, a navigation device, a smart book or reader, a media player, a combination of the above devices, a laptop computer with a wireless connection, or the like.

In this description, the terms "lossless" and "lossy" refer to different classes of compression algorithms or methods, and are not meant to refer to any particular algorithm. One skilled in the art will recognize whether a given particular compression algorithm is "lossless" or "lossy". In general, and as will be understood by those skilled in the art, "lossless" and "lossy" are terms that describe whether all of the original data can be recovered when a file is decompressed in the compression of a data set or frame. In the case of "lossless" compression, each bit of data originally in a frame is retained after the frame is decompressed, i.e., all information is fully recovered. Graphics interchange files ("GIFs") are one exemplary image format that provides lossless compression. In contrast, lossy compression algorithms reduce frames or data sets by permanently eliminating certain information (especially redundant information). Thus, when a file compressed with a lossy algorithm is decompressed, only a portion of the original information is retained (although the user experience may not be affected as such). Lossy compression algorithms may be suitable for video and sound based use cases, for example, because some amount of information loss may not be detectable by the user. A JPEG image file is an exemplary image format that provides lossy compression. Using lossy compression algorithms, the designer can decide how much loss to introduce and make a trade-off between file size and output image quality.

In this description, the terms "image," "data set," "data," "frame," "image frame," "buffer," "file," and the like may be used interchangeably. Although embodiments of the present solution are described herein in the context of a generator component generating a data set having the format of an image frame (such as may be generated by a camera or video subsystem), it will be understood that the solution described herein is not limited to application to frames. Indeed, it is envisioned that embodiments of the present solution may be applicable to any use case that may benefit from data compression in general.

In this description, the terms "tile" and "unit" are used interchangeably to refer to a block of data that forms a subset of a larger block of data, such as a frame.

In this description, the term "uncompressed" refers to a frame in its original, pre-compressed state, while the term "decompressed" refers to a frame that is first compressed from its uncompressed state using a compression algorithm and is decompressed later. Depending on the compression category used, the data set of the decompressed frame may be the same as the data set of the frame in its original, uncompressed state (lossless compression) or may be different (lossy compression).

As will be appreciated by those skilled in the art of frame compression, the resulting compression ratio generated by any given compression algorithm inevitably varies from frame to frame. The texture level, lighting conditions, ISO settings, etc. in a given frame of a digitally captured video sequence may differ significantly from different frames in the same sequence, and therefore, the relative level of compression for the frames will also differ significantly. For example, frames in a video sequence that capture several people standing and talking are more likely to experience a high level of compression than subsequent frames that capture the same several people rushing down the street to get away from the explosion in the background. In short, a frame with a running body and an explosion has much data, and if the frame is to achieve high quality rendering when decompressed later, the loss of the frame in compression is not affordable.

With sufficiently large processing components, memory components, bus bandwidth, and power, PCD designers will not need to consider the trade-off of one compression algorithm against another, they simply generate frames at the highest possible quality level, and compress with a lossless compression algorithm (if at all). In this way, they can ensure that the QoS experienced by the user is always at its maximum possible level. However, the limited form factor reality for PCDs forces designers to trade-off various compression algorithms when sizing components in the PCD to achieve a minimum acceptable QoS for all predicted use cases.

Thus, PCD designers have typically used an "actual worst case" compression ratio when evaluating the required bandwidth requirements and component sizes for all known use cases. Thus, the practical worst case is to maintain the compression ratio required for the minimum acceptable QoS for the most difficult frames that need to be compressed (e.g., frames that capture an explosion of people rushing down the street to get far away from the background). Using the actual worst case, the designer makes conservative size setting choices to ensure that memory and bus bandwidth will always be sufficient, regardless of the complexity of a given frame. It is noted that if the designer sets the size of the bandwidth too small, the resulting latency in processing during the problematic use case may result in dropped frames, a drop in the frame per second ("FPS") rate, and so forth. Conversely, for all use cases better than the actual worst case, the system may be significantly oversized for achieving the minimum acceptable QoS, resulting in a higher cost system.

Advantageously, embodiments of the present solution provide designers with the ability to "properly size" the generator, memory components, and bus bandwidth to optimize power consumption and QoS levels across a range of use cases. It is noted that embodiments of the present solution utilize a mixed mode compression approach. In doing so, and as will be explained more fully below in view of the figures, certain embodiments provide for the use of lossless and lossy compression algorithms in a single use case, the selection and duration of the algorithms being made in view of key performance indicators. Certain other embodiments provide a dynamic frame quality level as an input to the lossless compression block in order to create a more "compressible" frame.

Turning to fig. 1, shown is the effect of compressing an image frame comprising a plurality of data sub-units or tiles. In this description, various embodiments may be described within the context of an image frame consisting of 256-byte tiles. It is noted, however, that it will be appreciated that the 256 byte tile size and various compressed data transaction sizes are exemplary in nature, and that embodiments of the present solution are not suggested to be limited to application for 256 byte tile sizes. Furthermore, it will be understood that references to any particular minimum access length ("MAL") or access block size ("ABS") for DRAM in this description are used to facilitate the description of the present solution, and it is not suggested that embodiments of the present solution be limited to application to DRAM devices having particular MAL requirements. Thus, those skilled in the art will recognize that the particular data transfer sizes, block sizes, bus widths, MAL, etc. referenced in this description are provided for exemplary purposes only, and do not limit the scope of the contemplated solution to be applicable to applications having the same data transfer sizes, block sizes, bus widths, MAL, etc.

Returning to the fig. 1 illustration, an uncompressed image frame (also referred to as a "buffer") is depicted as including thirty uncompressed tiles or cells, each having a size "K", as indicated by the darkest shading. An exemplary size K may be 256 bytes, however, as explained above, the tiles are not limited to any particular size and may vary depending on the application. As will be understood by those skilled in the art, the uncompressed image frame may be reduced in size by applying the compressor block of the compression algorithm (depicted as image CODEC module 113 in fig. 1) tile-by-tile, thereby optimizing its transmission on the bus and minimizing its impact on memory capacity. The result of the compression is to compress the image frames plus the metadata file, as can be seen in the FIG. 1 illustration. The compressed image frame includes tiles in the original, uncompressed image frame that have been subjected to the compression algorithm performed by compression block 113.

In an uncompressed image frame, each tile may have a size K, and in a compressed image frame, each tile may have a size K or less (K (for no compression possible), K-1 byte, K-2 byte, K-3 byte,. -, K ═ 1 byte). In this illustration, the individual tiles that form the compressed image frame are represented by different levels of shading, depending on the degree of compression that results from compression block 113 having applied its compression algorithm to the data owned by a given tile. It is noted that the compression block 113 creates a companion buffer for compressing the image frame metadata, as will be understood by those skilled in the art. The compressed image frame metadata contains a record of the size, type, and attributes for each compressed tile in the compressed image frame. Because DRAM access may be limited to units of minimum access length MAL, the size of a given compressed tile may be represented in the metadata as the number of ABS required to represent the compressed tile size (e.g., 1MAL, 2 MAL. This size in the metadata describes the minimum required amount of data needed to allow future buffer readers to request from memory only to decompress each tile back to the original size K.

FIG. 2 illustrates compressed data transactions utilizing DRAM memory components that meet the requirement of being an integer multiple of the minimum access length per transaction ("MAL") for each transaction. As can be appreciated from the fig. 2 illustration, a compressed tile may have a length that is less than an integer multiple of the minimum access length requirement for the DRAM in which it is stored. Thus, the request requirements for compressed data include a certain amount of garbage or "filled" transactions needed to meet integer multiples of the MAL requirements. Padding is added to the compressed tile data that does not carry any information so that the transaction size is an integer multiple of the system MAL (i MAL). Exemplary MAL may be 32 bytes or 64 bytes depending on the particular chip technology (such as LPDDR2, LPDDR3, LPDDR4, etc.) and memory bus width (x16, x32, x 64). As an example, a compressed tile having a size of 63 bytes may be filled with 1 byte of padding data to produce a full 64 byte transaction size (2x32B MAL or 1x64B MAL). Similarly, if the MAL is 32 bytes, a compressed tile having a size of 65 bytes (3x32B MAL) may be filled with 31 bytes of padding data, or if the MAL is 64 bytes, a compressed tile having a size of 65 bytes may be filled with 63 bytes of padding data, so as to generate a complete 128 byte transaction size (2x64B MAL). Note that in the above example, the difference in compressed tile sizes is only 2 bytes; however, since a 65 byte compressed tile exceeds 64 bytes, its transaction must include significantly more padding.

Fig. 3 illustrates a series of compressed data transactions associated with an exemplary image frame. It is noted that the transactions or units in the fig. 3 illustration may be considered to have been compressed according to either lossless or lossy compression algorithms (as well as those units depicted in the fig. 1 and 2 illustrations).

The image frame is shown with "N" columns and "M" rows of tiles. The first four sequential tiles in the first row of tiles are shown in terms of their uncompressed length, compressed length, and transaction length (compressed length plus padding), according to methods known in the art. For ease of illustration, the illustration is made within the context of the first four sequential tiles, as will be understood by those skilled in the art, the concepts depicted are related to tile groups other than the first four sequential tiles in the first row of tiles of the image frame.

Consider an exemplary four sequential tiles in their uncompressed state, each tile (#1, 0; #2, 0; #3, 0; #4,0) having a length of 256 bytes (other lengths are contemplated). When compressed, the exemplary four sequential tiles have lengths of 112 bytes, 56 bytes, 33 bytes, and 177 bytes, respectively. Assuming that the MAL is 64 bytes, the transaction length for each of the exemplary four sequential tiles can be 128 bytes (112 bytes of compressed data plus 16 bytes of padding), 64 bytes (56 bytes of compressed data plus 8 bytes of padding), 64 bytes (33 bytes of compressed data plus 31 bytes of padding), and 192 bytes (177 bytes of compressed data plus 15 bytes of padding), respectively. It is noted that to process all four exemplary sequential tiles, the methods known in the art do four transactions: one transaction is performed for each compressed tile.

FIG. 4A illustrates an exemplary sequence of frames in a video stream, each frame having a different complexity and being compressed according to a lossless compression algorithm. FIG. 4B, which will be described in conjunction with the FIG. 4A diagram, shows the exemplary sequence of frames illustrated in FIG. 4A, the frames being compressed according to a smart compression method utilizing mixed-mode compression including lossless and lossy algorithms. It is noted that although the blocks in the fig. 4 illustration are referred to as frames, it is envisioned that these blocks may also represent tiles within a frame. That is, it is envisioned that embodiments of the present solution may be implemented on a frame-by-frame basis, or if so configured, on a tile-by-tile basis. Thus, the scope of the solution described herein will not be limited to applications on the frame level only, as embodiments of the present solution may also be applied on the tile level.

As can be appreciated from the legend associated with fig. 4, those frames depicted as solid blocks are frames compressed according to a lossless compression algorithm, while those frames depicted as transparent blocks are frames compressed according to a lossy compression algorithm. Those frames connected by solid lines are those frames that form a sequence of frames that are generated, compressed, and output to a memory device (e.g., such as a DDR memory component). The "y-axis" on both the graphs of fig. 4A and 4B represents the compression ratio of the frame, while the "x-axis" represents time.

Referring first to the fig. 4A diagram, a generator of frames (such as a camera or video subsystem, for example) is generating sequential frames that are compressed according to a lossless compression algorithm. For the first seven frames starting with frame 1, the compression ratio of the frame is above the minimum target compression ratio threshold as indicated by the horizontal dashed line in the graph. Thus, for the first seven frames, the complexity of the originally generated frames is such that application of the lossless compression algorithm does not produce a compression ratio that is so low as to overburden the system bandwidth and adversely affect the QoS level. However, starting from frame 8 and continuing to frame 20, the relative complexity of the originally generated frames increases, so that the lossless compression algorithm cannot reduce the frame size to a level above the desired target minimum compression ratio. The relative complexity of the originally generated frame is reduced at frame 21 so that the applied lossless compression algorithm reduces the size of the frame sufficiently to maintain a compression ratio above the target line. For frames 8 through 20 in the fig. 4A illustration, which are compressed according to a lossless compression algorithm that cannot reduce the frame size beyond the target minimum compression ratio, there is a risk that the user experiences an unacceptably low QoS. Insufficient bandwidth to accommodate frames 8 through 20 may cause the system to respond by: dropping one or more of the frames, increasing latency such that the effective frame rate is decreased, and so forth.

Referring to the fig. 4B diagram, the same sequence of originally generated frames shown in the fig. 4A diagram is compressed using the intelligent compression method according to an embodiment of the present solution. As in the graph of fig. 4A, frames 1 to 7 are compressed according to a lossless compression algorithm. With the relative complexity of frame 8 in its uncompressed state, it is minimally compressed using a lossless compression algorithm and, therefore, exhibits a compression ratio that is below the target minimum compression ratio. In response to the frame 8 compression ratio, the frame 9 may be compressed using a lossy compression algorithm that generates a compression ratio that exceeds a target minimum. The sequence of frames continues with subsequent frames being compressed according to a lossy compression algorithm until, at frame 21, it is identified that the preferred lossless compression algorithm will produce an acceptable compression ratio. At this point, the mixed mode method defaults back to the lossless compression algorithm and compression continues at frame 22 using the lossless algorithm.

The detection of reduced complexity in the frame 21 may be achieved in a number of ways. One example would be to encode each frame/tile in a lossless manner and record the resulting compression ratio, rather than writing the actual lossless compressed data to storage or DRAM; once the compressed frames recorded on the frame are above the target minimum, the system may switch to lossless compression for the frame 22, as shown in FIG. 4B. Alternatively, detection of reduced complexity in the frame 21 may be achieved by examining the compression ratio of the lossy frame. For low complexity frames, the lossy compression ratio will be significantly higher than for high complexity frames. The system may then use a predefined threshold for the lossy compression ratio to indicate: once the lossy compression ratio threshold is exceeded for a given frame (e.g., for frame 22 as shown in fig. 4B), the system may switch to using lossless compression.

Advantageously, by using a mixed-mode approach, embodiments of the intelligent compression solution may minimize the number of frames in the sequence that are compressed to a compression ratio below the target minimum compression ratio (e.g., frame 8 in the fig. 4B diagram), while also minimizing the number of frames in the sequence that are compressed using a lossy compression algorithm (when a lossless compression algorithm will produce an acceptable compression ratio) (e.g., frame 21 in the fig. 4B diagram).

Fig. 5 is a logic flow diagram illustrating a smart compression method 500 utilizing the mixed mode compression method of fig. 4B in view of any one or more key performance indicators ("KPIs"), such as, but not limited to, minimum target compression ratio, bandwidth utilization, voltage level, frame rate ("FPS"), compression latency, etc. It is noted that the mixed mode compression method illustrated in the graph of fig. 4B is described using the compression ratio of the output frame as the KPI, however, it is envisioned that the same method may be employed using KPIs other than the compression ratio as a trigger for switching the compressed mode.

Beginning at block 505, an original generated frame can be received into the image CODEC 113, the image CODEC 113 including one or more lossless compression blocks and one or more lossy compression blocks. At block 505, the received frame is compressed using a default or preferred lossless compression block. Next, at block 510, the relevant KPIs are monitored to determine whether the QoS remains at an acceptable level, and at decision block 515, the monitored KPIs are compared to predetermined thresholds. If the KPI is above (or below, as may be the case) the predetermined threshold, i.e., the KPI is at an acceptable level, the method 500 may branch along the YES branch back to block 505 and continue to compress the input uncompressed frames using the lossless compression algorithm. However, if the monitored KPI is not at an acceptable level when compared to the predetermined threshold, the no branch is taken from decision block 515 to block 520. In an exemplary implementation, the no branch is followed immediately after the first instance of a KPI at an unacceptable level is encountered. In other exemplary implementations, hysteresis is added to the "no" decision, where the KPI must be an unacceptable programmable sequence up to X consecutive times or up to X times in a window of Y consecutive sequences before invoking the "no" branch.

At block 520, the next resulting uncompressed frame is received into CODEC 113 and compressed using the lossy compression block. Advantageously, although frames compressed using a lossy compression algorithm may later generate decompressed frames having a relatively lower level of quality than the original generated uncompressed frames, the reduced bandwidth associated with lossy compression may cause the associated KPIs to have a tendency to return towards acceptable levels.

Next, at block 525, the relevant KPIs are monitored to determine whether the QoS has returned to an acceptable level, and at decision block 530, the monitored KPIs are compared to predetermined thresholds. If the KPI has not returned to an acceptable level, the method 500 may branch along the NO branch back to block 520 and continue to compress the input uncompressed frames using the lossy compression algorithm. However, if the monitored KPI, when compared to a predetermined threshold, indicates that the KPI has returned to an acceptable level, the yes branch is followed from decision block 530 to decision block 505.

At decision block 535, the method 500 may consider whether enough frames have been generated using the lossy compression algorithm to justify returning to the lossless compression algorithm. While some embodiments of the present solution may trigger a return to the preferred lossless compression mode as soon as the KPI indicates an acceptable level, it is envisioned that other embodiments may include hysteresis considerations for mitigating the probability of occurrence of: KPIs can bounce back and forth between thresholds and, in doing so, trigger too many events to switch between lossless and lossy modes. In an exemplary implementation, the yes branch is followed immediately after the first instance of a KPI above a threshold is encountered. For other exemplary implementations, hysteresis is added to the "yes" decision, where the KPI must be an unacceptable programmable sequence up to K consecutive times or up to K times in a window of L consecutive sequences before the "yes" branch is invoked.

Returning to method 500 at decision block 535, if a predetermined number of frames have been compressed using lossy compression while the KPI is at an acceptable level when compared to the threshold, the "YES" branch may be followed and the method returns to block 505 where the lossless compression algorithm is applied to the next incoming uncompressed frame at block 505. However, if at decision block 535 the predetermined number of frames have not been compressed using the lossy compression algorithm while the KPIs remain at an acceptable level, the no branch may be taken to block 540 and the next uncompressed frame may be received and compressed using the lossy compression block. At block 545, the KPIs are monitored, and the method 500 loops back to decision block 530.

In this way, embodiments of the solution for intelligent compression methods using mixed-mode compression methods may compromise the future quality of some decompressed frames in order to optimize QoS as quantified by the associated KPI or KPIs.

FIG. 6 is a logic flow diagram illustrating a smart compression method 600 utilizing the mixed mode compression method of FIG. 4B in view of thermal energy measurements and thresholds. It is noted that the mixed mode compression method illustrated in the graph of fig. 4B is described using the compression ratio of the output frame as a trigger for determining when to switch between the lossless compression mode and the lossy compression mode, however, it is envisioned that the same method may be employed as a way of mitigating and managing thermal energy generation in view of thermal budget constraints and/or thermal measurements. That is, it is envisioned that the mixed mode compression approach of FIG. 4B may be used as a thermal management technique.

Beginning at block 605, an original generated frame can be received into the image CODEC 113, the image CODEC 113 including one or more lossless compression blocks and one or more lossy compression blocks. At block 605, the received frame is compressed using a default or preferred lossless compression block. Next, at block 610, one or more temperature sensors are monitored to determine if the thermal energy level remains at an acceptable level, and at decision block 615, the temperature readings generated by the sensors are compared to a predetermined threshold. The monitored temperature sensors may be associated with a panel temperature of the PCD 100, a junction temperature of a processing core, a temperature of a PoP memory component, and the like. If the temperature is below the predetermined threshold, i.e., the monitored thermal condition is at an acceptable level, the method 600 may proceed along the NO branch back to block 605 and continue to compress the input uncompressed frame using the lossless compression algorithm. However, if the monitored thermal condition is not at an acceptable level when compared to the predetermined threshold, the yes branch is followed from decision block 615 to block 620. An alternative implementation may choose not to take the yes branch immediately after the hot KPI is met, but instead wait until the hot KPI is above the threshold for a predetermined period of time before taking the yes branch.

At block 620, the next resulting uncompressed frame is received into CODEC 113 and compressed using the lossy compression block. Advantageously, although frames compressed using a lossy compression algorithm may later generate decompressed frames with a relatively lower level of quality than the original produced uncompressed frames, the reduced bandwidth associated with lossy compression may reduce power consumption and cause the monitored thermal conditions to tend back toward acceptable levels.

Next, at block 625, the temperature sensor is monitored to determine whether the thermal condition has returned to an acceptable level, and at decision block 630, the temperature reading is compared to a predetermined threshold. If the thermal condition monitored by the temperature sensor has not returned to an acceptable level, the method 600 may branch along the YES branch back to block 620 and continue to compress the input uncompressed frame using the lossy compression algorithm. However, if the reading from the temperature sensor, when compared to the predetermined threshold, indicates that the thermal condition may have returned to an acceptable level, the no branch is taken from decision block 630 to decision block 635.

At decision block 635, the method 600 may consider whether enough frames have been generated using the lossy compression algorithm to justify returning to the lossless compression algorithm. While some embodiments of the present solution may trigger a return to the preferred lossless compression mode as soon as the temperature reading indicates an acceptable level for the thermal condition being monitored, it is envisioned that other embodiments may include hysteresis considerations for mitigating the probability of the following occurring: thermal conditions may bounce back and forth between thresholds and, in doing so, trigger too many events to switch between lossless and lossy modes.

Returning to method 600 at decision block 635, if a predetermined number of frames have been compressed using lossy compression, or if a predetermined amount of time has elapsed while the thermal conditions associated with the temperature sensor are at an acceptable level when compared to a threshold, the yes branch may be followed and the method returns to block 605 where the lossless compression algorithm is applied to the next incoming uncompressed frame at block 605. However, if at decision block 635, the predetermined number of frames have not been compressed using the lossy compression algorithm, or the predetermined amount of time has not elapsed while the thermal condition remains at an acceptable level, the "no" branch may be followed to block 640 and the next uncompressed frame may be received and compressed using the lossy compression block. At block 645, the thermal condition is monitored using the temperature readings generated by the temperature sensor, and the method 600 loops back to decision block 630.

In this way, embodiments of the solution for smart compression methods using mixed-mode compression methods may compromise the future quality of some decompressed frames in order to mitigate adverse thermal energy generation and subsequently the temperature of the device.

FIG. 7 is a logic flow diagram illustrating a smart compression method 700 utilizing the mixed mode compression method of FIG. 4B in view of system reliability targets associated with average voltage settings. It is noted that the mixed mode compression method illustrated in the graph of fig. 4B is described using the compression ratio of the output frame as a trigger for determining when to switch between the lossless compression mode and the lossy compression mode, however, it is envisioned that the same method may be employed in view of system reliability targets associated with the average voltage setting. That is, it is envisioned that the mixed mode compression method of fig. 4B may be used as a technique for maintaining the average supply voltage between the upper and lower thresholds. Those skilled in the art will appreciate that to ensure reliability of the chip 102, it may be desirable to keep the average supply voltage below the maximum average, but not so low that the average QoS experienced by the user is unnecessarily compromised.

Beginning at block 705, an originally generated frame can be received into an image CODEC 113, the image CODEC 113 including one or more lossless compression blocks and one or more lossy compression blocks. At block 705, the received frame is compressed using a default or preferred lossless compression block. Next, at block 710, one or more voltage sensors are monitored to calculate a moving average of the voltage supplied to the die 102 over time. Based on the moving average, method 700 may determine whether the long-term reliability of the chip is acceptable by comparing the voltage average to a predetermined threshold at decision block 715. If the supply voltage is below the predetermined threshold on average, i.e., the predicted long-term reliability of the chip is at an acceptable level, method 700 may branch along the "no" branch back to block 705 and continue to compress the incoming uncompressed frames using the lossless compression algorithm. However, if the supply voltage average is not at an acceptable level when compared to the predetermined threshold, the yes branch is followed from decision block 715 to block 720.

At block 720, the next resulting uncompressed frame is received into CODEC 113 and compressed using the lossy compression block. Advantageously, although frames compressed using a lossy compression algorithm may later generate decompressed frames with a relatively lower level of quality than the original produced uncompressed frames, the reduced bandwidth associated with lossy compression may reduce power consumption and cause the supply voltage to on average tend to return toward an acceptable level.

Next, at block 725, the voltage sensor is monitored to determine if the supply voltage average has returned to an acceptable level, and at decision block 730, the supply voltage average is compared to a predetermined lower threshold. If the supply voltage calculated from the monitored voltage sensor remains above the lower threshold on average, the method 700 may proceed along the "No" branch back to block 720 and continue to compress the input uncompressed frame using the lossy compression algorithm. However, if the supply voltage average indicates that the average voltage supplied to the chip 102 has dropped below the lower average threshold, the yes branch is followed from decision block 730 back to block 705 where lossless compression is applied to the next incoming uncompressed frame at block 705.

In this way, embodiments of the solution for smart compression methods that use mixed-mode compression methods may compromise the future quality of some decompressed frames in order to maintain the average supply voltage in a range associated with a target chip reliability target.

Fig. 8 shows an exemplary sequence of frames in a video stream, each frame having a different complexity and being compressed according to a lossless compression algorithm after dynamic adjustment of image quality settings. It is noted that although the blocks in the fig. 8 illustration are referred to as frames, it is envisioned that these blocks may also represent tiles within a frame. That is, it is envisioned that embodiments of the present solution may be implemented on a frame-by-frame basis, or if so configured, on a tile-by-tile basis. Thus, the scope of the present solution described herein will not be limited to applications on the frame level only, as embodiments of the present solution may also be applied on the tile level.

As can be appreciated from the legend associated with fig. 8, those frames depicted as solid squares are frames that are compressed according to a lossless compression algorithm if the uncompressed frames are generated using a default "high" image quality level. Conversely, those frames depicted as transparent squares are frames compressed according to a lossy compression algorithm if the uncompressed frames were generated using an adjusted image quality level that is "lower" in quality relative to the default image quality level. The filled circles indicate the level of image quality with which a generator (such as a camera or video subsystem) is generating uncompressed frames. Those frames connected by a solid line are those frames forming a sequence of frames that are generated, compressed, and output to a memory device (such as a DDR memory component, for example) according to the moving image quality level setting. The left "y-axis" on the graph of fig. 8 represents the compression ratio of the output frame, the right "y-axis" represents the image quality level setting of the generator for the uncompressed frame, and the "x-axis" represents time.

Referring to the fig. 8 diagram, a generator of frames (e.g., such as a camera or video subsystem) is generating sequential frames at a default image quality level that are then compressed according to a lossless compression algorithm. For the first seven frames starting with frame 1, the compression ratio of the frame is above the minimum target compression ratio threshold and below the maximum target compression ratio, as indicated by the horizontal dashed line in the graph. Thus, for the first seven frames, the complexity of the originally generated frames at the default image quality setting is such that application of the lossless compression algorithm does not produce a compression ratio that is so low as to overburden the system bandwidth and adversely affect the QoS level. However, starting from frame 8 and continuing to frame 20, the relative complexity of the frames when generated at the default image quality level is such that the lossless compression algorithm is not able to significantly reduce the frame size. In response, the image quality level setting associated with frame 9 may be reduced such that the relative complexity of the originally generated uncompressed frame is reduced so that the lossless compression algorithm may adequately maintain the compression ratio above the target line.

It is contemplated that any number and/or combination of "knobs" within the image generator can be adjusted to reduce the image quality level upstream of the uncompressed image of CODEC module 113. For example, image quality settings (such as, but not limited to, texture settings, color bit resolution, foveal settings, etc.) may be adjusted in order to reduce the complexity of the uncompressed frames generated by the generator.

Returning to the illustration of FIG. 8, at frame 21, the lower image quality level setting in the generator causes the lossless compression algorithm to generate compressed frames having a compression ratio higher than the predetermined maximum compression ratio. In response, the image quality settings used to generate the frame 22 in its uncompressed form may return to default settings. Moving forward in the frame sequence, the image quality setting may be maintained at a default level unless or until the following occurs: the lossless compression algorithm again generates compressed frames having a compression ratio below the minimum target.

Advantageously, by using the dynamic quality setting method, embodiments of the intelligent compression solution may minimize the number of frames in the sequence that are compressed to a compression ratio below the target minimum compression ratio (e.g., frame 8 in the fig. 8 diagram), while also minimizing the number of frames in the sequence that are compressed from uncompressed frames generated using lower quality settings (e.g., frame 21 in the fig. 8 diagram).

Fig. 9 is a logic flow diagram illustrating a smart compression method 900 utilizing the dynamic input compression method of fig. 8 in view of any one or more key performance indicators ("KPIs"), such as, but not limited to, minimum target compression ratio, bandwidth utilization, voltage level, frame rate ("FPS"), compression latency, etc. It is noted that the dynamic input frame quality method illustrated in the chart of fig. 8 is described using the compression ratio of the output frame as the KPI, however, it is envisioned that the same method may be employed using KPIs other than compression ratios as triggers for adjusting the generator quality settings.

Beginning at block 905, an uncompressed frame generated according to a default image quality setting can be received into the image CODEC 113 and compressed accordingly, the image CODEC 113 including one or more lossless compression blocks. Next, at block 910, the relevant KPIs are monitored to determine whether the KPI remains at an acceptable level, and at decision block 915, the monitored KPIs are compared to a predetermined threshold. If the KPI is above (or below, as may be the case) the predetermined threshold, i.e., the KPI is at an acceptable level, the method 900 may branch along the NO branch back to block 905 and continue compressing the input uncompressed frames produced at the default image quality setting. However, if the monitored KPI is not at an acceptable level when compared to the predetermined threshold, the yes branch is followed from decision block 915 to block 920.

At block 920, the next generated uncompressed frame is generated according to the reduced image quality level setting and received into the CODEC 113 and compressed using a lossless compression algorithm at block 925. Advantageously, the reduced bandwidth associated with lossless compression of relatively less complex uncompressed frames may cause the associated KPIs to tend back toward acceptable levels, although using a lower quality level setting to produce uncompressed frames will affect the final quality of the frames when later decompressed.

Next, at decision block 930, the monitored KPI is compared to a predetermined threshold to determine whether the QoS has returned to an acceptable level. If the KPI has not returned to an acceptable level, the method 900 may branch along the "No" branch back to block 925 (at block 925, maintaining the reduced picture quality setting) or block 920 (at block 920, further reducing the picture quality setting). However, if the monitored KPI, when compared to a predetermined threshold, indicates that the KPI has returned to an acceptable level, the "Yes" branch is followed from decision block 930 to decision block 935.

At decision block 935, the method 900 may consider whether enough frames have been compressed from among the uncompressed frames generated with the reduced image quality setting to justify returning to the default image quality setting. While some embodiments of the present solution may trigger a return to a default image quality setting upon KPI indicating an acceptable level, it is envisioned that other embodiments may include hysteresis considerations for mitigating the probability of the following occurring: KPIs can bounce back and forth between thresholds and, in doing so, trigger too many events to switch between default and reduced image quality settings.

Returning to method 900 at decision block 935, if a predetermined number of frames have been compressed from the uncompressed frames produced with the reduced image quality setting to justify returning to the default image quality setting, the "yes" branch may be followed to block 945 at which the image quality setting for the produced uncompressed frames is returned to the default setting. However, if at decision block 935 a predetermined number of frames have not been compressed from among the uncompressed frames produced with the reduced image quality setting to justify returning to the default image quality setting, the "no" branch may be followed to block 940 and the reduced image quality setting used to generate the next uncompressed frame prior to compression with the lossless algorithm. The method 900 loops back to decision block 930.

In this way, embodiments of the solution for intelligent compression methods using dynamic quality setting methods may compromise the future quality of some decompressed frames in order to optimize the QoS as quantified by the associated KPI or KPIs.

FIG. 10 is a logic flow diagram illustrating a smart compression method 1000 utilizing the dynamic input compression method of FIG. 8 in view of thermal energy measurements and thresholds. It is noted that the dynamic input frame quality approach illustrated in the chart of fig. 8 is described using the compression ratio of the output frame as a trigger for determining when to adjust the image quality setting of the generator, however, it is envisioned that the same approach may be used as a way to mitigate and manage thermal energy generation in view of thermal budget constraints and/or thermal measurements. That is, it is envisioned that the dynamic input compression method of FIG. 8 may be used as a thermal management technique.

Beginning at block 1005, an uncompressed frame generated according to a default image quality setting can be received into the image CODEC 113 and compressed accordingly, the image CODEC 113 including one or more lossless compression blocks. Next, at block 1010, one or more temperature sensors are monitored to determine if the thermal energy level remains at an acceptable level, and at decision block 1015, the temperature readings generated by the sensors are compared to a predetermined threshold. The monitored temperature sensors may be associated with a panel temperature of the PCD 100, a junction temperature of a processing core, a temperature of a PoP memory component, and the like. If the temperature is below the predetermined threshold, i.e., the monitored thermal condition is at an acceptable level, the method 1000 may branch along the "no" branch back to block 1005 and continue compressing the input uncompressed frames generated at the default image quality setting. However, if the monitored thermal condition is not at an acceptable level when compared to the predetermined threshold, the yes branch is followed from decision block 1015 to block 1020.

At block 1020, the next generated uncompressed frame is generated according to the reduced image quality level setting and received into the CODEC 113 at block 1025 and compressed using a lossless compression algorithm. Advantageously, the reduced bandwidth associated with lossless compression of relatively less complex uncompressed frames may cause the thermal conditions being monitored to tend back toward acceptable levels, although using a lower quality level setting to produce uncompressed frames will affect the final quality of the frames when later decompressed.

Next, at decision block 1030, the monitored thermal condition is compared to a predetermined threshold to determine whether the thermal condition has returned to an acceptable level. If the thermal condition has not returned to an acceptable level, the method 1000 may branch along the YES branch back to block 1025 (at block 1025, maintain the reduced image quality setting) or block 1020 (at block 1020, further reduce the image quality setting). However, if the monitored temperature sensor reading, when compared to the predetermined threshold, indicates that the thermal condition has returned to an acceptable level, the no branch is taken from decision block 1030 to decision block 1035.

At decision block 1035, the method 1000 may consider whether enough frames have been compressed from among the uncompressed frames generated with the reduced image quality setting to justify returning to the default image quality setting. While some embodiments of the present solution may trigger a return to a default image quality setting as soon as the temperature sensor readings indicate that the thermal condition being monitored has returned to an acceptable level, it is envisioned that other embodiments may include hysteresis considerations for mitigating the probability of the following occurring: thermal conditions may bounce back and forth between thresholds and, in doing so, trigger too many events to switch between default and reduced image quality settings.

Returning to method 1000 at decision block 1035, if a predetermined number of frames have been compressed from the uncompressed frames produced with the reduced image quality setting, or if a predetermined amount of time has elapsed to justify returning to the default image quality setting, the yes branch may be followed to block 1045, at block 1045, the image quality setting for the produced uncompressed frames is returned to the default setting. However, if at decision block 1035, a predetermined number of frames have not been compressed from among the uncompressed frames produced with the reduced image quality setting, or if a predetermined amount of time has not elapsed to justify returning to the default image quality setting, the "no" branch may be followed to block 1040 and the reduced image quality setting used to generate the next uncompressed frame prior to compression with the lossless algorithm. The method 1000 loops back to decision block 1030.

In this way, embodiments of the solution for intelligent compression methods using dynamic quality setting methods may compromise the future quality of some decompressed frames in order to mitigate adverse thermal energy generation.

FIG. 11 is a logic flow diagram illustrating a smart compression method 1100 utilizing the dynamic input compression method of FIG. 8 in view of system reliability goals associated with average voltage settings. It is noted that the dynamic input compression method illustrated in the graph of fig. 8 is described using the compression ratio of the output frame as a trigger for determining when to adjust the image quality setting of the generator, however, it is envisioned that the same method may be employed in view of the system reliability targets associated with the average voltage setting. That is, it is envisioned that the dynamic input compression method of fig. 8 may be used as a technique for maintaining an average supply voltage between an upper threshold and a lower threshold. Those skilled in the art will appreciate that to ensure reliability of the chip 102, it may be desirable to keep the average supply voltage below the maximum average, but not so low that the average QoS experienced by the user is unnecessarily compromised.

Beginning at block 1105, an uncompressed frame generated according to a default image quality setting can be received into the image CODEC 113 and compressed accordingly, the image CODEC 113 including one or more lossless compression blocks. Next, at block 1110, one or more voltage sensors are monitored to calculate a moving average of the voltage supplied to the chip 102 over time. Based on the moving average, method 1100 may determine whether the long-term reliability of the chip is acceptable by comparing the voltage average to a predetermined upper threshold at decision block 1115. If the supply voltage is below the predetermined upper threshold on average, i.e., the predicted long-term reliability of the chip is at an acceptable level, the method 1100 may branch along the "no" branch back to block 1105 and continue compressing the incoming uncompressed frames produced at the default quality setting. However, if the supply voltage average is not at an acceptable level when compared to the predetermined upper threshold, then the yes branch is followed from decision block 1115 to block 1120.

At block 1120, a next uncompressed frame is generated according to the reduced image quality level setting and received into the CODEC 113 at block 1125 and compressed using a lossless compression algorithm. Advantageously, the reduced bandwidth associated with lossless compression of relatively less complex uncompressed frames may reduce power consumption and cause the supply voltage to on average have a tendency to return toward an acceptable level, although using a lower quality level setting to produce uncompressed frames will affect the final quality of the frame when later decompressed.

Next, at decision block 1130, the supply voltage average is compared to a predetermined lower threshold. If the supply voltage calculated from the monitored voltage sensors remains above the lower threshold on average, the method 1100 may proceed along the "No" branch back to block 1125 (at block 1125, the reduced image quality setting is maintained) or to block 1120 (at block 1120, the image quality setting is further reduced). However, if the supply voltage average indicates that the average voltage supplied to chip 102 has dropped below the lower average threshold, the yes branch is followed from decision block 1130 to decision block 1135. At block 1135, the generator quality level setting may be increased back to a default level. The method 1100 loops back to block 1105.

In this way, embodiments of the solution for the intelligent compression method using the dynamic quality setting method may compromise the future quality of some decompressed frames in order to maintain the average supply voltage in a range associated with the target chip reliability target.

FIG. 12 is a functional block diagram illustrating an embodiment of a system-on-chip 102 for smart compression. The system 102 may be configured to implement any one or more of the intelligent compression methods described herein, including mixed mode methods and/or dynamic quality setting methods. Depending on the embodiment of the solution being implemented, the monitoring module 114 may monitor the power or voltage level associated with the power management integrated circuit ("PMIC") and supplied to the chip 102, the temperature readings from the sensor 157, the compression ratio of the compressed frames generated from the image CODEC module 113, or any other relevant KPI.

The intelligent compression ("IC") module 101 may work with a frame generator, such as the data/frame engine 201, to dynamically adjust the image quality settings employed to generate the uncompressed frames before they are input to the image CODEC module 113 for compression. Alternatively, the IC module 101 may work together with the image CODEC module 113 to switch between a lossless compression algorithm and a lossy compression algorithm. Consistent with what has been described above with respect to fig. 4-11, the IC module 101 may direct the generator engine 201 and/or the image CODEC module 113 based on input data from the monitor module 114 regarding: the compression ratio of the compressed frame, the temperature reading from sensor 157, the voltage level, or any related KPI.

Fig. 13A is a functional block diagram illustrating an embodiment of the image CODEC module of fig. 12 configured to implement a smart compression method using mixed-mode compression including a lossless algorithm and a lossy algorithm. When the system 102 is configured to perform the smart compression method using the mixed mode method, the generator engine 201A may provide its uncompressed frame input to the image CODEC 113A, which may then compress the frame according to either lossless compressor blocks or lossy compression blocks, consistent with what has been described with respect to fig. 4-7. The determination of which compressor block may be used to compress a given frame depends on the instructions received from the IC module 101A. The IC module 101A may generate instructions in view of any one or more inputs received from the monitor module 114.

FIG. 13B is a functional block diagram illustrating an embodiment of the image CODEC module of FIG. 12 configured to implement an intelligent compression method utilizing a lossless compression algorithm after dynamic adjustment of image quality settings. When the system 102 is configured to perform the intelligent compression method using the dynamic quality setting method, the generator engine 201B may provide its uncompressed frame input to the image CODEC 113B, which will then compress the frame according to the lossless compressor block, consistent with what has been described with respect to fig. 8-11. The complexity of the uncompressed frame generated by the engine 201B may be indicated by the image quality level setting indicated by the IC module 101B to the engine 201B.

As shown in the fig. 13B illustration, the image quality filter and multiplexer logic included within the image CODEC module 113B and upstream of the lossless compressor may be included within the engine 201B in other embodiments. Regardless of the particular arrangement, the IC module 101B indicates whether the image CODEC module 113B is ultimately compressing uncompressed frames produced at the default image quality setting or uncompressed frames produced at the reduced image quality setting. The IC module 101B may generate instructions for adjusting the image quality setting employed to produce the uncompressed frame in view of any one or more inputs received from the monitor module 114.

FIG. 14 is a functional block diagram illustrating exemplary, non-limiting aspects of a portable computing device ("PCD") 100 in the form of a wireless telephone for implementing the intelligent compression method and system. As shown, PCD 100 includes a system-on-chip 102, the system-on-chip 102 including a multicore central processing unit ("CPU") 110 and an analog signal processor 126 coupled together. As understood by those skilled in the art, the CPU 110 may include a zeroth core 222, a first core 224, and an nth core 230. Further, as will be appreciated by those skilled in the art, a digital signal processor ("DSP") may also be employed in place of the CPU 110.

In general, the smart compression ("IC") module 101 may be formed from hardware and/or firmware and may be responsible for dynamically adjusting the image quality settings employed by the generator to generate uncompressed frames and/or causing the image CODEC module 113 to switch between lossy and lossless compression algorithms. As shown in fig. 14, a display controller 128 and a touchscreen controller 130 are coupled to the digital signal processor 110. A touchscreen display 132 external to the on-chip system 102 is coupled to the display controller 128 and the touchscreen controller 130. The PCD 100 may also include a video encoder 134, such as a phase alternating line ("PAL") encoder, a sequential storage color ("SECAM") encoder, a national television systems Committee ("NTSC") encoder, or any other type of video encoder 134. A video encoder 134 is coupled to the multicore CPU 110. A video amplifier 136 is coupled to the video encoder 134 and the touchscreen display 132. A video port 138 is coupled to the video amplifier 136. As depicted in fig. 14, a universal serial bus ("USB") controller 140 is coupled to CPU 110. Further, a USB port 142 is coupled to the USB controller 140.

A memory 112 (which may include PoP memory, cache, mask ROM/boot ROM, boot OTP memory, DDR type DRAM memory) may also be coupled to the CPU 110. A subscriber identity module ("SIM") card 146 may also be coupled to CPU 110. Further, as shown in fig. 14, a digital camera 148 may be coupled to the CPU 110. In an exemplary aspect, the digital camera 148 is a charge coupled device ("CCD") camera or a complementary metal oxide semiconductor ("CMOS") camera.

As further shown in FIG. 14, a stereo audio CODEC 150 may be coupled to the analog signal processor 126. Further, an audio amplifier 152 can be coupled to the stereo audio CODEC 150. In an exemplary aspect, a first stereo speaker 154 and a second stereo speaker 156 are coupled to the audio amplifier 152. Fig. 14 shows that a microphone amplifier 158 can also be coupled to the stereo audio CODEC 150. Further, a microphone 160 may be coupled to the microphone amplifier 158. In a particular aspect, a frequency modulation ("FM") radio tuner 162 may be coupled to the stereo audio CODEC 150. Further, an FM antenna 164 is coupled to the FM radio tuner 162. Further, stereo headphones 166 can be coupled to the stereo audio CODEC 150.

Fig. 14 also indicates that a radio frequency ("RF") transceiver 168 can be coupled to the analog signal processor 126. An RF switch 170 may be coupled to the RF transceiver 168 and an RF antenna 172. As shown in fig. 14, a keypad 174 may be coupled to the analog signal processor 126. Further, a mono headset with a microphone 176 may be coupled to the analog signal processor 126. Further, a vibrator device 178 may be coupled to the analog signal processor 126. Fig. 14 also shows that a power supply 188 (e.g., a battery) is coupled to the on-chip system 102 through a power management integrated circuit ("PMIC") 180. In a particular aspect, the power supply 188 includes a rechargeable DC battery or a DC power source derived from an alternating current ("AC") to DC transformer connected to an AC power source.

CPU 110 may also be coupled to one or more internal, on-chip thermal sensors 157A and one or more external, off-chip thermal sensors 157B. The on-chip thermal sensor 157A may include one or more proportional to absolute temperature ("PTAT") temperature sensors, which are based on a vertical PNP structure and are typically dedicated to complementary metal oxide semiconductor ("CMOS") very large scale integration ("VLST") circuits. Off-chip thermal sensor 157B may include one or more thermistors. The thermal sensor 157 may generate a voltage drop that is converted to a digital signal using an analog-to-digital converter ("ADC") controller (not shown). However, other types of thermal sensors 157 may be employed.

The touchscreen display 132, the video port 138, the USB port 142, the camera 148, the first stereo speaker 154, the second stereo speaker 156, the microphone 160, the FM antenna 164, the stereo headphones 166, the RF switch 170, the RF antenna 172, the keypad 174, the mono headset 176, the vibrator 178, the thermal sensor 157B, PMIC 180, and the power supply 188 may be external to the on-chip system 102. However, it will be understood that one or more of these devices, depicted as external to the on-chip system 102 in the exemplary embodiment of the PCD 100 of fig. 14, may be located on the chip 102 in other exemplary embodiments.

In a particular aspect, one or more of the method steps described herein may be performed by executable instructions and parameters in the memory 112 or in the form of the IC module 101 and/or the image CODEC module 113. Further, the IC module 101, the image CODEC module 113, the memory 112, the instructions stored therein, or a combination thereof may serve as means for performing one or more of the method steps described herein.

FIG. 15 is a schematic diagram 1500 illustrating an exemplary software architecture of the PCD in FIG. 14 for performing the intelligent compression method. As shown in fig. 15, the CPU or digital signal processor 110 is coupled to the memory 112 via a main bus 211. As mentioned above, CPU 110 is a multi-core processor having N core processors. That is, the CPU 110 includes a first core 222, a second core 224, and an nth core 230. As known to those skilled in the art, each of the first kernel 222, the second kernel 224, and the nth kernel 230 may be used to support a dedicated application or program. Alternatively, one or more applications or programs may be distributed for processing across two or more of the available cores.

The CPU 110 may receive commands from the IC module 101, which may include software and/or hardware. If embodied as software, module 101 includes instructions executed by CPU 110, where CPU 110 issues commands to other applications executed by CPU 110 and other processors.

The first core 222, the second core 224, through the nth core 230 of the CPU 110 may be integrated on a single integrated circuit die, or they may be integrated or coupled on separate dies in a multi-circuit package. The designer may couple the first core 222, the second core 224, through the nth core 230 via one or more shared caches, and they may implement message or instruction passing via a network topology, such as a bus, ring, mesh, and crossbar topology.

As is known in the art and described above in the definitions, bus 211 may include multiple communication paths via one or more wired or wireless connections. The bus 211 may have additional elements for enabling communication, such as controllers, buffers (caches), drivers, repeaters, and receivers, which are omitted for simplicity. Further, bus 211 may include address, control, and/or data connections for enabling appropriate communications among the aforementioned components.

When the logic used by PCD 100 is implemented in software, as shown in fig. 15, it should be noted that one or more of startup logic 250, management logic 260, IC interface logic 270, applications in application store 280, and portions of file system 290 may be stored on any computer-readable medium for use by or in connection with any computer-related system or method.

In the context of this document, a computer readable medium is an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program and data for use by or in connection with a computer related system or method. The various logic elements and data stores can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

The computer readable medium may be, for example but not limited to: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a Random Access Memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM, EEPROM, or flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical). Note that the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

In alternative embodiments, where one or more of the initiation logic 250, the management logic 260, and possibly the IC interface logic 270 are implemented in hardware, the various logic may be implemented using any or a combination of the following techniques (all of which are well known in the art): discrete logic circuitry with logic gates for implementing logic functions on data signals, Application Specific Integrated Circuits (ASICs) with appropriate combinational logic gates, Programmable Gate Arrays (PGAs), Field Programmable Gate Arrays (FPGAs), etc.

The memory 112 is a non-volatile data storage device, such as a flash memory or solid state memory device. Although depicted as a single device, the memory 112 may be a distributed memory device with separate data storage coupled to the digital signal processor 110 (or additional processor cores).

The startup logic 250 includes one or more executable instructions for selectively identifying, loading, and executing a selection program for intelligent compression. The startup logic unit 250 may recognize, load, and execute the selected smart compressor. An exemplary selection program may be found in the program store 296 of the embedded file system 290. The example selection program, when executed by one or more of the core processors in the CPU 110, may operate according to one or more signals provided by the IC module 101 to implement the intelligent compression method.

The administrative logic unit 260 includes one or more executable instructions for terminating the IC program on one or more of the respective processor cores, and selectively identifying, loading, and executing a more suitable replacement program. The management logic unit 260 is arranged to perform these functions at run time or when the PCD 100 is powered up and in use by the operator of the device. Replacement programs may be found in the program store 296 of the embedded file system 290.

Interface logic 270 includes one or more executable instructions for presenting, managing, and interacting with external inputs to view, configure, or otherwise update information stored in embedded file system 290. In one embodiment, interface logic 270 may operate in conjunction with manufacturer inputs received via USB port 142. These inputs may include one or more programs to be deleted from the program store 296 or added to the program store 296. Alternatively, the input may include edits or alterations to one or more of the programs in the program store 296. Further, the input may identify one or more changes or a global replacement of one or both of the startup logic 250 and the management logic 260. By way of example, the input may include changes to the particular type of compression algorithm used and/or the target application of the active intelligent compression method at the frame level or tile level.

The interface logic 270 enables a manufacturer to controllably configure and adjust the end user experience under defined operating conditions on the PCD 100. When memory 112 is a flash memory, one or more of launch logic 250, management logic 260, interface logic 270, an application in application store 280, or information in embedded file system 290 may be edited, replaced, or otherwise modified. In some embodiments, the interface logic 270 may allow an end user or operator of the PCD 100 to search, locate, modify, or replace information in the startup logic 250, the management logic 260, applications in the application store 280, or the embedded file system 290. The operator may use the generated interface to make changes that will be implemented at the next boot of the PCD 100. Alternatively, the operator may use the generated interface to make changes that are implemented during runtime.

The embedded file system 290 includes a hierarchically arranged memory management store 292. In this regard, the file system 290 may include a reserved portion of its overall file system capacity for storing information for configuring and managing the various IC algorithms used by the PCD 100.

Certain steps in the processes or process flows described in this specification naturally precede others so that the invention operates as described. However, the invention is not limited to the order of the steps described if such order or sequence does not alter the functionality of the invention. That is, it is to be appreciated that certain steps may be performed before, after, or in parallel (substantially simultaneously) with other steps without departing from the scope and spirit of the present invention. In some instances, certain steps may be omitted or not performed without departing from the invention. Moreover, words such as "thereafter," "then," "next," etc. are not intended to limit the order of the steps. These words are simply used to guide the reader through the description of the exemplary methods.

Additionally, one of ordinary skill in the programming arts can, without difficulty, write computer code or identify appropriate hardware and/or circuitry to implement the disclosed invention based on, for example, the flowcharts and associated descriptions in this specification. Therefore, disclosure of a particular set of program code instructions or detailed hardware devices is not considered necessary to obtain a sufficient understanding of how to make and use the invention. The inventive functionality of the claimed computer-implemented processes is explained in more detail in the above description and in conjunction with the accompanying drawings, which may illustrate various process flows.

In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code in a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a program from one place to another. A storage media may be any available media that can be accessed by a computer or processor. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

Thus, while selected aspects have been illustrated and described in detail, it will be understood that various substitutions and alterations may be made herein without departing from the spirit and scope of the invention, as defined by the following claims.

Claims

1. A method for intelligent data compression in a portable computing device-PCD-, the method comprising:

defining a threshold value for a key performance indicator within the PCD;

receiving a first data block into a compression module and compressing the first data block according to a lossless compression algorithm, wherein the compression module is operable to switch between a lossless compression algorithm and a lossy compression algorithm;

monitoring an activity level for the key performance indicator based at least on compression of the first data block and comparing the activity level to a defined threshold value;

switching the compression module to a lossy compression mode based on the comparison of the activity level for the key performance indicator to a defined threshold value indicating a QoS level or PCD performance degradation resulting from lossless compression of the first data block; and

a second data block is received into the compression module and compressed according to the lossy compression algorithm.

2. The method of claim 1, wherein the key performance indicator is selected from the group consisting of: a compression ratio for a block of data compressed by the compression module, a voltage level, an average supply voltage level, a frame rate, and a processing time for compressing the block of data.

3. The method of claim 1, wherein the key performance indicator is a temperature reading generated by a temperature sensor within the PCD.

4. The method of claim 1, wherein switching the compression module is further based on a hysteresis calculation.

5. The method of claim 1, wherein the first data block and the second data block are image frames.

6. The method of claim 1, wherein the first data block and the second data block are units of an image frame.

7. The method of claim 1, wherein the PCD is in the form of a wireless telephone.

8. A system for intelligent data compression in a Portable Computing Device (PCD), the system comprising:

a smart compression module, a monitor module, and an image CODEC module collectively configured to:

defining a threshold value for a key performance indicator within the PCD;

9. The system of claim 8, wherein the key performance indicator is selected from the group consisting of: a compression ratio for a block of data compressed by the compression module, a voltage level, an average supply voltage level, a frame rate, and a processing time for compressing the block of data.

10. The system of claim 8, wherein the key performance indicator is a temperature reading generated by a temperature sensor within the PCD.

11. The system of claim 8, wherein switching the compression module is further based on a hysteresis calculation.

12. The system of claim 8, wherein the first data block and the second data block are image frames.

13. The system of claim 8, wherein the first data block and the second data block are units of an image frame.

14. The system of claim 8, wherein the PCD is in the form of a wireless telephone.

15. A system for intelligent data compression in a Portable Computing Device (PCD), the system comprising:

means for defining threshold values for key performance indicators within the PCD;

means for receiving a first data block into a compression module and compressing the first data block according to a lossless compression algorithm, wherein the compression module is operable to switch between a lossless compression algorithm and a lossy compression algorithm;

means for monitoring an activity level for the key performance indicator based at least on compression of the first data block and comparing the activity level to a defined threshold value;

means for switching the compression module to a lossy compression mode based on the comparison of the activity level for the key performance indicator to a defined threshold value indicating a QoS level or PCD performance degradation resulting from lossless compression of the first data block; and

Means for receiving a second data block into the compression module and compressing the second data block according to the lossy compression algorithm.

16. The system of claim 15, wherein the key performance indicator is selected from the group consisting of: a compression ratio for a block of data compressed by the compression module, a voltage level, an average supply voltage level, a frame rate, and a processing time for compressing the block of data.

17. The system of claim 15, wherein the key performance indicator is a temperature reading generated by a temperature sensor within the PCD.

18. The system of claim 15, wherein switching the compression module is further based on a hysteresis calculation.

19. The system of claim 15, wherein the first data block and the second data block are image frames.

20. The system of claim 15, wherein the first data block and the second data block are units of an image frame.

21. The system of claim 15, wherein the PCD is in the form of a wireless telephone.

22. A non-transitory computer-readable medium storing code for intelligent data compression in a Portable Computing Device (PCD), the code comprising instructions executable by a processor to:

Defining a threshold value for a key performance indicator within the PCD;

23. The non-transitory computer-readable medium of claim 22, wherein the code further comprises instructions executable by the processor to:

the key performance indicator is selected from the group consisting of: a compression ratio for a block of data compressed by the compression module, a voltage level, an average supply voltage level, a frame rate, and a processing time for compressing the block of data.

24. The non-transitory computer-readable medium of claim 22, wherein the key performance indicator is a temperature reading generated by a temperature sensor within the PCD.

25. The non-transitory computer-readable medium of claim 22, wherein the code further comprises instructions executable by the processor to:

switching the compression module is further based on a hysteresis calculation.

26. The non-transitory computer-readable medium of claim 22, wherein the first data block and the second data block are image frames.

27. The non-transitory computer-readable medium of claim 22, wherein the first data block and the second data block are units of an image frame.

28. The non-transitory computer-readable medium of claim 22, wherein the PCD is in the form of a wireless telephone.