TWI543627B - Method for adaptively performing video decoding, adaptive complexity video decoder and adaptive audio/video playback system - Google Patents

Description

Adaptive video decoding method, complexity adaptive video decoder and adaptive audio/video playback system

The present invention relates to an adaptively performing video decoding method, and more particularly to an adaptive video decoding method, an adaptive complexity video decoder, and an adaptive audio/video playing system.

When the terminal user watches the video program at home using the home theater system, the encoded data of the video program is decoded to be displayed on the display panel of the home theater system, and the decoding result of the encoded data can be expanded or reduced to conform to the size and resolution of the display panel. Degree requirements. In general, the decoding device of the home theater system can easily decode the encoded data regardless of the complexity of the algorithm for decoding the encoded data and whether the video program is high resolution.

It is convenient for the end user to have a portable electronic device that can view the same video program from anywhere, such as a mobile phone or a Personal Digital Assistant (PDA). However, when the manufacturer's Research and Development (R&D) team is designing such a portable electronic device, some problems may arise. For example, in the case where the algorithm for decoding the encoded data is too complicated and/or the resolution of the video program is too high, the decoding capability of the portable electronic device may be insufficient. As another example, when one or more processing circuits within the portable electronic device operate at the most At high available operating frequencies, power consumption may be too high. Therefore, there is a need for a highly efficient and affordable video decoder to play video of acceptable quality using portable electronic devices with limited computing power.

In view of this, the present invention provides an adaptive video decoding method, a complexity adaptive video decoder, and an adaptive audio/video playback system.

The present invention provides an adaptive video decoding method, including: performing decoding complexity management based on bit stream information of an input bit stream of a complexity adaptive video decoder to determine whether to reduce the complexity adaptive video decoding. Decoding complexity of at least one of the plurality of components within the device; and selectively reducing decoding complexity of a portion of the plurality of components within the complexity adaptive video decoder.

The present invention further provides a complexity adaptive video decoder comprising: a plurality of components for performing a plurality of operations related to video decoding; and a decoding complexity manager for using a bit based on an input bit stream The stream information performs decoding complexity management to determine whether to reduce the decoding complexity of the at least one component in the complexity adaptive video decoder; wherein the decoding complexity manager selectively reduces the complexity of the adaptive video decoder The decoding complexity of some of the components.

The invention also provides an adaptive audio/video playing system comprising: An audio decoder for performing audio decoding to generate audio information; and a video decoder for performing video decoding, wherein the video decoder includes a decoding manager, the decoding manager is configured according to an input bit stream The bit stream information delays the audio playback of the audio information.

The adaptive video decoding method, the complexity adaptive video decoder and the adaptive audio/video playing system provided by the invention can adaptively reduce the decoding complexity, thereby reducing the decoding workload and the corresponding power consumption, in order to realize digital video. The playback system offers great flexibility.

The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Certain terms are used throughout the description and claims to refer to particular components. It should be understood by those of ordinary skill in the art that hardware manufacturers may refer to the same component by different nouns. This specification and the scope of the patent application do not use the difference of the names as the means for distinguishing the components, but the difference in the function of the components as the criterion for distinguishing. The term "including" as used throughout the specification and the scope of the patent application is an open term and should be interpreted as "including but not limited to". "About" means that within the acceptable error range, those skilled in the art can solve the technical problem within a certain error range, and substantially achieve the technical effect. In addition, the term "coupled" is used herein to include any direct and indirect electrical connection. Therefore, if a first device is coupled to a The second device means that the first device can be directly electrically connected to the second device, or can be electrically connected to the second device indirectly through other devices or connection means. The description of the present invention is intended to be illustrative of the preferred embodiments of the invention. The scope of the invention is defined by the scope of the appended claims.

Referring to FIGS. 1A and 1B, FIGS. 1A and 1B are block diagrams of a complexity adaptive video decoder 100 in accordance with a first embodiment of the present invention. The complexity adaptive video decoder 100 includes a variable length decoding (VLD) and inverse quantization module 110, an inverse transform unit 120, and a motion compensation (MC) module 130 (in the first 1A is labeled as "MC Module", the calculation unit 140, the reconstructed frame output unit 150, and the frame storage unit 160, wherein the inverse transform unit 120 can be, for example, Inverse Discrete Cosine Transform (IDCT). The unit; the motion compensation module 130 includes a temporal prediction unit 132 and a spatial prediction unit 134; the computing unit 140 can be, for example, an adder; the reconstructed frame output unit 150 includes a de-blocking filter, such as a loop filter ( The in-loop filter 152; and the frame storage unit 160 may be, for example, a frame buffer. In particular, the aforementioned deblocking filter (e.g., loop filter 152) is a complexity adaptive deblocking filter. Note that the temporal prediction unit 132 and the spatial prediction unit 134 may be referred to as an inter prediction unit and an intra prediction unit, respectively. As shown in FIG. 1B, the complexity adaptive video decoder 100 further includes a scaler 170, a display time adjustment driver 175, The display buffer 180 and the decoding complexity manager 100M are displayed. The decoding complexity manager 100M can be implemented by using a hardware or by using a software. According to an embodiment of the present invention, the internal structure of the decoding complexity manager 100M can be configured as a hardware calculator/processor for calculating the decoding complexity allowed by the audio/video playback system, and the complexity is calculated according to the calculation result. The decoding complexity of the degree adaptive decoder 100 is adjusted. For example, the calculator/processor can selectively reduce the decoding complexity of at least some of the components of the complex adaptive video decoder 100. However, the foregoing is for illustrative purposes only, and the implementation of the decoding complexity manager 100M is not limited thereto.

In accordance with this embodiment, the fast variable length decoding and inverse quantization module 110 is configured to perform fast VLD and inverse quantization on the input bit stream 108 to generate a plurality of inverse quantization results 118, and the inverse transform unit 120 is used to The inverse quantization result 118 performs an inverse transform to produce a plurality of inverse transform results 128. In addition, the motion compensation module 130 is configured to perform motion compensation according to the input bit stream 108 and generate a corresponding plurality of predicted output results 138, wherein the temporal prediction unit 132 is configured to perform temporal prediction, and the spatial prediction unit 134 is Used to perform spatial prediction. As shown in FIG. 1A, the calculation unit 140 is configured to aggregate a plurality of inverse transform results 128 and a plurality of predicted output results 138 to generate a plurality of compensated output results 148, and the reconstructed frame output unit 150 is configured to generate a plurality of The reconstruction frame 158, wherein the spatial prediction unit 134 performs spatial prediction according to the reconstructed data output by the reconstructed frame output unit 150. In addition, the frame storage unit 160 is configured to temporarily store a plurality of reconstructed frames 158. At least a portion of the time prediction unit 132 performs temporal prediction in accordance with at least a portion of the plurality of reconstruction frames 158 described above.

Referring to Figure 1B, the scaler 170 is used to perform image scaling (e.g., zooming in or out of the image) as needed. In the case of performing image scaling, the plurality of video frames 178 include a scaled version of the plurality of reconstructed frames 158. In the case where image scaling is not performed, the plurality of video frames 178 include a plurality of reconstructed frames 158, that is, the scaler 170 bypasses the plurality of reconstructed frames 158. In addition, the display buffer 180 is used to temporarily store a plurality of video frames 178. Under the control of the display time adjustment driver 175, a plurality of video frames 178 temporarily stored in the display buffer 180 are output as a plurality of output frames 188 for display at appropriate points in time corresponding to audio playback of the audio information. Therefore, by using the display time adjustment driver 175, the complexity adaptive decoder 100 can synchronize the audio and video playback of the audio/video playback system (not shown in FIGS. 1A and 1B) to eliminate the delay of video decoding (if presence). For example, when there is a heavy video decoding effort, a delay in video decoding may occur. Additionally, the decoding complexity manager 100M can perform decoding complexity management based on the input bitstream 108 and the display buffer state from the display buffer 180. For example, the decoding complexity manager 100M can selectively reduce the decoding complexity of some of the various components within the complexity adaptive video decoder 100. More specifically, the decoding complexity manager 100M can generate a plurality of control signals, such as control signals C1, C2, C3, C5, C6, and C7, according to the input bit stream 108 and the aforementioned display buffer status, to separately control the complexity. The decoding complexity of the corresponding components within the adaptive video decoder 100. example For example, as shown in FIGS. 1A and 1B, the fast variable length decoding and inverse quantization module 110 is controlled by the control signal C1, the inverse transform unit 120 is controlled by the control signal C2, and the motion compensation module 130 is controlled by the control. The signal C3, the reconstructed frame output unit 150 is controlled by the control signal C5, the aforementioned deblocking filter (for example, the loop filter 152) is controlled by the control signal C6, and the display time adjustment driver 175 is controlled by the control signal C7. .

In particular, at least a portion of the complexity adaptive video decoder 100, such as temporal prediction unit 132, spatial prediction unit 134, inverse transform unit 120, reconstructed frame output unit 150, and/or fast VLD and inverse quantization module 110, The operation may be performed according to the resolution of the plurality of reconstructed frames 158 instead of merely relying on the resolution of the plurality of original frames represented by the input bit stream 108 to reduce the complexity of the decoding operation. The reduced resolution frame can be obtained via down sampling, for example, by selecting a particular pixel to represent each block of the frame, such as selecting the lower right pixel of each 2x2 pixel block. In an embodiment, the time prediction unit 132 can operate according to the resolution of the plurality of reconstructed frames 158 instead of operating according to the resolution of the plurality of original frames to reduce the complexity of the temporal prediction. For example, temporal prediction unit 132 may further estimate at least a portion of the information in the portion of the information that has been omitted that may be used to perform temporal prediction. In addition, the spatial prediction unit 134 can operate according to the resolution of the plurality of reconstructed frames 158 instead of operating according to the resolution of the plurality of original frames to reduce the complexity of the spatial prediction. For example, spatial prediction unit 134 can further estimate at least a portion of the information in the portion of the information that has been omitted that can be used to perform spatial prediction. According to this embodiment, the inverse transform unit 120 can operate according to the resolution of the plurality of reconstructed frames 158 instead of operating according to the resolution of the plurality of original frames to reduce the complexity of the inverse transform. For example, the inverse transform unit 120 may omit a portion of the plurality of calculations of the inverse transform via selecting one of a plurality of preset inverse transform functions F _IT for inverse transform. In addition, the reconstructed frame output unit 150 can operate according to the resolution of the plurality of reconstructed frames 158 instead of the resolution of the plurality of original frames to reduce the complexity of generating a plurality of reconstructed frames. Loop filter 152 may selectively perform loop filtering for deblocking corresponding to one of a plurality of complexity levels based on whether one or more conditions are met, wherein the deblocking operation may, for example, be all deblocking, partial Deblock the block and skip the solution block. In addition, the fast variable length decoding and inverse quantization module 110 can operate according to the resolution of the plurality of reconstructed frames 158 instead of operating according to the resolution of multiple original frames to reduce the execution of fast VLD and inverse quantization. the complexity. More specifically, to reduce complexity, the fast variable length decoding and inverse quantization module 110 may use a lookup table containing a primary table and at least one sub-table (eg, one or more sub-tables) during decoding, where due to complexity The design phase of the adaptive video decoder 100 has a probability of using the primary table for the preset arrangement of the lookup table greater than the probability of using at least one child table.

According to some variations of this embodiment, at least a portion of the components of the complexity adaptive video decoder 100 may omit portions of the at least some of the components to be processed. To reduce the complexity of decoding the input bit stream 108. For the sake of brevity, a similar description of such variations will not be repeated here.

1C is a block diagram of an adaptive audio/video playback system 50 in accordance with the first embodiment. The adaptive audio/video playback system 50 includes an audio decoder 52 and a complexity adaptive video decoder 100. Audio decoder 52 is operative to perform audio decoding to produce audio information 58. Additionally, the complexity adaptive video decoder 100 can perform the video decoding disclosed in the first embodiment. Note that the decoding complexity manager 100M is for performing decoding complexity management on the input bitstream 108 when needed, and the decoding complexity manager 100M selectively reduces multiples within the complexity adaptive video decoder 100. The decoding complexity of some of the components in the component. For example, the decoding complexity manager 100M reduces the decoding complexity of one or more components within the complexity adaptive video decoder 100. As another example, the decoding complexity manager 100M delays audio playback of the audio information 58 without reducing the complexity of decoding of any components within the adaptive video decoder 100. Therefore, in the case where the video decoder cannot decode and output a plurality of frames matching the audio playback in time, the decoding complexity manager 100M can appropriately delay the audio playback of the adaptive audio/video playback system 50 to maintain the self. Adapts to audio/video playback synchronization between video playback and audio playback of audio/video playback system 50. As shown in FIG. 1C, the decoding complexity manager 100M generates at least one control signal C0 to control whether to delay the amount of delay in audio playback of the audio information 58 and/or audio playback of the audio information 58. In fact, the decoding complexity manager 100M can depend on the bit stream information of the input bit stream 108, the parameters representing the processing capability, and A combination of display buffer states or one of them determines the audio playback of the delayed audio information 58. For example, depending on one or both of the bitstream information of the input bitstream 108, the parameter representing the processing capability, and the display buffer state, the decoding complexity manager 100M determines whether to delay the audio playback of the audio information 58. For another example, the decoding complexity manager 100M determines the amount of delay in audio playback of the audio information 58 based on either or both of the bitstream information of the input bitstream 108, the parameter representing the processing capability, and the display buffer state. For the sake of brevity, a similar description of such variations will not be repeated here.

1D is a block diagram of a complexity adaptive video decoder 100-1 in accordance with a second embodiment of the present invention. This embodiment is a modification of the first embodiment. As shown in FIG. 1D, the aforementioned deblocking filter (for example, loop filter 152) is disposed outside the reconstructed frame output unit 150 of this embodiment, instead of being disposed in the reconstructed frame output unit 150. Inside. Please note that other components in the complexity adaptive video decoder 100-1 (eg, fast variable length decoding and inverse quantization module 110, inverse transform unit 120, motion compensation module 130, computing unit 140, frame storage unit 160) The implementation may be the same as the corresponding components in the complexity adaptive video decoder 100 shown in FIG. 1A. In this embodiment, the plurality of elements in FIG. 1B can also be coupled to the first DD. For the sake of brevity, a similar description of this embodiment will not be repeated here.

According to some variations of this embodiment, at least a portion of the components of the complexity adaptive video decoder 100-1 may omit the at least a portion of the portion of the component to be processed To reduce the complexity of decoding the input bit stream 108. For the sake of brevity, a similar description of such variations will not be repeated here.

Figure 1E is a block diagram of an adaptive audio/video playback system 60 in accordance with a second embodiment. The adaptive audio/video playback system 60 includes an audio decoder 62 and a complexity adaptive video decoder 100-1. Audio decoder 62 is operative to perform audio decoding to produce audio information 68. Additionally, the complexity adaptive video decoder 100-1 can perform the video decoding disclosed in the second embodiment. Note that the decoding complexity manager 100M is used to perform decoding complexity management on the input bitstream 108 when needed, and the decoding complexity manager 100M selectively reduces the complexity within the adaptive video decoder 100-1. The decoding complexity of some of the multiple components. For example, the decoding complexity manager 100M reduces the decoding complexity of one or more components within the complexity adaptive video decoder 100-1. As another example, the decoding complexity manager 100M delays audio playback of the audio information 68 without reducing the decoding complexity of any component within the complexity adaptive video decoder 100-1. Accordingly, the decoding complexity manager 100M appropriately delays the audio playback of the adaptive audio/video playback system 60 to maintain the audio/video playback synchronization between the video playback and audio playback of the adaptive audio/video playback system 60, thereby solving any Video decoding delay problem. As shown in FIG. 1E, the decoding complexity manager 100M generates at least one control signal, such as the aforementioned at least one control signal C0, to control whether to delay the audio playback of the audio information 68 and/or the delay of the audio playback of the audio information 68. . In fact, the decoding complexity manager 100M may be based on input bits as described, for example, in the embodiment shown in FIG. 1C. The combination of bit stream information of the stream 108, a parameter representing the processing capability, and a display buffer state, or one of them, determines the audio playback of the delayed audio information 68. For example, based on either or both of the bitstream information of the input bitstream 108, the parameter representing the processing capability, and the display buffer state, the decoding complexity manager 100M determines whether to delay audio playback of the audio information 68. For another example, the decoding complexity manager 100M determines the amount of delay in audio playback of the audio information 68 based on either or both of the bitstream information of the input bitstream 108, the parameter representing the processing capability, and the display buffer state. For the sake of brevity, a similar description of this variant will not be repeated here.

The video decoding complexity of the various components can be adjusted via the architecture using any of the embodiments/variations disclosed above. For example, under the control of the decoding complexity manager 100M, the complexity of the first embodiment (or one of its variants) within the adaptive video decoder 100 or the second embodiment (or one of its variants) is One or more components within adaptive video decoder 100-1 may operate in accordance with lower levels of video decoding complexity. For another example, under the control of the decoding complexity manager 100M, when it is decided that there is no need to reduce the complexity of the first embodiment (or one of its variants) within the adaptive video decoder 100 or the second embodiment (or its variant a) Complexity When decoding the complexity of any component within video decoder 100-1, all components within complexity adaptive video decoder 100 or complexity adaptive video decoder 100-1 may be based on the highest level Video decoding complexity operates. Referring to Figure 2, further details are further described below.

2 is a flow diagram of an adaptive video decoding method 910 in accordance with an embodiment of the present invention. The adaptive video decoding method 910 can be applied to the complexity adaptive video decoder 100 shown in FIGS. 1A and 1B or the complexity adaptive video decoder 100-1 shown in FIG. 1D, and more particularly, The decoding complexity manager 100M disclosed in any of the embodiments/variations described above. The description of adaptive video decoding method 910 is as follows.

In step S912, the decoding complexity manager 100M adapts the video decoder based on the complexity in the consideration (for example, the complexity adaptive video decoder 100 of the first embodiment or the complexity adaptive video decoder of the second embodiment) The input bitstream 108 of 100-1) performs decoding complexity management to determine whether to reduce the decoding complexity of at least one of the plurality of components within the complexity adaptive video decoder. For example, the plurality of components may include a fast VLD and inverse quantization module 110, an inverse transform unit 120, a motion compensation module 130, a reconstructed frame output unit 150, and the aforementioned deblocking filter (eg, loop filter 152). And display time adjustment driver 175.

In step S914, the decoding complexity manager 100M selectively reduces the decoding complexity of some of the plurality of components within the complexity adaptive video decoder described in step S912. For example, the decoding complexity manager 100M can selectively reduce the fast VLD and inverse quantization module 110, the inverse transform unit 120, the motion compensation module 130, the reconstructed frame output unit 150, and the aforementioned solution block at a specific time point. Filter (eg loop filter 152) and display time adjustment driver The decoding complexity of at least a portion (e.g., a portion or all) of 175. For another example, the decoding complexity manager 100M does not reduce the fast VLD and inverse quantization module 110, the inverse transform unit 120, the motion compensation module 130, the reconstructed frame output unit 150, and the aforementioned deblocking filtering at another time point. The decoding complexity of any of the devices (e.g., loop filter 152) and display time adjustment driver 175.

According to this embodiment, the decoding complexity manager 100M can refer to the display buffer state of the display buffer 180, the parameters of the one or more representative processing capabilities of the complexity adaptive video decoder described in step S912, and the input bits. The decoding complexity management is performed by a combination or one of the bit stream information of the meta-stream 108, wherein the bit stream information may include a bit rate, a resolution, and/or a frame type. The bitstream information may include a plurality of motion vectors, a plurality of residuals, a plurality of IDCT coefficients, a bitstream length, or available bitstream information in some other instances, wherein the plurality of residuals may be, for example The direct current (DC) value, the available bit stream information can be obtained from the information of the known bit stream based on, for example, comparison and/or historical statistics. In an embodiment, the decoding complexity manager 100M adjusts the decoding complexity based on changes in the plurality of motion vectors. The parameter representing the processing capability may include statistics of the decoding time required for each frame. For example, the decoding complexity manager 100M may use the average decoding time of each frame to determine the current system capability, thereby adaptively adjusting the decoding complexity. . In particular, the decoding complexity manager 100M may perform decoding complexity management based on the aforementioned display buffer state of the display buffer 180, the aforementioned at least one parameter, and all of the bitstream information of the input bitstream 108.

3 is a schematic diagram of some operations that may be triggered by the decoding complexity manager 100M in accordance with an embodiment of the present invention, wherein the operations are related to the decoding complexity management of the adaptive video decoding method 910 as shown in FIG. For example, the operations that the decoding complexity manager 100M can trigger include display time adjustment, discard frame, deblocking block complexity adjustment, and motion compensation complexity adjustment (marked as "MC complexity adjustment" in FIG. ), intra-frame prediction complexity adjustment, inverse transform complexity adjustment, and VLD complexity adjustment. Based on the aforementioned display buffer status of the display buffer 180, a parameter representing the processing capability, and a combination of bit stream information of the input bit stream 108, the decoding complexity manager 100M performs decoding complexity management and selects One or more of these operations are triggered sexually.

4 is a schematic diagram showing the correlation quality and processing speed of some operations/steps that can be triggered by the decoding complexity manager 100M according to an embodiment of the present invention, wherein the operations are related to adaptive video decoding as shown in FIG. The decoding complexity management of method 910. For example, the operations/steps may include: a standard decoding process, a Bi-directional frame (B frame) (hereinafter referred to as a "B-frame") solution block (ie, adaptively executing all or part of the B-channel) Block the block or skip the B frame to solve the block, where the B frame is to solve the block operation on multiple B frames), and the B frame is simplified motion compensation (marked as "Figure 4" B frame simplifies MC"), fast B frame decoding (for example, decoding B frames in the reduced sampling domain), discarding B frames, and adaptive predictive frames (Predictive frames, P frame) (hereinafter referred to as "P-frame") solution block (for example, adaptively executing all or part of the P-frame deblocking or skipping the P-frame deblocking, where the P-frame deblocking refers to the block Multiple P-frames perform deblocking operations), P-frames simplify motion compensation (marked as "P-frame simplification MC" in Figure 4), fast P-frame decoding, discarding P-frames, adaptive frames Intra frame (I frame) (hereinafter referred to as "I frame") solution block (for example, adaptively executing all or part of the I frame deblocking or skipping the I frame deblocking block, where the I frame Deblocking refers to performing deblocking operations on multiple I-frames and fast I-frame decoding. The examples are for illustrative purposes only and the invention is not limited thereto. The "Quality" mark on the vertical axis can represent a comprehensive evaluation of image quality, fluency, user perception, system resource conservation, or various quality indicators. According to some variations of this embodiment, the quality and processing speed corresponding to these operations/steps may vary in different situations. For example, in the operation/step shown in FIG. 4, the operation/step of discarding the B frame can be inserted between the operation/step of the standard decoding process and the operation/step of the adaptive B frame demapping block. For another example, in the operation/step shown in FIG. 4, the operation/step of the adaptive I-frame deblocking can be inserted into the P-frame simplified motion compensation operation/step and the fast P-frame decoding operation/step between. For another example, in the operation/step shown in FIG. 4, the operation/step of discarding the P frame can be adjacent to the operation/step of the fast I frame decoding, and thus becomes the lowest level of the operation/step shown in FIG. An operation/step.

In this embodiment, based on the first policy - achieving the best quality available, when the video decoding delay problem is not the crux of the problem, the decoding complexity manager 100M The actions/steps that trigger the standard decoding process. However, when it is determined that a delay in video decoding will occur (eg, insufficient hardware resources are found and/or the decoding effort is greater than a predetermined threshold), the decoding complexity manager 100M may trigger one or more operations/steps to prevent serious The video decode delays and achieves the best quality that the decoder can provide in the current situation. For example, in the operation/step shown in FIG. 4, the decoding complexity manager 100M can trigger the next appropriate operation/step (for example, the operation/step of the adaptive B frame deblocking, the B frame simplifies the motion compensation). Operations/steps, etc.) to free up hardware resources and/or reduce decoding effort, wherein when decoding complexity is reduced, operations/steps corresponding to higher quality levels have higher execution priority, corresponding to Lower quality level operations/steps will be performed at the end. The examples are for illustrative purposes only and the invention is not limited thereto. According to a variation of this embodiment, the decoding complexity manager 100M can operate based on the second policy-implementing a predetermined quality level, rather than operating based on achieving optimal quality. According to some other variations of this embodiment, the decoding complexity manager 100M can operate based on some strategies that implement the fastest processing speed or preset processing speed.

5 is a schematic diagram showing the correlation quality and processing speed of some operations/steps that can be triggered by the decoding complexity manager 100M according to another embodiment of the present invention, wherein the operations are related to the adaptive video as shown in FIG. Decoding complexity management of decoding method 910. In particular, the operations/steps considered in this embodiment include: normal deblocking, simple deblocking, and disabling of the deblocking. Please note that the control mechanism of the decoding complexity manager 100M can be based on a plurality of preset thresholds. (For example, the two thresholds T1 and T2 shown in FIG. 5) are switched between these operations/steps, wherein the preset thresholds T1 and T2 represent respective levels of quality.

Figure 6 is a schematic diagram of some operations/steps that the decoding complexity manager 100M can trigger in different situations in accordance with another embodiment of the present invention, relating to the adaptive video decoding method 910 as shown in Figure 2; Decoding complexity management. In particular, the operations/steps considered in this embodiment include: stopping the B frame to solve the block, stopping the P frame to solve the block, starting the adaptive P frame to solve the block, starting the adaptive I frame solution block, and Stop solving the block. Please note that the control mechanism of the decoding complexity manager 100M can switch between these operations/steps according to a plurality of preset thresholds (for example, three thresholds T _I , T _P and T _B shown in FIG. 6 ). Wherein the preset thresholds T _I , T _P and T _B represent respective levels of display buffer status. In this embodiment, the preset thresholds T _I , T _P and T _B are respectively used to determine whether to start/stop the B frame deblock, the P frame deblock, and the I frame deblock. In particular, in the event that display buffer 180 is full or nearly full, decoding complexity manager 100M does not stop any of the I-frame deblocking, P-frame deblocking, and B-frame deblocking. Conversely, in the event that display buffer 180 is empty or nearly empty, decoding complexity manager 100M stops each of the I-frame deblocking, P-frame deblocking, and B-frame deblocking.

More specifically, in the case where the buffer state of the display buffer 180 (more specifically, the buffer level of the display buffer 180) is between the preset thresholds T _P and T _B , the decoding complexity manager 100M stops B. The frame deblocks the block and triggers the operation/step of starting the adaptive P-frame to solve the block (ie, the P-frame solution block with complexity adaptation). In addition, in the case where the buffer state of the display buffer 180 (more specifically, the buffer level of the display buffer 180) is between the preset thresholds T _I and T _P , the decoding complexity manager 100M stops the P frame. The block and the B frame are decoded, and the operation/step of starting the adaptive I frame decoding block (ie, the I-frame solution block with complexity adaptation) is triggered. In addition, in the case where the buffer state of the display buffer 180 (more specifically, the buffer level of the display buffer 180) is between the preset thresholds T _I and 0, the decoding complexity manager 100M stops the I frame solution. Block, P-frame solution block and B-frame solution block.

In this embodiment, with respect to the control mechanism of the complexity adaptive deblocking, the decoding complexity manager 100M can operate based on a macroblock (MB) based deblocking scheme. For example, the decoding complexity manager 100M can adaptively start/stop the I-frame deblocking, P-frame solution according to the buffer status and the bit stream length of at least a portion (eg, part or all) of the input bit stream 108. Block and / or B frame to solve the block. The examples are for illustrative purposes only and the invention is not limited thereto. According to a variation of this embodiment, with respect to the control mechanism of the complexity adaptive deblocking, the decoding complexity manager 100M can operate based on the frame-based deblocking scheme. For example, the decoding complexity manager 100M can be adaptively enabled according to the buffer status, the bit stream length of at least a portion (eg, a portion or all) of the input bit stream 108, the quantization step size, the MB type, and/or the edge strength. Stop I frame decryption block, P frame solution block and / or B frame solution A block in which the quantization step size is a so-called Quantization Parameter (QP) value.

An advantage of the present invention is that each of the embodiments/variable adaptive video decoding methods, associated complexity adaptive video decoders, and corresponding adaptive audio/video playback systems disclosed above are applicable to a variety of digital video applications, wherein At least a portion (eg, a portion or all) of the plurality of components within the complexity adaptive video decoder may operate at a low level of video decoding complexity. Since the video decoding complexity of one or more components can be adjusted, the decoding effort and corresponding power consumption can be adaptively reduced. Thus, the present invention provides great flexibility in implementing digital video playback systems as compared to related art.

The embodiments described above are only intended to illustrate the embodiments of the present invention, and to illustrate the technical features of the present invention, and are not intended to limit the scope of the present invention. It is intended that the present invention be construed as being limited by the scope of the invention.

100‧‧‧Complexity Adaptive Video Decoder

108‧‧‧Input bit stream

110‧‧‧Fast VLD and inverse quantization module

118‧‧‧ inverse quantified results

120‧‧‧ inverse transformation unit

128‧‧‧ inverse transformation results

130‧‧‧MC module

132‧‧‧Time prediction unit

134‧‧‧ Spatial Prediction Unit

138‧‧‧Predicted output

140‧‧‧ Calculation unit

148‧‧‧Compensation output

150‧‧‧Reconstruction frame output unit

152‧‧‧ Loop Filter

158‧‧‧Reconstruction frame

160‧‧‧ Frame storage unit

170‧‧‧Scaler

175‧‧‧Display time adjustment drive

178‧‧‧Image frame

180‧‧‧ display buffer

188‧‧‧Output frame

100M‧‧‧Decoding Complexity Manager

50‧‧‧Adaptive audio/video playback system

52‧‧‧Audio decoder

58‧‧‧Audio information

100-1‧‧‧Complexity Adaptive Video Decoder

62‧‧‧Audio decoder

68‧‧‧Audio information

910‧‧‧Adaptive video decoding method

C1-C7‧‧‧ control signal

S912-S914‧‧‧Steps

Figure 1A is a block diagram of a complexity adaptive video decoder in accordance with a first embodiment of the present invention.

1B is a block diagram of a complexity adaptive video decoder in accordance with a first embodiment of the present invention.

1C is a block diagram of an adaptive audio/video playback system in accordance with the first embodiment.

1D is a block diagram of a complexity adaptive video decoder in accordance with a second embodiment of the present invention.

Figure 1E is a block diagram of an adaptive audio/video playback system in accordance with a second embodiment.

2 is a flow chart of an adaptive video decoding method in accordance with an embodiment of the present invention.

FIG. 3 is a schematic diagram of some operations that can be triggered by the decoding complexity manager according to an embodiment of the invention.

Figure 4 is a schematic illustration of the correlation quality and processing speed of some of the operations/steps that the decoding complexity manager can trigger in accordance with an embodiment of the present invention.

Figure 5 is a schematic illustration of the correlation quality and processing speed of some of the operations/steps that the decoding complexity manager can trigger in accordance with another embodiment of the present invention.

Figure 6 is a schematic illustration of some of the operations/steps that the decoding complexity manager can trigger in different situations in accordance with another embodiment of the present invention.

100‧‧‧Complexity Adaptive Video Decoder

108‧‧‧Input bit stream

110‧‧‧Fast VLD and inverse quantization module

118‧‧‧ inverse quantified results

120‧‧‧ inverse transformation unit

128‧‧‧ inverse transformation results

130‧‧‧MC module

132‧‧‧Time prediction unit

134‧‧‧ Spatial Prediction Unit

138‧‧‧Predicted output

140‧‧‧Computation unit

148‧‧‧Compensation output

150‧‧‧Reconstruction frame output unit

152‧‧‧ Loop Filter

158‧‧‧Reconstruction frame

C1, C2, C3, C5, C6‧‧‧ control signals

160‧‧‧ Frame storage unit

Claims (27)

An adaptive video decoding method includes: performing different decoding complexity management based on bit stream information of an input bit stream of a complexity adaptive video decoder to determine whether to reduce the complexity adaptive video decoder Decoding complexity of multiple components; and selectively reducing decoding complexity in the plurality of components in the adaptive video decoder; wherein the plurality of components include: a fast variable length decoding and inverse quantization module And performing fast variable length decoding and inverse quantization on the input bit stream to generate a plurality of inverse quantization results; and a motion compensation module configured to perform motion compensation according to the input bit stream and generate corresponding multiples Predicting an output result, wherein the motion compensation module includes a temporal prediction unit and a spatial prediction unit, the temporal prediction unit is configured to perform temporal prediction to generate at least a portion of the plurality of predicted output results; and the spatial prediction unit is configured to perform Spatial prediction to generate at least a portion of the plurality of predicted output results. The adaptive video decoding method of claim 1, wherein the plurality of components comprise: an inverse transform unit, configured to perform inverse transform on the plurality of inverse quantization results to generate a plurality of inverse transform results; a computing unit, configured to aggregate the plurality of inverse transform results and the plurality of predictions Outputting a result to generate a plurality of compensated output results; a reconstructed frame output unit, configured to generate a plurality of reconstructed frames according to the plurality of compensated output results, wherein the spatial prediction unit is configured according to the plurality of reconstructed frames a current reconstructed frame performs spatial prediction; and a frame storage unit configured to temporarily store at least a portion of the plurality of reconstructed frames, wherein the temporal prediction unit is based on the at least a portion of the plurality of reconstructed frames Perform time predictions. The adaptive video decoding method of claim 1, wherein the plurality of components comprise: a display buffer for temporarily storing a plurality of video frames; wherein the step of performing decoding complexity management comprises : Performing decoding complexity management based on a display buffer state of the display buffer. The adaptive video decoding method of claim 1, wherein the step of performing decoding complexity management comprises: generating at least one control signal to control the plurality of components in the complexity adaptive video decoder The decoding complexity of this part of the component. The adaptive video decoding method of claim 1, wherein the plurality of components comprise: a display time adjustment driver, wherein the output is temporarily stored in a display buffer under the control of the display time adjustment driver Multiple video messages The frame serves as a plurality of output frames for respectively displaying at a plurality of time points of audio playback of the corresponding audio information. The adaptive video decoding method of claim 2, wherein the time prediction unit operates according to a resolution of the plurality of reconstructed frames to reduce the complexity of performing time prediction. The adaptive video decoding method according to claim 2, wherein the spatial prediction unit operates according to a resolution of the plurality of reconstructed frames to reduce complexity of performing spatial prediction; and the spatial prediction The unit further estimates that at least a portion of the partially excluded information can be used to perform spatial prediction. The adaptive video decoding method of claim 2, wherein the inverse transform unit operates according to a resolution of the plurality of reconstructed frames to reduce complexity of performing the inverse transform; and the inverse The transform unit omits a part of the plurality of calculations of the inverse transform. The adaptive video decoding method of claim 2, wherein the reconstructed frame output unit operates according to a resolution of the plurality of reconstructed frames to reduce generation of the plurality of reconstructed frames The complexity. The adaptive video decoding method according to claim 2, wherein the fast variable length decoding and inverse quantization module are based on one of the plurality of reconstructed frames The resolution operates to reduce the complexity of performing fast variable length decoding and inverse quantization; and the fast variable length decoding and inverse quantization module uses a lookup table including a primary table and at least one child table during decoding, and the primary A usage probability of the table is greater than a usage probability of the at least one sub-table. The adaptive video decoding method of claim 1, wherein the performing the decoding complexity management comprises: performing decoding complexity based on at least one parameter representing a processing capability of the complexity adaptive video decoder management. The adaptive video decoding method of claim 1, wherein the bit stream information of the input bit stream comprises a combination of one bit rate, one resolution, and one frame type or one of . The adaptive video decoding method of claim 1, wherein the bit stream information of the input bit stream comprises a plurality of motion vectors, a plurality of residuals, and a combination of bit stream lengths or One. A complexity adaptive video decoder comprising: a plurality of components for performing a plurality of operations related to video decoding; and a decoding complexity manager for performing differently based on bitstream information of an input bitstream Decoding complexity management to determine whether to reduce the decoding complexity of the plurality of components in the complexity adaptive video decoder; wherein the decoding complexity manager selectively reduces the complexity adaptation Decoding complexity of the plurality of components in the video decoder; wherein the plurality of components comprise: a fast variable length decoding and inverse quantization module, configured to perform fast variable length decoding and inverse quantization on the input bit stream, Generating a plurality of inverse quantization results; and a motion compensation module for performing motion compensation according to the input bit stream and generating a corresponding plurality of predicted output results, wherein the motion compensation module includes a temporal prediction unit and a spatial prediction a unit, the temporal prediction unit configured to perform temporal prediction to generate at least a portion of the plurality of predicted output results; and the spatial prediction unit to perform spatial prediction to generate at least a portion of the plurality of predicted output results. The complexity adaptive video decoder of claim 14, wherein the plurality of components comprise: an inverse transform unit, configured to perform inverse transform on the plurality of inverse quantization results to generate a plurality of inverse transform results a calculating unit, configured to aggregate the plurality of inverse transform results and the plurality of predicted output results to generate a plurality of compensated output results; a reconstructed frame output unit, configured to generate multiple weights according to the plurality of compensated output results a frame, wherein the spatial prediction unit performs spatial prediction according to a current reconstructed frame of the plurality of reconstructed frames; and a frame storage unit configured to temporarily store at least a portion of the plurality of reconstructed frames The time prediction unit according to the plurality of reconstructed frames A small portion of the execution time prediction. The complexity adaptive video decoder of claim 14, wherein the plurality of components comprise: a display buffer for temporarily storing a plurality of video frames; wherein the decoding complexity manager is based on A display buffer state of the display buffer performs decoding complexity management. The complexity adaptive video decoder according to claim 14, wherein the decoding complexity manager generates at least one control signal according to the bit stream information of the input bit stream to control the complexity. Adapting to the decoding complexity of the portion of the plurality of components within the video decoder, the plurality of components being separately controlled by the at least one control signal. The complexity adaptive video decoder according to claim 14, wherein the plurality of components comprise: a display time adjustment driver, wherein the retrieval is temporarily stored in a control under the control of the display time adjustment driver A plurality of image frames in the display buffer are used as a plurality of output frames, and the plurality of output frames are respectively displayed at a plurality of time points of audio playback of the corresponding audio information. The complexity adaptive video decoder according to claim 15, wherein the time prediction unit operates according to a resolution of the plurality of reconstructed frames to reduce the complexity of performing time prediction. The complexity adaptive video decoder according to claim 15, wherein the spatial prediction unit operates according to a resolution of the plurality of reconstructed frames to reduce the complexity of performing spatial prediction; The spatial prediction unit further estimates that at least a portion of the partially excluded information is available for performing spatial prediction. The complexity adaptive video decoder according to claim 15, wherein the inverse transform unit operates according to a resolution of the plurality of reconstructed frames to reduce the complexity of performing the inverse transform; The inverse transform unit omits a part of the plurality of calculations of the inverse transform. The complexity adaptive video decoder according to claim 15, wherein the reconstructed frame output unit operates according to a resolution of the plurality of reconstructed frames to reduce the plurality of reconstructions. The complexity of the frame. The complexity adaptive video decoder according to claim 15, wherein the fast variable length decoding and the inverse quantization module operate according to a resolution of the plurality of reconstructed frames to reduce the fast change of execution. The complexity of long decoding and inverse quantization; and the fast variable length decoding and inverse quantization module uses a lookup table including a primary table and at least one child table during decoding, and a usage probability of the primary table is greater than the at least one A probability of use of a child table. Complexity adaptive video decoding as described in claim 14 And the decoding complexity manager performs decoding complexity management based on at least one parameter representing a processing capability of the complexity adaptive video decoder. The complexity adaptive video decoder of claim 14, wherein the bit stream information of the input bit stream comprises a combination of a bit rate, a resolution, and a frame type or one. The complexity adaptive video decoder of claim 14, wherein the bit stream information of the input bit stream comprises a plurality of motion vectors, a plurality of residuals, and a combination of bit stream lengths or one of them. The complexity adaptive video decoder of claim 14, wherein the decoding complexity manager delays audio playback of the audio information.