CN102316319B - Adaptive video coding/decoding method, decoder and audio/video playback system - Google Patents
Adaptive video coding/decoding method, decoder and audio/video playback system Download PDFInfo
- Publication number
- CN102316319B CN102316319B CN201110122127.7A CN201110122127A CN102316319B CN 102316319 B CN102316319 B CN 102316319B CN 201110122127 A CN201110122127 A CN 201110122127A CN 102316319 B CN102316319 B CN 102316319B
- Authority
- CN
- China
- Prior art keywords
- decoding
- adaptive video
- complex degree
- bit stream
- adaptive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention provides a kind of adaptive video coding/decoding method, decoder and audio/video playback system, and wherein, complicacy self-adaptive Video Decoder includes: multiple elements, for performing the multiple operations relevant with video decoding；And decoding complex degree manager, perform decoding complex degree management for bit stream information based on incoming bit stream, to decide whether to reduce the decoding complex degree of at least one element in complicacy self-adaptive Video Decoder.The complicacy self-adaptive Video Decoder utilizing the present invention can reduce decoding complex degree adaptively, thus reduces decoding effort amount and corresponding power consumption, for realizing the great flexibility that Digital Video Broadcasting System provides.
Description
Technical field
The present invention is related to a kind of adaptive video decoding (adaptively performing video decoding) side
Method, more particularly in a kind of adaptive video coding/decoding method, complicacy self-adaptive Video Decoder (adaptive
Complexity video decoder) and adaptive audio/audio/video player system.
Background technology
When terminal use's household audio and video system use at home viewing video frequency program, the coded data of video frequency program
It is decoded with display on the display screen of household audio and video system, and the decoded result of coded data can expand or contract
The requirement of little size and resolution to meet display screen.Generally, the calculation no matter coded data being decoded
Whether fado is complicated, be high-resolution regardless of video frequency program, and the decoding apparatus of household audio and video system can be light
Change places and coded data is decoded.
For terminal use, have the portable electronic dress that can watch same video frequency program anywhere
Putting is very easily, and this portable electron device can such as mobile phone or personal digital assistant (Personal
Digital Assistant, PDA).But, when manufacturer research and development (Research and Development,
R&D) team is when designing this portable electron device, in fact it could happen that some problems.Such as, to coding
In the case of the most complicated and/or video frequency program the resolution of the algorithm of decoding data is too high, portable electronic
The decoding capability of device is the most not enough.The most such as, when the one or more process in portable electron device
Circuit be operated in High Availabitity operating frequency time, power consumption is the most too high.Accordingly, it would be desirable to high efficiency and price
Desirable Video Decoder, has can connect to use the portable electron device with finite computational abilities to play
Video by quality.
Summary of the invention
In view of this, the present invention provides a kind of adaptive video coding/decoding method, the decoding of complicacy self-adaptive video
Device and adaptive audio/audio/video player system.
The present invention provides a kind of adaptive video coding/decoding method, including: decode based on complicacy self-adaptive video
The bit stream information of the incoming bit stream of device, performs decoding complex degree management, to decide whether to reduce complexity
The decoding complex degree of at least one element in multiple elements in adaptive video decoder；And selectivity reduces multiple
The decoding complex degree of a part of element in multiple elements in miscellaneous degree adaptive video decoder.
The present invention separately provides a kind of complicacy self-adaptive Video Decoder, including: multiple elements, it is used for performing
The multiple operations relevant to video decoding；And decoding complex degree manager, for based on incoming bit stream
Bit stream information performs decoding complex degree management, to decide whether to reduce in complicacy self-adaptive Video Decoder
The decoding complex degree of at least one element；Wherein, decoding complex degree manager selectivity reduces complicacy self-adaptive
The decoding complex degree of a part of element in multiple elements in Video Decoder.
The present invention also provides for a kind of adaptive audio/audio/video player system, including: audio decoder, it is used for holding
Row audio decoder is to produce audio-frequency information；And Video Decoder, it is used for performing video and decodes, wherein, depending on
Frequently decoder includes decoding management device, and decoding management device postpones audio frequency according to the bit stream information of incoming bit stream
The audio frequency of information is play.
Adaptive video coding/decoding method provided by the present invention, complicacy self-adaptive Video Decoder and adaptive
Answer audio/video playback system can reduce decoding complex degree adaptively, thus reduce decoding effort amount with corresponding
Power consumption, for realizing the great flexibility that Digital Video Broadcasting System provides.
Below for presently preferred embodiments of the present invention is described in detail according to multiple diagrams, the skill of art
Art personnel should be expressly understood that the purpose of the present invention after reading.
Accompanying drawing explanation
Figure 1A is the block schematic diagram of the complicacy self-adaptive Video Decoder according to first embodiment of the invention.
Figure 1B is the block schematic diagram of the complicacy self-adaptive Video Decoder according to first embodiment of the invention.
Fig. 1 C is the block schematic diagram of the adaptive audio/audio/video player system according to first embodiment.
Fig. 1 D is the block schematic diagram of the complicacy self-adaptive Video Decoder according to second embodiment of the invention.
Fig. 1 E is the block schematic diagram of the adaptive audio/audio/video player system according to the second embodiment.
Fig. 2 is the flow chart of the adaptive video coding/decoding method according to one embodiment of the present of invention.
Fig. 3 is the certain operations signal that the decoding complex degree manager according to one embodiment of the invention can trigger
Figure.
Fig. 4 is certain operations/step that the decoding complex degree manager according to one embodiment of the invention can trigger
Correlated quality and the schematic diagram of processing speed.
Fig. 5 is certain operations/step that the decoding complex degree manager according to another embodiment of the present invention can trigger
Rapid correlated quality and the schematic diagram of processing speed.
Fig. 6 is that the decoding complex degree manager according to another embodiment of the present invention can trigger in varied situations
The schematic diagram of certain operations/step.
Detailed description of the invention
Some vocabulary is employed to censure specific components in the middle of description and claim.Art
Technical staff it is to be appreciated that same assembly may be called with different nouns by manufacturer.This specification
And claim is not using the difference of title as in the way of distinguishing assembly, but with assembly difference functionally
Different as distinguishing criterion." comprising " mentioned in description and claim in the whole text is open language,
Therefore " comprise but be not limited to " should be construed to." substantially " for referring in acceptable range of error, this area
Technical staff can solve described technical problem in the range of certain error, basically reaches described technique effect.
Additionally, " coupling " word includes at this any directly and being indirectly electrically connected with means.Therefore, if literary composition is retouched
State a first device and be coupled to one second device, then represent this first device can directly be electrically connected at this second
Device, or indirectly it is electrically connected to this second device by other device or connection means.Description is follow-up
Being described as implementing the better embodiment of the present invention, so this description is with the rule that the present invention is described as mesh
, it is not intended to limit the scope of the present invention.Protection scope of the present invention is when being defined depending on claims
Person is as the criterion.
It is that the complexity according to first embodiment of the invention is adaptive with reference to Figure 1A and Figure 1B, Figure 1A and Figure 1B
Answer the block schematic diagram of Video Decoder 100.Complicacy self-adaptive Video Decoder 100 includes the most elongated
Decoding (Variable Length Decoding, VLD) and inverse quantization module 110, inverse transformation (inverse transform)
Unit 120, motion compensation (Motion Compensation, MC) module 130 (are labeled as " MC in figure ia
Module "), computing unit 140, reconstruction frames output unit 150 and frame memory element 160, wherein, inverse transformation
Unit 120 can such as inverse discrete cosine transform (Inverse Discrete Cosine Transform, IDCT) unit；
Motion compensating module 130 includes time prediction unit 132 and spatial prediction unit 134；Computing unit 140
Can such as adder；Reconstruction frames output unit 150 includes (de-blocking) wave filter that deblocks, such as loop filter
Ripple device (in-loop filter) 152；And frame memory element 160 can such as frame buffer.Especially, aforesaid
Deblocking filter (such as loop filter 152) is complicacy self-adaptive deblocking filter.Note that can be by
Time prediction unit 132 and spatial prediction unit 134 be called inter prediction (inter prediction) unit with
Infra-frame prediction (intra prediction) unit.As shown in Figure 1B, complicacy self-adaptive Video Decoder 100 enters
One step includes that scaler (resizer) 170, display time adjust driver 175, display buffer 180 and solve
Code complexity manager 100M.Wherein, decoding complex degree manager 100M can employ hardware to realize, also
Software can be used to realize.According to one embodiment of the present invention, decoding complex degree manager 100M's is interior
Portion's structure can be configured to hardware computer/processor, for calculating the solution that audio/video playback system can allow
Code complexity, and according to result of calculation, the decoding complex degree of complicacy self-adaptive decoder 100 is adjusted.
Such as in this computer/processor alternative reduces complicacy self-adaptive Video Decoder 100 in multiple elements
The decoding complex degree of at least some of element.It is for illustration purposes only more than but, decoding complex degree manager
The embodiment of 100M is not limited thereto.
According to this embodiment, quick variable length decoding and inverse quantization module 110 are for holding incoming bit stream 108
The quick VLD of row and inverse quantization are to produce multiple inverse quantization results 118, and inverse transformation unit 120 is for many
Individual inverse quantization result 118 performs inverse transformation to produce multiple inverse transformation results 128.It addition, motion compensating module
130 for performing motion compensation according to incoming bit stream 108, and produces corresponding multiple prediction output result
138, wherein, time prediction unit 132 is for running time prediction, and spatial prediction unit 134 is used for
Perform spatial prediction.As shown in Figure 1A, computing unit 140 is used for adding up to multiple inverse transformation result 128 with many
Individual prediction output result 138 exports result 148 to produce multiple compensation, and reconstruction frames output unit 150 is used for
Produce multiple reconstruction frames 158, the weight that wherein spatial prediction unit 134 exports according to reconstruction frames output unit 150
Build data and perform spatial prediction.It addition, frame memory element 160 is for the multiple reconstruction frames 158 of interim storage
At least partially, wherein, time prediction unit 132 is according at least one of aforesaid multiple reconstruction frames 158
Divide running time prediction.
With reference to Figure 1B, scaler 170 is for (such as zooming in or out figure performing image scaling whenever necessary
Picture).In the case of performing image scaling, multiple picture frames 178 include the scaled version of multiple reconstruction frames 158
This.In the case of not performing image scaling, multiple picture frames 178 include multiple reconstruction frames 158, i.e. contracting
Put device 170 and bypass (bypass) multiple reconstruction frames 158.It addition, display buffer 180 is many for interim storage
Individual picture frame 178.Adjusting under the control of driver 175 in the display time, output is temporarily stored in display buffer
Multiple picture frames 178 in device 180 are as multiple output frames 188, with display at the sound corresponding to audio-frequency information
On the appropriate time point that frequency is play.Therefore, by using the display time to adjust driver 175, complexity is adaptive
Answer the decoder 100 can the Voice & Video of isochronous audio/audio/video player system (not showing in Figure 1A and Figure 1B)
Play, to eliminate the delay (if present) of video decoding.Such as, when there is heavy video decoding effort amount
Time, it may occur that the delay of video decoding.It addition, decoding complex degree manager 100M can be based on input ratio
Special stream 108 and the display buffer state from display buffer 180 perform decoding complex degree management.Such as,
Decoding complex degree manager 100M alternative reduces multiple elements in complicacy self-adaptive Video Decoder 100
The decoding complex degree of middle a part of element.More specifically, decoding complex degree manager 100M can be according to input ratio
Special stream 108 and aforesaid display buffer state, produce multiple control signal, such as control signal C1, C2,
C3, C5, C6 and C7, to control the solution of respective element in complicacy self-adaptive Video Decoder 100 respectively
Code complexity.Such as, as shown in FIG. 1A and 1B, quick variable length decoding is controlled with inverse quantization module 110
In control signal C1, inverse transformation unit 120 is controlled by control signal C2, and motion compensating module 130 is controlled by
Control signal C3, reconstruction frames output unit 150 is controlled by control signal C5, aforesaid deblocking filter (example
Such as loop filter 152) it is controlled by control signal C6, and the display time adjusts driver 175 and is controlled by
Control signal C7.
Especially, complicacy self-adaptive Video Decoder 100 at least some of, such as time prediction unit
132, spatial prediction unit 134, inverse transformation unit 120, reconstruction frames output unit 150 and/or quick VLD
With inverse quantization module 110, can carry out operating rather than according only to defeated according to the resolution of multiple reconstruction frames 158
The resolution entering the multiple primitive frames representated by bit stream 108 operates, to reduce the complexity of decoding operation
Degree.Can be by frame down-sampling (down sampling) being obtained the frame that resolution reduces, such as, by selecting spy
Fixation element to represent each block of frame, such as, selects the lower right corner pixel of each 2 × 2 block of pixels.One
In individual embodiment, time prediction unit 132 can operate according to the resolution of multiple reconstruction frames 158, and
It not that the resolution according to multiple primitive frames operates, to reduce the complexity of time prediction.Such as, time
Between predicting unit 132 can estimate further the partial information having been omitted from can be used for running time prediction at least
A part of information.It addition, spatial prediction unit 134 can be grasped according to the resolution of multiple reconstruction frames 158
Make rather than operate according to the resolution of multiple primitive frames, to reduce the complexity of spatial prediction.Example
As, spatial prediction unit 134 can be estimated to can be used in the partial information having been omitted from performing spatial prediction further
At least some of information.According to this embodiment, inverse transformation unit 120 can be according to multiple reconstruction frames 158
Resolution carries out operating rather than operating according to the resolution of multiple primitive frames, to reduce inverse transformation
Complexity.Such as, inverse transformation unit 120 can be by selecting multiple predetermined contravariant exchange the letters for inverse transformation
In number FIT one is to omit a part for multiple calculating of inverse transformation.It addition, reconstruction frames output unit 150
Can carry out operating according to the resolution of multiple reconstruction frames 158 rather than the resolution of multiple primitive frame is grasped
Make, to reduce the complexity producing multiple reconstruction frames.Based on whether meet one or more condition, loop is filtered
Ripple device 152 can perform loop filtering corresponding to a selectivity in multiple levels of complexity and be used for deblocking, its
In, deblocking operation such as can all deblock, partly deblock and skip (skip) and deblock.It addition, quickly become
Long decoding can carry out operating rather than root according to the resolution of multiple reconstruction frames 158 with inverse quantization module 110
Operate according to the resolution of multiple primitive frames, to reduce the complexity performing quick VLD with inverse quantization.More
Especially, for reducing complexity, quick variable length decoding and inverse quantization module 110 can use bag during decoding
Include master meter and the look-up table of at least one sublist (the most one or more sublist), wherein, due in complexity
The design phase of adaptive video decoder 100 preset schedule to look-up table, uses the probability of master meter to be more than
Use the probability of at least one sublist.
According to some deformation of this embodiment, at least some of unit of complicacy self-adaptive Video Decoder 100
Part can omit the partial information that this at least some of element need to process, and carries out incoming bit stream 108 to reduce
The complexity of decoding.For purpose of brevity, the similar description about this type of deformation is not repeated herein.
Fig. 1 C is the block schematic diagram of the adaptive audio/audio/video player system 50 according to first embodiment.From
Adapt to audio/video playback system 50 and include audio decoder 52 and complicacy self-adaptive Video Decoder 100.
Audio decoder 52 is used for performing audio decoder to produce audio-frequency information 58.It addition, complicacy self-adaptive video
Decoder 100 can perform the video decoding disclosed in first embodiment.Note that decoding complex degree manages
Device 100M for performing decoding complex degree management whenever necessary to incoming bit stream 108, and decoding complex degree
A part of element in multiple elements in manager 100M selectivity reduction complicacy self-adaptive Video Decoder 100
Decoding complex degree.Such as, decoding complex degree manager 100M reduces complicacy self-adaptive Video Decoder
The decoding complex degree of one or more elements in 100.The most such as, decoding complex degree manager 100M postpones sound
Frequently the audio frequency of information 58 is play, and the solution of any element in not reducing complicacy self-adaptive Video Decoder 100
Code complexity.Therefore, Video Decoder can not decode in time and export with audio frequency play match many
In the case of individual frame, decoding complex degree manager 100M can suitable delayed adaptation audio/video playback system
The audio frequency of 50 is play, and plays it with the video playback and audio frequency keeping adaptive audio/audio/video player system 50
Between audio/video playback synchronize.As shown in Figure 1 C, decoding complex degree manager 100M produces at least one control
Signal C0 processed is to control whether that the audio frequency postponing the audio frequency broadcasting of audio-frequency information 58 and/or audio-frequency information 58 is broadcast
The retardation put.It practice, decoding complex degree manager 100M can be according to the bit stream of incoming bit stream 108
Information, the parameter representing disposal ability and the combination of display buffer state or one of them, determine postpone sound
Frequently the audio frequency of information 58 is play.Such as, energy is processed according to bit stream information, the representative of incoming bit stream 108
The parameter of power and the combination of display buffer state or one of them, decoding complex degree manager 100M determine be
The audio frequency of no delay audio-frequency information 58 is play.The most such as, according to the bit stream information of incoming bit stream 108,
Represent the parameter of disposal ability and the combination of display buffer state or one of them, decoding complex degree manager
100M determines the retardation that the audio frequency of audio-frequency information 58 is play.For purpose of brevity, being similar to about this type of deformation
Describe and be not repeated herein.
Fig. 1 D is that the square of the complicacy self-adaptive Video Decoder 100-1 according to second embodiment of the invention shows
It is intended to.This embodiment is the deformation of first embodiment.As shown in figure ip, by aforesaid deblocking filter (example
Such as loop filter 152) be placed in the reconstruction frames output unit 150 of this embodiment outside rather than be placed in
Within reconstruction frames output unit 150.Note that other yuan in complicacy self-adaptive Video Decoder 100-1
Part (such as quickly variable length decoding and inverse quantization module 110, inverse transformation unit 120, motion compensating module 130,
Computing unit 140, frame memory element 160) enforcement can be with the complicacy self-adaptive video solution shown in Figure 1A
In code device 100, corresponding element is identical.In this embodiment, in Figure 1B, multiple elements also can be with Fig. 1 D coupling
Connect.For purpose of brevity, the similar description about this embodiment is not repeated herein.
According to some deformation of this embodiment, at least some of unit of complicacy self-adaptive Video Decoder 100-1
Part can omit this partial information that need to process at least partially, to reduce the complexity of decoding incoming bit stream 108
Degree.For purpose of brevity, the similar description about this type of deformation is not repeated herein.
Fig. 1 E is the block schematic diagram of the adaptive audio/audio/video player system 60 according to the second embodiment.From
Adapt to audio/video playback system 60 and include audio decoder 62 and complicacy self-adaptive Video Decoder
100-1.Audio decoder 62 is used for performing audio decoder to produce audio-frequency information 68.It addition, complexity is certainly
Adaptive video decoder 100-1 can perform the video decoding disclosed in the second embodiment.Note that decoding is multiple
Miscellaneous degree manager 100M for performing decoding complex degree management whenever necessary to incoming bit stream 108, and solves
In code complexity manager 100M selectivity reduction complicacy self-adaptive Video Decoder 100-1 in multiple elements
The decoding complex degree of a part of element.Such as, decoding complex degree manager 100M reduction complicacy self-adaptive regards
Frequently the decoding complex degree of one or more elements in decoder 100-1.The most such as, decoding complex degree manager
100M postpones the audio frequency of audio-frequency information 68 and plays, and does not reduce complicacy self-adaptive Video Decoder 100-1
The decoding complex degree of interior any element.Therefore, decoding complex degree manager 100M suitable delayed adaptation audio frequency
The audio frequency of/audio/video player system 60 is play, to keep the video playback of adaptive audio/audio/video player system 60
With the audio/video playback between audio frequency broadcasting synchronizes, thus solve any video decoding delay problem.Such as figure
Shown in 1E, decoding complex degree manager 100M produces at least one control signal, the most aforesaid at least one control
Signal C0 processed, to control whether that the audio frequency postponing the audio frequency broadcasting of audio-frequency information 68 and/or audio-frequency information 68 is broadcast
The retardation put.It practice, decoding complex degree manager 100M can be according to the enforcement shown in such as Fig. 1 C
The bit stream information of the incoming bit stream 108 described in example, the parameter representing disposal ability and display buffer
The combination of state or one of them, determine postpone audio-frequency information 68 audio frequency play.Such as, according to input ratio
Bit stream information, the parameter representing disposal ability and the combination of display buffer state of special stream 108 or wherein
One of, decoding complex degree manager 100M decides whether that the audio frequency postponing audio-frequency information 68 is play.The most such as,
Bit stream information according to incoming bit stream 108, the parameter representing disposal ability and display buffer state
Combining or one of them, decoding complex degree manager 100M determines the delay that the audio frequency of audio-frequency information 68 is play
Amount.For purpose of brevity, the similar description about this deformation is not repeated herein.
By using the framework of any embodiment/deformation of the disclosure above, the video decoding of each element of adjustable
Complexity.Such as, under the control of decoding complex degree manager 100M, first embodiment (or the first enforcement
One of deformation of example) complicacy self-adaptive Video Decoder 100 in or the second embodiment (or first implement
One of deformation of example) complicacy self-adaptive Video Decoder 100-1 in one or more elements can be according to relatively
Low-level video decoding complexity operates.The most such as, in the control of decoding complex degree manager 100M
Under, when determining that the complexity that need not reduce first embodiment one of (or the deformation of first embodiment) is adaptive
In answering Video Decoder 100 or the complexity of the second embodiment one of (or the deformation of the second embodiment) is adaptive
In answering Video Decoder 100-1 during the decoding complex degree of any element, complicacy self-adaptive Video Decoder 100
Or all elements in complicacy self-adaptive Video Decoder 100-1 can decode multiple according to the video of highest level
Miscellaneous degree operates.With reference to Fig. 2, more details is further described below.
Fig. 2 is the flow chart of the adaptive video coding/decoding method 910 according to one embodiment of the invention.Adaptive
Video encoding/decoding method 910 is answered to be applicable to the complicacy self-adaptive Video Decoder 100 shown in Figure 1A and Figure 1B
Or the complicacy self-adaptive Video Decoder 100-1 shown in Fig. 1 D, more specifically, it is adaptable to any of the above described reality
Execute the decoding complex degree manager 100M disclosed in example/deformation.The description of adaptive video coding/decoding method 910 is such as
Under.
In step S912, decoding complex degree manager 100M is based on the complicacy self-adaptive video in considering
Decoder (the complicacy self-adaptive Video Decoder 100 of such as first embodiment or the complexity of the second embodiment
Degree adaptive video decoder 100-1) incoming bit stream 108 perform decoding complex degree management, with determine be
The decoding complex degree of at least one element in multiple elements in no reduction above-mentioned complicacy self-adaptive Video Decoder.
Such as, the plurality of element can include quick VLD and inverse quantization module 110, inverse transformation unit 120, motion
Compensating module 130, reconstruction frames output unit 150, aforesaid deblocking filter (such as loop filter 152)
And the display time adjusts driver 175.
In step S914, decoding complex degree manager 100M selectivity reduces described in step S912
The decoding complex degree of a part of element in multiple elements in complicacy self-adaptive Video Decoder.Such as, decoding
Complexity manager 100M particular point in time optionally reduce quick VLD and inverse quantization module 110,
Inverse transformation unit 120, motion compensating module 130, reconstruction frames output unit 150, aforesaid deblocking filter
(such as loop filter 152) and display time adjust (such as a part at least some of in driver 175
Or all) decoding complex degree.The most such as, decoding complex degree manager 100M does not reduces at another time point
Quickly VLD and inverse quantization module 110, inverse transformation unit 120, motion compensating module 130, reconstruction frames output
Unit 150, aforesaid deblocking filter (such as loop filter 152) and display time adjust driver 175
In any one decoding complex degree.
According to this embodiment, decoding complex degree manager 100M can be by the display with reference to display buffer 180
One or more representative office of the complicacy self-adaptive Video Decoder described in buffer state, step S912
The parameter of reason ability and the combination of the bit stream information of incoming bit stream 108 or one of them perform decoding
Complexity manages, and wherein, bit stream information can include bit rate, resolution and/or frame type.Bit stream is believed
Breath can include in some other example multiple motion vector, multiple residual error (residual), multiple IDCT coefficient,
Bitstream length or obtainable bit stream information, wherein, multiple residual errors can such as DC current (Direct
Current, DC) value, obtainable bit stream information can based on such as comparing and/or historical statistics, by above-mentioned
Know that the information of bit stream is obtained.In one embodiment, decoding complex degree manager 100M is according to multiple fortune
The change of moving vector adjusts decoding complex degree.The parameter representing disposal ability can include the decoding needed for each frame
The statistics of time, such as, decoding complex degree manager 100M can use the average decoding time of each frame to sentence
Disconnected current system ability, thus accommodation decoding complex degree.Especially, decoding complex degree manager 100M
Can display buffer state based on aforesaid display buffer 180, aforesaid at least one parameter and input
Whole execution decoding complex degree management of the bit stream information of bit stream 108.
Fig. 3 is the certain operations that the decoding complex degree manager 100M according to one embodiment of the invention can trigger
Schematic diagram, wherein, these operations are multiple about the decoding of adaptive video coding/decoding method 910 as shown in Figure 2
Miscellaneous degree management.Such as, the operation that decoding complex degree manager 100M can trigger includes: the adjustment of display time,
Abandon frame, deblocking filter complexity adjusts, motion compensation complexity adjusts and (is labeled as " MC in figure 3
Complexity adjust "), infra-frame prediction complexity adjust, inverse transformation complexity adjust and VLD complexity adjust.
Display buffer state based on aforesaid display buffer 180, the parameter representing disposal ability and input
The combination of the bit stream information of bit stream 108 or one of them, decoding complex degree manager 100M performs decoding
Complexity manage, and optionally trigger these operation in one or more.
Fig. 4 is the certain operations that the decoding complex degree manager 100M according to one embodiment of the invention can trigger
The correlated quality of/step and the schematic diagram of processing speed, wherein, adaptive about as shown in Figure 2 of these operations
The decoding complex degree answering video encoding/decoding method 910 manages.Such as, these operation/steps comprise the steps that canonical solution
Code stream journey, self adaptation bidirectional frame (Bi-directional frame, B frame) (hereinafter referred to as " B frame ") deblock
(i.e. all or part of B frame of self adaptation execution deblocks or skips B frame and deblocks, and wherein B frame deblocks finger to multiple
B frame performs deblocking operation), B frame simplify motion compensation (being labeled as in the diagram " B frame simplify MC "),
Fast B frame decoding (such as in down-sampling territory decode B frame), abandon B frame, adaptive prediction frame (Predictive
Frame, P frame) (hereinafter referred to as " P frame ") deblocks, and (such as, self adaptation performs all or part of P frame
Deblock or skip P frame and deblock, wherein P frame deblock finger to multiple P frames perform deblocking operation), P frame simplify fortune
Move and compensate (being labeled as " P frame simplifies MC " in the diagram), quick P frame decoding, abandon P frame, self adaptation
In frame (Intra frame, I frame) (hereinafter referred to as " I frame ") deblock (such as, self adaptation perform all or
Part I frame deblocks or skips I frame and deblocks, and wherein I frame deblocks finger to multiple I frames execution deblocking operation) and
Quickly I frame decoding.It is used that institute's illustrated example is only explanation, and the present invention is not so limited." matter on vertical axis
Amount " labelling can represent picture quality, fluency, user perceptibility, system resource saving degree or various quality
The overall merit of index.According to some deformation of this embodiment, corresponding to quality and the place of these operation/steps
Reason speed in varied situations can be different.Such as, in the operation/step shown in Fig. 4, B frame is abandoned
Operation/step that operation/step may be inserted into the operation/step of standard decoding process and self adaptation B frame deblocks
Between.The most such as, in the operation/step shown in Fig. 4, operation/step that self adaptation I frame deblocks can be inserted into
Simplify between the operation/step of motion compensation and the operation/step of quick P frame decoding at P frame.The most such as, exist
In operation/step shown in Fig. 4, abandon P frame operation/step can the operation/step of neighbouring quick I frame decoding,
Therefore, an operation/step of the bottom in the operation/step shown in Fig. 4 is become.
In this embodiment, based on first strategy-realize available best in quality, when video decoding delay problem
When being not the very crux, decoding complex degree manager 100M can the operation/step of trigger criteria decoding process.
But, (such as find hardware resource deficiency and/or decoding effort when determining the delay that video can be occurred to decode
Amount is more than predetermined threshold value), decoding complex degree manager 100M can trigger one or more operation/step, with
Prevent serious video decoding delay, and realize the available best in quality of decoder under present case.Such as,
In the operation/step shown in Fig. 4, decoding complex degree manager 100M can trigger next suitable operation/
Step (operation/step that such as self adaptation B frame deblocks, B frame simplifies the operation/step etc. of motion compensation),
With release hardware resource and/or minimizing decoding effort amount, wherein, when reducing decoding complex degree, corresponding to relatively
Operation/the step of high-quality rank has a higher execution priority, and corresponding to lower quality rank operation/
Step can perform last.It is used that institute's illustrated example is only explanation, and the present invention is not so limited.According to this
The deformation of embodiment, decoding complex degree manager 100M can based on second strategy-realize predetermined quality scale
Carry out operating rather than operating based on realizing best in quality.According to some other deformation of this embodiment,
Decoding complex degree manager 100M can be based on realizing the fastest processing speed or some plans of predetermined processing speed
Slightly operate.
Fig. 5 is some behaviour that the decoding complex degree manager 100M according to another embodiment of the present invention can trigger
The correlated quality of work/step and the schematic diagram of processing speed, wherein, these operation about as shown in Figure 2 from
The decoding complex degree management of adaptive video coding/decoding method 910.Specifically, the operation/step considered in this embodiment
Suddenly include: normally deblock, simply deblock and disable and deblock.Note that decoding complex degree manager 100M
Control mechanism can exist according to multiple predetermined threshold values (such as, two shown in Fig. 5 threshold value T1 and T2)
Switch between these operation/steps, wherein, predetermined threshold value T1 and the appropriate level of T2 representation quality.
Fig. 6 is that the decoding complex degree manager 100M according to another embodiment of the present invention in varied situations may be used
The schematic diagram of the certain operations/step triggered, these operations are about adaptive video decoding side as shown in Figure 2
The decoding complex degree management of method 910.Specifically, the operation/step considered in this embodiment includes: stop B
Frame deblocks, stop P frame deblocks, start self adaptation P frame deblocks, start self adaptation I frame deblocks and stops
Deblock.Note that the control mechanism of decoding complex degree manager 100M can be according to multiple predetermined threshold value (examples
As, three shown in Fig. 6 threshold value TI, TP and TB) switch between these operation/steps, wherein,
Predetermined threshold value TI, TP and TB represent the appropriate level of display buffer state.In this embodiment, in advance
Fixed threshold value TI, TP and TB are respectively used to decide whether that startup/B frame deblocks, P frame deblocks and I in stopping
Frame deblocks.Especially, in the case of display buffer 180 is full or the fullest, decoding complex degree pipe
Reason device 100M do not stop I frame deblock, P frame deblock and B frame deblocks any one.On the contrary, aobvious
Show buffer 180 for empty or almost in the case of sky, decoding complex degree manager 100M stop I frame deblock,
Each that P frame deblocks and B frame deblocks.
More specifically, at the buffer state of display buffer 180 (more specifically, display buffer 180
Buffering rank) between predetermined threshold value TP and TB in the case of, decoding complex degree manager 100M
Stop B frame to deblock, and trigger and start operation/step of deblocking of self adaptation P frame and (i.e. there is complicacy self-adaptive
P frame deblock).It addition, at the buffer state of display buffer 180 (more specifically, display buffer
The buffering rank of 180) between predetermined threshold value TI and TP in the case of, decoding complex degree manager 100M
Stop P frame to deblock and B frame deblocks, and trigger the operation/step starting self adaptation I frame and deblocking and (i.e. have multiple
Miscellaneous degree adaptive I frame deblocks).It addition, display buffer 180 buffer state (more specifically,
The buffering rank of display buffer 180) between predetermined threshold value TI and 0 in the case of, decoding complex degree
I frame deblocks, P frame deblocks and B frame deblocks in manager 100M stopping.
In this embodiment, the control mechanism deblocked about complicacy self-adaptive, decoding complex degree manager
100M can operate based on the solution based on macro block (Macroblock, MB).Such as, decoding is multiple
Miscellaneous degree manager 100M can be according to buffer state and at least some of (such as one of incoming bit stream 108
Point or all) bitstream length, self-adapting start/stopping I frame deblocks, P frame deblocks and/or B frame deblocks.
It is used that institute's illustrated example is only explanation, and the present invention is not so limited.According to the deformation of this embodiment, about
The control mechanism that complicacy self-adaptive deblocks, decoding complex degree manager 100M can be based on deblocking based on frame
Scheme operates.Such as, decoding complex degree manager 100M can be according to buffer state, incoming bit stream
The bitstream length of at least some of (such as part or all) of 108, quantization step, MB type and
/ or edge strength start/stop that I frame deblocks, P frame deblocks and/or B frame deblocks adaptively, wherein, amount
Change the most so-called quantization parameter of step-length (Quantization Parameter, QP) value.
It is an advantage of the present invention that each embodiment/deformation of the disclosure above adaptive video coding/decoding method,
Relevant complicacy self-adaptive Video Decoder and corresponding adaptive audio/audio/video player system are applicable to respectively
Plant digital video application, wherein, at least some of (example in multiple elements in complicacy self-adaptive Video Decoder
Such as part or all) can operate with low-level video decoding complexity.Due to adjustable one or
The video decoding complexity of multiple elements, decoding effort amount and corresponding power consumption can reduce adaptively.Therefore,
Compared with relevant technology, the present invention is to realize the great flexibility that Digital Video Broadcasting System provides.
The above embodiments only are used for enumerating embodiments of the present invention, and the technical characteristic of the explaination present invention,
Not it is used for limiting scope of the invention.Any person of ordinary skill in the field is according to the spirit of the present invention
And unlabored change or isotropism arrangement belong to the scope that the present invention is advocated, the right model of the present invention
Enclose and should be as the criterion with claim.
Claims (25)
1. an adaptive video coding/decoding method, described adaptive video coding/decoding method includes:
The bit stream information of incoming bit stream based on complicacy self-adaptive Video Decoder, performs decoding complex degree management, to decide whether to reduce the decoding complex degree of multiple elements in described complicacy self-adaptive Video Decoder；And
The decoding complex degree of the plurality of element in the selectivity described complicacy self-adaptive Video Decoder of reduction；
Wherein, the plurality of element includes: quickly variable length decoding and inverse quantization module, for described incoming bit stream is performed quick variable length decoding and inverse quantization, to produce multiple inverse quantization result；Motion compensating module, for performing motion compensation according to described incoming bit stream and producing corresponding multiple prediction output result, wherein said motion compensating module comprises time prediction unit and spatial prediction unit, and described time prediction unit exports a part for result for running time prediction to produce the plurality of prediction；And described spatial prediction unit is for performing spatial prediction to produce a part for the plurality of prediction output result；Computing unit, is used for adding up to multiple inverse transformation result and the plurality of prediction output result to export result to produce multiple compensation；Reconstruction frames output unit, for producing multiple reconstruction frames according to the plurality of output result that compensates, wherein, described spatial prediction unit performs spatial prediction according to the current reconstructed frame of the plurality of reconstruction frames；And frame memory element, the at least some of of the plurality of reconstruction frames is stored for interim, wherein, described time prediction unit is according to described at least some of running time prediction of the plurality of reconstruction frames, and described time prediction unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity of running time prediction.
2. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that the plurality of element includes:
Inverse transformation unit, for performing inverse transformation to the plurality of inverse quantization result, to produce multiple inverse transformation result.
3. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that the plurality of element includes:
Display buffer, stores multiple picture frames for interim；
Wherein, the step of described execution decoding complex degree management includes:
Display buffer state based on described display buffer, performs decoding complex degree management.
4. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that the step of described execution decoding complex degree management includes:
Produce at least one control signal, to control the decoding complex degree of the plurality of element in described complicacy self-adaptive Video Decoder.
5. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that the plurality of element includes:
The display time adjusts driver, wherein, adjusting under the control of driver in the described display time, multiple picture frames that output is temporarily stored in display buffer are as multiple output frames, and the plurality of output frame is for showing respectively on multiple time points that the audio frequency of respective audio information is play.
6. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that described spatial prediction unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity performing spatial prediction；And described spatial prediction unit estimates to can be used in the partial information having been omitted from performing at least some of of spatial prediction further.
7. adaptive video coding/decoding method as claimed in claim 2, it is characterised in that described inverse transformation unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity performing described inverse transformation；And described inverse transformation unit omits the part of multiple calculating for described inverse transformation.
8. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that described reconstruction frames output unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity producing the plurality of reconstruction frames.
9. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that described quick variable length decoding operates according to the resolution of the plurality of reconstruction frames with inverse quantization module, to reduce the complexity performing quick variable length decoding with inverse quantization；And described quick variable length decoding uses during decoding with inverse quantization module and includes master meter and the look-up table of at least one sublist, and the use probability of described master meter is more than the use probability of described at least one sublist.
10. adaptive video coding/decoding method as claimed in claim 1, it is characterised in that the step of described execution decoding complex degree management includes:
At least one parameter based on the disposal ability representing described complicacy self-adaptive Video Decoder, performs decoding complex degree management.
11. adaptive video coding/decoding methods as claimed in claim 1, it is characterised in that the described bit stream information of described incoming bit stream include the combination of bit rate, resolution and frame type or one of them.
12. adaptive video coding/decoding methods as claimed in claim 1, it is characterised in that the described bit stream information of described incoming bit stream include multiple motion vector, multiple residual error and the combination of bitstream length or one of them.
13. 1 kinds of complicacy self-adaptive Video Decoders, including:
Multiple elements, for performing the multiple operations relevant to video decoding；And
Decoding complex degree manager, performs decoding complex degree management for bit stream information based on incoming bit stream, to decide whether to reduce the decoding complex degree of multiple elements in described complicacy self-adaptive Video Decoder；
Wherein, the decoding complex degree of the plurality of element in described decoding complex degree manager selectivity reduces described complicacy self-adaptive Video Decoder；
Wherein, the plurality of element includes: quickly variable length decoding and inverse quantization module, for described incoming bit stream is performed quick variable length decoding and inverse quantization, to produce multiple inverse quantization result；Motion compensating module, for performing motion compensation according to described incoming bit stream and producing corresponding multiple prediction output result, wherein said motion compensating module comprises time prediction unit and spatial prediction unit, and described time prediction unit exports a part for result for running time prediction to produce the plurality of prediction；And described spatial prediction unit is for performing spatial prediction to produce a part for the plurality of prediction output result；Computing unit, is used for adding up to multiple inverse transformation result and the plurality of prediction output result to export result to produce multiple compensation；Reconstruction frames output unit, for producing multiple reconstruction frames according to the plurality of output result that compensates, wherein, described spatial prediction unit performs spatial prediction according to the current reconstructed frame of the plurality of reconstruction frames；And frame memory element, the at least some of of the plurality of reconstruction frames is stored for interim, wherein, described time prediction unit is according to described at least some of running time prediction of the plurality of reconstruction frames, and described time prediction unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity of running time prediction.
14. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that the plurality of element includes:
Inverse transformation unit, for performing inverse transformation to produce multiple inverse transformation results to the plurality of inverse quantization result.
15. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that the plurality of element includes: display buffer, store multiple picture frames for interim；
Wherein, described decoding complex degree manager display buffer based on described display buffer state, perform decoding complex degree management.
16. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterized in that, described decoding complex degree manager produces at least one control signal according to the described bit stream information of described incoming bit stream, to control the decoding complex degree of the plurality of element in described complicacy self-adaptive Video Decoder, the plurality of element be controlled by described at least one control signal.
17. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that the plurality of element includes:
The display time adjusts driver, wherein, adjusting under the control of driver in the described display time, multiple picture frames that output is temporarily stored in display buffer are as multiple output frames, and the plurality of output frame is for showing respectively on multiple time points that the audio frequency of respective audio information is play.
18. complicacy self-adaptive Video Decoders as claimed in claim 14, it is characterised in that described spatial prediction unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity performing spatial prediction；And described spatial prediction unit estimates to can be used in the partial information having been omitted from performing at least some of of spatial prediction further.
19. complicacy self-adaptive Video Decoders as claimed in claim 14, it is characterised in that described inverse transformation unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity performing described inverse transformation；And described inverse transformation unit omits the part of multiple calculating for described inverse transformation.
20. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that described reconstruction frames output unit operates according to the resolution of the plurality of reconstruction frames, to reduce the complexity producing the plurality of reconstruction frames.
21. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that described quick variable length decoding operates according to the resolution of the plurality of reconstruction frames with inverse quantization module, to reduce the complexity performing quick variable length decoding with inverse quantization；And described quick variable length decoding uses during decoding with inverse quantization module and includes master meter and the look-up table of at least one sublist, and the use probability of described master meter is more than the use probability of described at least one sublist.
22. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that described decoding complex degree manager at least one parameter based on the disposal ability representing described complicacy self-adaptive Video Decoder, perform decoding complex degree management.
23. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that the described bit stream information of described incoming bit stream include the combination of bit rate, resolution and frame type or one of them.
24. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that the described bit stream information of described incoming bit stream include multiple motion vector, multiple residual error and the combination of bitstream length or one of them.
25. complicacy self-adaptive Video Decoders as claimed in claim 13, it is characterised in that described decoding complex degree manager postpones the audio frequency of audio-frequency information and plays.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US12/830,419 US9172980B2 (en) | 2010-03-25 | 2010-07-05 | Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system |
| US12/830,419 | 2010-07-05 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102316319A CN102316319A (en) | 2012-01-11 |
| CN102316319B true CN102316319B (en) | 2016-08-10 |
Family
ID=45429092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110122127.7A Expired - Fee Related CN102316319B (en) | 2010-07-05 | 2011-05-12 | Adaptive video coding/decoding method, decoder and audio/video playback system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN102316319B (en) |
| TW (1) | TWI543627B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104093072B (en) | 2014-06-30 | 2017-06-16 | 京东方科技集团股份有限公司 | A kind of video information Play System and method |
| CN106550269A (en) * | 2015-09-17 | 2017-03-29 | 联发科技股份有限公司 | Adaptive video coding/decoding method, decoder and audio/video playback system |
| US10574988B2 (en) * | 2015-11-19 | 2020-02-25 | Qualcomm Incorporated | System and methods for reducing slice boundary visual artifacts in display stream compression (DSC) |
| US10244248B2 (en) * | 2016-02-25 | 2019-03-26 | Mediatek Inc. | Residual processing circuit using single-path pipeline or multi-path pipeline and associated residual processing method |
| CN105847821A (en) * | 2016-04-01 | 2016-08-10 | 乐视控股（北京）有限公司 | Video decoding method and device |
| SG11202013015PA (en) * | 2018-07-02 | 2021-01-28 | Huawei Tech Co Ltd | Apparatus and method for filtering in video coding |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1902939A (en) * | 2004-01-05 | 2007-01-24 | 汤姆森许可贸易公司 | Encoding method, decoding method, and encoding apparatus for a digital picture sequence |
| CN101277448A (en) * | 2008-04-25 | 2008-10-01 | 清华大学 | Method for encoding video with lower limitation of decoding complex degree |
| US20090003447A1 (en) * | 2007-06-30 | 2009-01-01 | Microsoft Corporation | Innovations in video decoder implementations |
| CN101729883A (en) * | 2008-10-14 | 2010-06-09 | 上海炜寅信息科技有限公司 | Method for adaptively adjusting video decoding complexity |
-
2011
- 2011-05-10 TW TW100116326A patent/TWI543627B/en not_active IP Right Cessation
- 2011-05-12 CN CN201110122127.7A patent/CN102316319B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1902939A (en) * | 2004-01-05 | 2007-01-24 | 汤姆森许可贸易公司 | Encoding method, decoding method, and encoding apparatus for a digital picture sequence |
| US20090003447A1 (en) * | 2007-06-30 | 2009-01-01 | Microsoft Corporation | Innovations in video decoder implementations |
| CN101277448A (en) * | 2008-04-25 | 2008-10-01 | 清华大学 | Method for encoding video with lower limitation of decoding complex degree |
| CN101729883A (en) * | 2008-10-14 | 2010-06-09 | 上海炜寅信息科技有限公司 | Method for adaptively adjusting video decoding complexity |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201204049A (en) | 2012-01-16 |
| CN102316319A (en) | 2012-01-11 |
| TWI543627B (en) | 2016-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102316319B (en) | Adaptive video coding/decoding method, decoder and audio/video playback system | |
| KR100994979B1 (en) | Parallel decoding of intra-encoded video | |
| JP6120390B2 (en) | Predicted characteristics compensated for next-generation video content | |
| JP5831587B2 (en) | Image processing apparatus and method | |
| US9930361B2 (en) | Apparatus for dynamically adjusting video decoding complexity, and associated method | |
| JP2008506294A (en) | Method and system for performing deblocking filtering | |
| EP3748967A1 (en) | Methods and apparatus for collaborative partition coding for region based filters | |
| CN104221389A (en) | Image-processing device and method | |
| CN103155564A (en) | Image processing device and method | |
| CN108810552A (en) | Image prediction method and related product | |
| EP2651135A1 (en) | Image processing device, image processing method, and program | |
| US20230239464A1 (en) | Video processing method with partial picture replacement | |
| US9172980B2 (en) | Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system | |
| US20080170628A1 (en) | Video decoding method and apparatus having image scale-down function | |
| US20220150515A1 (en) | Adaptive temporal filter for an unavailable reference picture | |
| US6996185B2 (en) | Image signal decoding apparatus | |
| CN106550269A (en) | Adaptive video coding/decoding method, decoder and audio/video playback system | |
| US20060291743A1 (en) | Configurable motion compensation unit | |
| US20110280321A1 (en) | Deblocking filter and method for controlling the deblocking filter thereof | |
| US10063873B2 (en) | Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system | |
| CN102986225A (en) | Information processing device and information processing method | |
| WO2021248135A1 (en) | Methods and apparatuses for video coding using satd based cost calculation | |
| US20160044325A1 (en) | Image coding apparatus, image coding method and imaging apparatus | |
| WO2020159993A1 (en) | Explicit signaling of extended long term reference picture retention | |
| KR100821922B1 (en) | Local memory controller for mpeg decoder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20160810 |