WO2006109141A9 - Procede et systeme de codage video evolutif a granularite fine et compensation de mouvement, avec commande de derive - Google Patents

Procede et systeme de codage video evolutif a granularite fine et compensation de mouvement, avec commande de derive

Info

Publication number
WO2006109141A9
WO2006109141A9 PCT/IB2006/000849 IB2006000849W WO2006109141A9 WO 2006109141 A9 WO2006109141 A9 WO 2006109141A9 IB 2006000849 W IB2006000849 W IB 2006000849W WO 2006109141 A9 WO2006109141 A9 WO 2006109141A9
Authority
WO
WIPO (PCT)
Prior art keywords
reference block
base layer
block
layer
transform coefficients
Prior art date
Application number
PCT/IB2006/000849
Other languages
English (en)
Other versions
WO2006109141A1 (fr
Inventor
Yiliang Bao
Marta Karczewicz
Justin Ridge
Xianglin Wang
Original Assignee
Nokia Corp
Nokia Inc
Yiliang Bao
Marta Karczewicz
Justin Ridge
Xianglin Wang
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Corp, Nokia Inc, Yiliang Bao, Marta Karczewicz, Justin Ridge, Xianglin Wang filed Critical Nokia Corp
Priority to EP06744488A priority Critical patent/EP1878257A1/fr
Publication of WO2006109141A1 publication Critical patent/WO2006109141A1/fr
Publication of WO2006109141A9 publication Critical patent/WO2006109141A9/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/29—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving scalability at the object level, e.g. video object layer [VOL]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/48—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • This invention relates to the field of video coding, and more specifically to scalable video coding.
  • temporal redundancy existing among video frames can be minimized by predicting a video frame based on other video frames. These other frames are called the reference frames.
  • Temporal prediction can be carried out in different ways: The decoder uses the same reference frames as those used by the encoder. This is the most common method in conventional non-scalable video coding. In normal operations, there should not be any mismatch between the reference frames used by the encoder and those by the decoder.
  • the encoder uses the reference frames that are not available to the decoder.
  • One example is that the encoder uses the original frames instead of reconstructed frames as reference frames.
  • the decoder uses the reference frames that are only partially reconstructed compared to the frames used in the encoder.
  • a frame is partially reconstructed if either the bitstream of the same frame is not fully decoded or its own reference frames are partially reconstructed.
  • mismatch is likely to exist between the reference frames used by the encoder and those by the decoder. If the mismatch accumulates at the decoder side, the quality of reconstructed video suffers.
  • drift Mismatch in the temporal prediction between the encoder and the decoder is called a drift.
  • Many video coding systems are designed to be drift-free because the accumulated errors could result in artifacts in the reconstructed video.
  • a signal-to-noise ratio (SNR) scalable video stream has the property that the video of a lower quality level can be reconstructed from a partial bitstream.
  • Fine granularity scalability (FGS) is one type of SNR scalability in which the scalable stream can be arbitrarily truncated.
  • Figure 1 illustrates how a stream of FGS property is generated in MPEG-4. Firstly a base layer is coded in a non-scalable bitstream. FGS layer is then coded on top of that. MPEG-4 FGS does not exploit any temporal correlation within the FGS layer. As shown in Figure 2, when no temporal prediction is used in FGS layer coding, the FGS layer is predicted from the base layer reconstructed frame. This approach has the maximal bitstream flexibility since truncation of the FGS stream of one frame will not affect the decoding of other frames, but the coding performance is not competitive.
  • Leaky prediction is a technique that has been used to seek a balance between coding performance and drift control in SNR enhancement layer coding (see, for example, Huang et al. "A robust fine granularity scalability using trellis-based predictive leak", . IEEE Transaction on Circuits and Systems for Video Technology", pp.372-385, vol.12, Issue 6, June 2002).
  • the actual reference frame is formed with a linear combination of the base layer reconstructed frame ⁇ x b n ) and the enhancement layer reference frame ( r" ⁇ ).
  • the leaky prediction method will limit the propagation of the error caused by the mismatch between the reference frame used by the encoder ( r" ⁇ l ' E ) and that used by the decoder ( r" ⁇ hD ), since the error ( ⁇ n ⁇ x ) will be attenuated every time a new reference signal is formed.
  • r ⁇ " >X) , r ⁇ ' E are the actual reference frames used in FGS layer coding in the decoder and the encoder, respectively, and a is a value given by 0 ⁇ a ⁇ 1 in order to achieve attenuation of error signal.
  • the third technique is to classify the DCT coefficients in a block to be encoded in the enhancement layer according to the value of the corresponding quantized coefficients in the base layer (see Comer "Conditional replacement for improved coding efficiency in fine-grain scalable video coding", International Conference on Image Processing, vol.2, pp. 57-60, 2002). The decision as to whether the base or enhancement layer is used for prediction is made per each coefficient.
  • a quantized coefficient in the base layer is zero, the corresponding DCT coefficient in the enhancement layer will be predicted using the DCT coefficient calculated from the enhancement layer reference frame. If this quantized coefficient in the base layer is nonzero, the corresponding DCT coefficient in the enhancement layer will be predicted using the DCT coefficient calculated from the reference block from the base layer reconstructed frame.
  • JSVM 1.0 Joint Scalable Video Model 1.0
  • JSVMl.0 FGS coder is not designed to manage the drift.
  • the FGS layer of an anchor frame which is at the boundary of a GOP (group of picture), is coded in a way similar to MPEG-4 FGS where the temporal redundancy is not exploited.
  • the length of GOP could be as short as 1 frame.
  • the JSVMl .0 FGS coder is very inefficient because every frame is coded as an anchor frame.
  • Figure 4 gives the prediction paths in a 3-layer scalable video stream.
  • An FGS layer is inserted in between 2 discrete layers.
  • the upper discrete enhancement layer can be a spatial enhancement layer or a coarse SNR scalability layer.
  • This upper enhancement layer is usually coded based on the FGS layer using either the base layer texture prediction mode or the residual prediction mode.
  • the base layer texture prediction mode a block in the reconstructed FGS layer is used as the reference for coding a block in the upper discrete enhancement layer.
  • the residual prediction mode the residual reconstructed from both the base layer and FGS layer is used as the prediction for coding the prediction residual in the enhancement layer.
  • the decoding of upper enhancement layer can still be performed even if the FGS layer in the middle is only partially reconstructed. However, the upper enhancement layer has a drift problem because of the partial decoding of the FGS layer.
  • the present invention provides a fine granularity SNR scalable video codec that exploits the temporal redundancy in the FGS layer in order to improve the coding performance while the drift is controlled. More specifically the present invention focuses on how the reference blocks used in predictive coding in FGS layer should be formed, and the signaling and mechanism that are needed to control the process.
  • the present invention improves the efficiency of FGS coding, especially under low-delay constraints.
  • the present invention is effective in controlling the drift, and thus a fine granularity scalability (FGS) coder of better performance can be designed accordingly.
  • an adaptively formed reference block is used.
  • the reference block is formed from a reference block in base layer reconstructed frame and a reference block in the enhancement layer reference frame together with a base layer reconstructed prediction residual block.
  • the reference block for coding is adjusted depending on the coefficients coded in the base layer.
  • the actual reference signal used for coding is a weighted average of a reference signal from the reconstructed frame in the base layer and a reference signal from the enhancement layer reference frame together with a base layer reconstruction prediction residual.
  • the first aspect of the present invention provides a method for motion compensated scalable video coding, wherein the method comprises forming the reference block and adjusting the reference block. The method further comprises choosing a weighting factor so that the reference block is formed as weighted average of the base layer reference block and the enhancement layer reference block.
  • the second aspect of the present invention provides a software application product having a storage medium to store program codes to carry out the method of the present invention.
  • the third aspect of the present invention provides an electronic module for use in motion compensated video coding.
  • the electronic module comprises a formation block for forming the reference block and an adjustment block for adjust the reference block according to the method of the present invention.
  • the fourth aspect of the present invention provides an electronic device, such as a mobile terminal, having one or both of a decoding module and an encoding module having a module for motion compensated video coding.
  • the electronic module comprises a formation block for forming the reference block and an adjustment block for adjust the reference block according to the method of the present invention.
  • Figure 1 illustrates fine granularity scalability with no temporal prediction in the FGS layer (MPEG-4).
  • Figure 2 illustrates reference blocks being used in coding the base layer and FGS layer, when no temporal prediction is used in the FGS layer coding.
  • Figure 3 illustrates fine granularity scalability with temporal prediction.
  • Figure 4 show the use of FGS information in predicting the upper enhancement layer.
  • Figure 5 illustrates generation of reference block with FGS layer temporal prediction and drift control, according to the present invention.
  • Figure 6 illustrates base-layer dependent adaptive reference block formation, according to the present invention.
  • Figure 7 illustrates reference block formation by performing interpolation on differential reference frame, according to the present invention.
  • Figure 8 illustrates base-layer dependent differential reference block adjustment, according to the present invention.
  • FIG. 9 illustrates an FGS encoder with base-layer-dependent formation of reference block, according to the present invention.
  • Figure 10 illustrates an FGS decoder with base-layer-dependent formation of reference block, according to the present invention.
  • Figure 11 illustrates an electronic device having at least one of the scalable encoder and the scalable decoder, according to the present invention.
  • a reference block R" is used to code a block X" of size M x N pixels in the FGS layer.
  • R a " is formed adaptively from a reference block X ⁇ from the base layer reconstructed frame and a reference block R" '1 from the enhancement layer reference frame based on the coefficients coded in the base layer, Ql .
  • Figure 5 gives the relationship among these blocks.
  • a block is a rectangular area in the frame.
  • the size of a block in the spatial domain is the same as the size of the corresponding block in the coefficient domain.
  • the same original frame is coded in the enhancement layer and the base layer, but at different quality levels.
  • the base layer collocated block refers to the block coded in the base layer that corresponds to the same original block that is being processed in the enhancement layer.
  • Ql is a block of quantized coefficients coded in the base layer corresponding to the same original block being coded in the enhancement layer.
  • Ql (u,v) the information whether individual coefficients, Ql (u,v) , are zero or not is important.
  • ⁇ is a weighting factor
  • AU weighting factors are always in the range [0, 1].
  • the base-layer dependent adaptive reference block formation is illustrated in Figure 6.
  • weighting factor CC is set to 0
  • weighting factor ⁇ is set to 1.
  • the base layer reconstructed block will be selected as the actual reference block if the block being coded in the FGS layer has some nonzero coefficients in the base layer, or the enhancement layer reference block will be selected as the actual reference block if the block being coded does not have any non-zero coefficients in the base layer.
  • This is a simple design. Decision on whether the data of a reference block should be from the base layer reconstructed frame or the enhancement layer reference frame is only made at the block level and no additional transform or weighted averaging operations are needed.
  • weighting factor OC is not restricted, and the value of weighting factor/? depends on the frequency of the coefficient being considered.
  • weighting factor OC is not restricted, the value of weighting factor/? depends on the FGS coding cycle in which the current coefficient is coded.
  • the lower case variables such as x b n and r ⁇ " , are used for general discussion.
  • the upper case variables, such as X b and R ⁇ are used for representing the signals in the spatial domain, while their corresponding transform coefficients are denoted as F and F , etc.
  • the present invention provides a number of algorithms for generating the optimal reference signals to be used in FGS layer coding. With these algorithms, the temporal prediction is efficiently incorporated in FGS layer coding to improve the coding performance while the drift is effectively controlled.
  • the actual reference signal is a weighted average of the reference signal from the reconstructed frame in the base layer and that from the enhancement layer reference frame:
  • the base layer reconstructed signal x b n itself is calculated from the base layer reference signal r ⁇ ⁇ x , and the base layer reconstructed prediction residual p b n :
  • this relationship is generalized by introducing an independent scaling factor a p for the base layer reconstructed prediction residual, pi :
  • the independent scaling factor a p has a value between 0 and 1. When the scaling factor is equal to 1, the base layer reconstructed prediction residual is not scaled.
  • a coefficient block F n (u,v), 0 ⁇ u ⁇ M, 0 ⁇ v ⁇ N is formed with each coefficient being a weighted average of coefficients from the base layer reference frame and that from the enhancement layer reference frame. The weighting factor depends on the base layer coefficient value.
  • Equations 9, 10 and 11 can be reorganized as follows:
  • F ⁇ (u,v) F R ⁇ u,v) + r p -F pS (u,v) + (l- ⁇ ).F Rr (u,v) ifQ b n (u,v) ⁇ 0 (14)
  • the differences between the transform coefficients can be calculated by performing the transform on the differential reference block.
  • R d n ⁇ l can be generated by performing the motion compensation on the differential reference frame which is calculated by subtracting the base layer reference frame from the enhancement layer reference frame.
  • Reference block formation by performing interpolation on differential reference frame is shown in Figure 7, and the base-layer dependence differential reference block adjustment is illustrated in Figure 8.
  • One example of the interpolation filter is the filter for bilinear interpolation.
  • the base layer reconstruction ( x b '' ) is the same as the base layer reference signal ( r b ⁇ ).
  • the application may choose the following equations instead of Equations 12, 13 and 14 to simplify the implementation.
  • Equations 20, 21 and 22 can be used even if additional operations, such as loop filtering, are performed on the base layer reconstruction, although X b is now not always equal to R n ⁇ x + P b because of the additional operations on "R b n ⁇ x +P b " ".
  • One classification technique is to classify a block depending on whether it has any neighboring blocks that have non-zero base layer coefficients.
  • One way of performing such a classification is to use the coding context index for coding the coded block flag in the base layer as defined in H.264.
  • the coding context index is 0 if the coded block flags of both the left neighboring block and the top neighboring block are zero.
  • the coding context index is 1 if only coded block flag of the left neighboring block is nonzero.
  • the coding context index is 2 if only coded block flag of the top neighboring block is nonzero. Otherwise, the coding text index is 3.
  • Another classification technique is to use explicit signaling to indicate whether the reference block is only from the base layer reconstructed frame or from the weighted average between the base layer reconstructed frame and the enhancement reference frame in a way as described in this invention, or from the enhancement layer.
  • the signaling can be performed at Macroblock (MB) level, and only for those MBs that do not have any nonzero coefficients in the base layer.
  • MB Macroblock
  • the transform operations are needed because different weighting factors are used for different coefficients within a block if the block in the base layer has any nonzero coefficients. If the same weighting factor is used for all the coefficients within a block, the transform operations are not necessary.
  • the number of nonzero coefficients is counted in the base layer block. If the number of nonzero coefficients is larger than or equal to a pre-determined number Tc, all the coefficients in this block use a single weighting factor.
  • the value of weighting factor may depend on the number of nonzero coefficients in the base layer.
  • This threshold number Tc determines whether the entire block should use the same weighting factor.
  • Tc is in the range between 1 and BlockSize. For example, for a 4x4 block, there are 16 coefficients in a block, and Tc is in the range between 1 and 16.
  • Tc 1, i.e., all the coefficients in a block always use the same weighting factor. In this case, no additional transform is needed.
  • the value of weighting factor may depend on the number of nonzero coefficients in a block in the base layer.
  • the weighting factors can change from frame to frame or slice to slice, or they can be fixed for certain amount of frames or slices.
  • the weighting factor ⁇ may depend on the number of nonzero coefficients in the base layer.
  • ⁇ is a constant for the slice, ⁇ is equal to ⁇ ⁇ , ⁇ x ⁇ ⁇ , when there is only one nonzero coefficient in the base layer.
  • ⁇ is equal to ⁇ .
  • the user may choose to use the discrete base layer as the "base layer” and the top-most FGS layer as the "enhancement layer” to implement the algorithms mentioned above. This is referred to as a two-loop structure.
  • the user may also use a multi-loop coding structure as follows:
  • the first coding loop is the normal discrete base layer coding loop.
  • the second coding loop is for coding the first FGS layer using the algorithms described in this disclosure.
  • the "base layer” is the discrete base layer and the
  • “enhancement layer” is the first FGS layer”.
  • the "base layer” is the first FGS layer and the “enhancement layer” is the second FGS layer, and so on.
  • the “base layer” is an FGS layer
  • the “base layer” coefficients considered are those in this FGS layer as well as other layers below this layer.
  • Q b " (u, v) is considered nonzero if the coefficient at the same position in any of these layers is nonzero.
  • Base layer still uses the subscript "b", but the first FGS enhancement layer, which is the layer immediately on top of the base layer, uses the subscript "ei”, and the second FGS enhancement layer uses the subscript "C 2 " and so on.
  • R j '1 ' is the adjusted differential reference signal calculated for coding the first FGS enhancement layer. Equation (23) is equivalent to equation (16) except for the changes of subscripts.
  • the actual reference signal can be calculated as in equation (24).
  • R j '1 ' is calculated from the differential reference frame which is the difference between the reference frame of the second FGS enhancement layer and the reference frame of the first enhancement layer. It can be seen that except there is one more reconstructed residual term, the equation is not much different from (23).
  • K 2 K 1 + R V y n Pb ⁇ P b " + r, n ⁇ K (24)
  • the first method is to perform motion compensation on the reference frame of the first FGS enhancement layer.
  • one of two other methods for calculating the R" '1 can be used.
  • R" '1 is set to be equal to R ⁇ 1 + R ⁇ '1 .
  • R" ⁇ l is set to be equal to R b " '1 + R j '1 ' .
  • the choice of the two-loop or multi-loop FGS coding can be an encoder choice and signaled in the bitstream.
  • this frame When the bitstream is changed from two-loop FGS coding to multi-loop coding for a frame, this frame has access to 2 reference frames, the base layer reference frame and the highest enhancement reference frame. However, all layers of this frame will be fully reconstructed so the next frame has all the reference frames needed for multi-loop FGS coding. If the bitstream is changed from multi-loop to two-loop, the current frame will be coded in multi-loop, but only the base layer and the highest FGS layer are fully reconstructed, since frames of any intermediate layers are no longer needed for the next frame.
  • the new predictors are calculated using motion compensation for coding the FGS layers. This requires the reconstruction of the enhancement layer reference frame needed. However, if the decoder wants to decode the layers above these FGS layers, the full reconstruction of the FGS layers can be avoided under certain constraints. For example, assume that there is a discrete base layer (LO), 2 FGS layers (Fl, F2) on top of LO, and there is a discrete enhancement layer (L3) on top of FGS layer F2.
  • LO discrete base layer
  • Fl, F2 discrete base layer
  • L3 discrete enhancement layer
  • the layer L3 does not use the fully reconstructed Macroblock that is coded as inter-MB in the base layer LO as predictors, instead it uses only the reconstructed residual in the prediction, when the decoder wants to reconstruct layer L3, it is only necessary to decode the residual information of layer Fl and F2 and no motion compensation is needed.
  • FIGS 9 and 10 are block diagrams of the FGS encoder and decoder of the present invention wherein the formation of reference blocks is dependent upon the base layer. In these block diagrams, only one FGS layer is shown. However, it should be appreciated that the extension of one FGS layer to a structure having multiple FGS layers is straightforward.
  • the FGS coder is a 2-loop video coder with additional "reference block formation module”.
  • FIG 11 depicts a typical mobile device according to an embodiment of the present invention.
  • the mobile device 1 shown in Figure 11 is capable of cellular data and voice communications. It should be noted that the present invention is not limited to this specific embodiment, which represents one of a multiplicity of different embodiments.
  • the mobile device 1 includes a (main) microprocessor or microcontroller 100 as well as components associated with the microprocessor controlling the operation of the mobile device.
  • These components include a display controller 130 connecting to a display module 135, a non-volatile memory 140, a volatile memory 150 such as a random access memory (RAM), an audio input/output (I/O) interface 160 connecting to a microphone 161, a speaker 162 and/or a headset 163, a keypad controller 170 connected to a keypad 175 or keyboard, any auxiliary input/output (I/O) interface 200, and a short-range communications interface 180.
  • a display controller 130 connecting to a display module 135, a non-volatile memory 140, a volatile memory 150 such as a random access memory (RAM), an audio input/output (I/O) interface 160 connecting to a microphone 161, a speaker 162 and/or a headset 163, a keypad controller 170 connected to a keypad 175 or keyboard, any auxiliary input/output (I/O) interface 200, and a short-range communications interface 180.
  • Such a device also typically includes other device subsystems shown generally at 190.
  • the mobile device 1 may communicate over a voice network and/or may likewise communicate over a data network, such as any public land mobile networks (PLMNs) in form of e.g. digital cellular networks, especially GSM (global system for mobile communication) or UMTS (universal mobile telecommunications system).
  • PLMNs public land mobile networks
  • GSM global system for mobile communication
  • UMTS universal mobile telecommunications system
  • the voice and/or data communication is operated via an air interface, i.e. a cellular communication interface subsystem in cooperation with further components (see above) to a base station (BS) or node B (not shown) being part of a radio access network (RAN) of the infrastructure of the cellular network.
  • BS base station
  • node B not shown
  • RAN radio access network
  • the cellular communication interface subsystem as depicted illustratively in Figure 11 comprises the cellular interface 110, a digital signal processor (DSP) 120, a receiver (RX) 121, a transmitter (TX) 122, and one or more local oscillators (LOs) 123 and enables the communication with one or more public land mobile networks (PLMNs).
  • the digital signal processor (DSP) 120 sends communication signals 124 to the transmitter (TX) 122 and receives communication signals 125 from the receiver (RX) 121.
  • the digital signal processor 120 also provides for the receiver control signals 126 and transmitter control signal 127.
  • the gain levels applied to communication signals in the receiver (RX) 121 and transmitter (TX) 122 maybe adaptively controlled through automatic gain control algorithms implemented in the digital signal processor (DSP) 120.
  • Other transceiver control algorithms could also be implemented in the digital signal processor (DSP) 120 in order to provide more sophisticated control of the transceiver 121/122.
  • LO local oscillator
  • a single local oscillator (LO) 123 may be used in conjunction with the transmitter (TX) 122 and receiver (RX) 121.
  • LO local oscillator
  • a plurality of local oscillators can be used to generate a plurality of corresponding frequencies.
  • the mobile device 1 depicted in Figure 11 is used with the antenna 129 as or with a diversity antenna system (not shown), the mobile device 1 could be used with a single antenna structure for signal reception as well as transmission.
  • Information which includes both voice and data information, is communicated to and from the cellular interface 110 via a data link between the digital signal processor (DSP) 120.
  • DSP digital signal processor
  • the detailed design of the cellular interface 110 such as frequency band, component selection, power level, etc., will be dependent upon the wireless network in which the mobile device 1 is intended to operate.
  • the mobile device 1 may then send and receive communication signals, including both voice and data signals, over the wireless network.
  • SIM subscriber identification module
  • Signals received by the antenna 129 from the wireless network are routed to the receiver 121, which provides for such operations as signal amplification, frequency down conversion, filtering, channel selection, and analog to digital conversion.
  • Analog to digital conversion of a received signal allows more complex communication functions, such as digital demodulation and decoding, to be performed using the digital signal processor (DSP) 120.
  • signals to be transmitted to the network are processed, including modulation and encoding, for example, by the digital signal processor (DSP) 120 and are then provided to the transmitter 122 for digital to analog conversion, frequency up conversion, filtering, amplification, and transmission to the wireless network via the antenna 129.
  • the microprocessor / microcontroller ( ⁇ C) 110 which may also be designated as a device platform microprocessor, manages the functions of the mobile device 1.
  • Operating system software 149 used by the processor 110 is preferably stored in a persistent store such as the non- volatile memory 140, which may be implemented, for example, as a Flash memory, battery backed-up RAM, any other non-volatile storage technology, or any combination thereof.
  • the non- volatile memory 140 includes a plurality of high-level software application programs or modules, such as a voice communication software application 142, a data communication software application 141, an organizer module (not shown), or any other type of software module (not shown).
  • These modules are executed by the processor 100 and provide a high-level interface between a user of the mobile device 1 and the mobile device 1.
  • This interface typically includes a graphical component provided through the display 135 controlled by a display controller 130 and input/output components provided through a keypad 175 connected via a keypad controller 170 to the processor 100, an auxiliary input/output (I/O) interface 200, and/or a short-range (SR) communication interface 180.
  • I/O auxiliary input/output
  • SR short-range
  • the auxiliary I/O interface 200 comprises especially USB (universal serial bus) interface, serial interface, MMC (multimedia card) interface and related interface technologies/standards, and any other standardized or proprietary data communication bus technology, whereas the short-range communication interface radio frequency (RF) low- power interface includes especially WLAN (wireless local area network) and Bluetooth communication technology or an IRDA (infrared data access) interface.
  • the RF low- power interface technology referred to herein should especially be understood to include any IEEE 801. xx standard technology, which description is obtainable from the Institute of Electrical and Electronics Engineers.
  • the auxiliary I/O interface 200 as well as the short-range communication interface 180 may each represent one or more interfaces supporting one or more input/output interface technologies and communication interface technologies, respectively.
  • the operating system, specific device software applications or modules, or parts thereof, may be temporarily loaded into a volatile store 150 such as a random access memory (typically implemented on the basis of DRAM (direct random access memory) technology for faster operation).
  • received communication signals may also be temporarily stored to volatile memory 150, before permanently writing them to a file system located in the non-volatile memory 140 or any mass storage preferably detachably connected via the auxiliary I/O interface for storing data.
  • a volatile store 150 such as a random access memory (typically implemented on the basis of DRAM (direct random access memory) technology for faster operation).
  • received communication signals may also be temporarily stored to volatile memory 150, before permanently writing them to a file system located in the non-volatile memory 140 or any mass storage preferably detachably connected via the auxiliary I/O interface for storing data.
  • An exemplary software application module of the mobile device 1 is a personal information manager application providing PDA functionality including typically a contact manager, calendar, a task manager, and the like. Such a personal information manager is executed by the processor 100, may have access to the components of the mobile device 1, and may interact with other software application modules. For instance, interaction with the voice communication software application allows for managing phone calls, voice mails, etc., and interaction with the data communication software application enables for managing SMS (soft message service), MMS (multimedia service), e-mail communications and other data transmissions.
  • the non-volatile memory 140 preferably provides a file system to facilitate permanent storage of data items on the device including particularly calendar entries, contacts etc.
  • the ability for data communication with networks e.g. via the cellular interface, the short-range communication interface, or the auxiliary I/O interface enables upload, download, and synchronization via such networks.
  • the application modules 141 to 149 represent device functions or software applications that are configured to be executed by the processor 100.
  • a single processor manages and controls the overall operation of the mobile device as well as all device functions and software applications.
  • Such a concept is applicable for today's mobile devices.
  • the implementation of enhanced multimedia functionalities includes, for example, reproducing of video streaming applications, manipulating of digital images, and capturing of video sequences by integrated or detachably connected digital camera functionality.
  • the implementation may also include gaming applications with sophisticated graphics and the necessary computational power.
  • One way to deal with the requirement for computational power which has been pursued in the past, solves the problem for increasing computational power by implementing powerful and universal processor cores.
  • a multi-processor arrangement may include one or more universal processors and one or more specialized processors adapted for processing a predefined set of tasks. Nevertheless, the implementation of several processors within one device, especially a mobile device such as mobile device 1, requires traditionally a complete and sophisticated re-design of the components. hi the following, the present invention will provide a concept which allows simple integration of additional processor cores into an existing processing device implementation enabling the omission of expensive complete and sophisticated redesign.
  • SoC system-on-a-chip
  • SoC is a concept of integrating at least numerous (or all) components of a processing device into a single high-integrated chip.
  • Such a system-on-a-chip can contain digital, analog, mixed-signal, and often radio-frequency functions - all on one chip.
  • a typical processing device comprises a number of integrated circuits that perform different tasks. These integrated circuits may include especially microprocessor, memory, universal asynchronous receiver-transmitters (UARTs), serial/parallel ports, direct memory access (DMA) controllers, and the like.
  • UART universal asynchronous receiver- transmitter
  • VLSI very-large-scale integration
  • the device 1 is equipped with a module for scalable encoding 105 and scalable decoding 106 of video data according to the inventive operation of the present invention.
  • said modules 105, 106 may individually be used.
  • the device 1 is adapted to perform video data encoding or decoding respectively. Said video data may be received by means of the communication modules of the device or it also may be stored within any imaginable storage means within the device 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

Un bloc de référence obtenu de manière adaptative est utilisé pour coder un bloc dans une trame en cours dans la couche d'amélioration. En particulier, le bloc de référence est obtenu à partir d'un bloc de référence dans la trame reconstituée de la couche de base et d'un bloc de référence dans la trame de référence de la couche d'amélioration, ainsi que d'un bloc résiduel de prédiction reconstitué par la couche de base. Par ailleurs, le bloc de référence servant au codage est ajusté en fonction des coefficients de transformées de la couche résiduelle reconstituée par la couche de base. De plus, le signal de référence effectif utilisé dans le codage constitue une moyenne pondérée d'un signal de référence de la trame reconstituée dans la couche de base et d'un signal de référence de la trame de référence de la couche d'amélioration, ainsi que d'une erreur résiduelle de prédiction de la reconstitution de la couche de base.
PCT/IB2006/000849 2005-04-12 2006-04-12 Procede et systeme de codage video evolutif a granularite fine et compensation de mouvement, avec commande de derive WO2006109141A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP06744488A EP1878257A1 (fr) 2005-04-12 2006-04-12 Procede et systeme de codage video evolutif a granularite fine et compensation de mouvement, avec commande de derive

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US67079405P 2005-04-12 2005-04-12
US60/670,794 2005-04-12
US67126305P 2005-04-13 2005-04-13
US60/671,263 2005-04-13
US72452105P 2005-10-06 2005-10-06
US60/724,521 2005-10-06

Publications (2)

Publication Number Publication Date
WO2006109141A1 WO2006109141A1 (fr) 2006-10-19
WO2006109141A9 true WO2006109141A9 (fr) 2007-12-27

Family

ID=37086635

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/000849 WO2006109141A1 (fr) 2005-04-12 2006-04-12 Procede et systeme de codage video evolutif a granularite fine et compensation de mouvement, avec commande de derive

Country Status (5)

Country Link
US (1) US20070014348A1 (fr)
EP (1) EP1878257A1 (fr)
KR (1) KR20080006607A (fr)
TW (1) TW200704202A (fr)
WO (1) WO2006109141A1 (fr)

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1977608B1 (fr) * 2006-01-09 2020-01-01 LG Electronics, Inc. Methode de prediction inter-couche de signal video
CN101043619A (zh) * 2006-03-24 2007-09-26 华为技术有限公司 视频编码的误差控制系统和方法
WO2008010157A2 (fr) * 2006-07-17 2008-01-24 Nokia Corporation Codage et décodage vidéo évolutif
EP2060122A4 (fr) * 2006-09-07 2016-04-27 Lg Electronics Inc Procédé et dispositif de décodage/codage d'un signal vidéo
AU2007318376B2 (en) * 2006-11-09 2010-07-01 Lg Electronics Inc. Method and apparatus for decoding/encoding a video signal
KR101349836B1 (ko) * 2006-11-17 2014-01-10 엘지전자 주식회사 비디오 신호의 디코딩/인코딩 방법 및 장치
US20100046620A1 (en) * 2007-01-09 2010-02-25 Jung-Won Kang Scalable video coding encoder with adaptive reference fgs and fgs motion refinement mechanism and method thereof
US8488668B2 (en) * 2007-06-15 2013-07-16 Qualcomm Incorporated Adaptive coefficient scanning for video coding
US8571104B2 (en) * 2007-06-15 2013-10-29 Qualcomm, Incorporated Adaptive coefficient scanning in video coding
KR101103724B1 (ko) 2007-07-02 2012-01-11 니폰덴신뎅와 가부시키가이샤 동화상 스케일러블 부호화 방법 및 복호 방법, 그 장치, 그 프로그램 및 프로그램을 기록한 기록 매체
FR2932637B1 (fr) * 2008-06-17 2010-08-20 Canon Kk Procede et dispositif de codage d'une sequence d'images
KR101483497B1 (ko) * 2008-09-25 2015-01-20 에스케이텔레콤 주식회사 임펄스 신호를 고려한 영상 부호화/복호화 장치 및 방법
US20100098156A1 (en) * 2008-10-16 2010-04-22 Qualcomm Incorporated Weighted prediction based on vectorized entropy coding
JP5344238B2 (ja) * 2009-07-31 2013-11-20 ソニー株式会社 画像符号化装置および方法、記録媒体、並びにプログラム
EP2617196A2 (fr) 2010-09-14 2013-07-24 Samsung Electronics Co., Ltd Procédé et appareil de codage et décodage d'image hiérarchiques
US9392274B2 (en) * 2012-03-22 2016-07-12 Qualcomm Incorporated Inter layer texture prediction for video coding
US9516309B2 (en) * 2012-07-09 2016-12-06 Qualcomm Incorporated Adaptive difference domain spatial and temporal reference reconstruction and smoothing
US9843801B2 (en) * 2012-07-10 2017-12-12 Qualcomm Incorporated Generalized residual prediction for scalable video coding and 3D video coding
US9467692B2 (en) 2012-08-31 2016-10-11 Qualcomm Incorporated Intra prediction improvements for scalable video coding
US20140092977A1 (en) * 2012-09-28 2014-04-03 Nokia Corporation Apparatus, a Method and a Computer Program for Video Coding and Decoding
EP2904803A1 (fr) 2012-10-01 2015-08-12 GE Video Compression, LLC Codage vidéo échelonnable utilisant la dérivation de subdivision en sous-blocs pour la prédiction à partir d'une couche de base
WO2014058110A1 (fr) * 2012-10-09 2014-04-17 광운대학교 산학협력단 Procédé de prédiction inter-couche pour une vidéo multicouche et son dispositif
US9357212B2 (en) 2012-12-07 2016-05-31 Qualcomm Incorporated Advanced residual prediction in scalable and multi-view video coding
GB2512563B (en) * 2013-01-04 2015-10-14 Canon Kk Method and apparatus for encoding an image into a video bitstream and decoding corresponding video bitstream with weighted residual predictions
US20140254681A1 (en) * 2013-03-08 2014-09-11 Nokia Corporation Apparatus, a method and a computer program for video coding and decoding
US9924191B2 (en) * 2014-06-26 2018-03-20 Qualcomm Incorporated Filters for advanced residual prediction in video coding
US10306229B2 (en) 2015-01-26 2019-05-28 Qualcomm Incorporated Enhanced multiple transforms for prediction residual
US10623774B2 (en) 2016-03-22 2020-04-14 Qualcomm Incorporated Constrained block-level optimization and signaling for video coding tools
US11323748B2 (en) 2018-12-19 2022-05-03 Qualcomm Incorporated Tree-based transform unit (TU) partition for video coding
US20220279185A1 (en) * 2021-02-26 2022-09-01 Lemon Inc. Methods of coding images/videos with alpha channels

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6700933B1 (en) * 2000-02-15 2004-03-02 Microsoft Corporation System and method with advance predicted bit-plane coding for progressive fine-granularity scalable (PFGS) video coding
US20020037046A1 (en) * 2000-09-22 2002-03-28 Philips Electronics North America Corporation Totally embedded FGS video coding with motion compensation
US7042944B2 (en) * 2000-09-22 2006-05-09 Koninklijke Philips Electronics N.V. Single-loop motion-compensation fine granular scalability
US6944222B2 (en) * 2002-03-04 2005-09-13 Koninklijke Philips Electronics N.V. Efficiency FGST framework employing higher quality reference frames
EP1530829B1 (fr) * 2002-07-09 2018-08-22 Nokia Technologies Oy Procede et dispositif permettant de selectionner le type de filtre d'interpolation pour un codage video
US7072394B2 (en) * 2002-08-27 2006-07-04 National Chiao Tung University Architecture and method for fine granularity scalable video coding

Also Published As

Publication number Publication date
EP1878257A1 (fr) 2008-01-16
US20070014348A1 (en) 2007-01-18
WO2006109141A1 (fr) 2006-10-19
TW200704202A (en) 2007-01-16
KR20080006607A (ko) 2008-01-16

Similar Documents

Publication Publication Date Title
US20070014348A1 (en) Method and system for motion compensated fine granularity scalable video coding with drift control
US20070201551A1 (en) System and apparatus for low-complexity fine granularity scalable video coding with motion compensation
US20070160137A1 (en) Error resilient mode decision in scalable video coding
KR100950273B1 (ko) 2계층 인코딩 및 단일 계층 디코딩을 이용한 스케일러블비디오 코딩
US20070009050A1 (en) Method and apparatus for update step in video coding based on motion compensated temporal filtering
US20080240242A1 (en) Method and system for motion vector predictions
US20070053441A1 (en) Method and apparatus for update step in video coding using motion compensated temporal filtering
US20070217502A1 (en) Switched filter up-sampling mechanism for scalable video coding
US20070030894A1 (en) Method, device, and module for improved encoding mode control in video encoding
US20080107185A1 (en) Complexity scalable video transcoder and encoder
US20070110159A1 (en) Method and apparatus for sub-pixel interpolation for updating operation in video coding
US20060256863A1 (en) Method, device and system for enhanced and effective fine granularity scalability (FGS) coding and decoding of video data
US20090279602A1 (en) Method, Device and System for Effective Fine Granularity Scalability (FGS) Coding and Decoding of Video Data
US20070201550A1 (en) Method and apparatus for entropy coding in fine granularity scalable video coding
US20080013623A1 (en) Scalable video coding and decoding
Nakagawa et al. Dynamic resolution conversion method for low bit rate video transmission
WO2008010157A2 (fr) Codage et décodage vidéo évolutif
CN101185340A (zh) 具有漂移控制的运动补偿精细粒度可伸缩视频编码方法和系统
Fakeh et al. Low-bit-rate scalable compression of mobile wireless video
Liu et al. Rate distortion performance of leaky prediction layered video coding: Theoretic analysis and results

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 7759/DELNP/2007

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Ref document number: DE

NENP Non-entry into the national phase

Ref country code: RU

WWE Wipo information: entry into national phase

Ref document number: 2006744488

Country of ref document: EP

Ref document number: 1020077026289

Country of ref document: KR

WWW Wipo information: withdrawn in national office

Ref document number: RU

WWE Wipo information: entry into national phase

Ref document number: 200680018852.7

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 2006744488

Country of ref document: EP