JP2003283340A - Encoding method and decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an improved method for switching streaming data bit streams. <P>SOLUTION: After decoding S12 entropy, a Lerr<SB>12</SB>and a movement vector are generated for a macro block. The Lerr<SB>12</SB>is dequantized using a Qs<SB>2</SB><SP>-1</SP>. A Kserr<SB>12</SB>is quantized using a Qs=Qs<SB>2</SB>to obtain a Lserr<SB>12</SB>. After movement compensation, forward DCT (discrete cosine transformation) transformation is conducted for a predicted macro block to obtain a Kpred<SB>1</SB>. The Kpred<SB>1</SB>is quantized by a Qs<SB>1</SB>. A Lpred<SB>1</SB>is dequantized by a Qs<SB>1</SB><SP>-1</SP>. A coefficient Kspred<SB>1</SB>after dequantizing is quantized by a Qs=Qs<SB>2</SB>to obtain a Lspred<SB>12</SB>. A Lrec<SB>2</SB>after rebuild-up is obtained from an equation (Lrec<SB>2</SB>=Lspred<SB>12</SB>+Lserr<SB>12</SB>). The Lrec<SB>2</SB>is dequantized using a Qs<SP>-1</SP>=Qs<SB>2</SB><SP>-1</SP>and inverse DCT transformation is conducted to obtain an image after rebuild-up. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a coding method and a decoding method, and more particularly to a coding method, a recording medium, an apparatus and a decoding method for switching between different streaming bitstreams related to a data bitstream. Method and decoder.

[0002]

BACKGROUND OF THE INVENTION With the steady increase in access bandwidth, more and more Internet applications are using streaming audio and video content. Since the current Internet is a heterogeneous network in nature and a dynamic best effort network, channel bandwidths typically range from low bit rates below 64 kbps to
It varies over a wide range up to bit rates well above 1 Mbps. This is a smooth playback experience,
In providing users with the best available video quality, it poses a significant challenge to video coding and streaming technologies. Two major approaches to address network bandwidth fluctuations have been extensively researched in recent years: switching between multiple non-scalable bitstreams, and streaming in a single scalable bitstream. .

In the first approach, a video sequence is compressed into several non-scalable bitstreams at different bit rates. Certain special frames, known as keyframes, are compressed without prediction or coded with an extra switching bitstream. Keyframes provide an access point for switching between these bitstreams and adapting to the available bandwidth. One advantage of this method is high coding efficiency for non-scalable bitstreams.
However, since both the number of bitstreams and the number of switching points are limited, this method provides only coarse and dull capabilities in adapting to variations in channel bandwidth.

In the second approach, a video sequence is compressed into a single scalable bitstream that can be flexibly truncated to accommodate bandwidth fluctuations. Among the many scalable coding techniques, MPEG-4 Granularity Scalable (FGS; Fine G)
Ranularity Scalable coding has come to the fore because of its fine scalability. Since the enhanced bitstream can be arbitrarily truncated at any frame, FGS provides outstanding ability in adapting easily and accurately to variations in channel bandwidth. However, the low coding efficiency is a serious drawback, which is a drawback of FGS.
Has hindered widespread use in video streaming applications. Progressive granularity scalable
The (PFGS) coding scheme is a significant improvement over FGS by introducing two prediction loops with different quality criteria. On the other hand, since only one high quality criterion is used in enhancement layer coding, the highest coding efficiency gains appear within some bit rate range around the high quality criterion.
In general, with today's technology, there is still a loss of coding efficiency as compared to the case of non-scalability at constant bit rate.

Nevertheless, bandwidth fluctuations still remain a problem when streaming video over the current Internet. For example, as mentioned above, conventional streaming video systems generally attempt to address this problem by switching between different video bitstreams having different bitrates.
However, these existing video coding schemes limit the switching points to only keyframes (eg, typically I-frames) to avoid drifting problems. Such keyframes are typically coded at a distance from each other in order to maintain high coding efficiency, so that bitstream switching occurs only periodically. This significantly reduces the adaptability of existing streaming systems. Therefore, frequent pauses and rebuffering may occur when a viewer watches a streaming video.

[0006] In the literature related to such prior art, a switching system which enables seamless switching between bit streams having different bit rates is proposed.
(See, for example, Non-Patent Documents 1, 2, and 3).

[0007]

[Non-Patent Document 1] Ragip Kurceren and Marta Karczewic
z, "Improved SP-fRAMe Encoding", VCEG-M-73, I
TU-T Video Coding Experts Group Meeting, Austin, T
X, 02-04 April 2001

[0008]

[Non-Patent Document 2] Xiaoyan Sun, Feng Wu, Shipeng, Ya-
Qin Zhang, "Improved SP coding technique", Micro
soft Patent Predisclosure, Jan., 2002

[0009]

[Non-Patent Document 3] Shipeng Li, Feng Wu, Xiaoyan Sun,
Jacky Shen, `` Method for Seamless Switching of Vid
eo Bitstreams ", Microsoft Patent Predisclosure, J
an., 2002

[0010]

There are various problems with conventional systems, such as those mentioned above, and there is therefore a need for improved methods and apparatus for use in switching streaming bitstreams.

The present invention has been made in view of the above circumstances, and an object thereof is to provide an improved encoding method and decoding method for switching streaming data bit streams. .

[0012]

SUMMARY OF THE INVENTION Improved methods and apparatus are provided for switching streaming data bitstreams, such as are used in video streaming and other related applications. Some desired features provided herein include random access, fast forward, reverse fast forward, error resilience, and bandwidth adaptation. The improved method and apparatus can be configured to improve coding efficiency and / or reduce the amount of data required to code the switched bitstream.

According to an exemplary implementation of the invention, an encoding method is provided. The method includes encoding data into a first bitstream using a first quantization parameter and using a second quantization parameter that is different from the first quantization parameter. To a second bitstream. The method also uses the first quantization parameter to support up-switching between the first bitstream and the second bitstream, the first bitstream and the second bitstream being used. The second quantization parameter to support down-switching between the first bitstream and the second bitstream, and the encoded switching bitstream associated with the first bitstream and the second bitstream. It also includes the step of generating.

The exemplary apparatus includes a first bitstream encoder configured to encode data into an encoded first bitstream using the first quantization parameter, and a first quantization parameter. A second bitstream encoder configured to encode the data into a second encoded bitstream using a second quantization parameter different from. The apparatus is also operatively coupled to the first bitstream encoder and the second bitstream encoder and based on information processed using the first quantization parameter and the second quantization parameter. It also includes a switched bitstream encoder configured to output a coded switched bitstream that supports upward and downward switching between the first encoded bitstream and the second encoded bitstream.

The exemplary decoding method uses a first bitstream generated using the first quantization parameter and / or a second quantization parameter different from the first quantization parameter. Receiving at least one encoded bitstream, such as a second generated bitstream. The received encoded bitstream is decoded. The decoding method uses a first quantization parameter to support upward switching between a first bitstream and a second bitstream, and the first bitstream and the second bitstream. Further comprising receiving a coded switch bitstream associated with the first bitstream and the second bitstream generated using the second quantization parameter to support downward switching between the two. The method also includes decoding the received coded switch bitstream using the first quantization parameter and the second quantization parameter.

Another exemplary apparatus is a first decoder configured to decode a first coded bitstream into a decoded first bitstream using a first quantization parameter, A second decoder configured to decode the second bitstream into a second decoded bitstream using a second quantization parameter that is different from the first quantization parameter. This device also
A first decoded bitstream and a first decoded bitstream based on information operably coupled to the first decoder and the second decoder and processed using the first quantization parameter and the second quantization parameter. It also includes a switched bitstream decoder configured to output a decoded switched bitstream that supports upward switching and downward switching between two decoded bitstreams.

The present invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings. The same numbers are used throughout the figures to facilitate referencing the same components and / or functions.

[0018]

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. In the drawings, parts having the same function are denoted by the same reference numerals, and duplicate description will be omitted.

[First Embodiment] A method for seamlessly switching video bitstreams relating to a video streaming application example will be described. This technique separates the Qs for upward switching from a low bit rate to a high bit rate (ie the speed of the switching rate) and the Qs for downward switching from a high bit rate to a low bit rate, and advantageously Separating each switching point provides advantages over prior art switching schemes.

This disclosure proposes an improved method for seamless switching of video bitstreams for video streaming and other related applications.
Random access, fast forward, reverse fast forward, error resilience,
And some desired features such as bandwidth adaptation also
It is provided by the method proposed here. The method proposed here improves the coding efficiency of a normally coded bitstream or reduces the amount of bits for coding a switched bitstream. This method is an improvement over Non-Patent Document 1 of the prior art, but there are considerable differences between the two.

A switching method has been proposed which enables seamless switching between bit streams having different bit rates (for example, refer to Non-Patent Document 1; hereinafter, simply referred to as Kurceren et al.). Kurceren et al. Introduced a special frame called an SP picture that acts as a switching point in a video sequence.

A similar exemplary switching process 200
This is shown in the exemplary diagram of FIG. Here, it is assumed that switching from bitstream 1 to bitstream 2 is performed using SP pictures.

Streaming systems typically send either bitstream 1 or bitstream 2 depending on, for example, the current channel bandwidth. However, when the channel bandwidth changes, the sending bitstream is switched to a bitrate that matches the current channel conditions, for example to improve video quality when bandwidth increases or smooth when bandwidth decreases. It is possible to maintain good reproduction.

When switching from bitstream 1 to bitstream 2, the streaming system does not have to wait for a keyframe to initiate the switching process. Instead of waiting for keyframes, the streaming system can switch on SP frames. In SP frame, the streaming system switches bitstream
S12 is sent and the switching bitstream is S1 or S2
The decoder does not know whether it is S12 or S12 and uses the same technique to decode the switched bitstream. Therefore, bitstream switching is transparent to the decoder. The decoded frame will be exactly the same as the reference frame for the next frame prediction in bitstream 2. Therefore, there is no drifting problem.

An exemplary conventional decoder 300 and encoder 400 are shown in FIGS. 3 and 4, respectively. A more detailed description of this scheme can be obtained from Kurceren et al. There are some potential problems with the scheme in Kurceren et al.

For example, in real streaming applications, it is usually desirable to switch down from a high bitrate bitstream to a low bitrate bitstream very quickly. This is the TC currently used in many existing streaming systems.
This is a desirable feature for P-friendly protocols. On the other hand, switching up from a low bitrate video bitstream to a high bitrate video bitstream typically does not have to be as fast as switch down. Again, this is a feature of TCP friendly protocols, for example.

Therefore, it would be useful to support faster and more frequent down-switching. In fact, as mentioned above, the only reason to do the down-switching is often related to the reduced / reduced channel bandwidth capacity. The size of the lower switching bitstream is often much smaller than the size of the upper switching bitstream. The high bit rate bit stream generally contains most of the information of the low bit rate bit stream, so theoretically a scheme could be constructed to make the size of the switching bit stream sufficiently small.

However, in the Kurceren et al. Scheme, only the same Qs is possible for the lower switching bitstream and the upper switching bitstream (see, eg, FIG. 4), which is included in the prediction loop and the reconstruction loop. Be done. The introduction of quantized Qs into the prediction loop and the reconstruction loop necessarily reduces the coding efficiency of the original bitstream without SP frames. When Qs is set to be very small, high coding efficiency can be achieved for both bitstreams 1 and 2. However, the difference for down-switching is also finely quantized, which makes the down-switching bitstream very large. Conversely, if Qs is set too large, a very compact switched bitstream is obtained, but the coding efficiency of bitstreams 1 and 2 is severely reduced. Neither of these is desirable. It is clear that this clear contradiction cannot be resolved by the technique proposed by Kurceren et al., Which trades off between coding efficiency and the size of the switching bitstream.

Furthermore, in the encoder proposed by Kurceren et al., There are many quantization and dequantization processes in the signal flow (see eg FIG. 4).
This process tends to further reduce the coding efficiency of bitstreams 1 and 2. In Kurceren et al., There is also a mismatch between the prediction criterion and the reconstruction criterion, and the mismatch may contribute to the deterioration of the coding efficiency of bitstreams 1 and 2.

To address these and other issues,
In this specification, different for upper switching and lower switching
An improved method and apparatus capable of Qs is provided. Figure 5
6 and 6 respectively show an improved decoder and encoder according to an implementation of the invention.

According to an aspect of the present invention, the proposed technique resolves the contradiction existing in the scheme proposed by Kurceren et al., And consequently maintains good coding efficiency of bitstreams 1 and 2. However, the down-switching bitstream can be encoded to have a significantly, if not a minimally reduced size.

According to another aspect of the invention, the switching points for upper and lower switching can be separated. This means that the lower switching points can be coded more than the upper switching points in order to fit eg TCP friendly protocols. Furthermore, such a separation makes it possible to further improve the coding efficiency of the original bitstream to which the system switches, for example by individually setting Qs in the reconstruction loop to an appropriately small value.

According to certain other aspects of the invention, the improved method and apparatus can be further simplified and additional quantization and dequantization processes can be easily eliminated. For example, FIG. 7 and FIG. 8 respectively show high coding efficiency for a normal bit stream,
1 illustrates an exemplary decoder and encoder that supports both compact sizes for switched bitstreams.

Exemplary Operating Environment With reference to the drawings, like reference numbers refer to like elements and the invention is shown as implemented in a suitable computing environment. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a personal computer.

Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, the book can be used with other computer system configurations including handheld devices, multiprocessor systems, microprocessor-based consumer electronics or programmable consumer electronics, network PCs, minicomputers, mainframe computers, portable communication devices, etc. Those skilled in the art will understand that the invention can be practiced.

The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

FIG. 1 illustrates an example of a suitable computing environment 120 on which the systems, devices, and methods described below may be implemented. The exemplary computing environment 120 is merely one example of a suitable computing environment and does not suggest any limitation with regard to the scope of use and functionality of the improved methods and systems described herein. Neither should the computing environment 120 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the computing environment 120.

The improved methods and systems herein are operational with numerous other general purpose computing system environments or configurations, or numerous other special purpose computing system environments or configurations. Examples of suitable well-known computing systems, environments, and / or configurations include, but are not limited to, personal computers, server computers, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based. Systems, set-top boxes, programmable consumer electronics, networked PCs, minicomputers, mainframe computers, as well as distributed computing environments including any of the above systems or devices.

As shown in FIG. 1, computing environment 120 includes a general purpose computing device in the form of computer 130. The components of computer 130 include one or more processors or processing units 132.
, A system memory 134, and a bus 136 coupling various system components including the system memory 134 to the processor 132.

Bus 136 may be any of several types of buses including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. One of the structures
Represents one or more. For example, but not limiting of, such architectures include ISA (Industry Standard Arch
itecture) bus, MCA (Micro Channel Architecture) bus, EISA (Enhanced ISA) bus, VESA (Video Electronics)
PCIs (Peripheral Component Interco), also called Standards Association (Local Association Bus) and mezzanine bus
nnect) Bus included.

Computer 130 typically includes a variety of computer readable media. Such media can be any available media that can be accessed by computer 130 and includes both volatile and nonvolatile media, removable and non-removable media.

In FIG. 1, system memory 134 is a computer-readable medium in the form of volatile memory, such as random access memory (RAM) 140, and / or a computer in the form of non-volatile memory, such as read-only memory (ROM) 138. Includes readable media. A basic input / output system (BIOS) 142 that contains the basic routines that help to transfer information between elements within computer 130, such as during start-up.
Are stored in ROM 138. RAM 140 is generally immediately accessible to processor 132,
And / or contains data and / or program modules that processor 132 is currently operating.

The computer 130 is not removable.
It may further include non-removable, volatile / nonvolatile computer storage media. For example, in FIG. 1, a hard disk drive 144 for reading and writing a non-removable nonvolatile magnetic medium (not shown; generally called a “hard drive”) and a removable nonvolatile magnetic disk 148 (for example, “Floppy (registered trademark)” Magnetic disk drive 146 for reading / writing ")", CD-ROM / R / RW, DVD-ROM / R / RW / + R
/ RAM, or an optical disk drive 150 for reading and writing removable non-volatile optical disks 152, such as other optical media. Hard disk drive 14
4, magnetic disk drive 146, and optical disk drive 150 are each connected to bus 136 by one or more interfaces 154.

The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules, and other data for computer 130. In the exemplary environment described herein, a hard disk, removable magnetic disk 148, and removable optical disk 152.
Utilizes magnetic cassettes, flash memory cards, digital video disks, random access memory
Those skilled in the art should appreciate that other types of computer-readable media capable of storing data, such as (RAM), read-only memory (ROM), etc., capable of storing data may also be used in this exemplary operating environment.

For example, operating system 158
a, 158b, one or more application programs 160a, 160b, other program modules 162a, 162b, and program data 164.
Some program modules, including a, 164b, can be stored on a hard disk, magnetic disk 148, optical disk 152, ROM 138, or RAM 140.

The improved method and system described herein includes operating systems 158a, 158.
b, one or more application programs 16
0a, 160b, another program module 162a,
162b and / or program data 164a,
It can be implemented in 164b.

A user can provide commands and information to computer 130 via input devices such as keyboard 166 and pointing device 168 (such as a "mouse"). Other input devices (not shown) include
It can include a microphone, joystick, gamepad, satellite dish, serial port, scanner, camera, etc. These and other input devices are connected to the processor 132 via a user input interface 170 that is coupled to the bus 136, but other interface structures such as parallel ports, game ports, universal serial buses (USB). It is also possible to connect with a bus structure.

A monitor 172 or other type of display device is also connected to the bus 136 via an interface such as a video adapter 174. Monitor 172
In addition, personal computers typically include other peripheral output devices (not shown) such as speakers and printers. These can be connected via the output peripheral interface 175.

Computer 130 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer 182. Remote computer 182 may include many or all of the elements and features described herein with respect to computer 130.

The logical connections shown in FIG. 1 are a local area network (LAN) 177 and a general-purpose wide area network (WAN) 179. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN networking environment, computer 130 is connected to LAN 177 via a network interface or adapter 186. When used in a WAN networking environment,
Computers typically have a modem 178, or WAN.
Other means for establishing communication via 179 are included. The modem 178, which may be internal or external, may be connected to the system bus 136 via the user input interface 170 or other suitable mechanism.

Illustrated in FIG. 1 is a particular implementation of WAN over the Internet. In this case, computer 130 utilizes modem 178 to establish communication with at least one remote computer 182 via Internet 180.

In the network environment, the computer 13
The program modules illustrated in relation to 0, or portions thereof, may be stored in remote memory storage. Thus, for example, as shown in FIG. 1, the remote application program 189 can reside on the memory device of the remote computer 182. It will be appreciated that the network connections shown and described are exemplary and other means of establishing a communications link between the computers may be used.

Exemplary Switching Schemes In this section, exemplary encoder and decoder methods and apparatus are described in more detail with reference to FIGS. 5-8. For comparison, an additional explanation of the architecture proposed by Kurceren et al. Is also given.

The modules and notations used in FIGS. 3-8 are defined as follows. DCT: Discrete cosine transform. IDCT: Inverse discrete cosine transform. Entropy coding: Entropy coding of quantized coefficients. This can be arithmetic coding or variable length coding. Entropy decoding: Entropy decoding of quantized coefficients. This can be an arithmetic or variable length decoding that matches the corresponding module in the encoder. Q: Quantization. Q- ¹ : Inverse quantization, that is, dequantization. MC: Motion compensation module. However, the predicted frame is formed according to the motion vector and the criteria in the frame buffer. ME: Motion estimation module. However, the motion vector is searched to best predict the current frame. Loop Filter: A smoothing filter in the motion compensation loop to reduce blocking artifacts. FRAMeBuffer 0: Frame buffer that holds the reference frame for the next frame encoding / decoding. P-picture: A frame encoded using conventional motion-compensated predictive coding. SP picture: A frame coded as a switching frame using the proposed motion compensated predictive coding. Switched bitstream: A bitstream sent for a seamless transition from one bitstream to another.

There are some basic assumptions regarding quantization and dequantization. If L ₁ = Q (K ₁ ), then Q (Q ⁻¹ (K ₁ )) = L ₁ L ₁ = Q (K ₁ ), and if L ₂ = Q (K ₂ ), then Q (Q ^{− 1} (L ₁ ) + Q
^{If -1} (L ₂ )) = L ₁ + L ₂ L ₁ = Q (K ₁ ), then Q (Q ⁻¹ (L ₁ ) + K ₂ )) = L ₁ + Q (K ₂ ) The description applies to inter macroblocks in a frame. A simple "copy" operation can be used for intra macroblocks.

Reference is now made to the conventional decoding process shown, for example, in FIG. Here, the decoding of SP frames S1 or S2 in a normal bitstream is shown. Using S1 as an example, after entropy decoding of the bitstream S1, the prediction error coefficient level L _err1 and the motion vector are generated for the _macroblock . L _err1 is dequantized using the dequantizer QP ₁ ^{-1. _{K serr1 = QP 1 -1 (L}} err1) after motion compensation, forward DCT to the prediction macroblock
The transformation is performed to obtain K _pred1 and the reconstructed coefficient K _rec1
Is obtained by K _rec1 = K _pred1 + K _serr1 Reconstructed coefficient K _rec1 is quantized by Qs, and reconstructed level L
get _rec1 . _Dequantize the level L _rec1 using L _rec1 = Qs (K _rec1 ) Qs ⁻¹ and perform an inverse DCT transform to obtain the reconstructed image. The post-reconstructed image will pass through a loop filter to smooth out certain uneven artifacts, output them to the display, and output them to the frame buffer for the next frame decoding.

[0058] When decoding a switching bitstream S12, for example, when switching from bitstream 1 to bitstream 2, the decoding process, input is next bitstream S12, QP ₁ ^-1 is replaced by Qs ^-1, K _serr1 is replaced by K _serr12 , L _rec1 is replaced by L _rec2 , K _rec1 is replaced by K _rec12 , L _err1 is replaced by L _err
It is the same as the decoding of S1 except that it is replaced by ₁₂ .

The picture obtained is the same as the picture decoded from S2. Thus, a drift-free switch from bitstream 1 to bitstream 2 is achieved. Qs is encoded in the S12 bit stream.

Next, referring to FIG. 4, in the normal bit stream
Reference is made to the exemplary conventional encoding process shown for the encoding of SP frames S1 or S2. Where S
1 is used as an example.

DCT for macroblocks of original video
The transformation is performed and the obtained coefficient is represented as K _{orig 1} . After motion compensation, DCT transform is performed on the prediction macroblock, and the obtained coefficient is represented as K _pred1 . The next step is Q
The K _pred1 quantized using s, it is to obtain a level L _pred1. Use L _{pred1 =} Qs (K _pred1) then dequantizer Qs ^-1, the L _pred1 released quantization ^{_{(K spred1 = Qs - 1 (}} L pred1)), K from K _Orig1
The error coefficient K _err1 is obtained by subtracting _spred1 . K _err1 = K _orig1 -K _pred1 then quantizes K _err1 using QP ₁ and error level L
_{Get err1} . L _err1 = QP ₁ (K _err1 ) Next, entropy coding is performed on L _err1 to obtain the bit stream S1. _Reconstruct the level L _rec1 and the reference for the next frame encoding, for example using the S1 decoder described above. Note that in this example there is a quantizer Qs and a dequantizer Qs ^{-1 in the} reconstruction loop.

Note that in this scheme there is a mismatch between the prediction criteria and the reconstruction criteria. The encoding of the switching bitstream S12 (switching from bitstream 1 to bitstream 2) and the encoding of S12 are S
1 and S2 encoding. With the S1 encoder,
L _pred1 is subtracted from the reconstructed level L _rec2 in the S2 encoder. L _err12 = L _rec2 -L _{pred1 Performs} entropy coding using L _err12 and bit stream S12.

Next, FIG. 5 according to an exemplary implementation of the present invention.
The improved decoding process 500 of will be described in more detail.

SP frame S in a normal bitstream
S1 is used as an example to describe the decoding of 1 and S2. After entropy decoding of the bitstream S1, the level L of the prediction error coefficient for the macroblock
_{Generate err1} and the motion vector. Dequantizer QP
The level L _err1 using ₁ ^-1 to dequantization. ^{_{K serr1 = QP 1 -1 (L}} err1) using quantizer Qs = Qs ₁ quantizes the error coefficient _K serr1, obtain level _L serr1. L _serr1 = Qs (K _serr1 ) Forward DCT for prediction macroblock after motion compensation
Perform a transformation to obtain K _pred1 . Then, K _pred1 is quantized with Qs ₁ . L _pred1 = the Qs ₁ (K _pred1) then L _pred1 releasing quantized by Qs ₁ ^{-1. _{K spred1 = Qs 1 -1 (L}} pred1) further quantizing the coefficient K _Spred1 after dequantization in the quantizer Qs = Qs _1, to obtain a level L _{sp red1.} L _spred1 = Qs (K _spred1 ) The post-reconstruction level L _rec1 is obtained by: _Dequantize the level L _rec1 using L _rec1 = L _spred1 + L _serr1 Qs ⁻¹ = Qs ₁ ⁻¹ and perform an inverse DCT transform to obtain the reconstructed image. The post-reconstructed image will pass through a loop filter, smoothing out certain constant artifacts, output to the display, and output to the frame buffer for the next frame decoding.

For example, from bitstream 1 to bitstream
Switching bitstream when switching to Ream 2
Decoding S12 requires that the input is a bitstream S12.
QP ₁ ^-1Is Qs_Two ^-1Replaced by, Qs becomes Qs_TwoReplaced by
Qs^-1Is Qs_Two ^-1Replaced by L_rec1Is L_rec2Replace with
Is L_err1Is L_err12Replaced by K_serr1Is K
_serr12Replaced by L_spred1Is L_spred12Replaced by
Follow a similar decryption process except that:

Note that the information about Qs ₁ and Qs ₂ is encoded in bitstream S12.

The picture obtained is the same as the picture decoded from S2. Thus, a drift-free switch from bitstream 1 to bitstream 2 is achieved.

The improved encoding process 600 of FIG. 6, according to one exemplary implementation of the present invention, will now be described in more detail.

SP frame S in a normal bitstream
S1 is used as an example to describe the encoding of 1 or S2. Here, for example, DCT conversion is performed on the macroblock of the original video, and the obtained coefficient is represented as K _orig1 .

After motion compensation, DCT conversion is performed on the prediction macroblock, and the obtained coefficient is represented by K _pred1 . Then quantize K _{pred 1} using Qs ₁ to _obtain level L _{pred 1}
To get L _{pred 1} = Qs ₁ (K _{pred 1} ) Then the next step is to dequantize L _pred ¹ using the dequantizer Qs ₁ ⁻¹ . ^{_{K spred1 = Qs 1 -1 (L}} pred1) then subtracts the K _Spred1 from _K orig1, error coefficient K _err1
To get K _err1 = K _orig1 -K _pred1 Then, QP ₁ is used to quantize K _err1 and the error level L
_{Get err1} . L _err1 = QP ₁ (K _err1 ), then _perform entropy coding on L _err1 ,
Get the bitstream S1.

_Reconstruct the level L _rec1 and the reference for the next frame encoding, for example using the S1 decoder described above.

In this case, the quantizer Qs ₁
And it should be noted that there is dequantizer Qs ₁ ^-1.

Switching bitstream when switching from bitstream 1 to bitstream 2, for example
The coding of S12 is based on the coding of S1 and S2.

In this case, the process is the quantizer Qs_Twouse
Prediction coefficient K in S1 encoder _spred1Quantize
Including that. L_spred12= Qs_Two(K_spred1) Then, the level L after reconstruction in the S2 encoder_rec2To L
_spred12Subtract. L_err12= L_rec2-L_spred12 Then L_err12Perform entropy coding of the bits
Generate the stream S12. Then another of the invention
The improved decoding process 70 of FIG. 7, according to an example implementation.
0 will be described in more detail.

SP frame S in a normal bitstream
S1 is used as an example to describe the decoding of 1 or S2.

Here, after entropy decoding of the bitstream S1, the level L _{err1 of} the prediction error coefficient and the motion vector are generated for the macroblock. The level L _err1 to dequantized using dequantizer QP ₁ ^{-1. _{K serr1 = QP 1 -1 (L}} err1) after motion compensation, forward DCT to the prediction macroblock
Perform a transformation to obtain K _pred1 . The post-reconstruction coefficient K _rec1 is obtained by K _rec1 = K _pred1 + K _serr1 Reconstructed coefficient K _rec1 is quantized with Qs ₁ and reconstructed level
Get L _rec1 . _Dequantize the level L _rec1 using L _rec1 = Qs ₁ (K _rec1 ) Qs ₁ ^-1, and use the inverse DCT
The transformation is performed to obtain the reconstructed image. After reconstruction, the image is passed through a loop filter to smooth out certain uneven artifacts and output it to the display.
It will be output to the frame buffer for the next frame decoding.

For example, from bitstream 1 to bitstream
Switching bitstream when switching to Ream 2
Decoding S12 requires that the input is a bitstream S12.
QP ₁ ^-1Is Qs_Two ^-1Replaced by Qs₁Is Qs_TwoReplaced by
Qs₁ ^-1Is Qs_Two ^-1Replaced by K_serr1Is K_serr12
Replaced by K_rec1Is K_rec12Replaced by L_rec1Is L_r
ec2Replaced by L_err1Is L_err12To be replaced with
Follow the same decryption process except.

The picture obtained is the same as the picture decoded from S2. Thus, a drift-free switch from bitstream 1 to bitstream 2 is achieved.

The improved encoding process 800 of FIG. 8 according to another exemplary implementation of the invention will now be described in more detail.

SP frame S in a normal bitstream
S1 is used as an example to describe the encoding of 1 or S2.

Here, the coefficient obtained by performing the DCT transform on the macroblock of the original video is represented as K _orig1 .

Next, after motion compensation, DCT transform is performed on the prediction macroblock, and the obtained coefficient is represented as K _pred1 .

The process then _involves subtracting K _pred1 from K _orig1 to obtain the error coefficient K _err1 . K _err1 = K _orig1 -K _pred1 Then, QP ₁ is used to quantize K _err1 and the error level L
_{Get err1} . L _err1 = QP ₁ (K _err1 ) Next, entropy coding is performed on L _err1 to obtain the bit stream S1.

Using, for example, the S1 decoder described above, the process involves _{reconstructing} the level L _rec1 and the reference for the next frame encoding. In this case, it should be noted that there are quantizers Qs ₁ and dequantizer Qs ₁ ^-1 during the rebuild loop.

For example, when switching from bitstream 1 to bitstream 2, a switching bitstream
The coding of S12 is based on the coding of S1 and S2.

For example, quantizer Qs ₂ is used to quantize prediction coefficient K _pred1 in the S1 encoder. L _pred12 = Qs ₂ (K _pred1 ) Then the process is the reconstructed level L in the S2 encoder.
_Includes subtracting L _pred12 from _rec2 . L _err12 = L _rec2 −L _pred12 Next, entropy coding of L _err12 is performed to generate a bitstream S12.

Next, referring to FIG. FIG. 9 is a block diagram illustrating a decoder 900 for S1 and S2, according to another implementation of the invention. In this case, the quantized Qs
Post-reconstruction DCT, not for DCT remainder and DCT prediction
Note that it acts on the criteria. The quantization in this example can be described as: Y = [X * A (Qs) +2 ¹⁹ ] / 2 ^{20 In the} above formula, X is the DCT coefficient after reconstruction and Y is the quantized DCT coefficient. A (.) Is a quantization table. Qs is the quantization step.

If the dequantization QP and the quantization QS are merged into one step, the operation can be formulated, for example, as follows.

[0089]

[Equation 7]

In the above equation, L _err is the level of the prediction error coefficient, and K _pred is the prediction coefficient. This is very different from that used in conventional SP coding. One advantage is that a high quality display can be reconstructed from the [...] part of the above equation.

Therefore, the decoder 900 provides two methods for reconstructing the display image.
In the first case, the post-reconstruction criteria are used directly for display. In this case, there is little or no complexity, even as complexity increases. In the second case, if the decoder is powerful enough, another high quality image can be reconstructed for display. This process is box 9
02 contains modules. These modules are non-standard parts with respect to the current JVT (Joint Video Team) standard, for example.

FIG. 10 illustrates another implementation of the present invention.
A decoder 1000 for a switched bitstream S12 is shown. In this example, the decoder 1000 for switched bitstream S12 is slightly different than the decoder for S1 and S2 presented in the previous section. In this case, for example, quantization
Qs is needed only for DCT prediction. Quantization in this example can also be described as follows. Y = [X * A (Qs) +2 ¹⁹ ] / 2 ²⁰

The decoder 1000 is arranged to recognize which SP bitstream was received. Thus, for example, the 1-bit syntax can be used to notify the decoder 1000. Exemplary modifications in SP syntax and semantics include switching bitstream flags (eg 1 bit) and quantization parameters (eg 5 bits).

Therefore, for example, when the P type indicates an SP frame, the 1-bit syntax element "switch bitstream flag" is set to the syntax element "slice Qp".
Insert before. Here, when the switching bitstream flag is 1, the current bitstream is decoded as the bitstream S12, and the syntax element "slice QP" is skipped. When the switching bitstream flag is not 1, the current bitstream is decoded as the bitstream S1 or S2, and the syntax element "slice QP" becomes the quantization parameter Qp.

When the P type indicates an SP frame, the syntax element "SP slice QP" is inserted after the syntax element "slice QP" and the quantization parameter Qs is encoded.

The block diagram of FIG. 11 shows an encoder 1100 according to another exemplary implementation of the invention.

In this case, for example, the encoder 1100 includes a switch 1102. Therefore, the DCT prediction can be subtracted directly from the original DCT image without quantization and dequantization, or after quantization and dequantization, DC
The T prediction can be subtracted from the original DCT image. DC
Whether or not the T prediction is quantized can be determined for each coefficient using, for example, a rate distortion criterion.

Thus, some exemplary refinements of
An SP picture coding method and device is presented. Separate Qs can be provided for the upper switching bitstream and the lower switching bitstream. Qs for switched bitstream coding can be separated from the prediction loop and the reconstruction loop. This eliminates the contradiction between, for example, reducing the size of the switching bitstream and improving the coding efficiency of a normal bitstream. By optimizing different Qs separately, the switching bitstream size can also be significantly reduced while maintaining the high coding efficiency of a normal bitstream. To improve coding efficiency, some of the quantization / dequantization process can be removed according to some implementations. Coding efficiency can also be improved by using the same criteria for prediction and reconstruction. The method and apparatus can also be configured to allow more downward switching points than upward switching points.

Conclusion The above description uses terms specific to structural features and / or operations, but the invention defined in the appended claims is not limited to the particular functions and operations described herein. I want you to understand. Rather, the specific features and operations are disclosed as exemplary forms of implementing the invention.

Embodiment 2 This disclosure describes a further improved method for seamless switching of video bitstreams for video streaming and other related applications. This improvement is based on the above Non-Patent Documents 2 and 3. See these documents for some basic background on seamless switching schemes.

Non-Patent Document 1 has proposed a method for seamlessly switching between video bit streams. This proposal has some obvious drawbacks.

As an improvement, Non-Patent Document 2 proposed an alternative method for seamlessly switching between video bit streams. The method proposed in Non-Patent Document 2 is
Non-Patent Document 1 presents some advantages that do not exist. First, the method proposed in Non-Patent Document 2 simplifies implementation. The structure of the encoder and decoder is normal
It is similar for P-pictures, so many common modules can be shared with P-pictures. Each DCT
Modules are tightly coupled with quantization. Each IDCT module is tightly coupled with dequantization. In this case, the integrated DCT / quantization module and dequantization / IDCT module can be easily applied. Secondly, the method proposed in Non-Patent Document 2 improves coding efficiency. This scheme reduces the number of quantization modules in the coding path and thus reduces quantization error. With this scheme, the mismatch between the prediction criterion and the reconstruction criterion can be eliminated and thus the quality of the reconstructed image can be improved. The final, but perhaps more important, advantage of this scheme that distinguishes it from other schemes is the ability to output a high quality display image prior to quantization in the reconstruction loop. This significantly improves the video quality when many switching points are inserted in the video bitstream.

However, the method proposed in Non-Patent Document 2 also has drawbacks. First, this scheme requires two bitstreams to complete the seamless switching process. The decoder needs to capture both the regular bitstream and the delta bitstream of the current frame. A delta bitstream is one that encodes the difference in each quantization criterion for the next frame and reconstructs exactly the same as the reference picture in the other bitstream. This can cause certain problems with decoder complexity. Furthermore, the total number of bits used for switching is the number of bits in these two combined bitstreams. This number is bounded by the number of bits in the current frame of the original bitstream, which is generally expected to be much larger than this lower bound. Second, the same Qs
Is used for both sequences, so the same delta bitstream is used for both the upper and lower switches. Recalling that two bitstreams are needed for switching, such a scheme
Contradicts the TCP friendly transport protocol, which requires shorter bitstreams for down-switching. In addition, the same Qs hinders optimizing the coding efficiency of a normal bitstream separately.
Third, it is not possible to have more switching points for down-switching, which is another desirable feature that is also needed for TCP-friendly protocols, since the up-switching points and down-switching points combine in the same frame. .

On the other hand, in Non-Patent Document 3, Qs is separated for the upper switching and the lower switching, the upper switching point and the lower switching point are separated, and an unnecessary quantization process is removed. A new technique was proposed to improve the SP scheme proposed in 1. Finally, a more efficient and flexible seamless switching scheme is designed.

By combining these proposals in Non-Patent Document 2 and Non-Patent Document 3, all the advantages of the methods proposed in Non-Patent Document 1, Non-Patent Document 2 and Non-Patent Document 3 are retained, and disadvantages are It is natural to ask how to get a better integrated method that eliminates everything. Fortunately, we have designed such a "very good" scheme. This scheme will be described in detail below.

FIG. 12 shows a high level diagram of the seamless switching scheme proposed here. Note that this scheme is slightly different than the previous switching scheme. In previous switching schemes, S-pictures must be encoded in both the original and target bitstreams to enable seamless switching. However, in this new proposal and the scheme proposed in Non-Patent Document 3, this requirement is no longer necessary. The switching point frame in the original bitstream is either an S-picture or a P-picture.
It can be encoded as a picture. In general, only the predicted image from the previous frame in the original bitstream is needed to encode the switched bitstream, so the frame located at the switch point in the original bitstream can be of any picture type. Can also be encoded as Moreover, the original bitstream may be from a different video sequence. However, in a general case, an extra motion estimation module and motion estimation module are required (FIG. 17).

For comparison, FIGS. 13 and 14 show the normal P picture decoding process and coding process, respectively.

FIG. 15 and FIG. 16 respectively show the decoding process and the coding process for the switching frame proposed here.

FIG. 17 shows a general encoder using the switching method proposed here.

The modules and notations used in FIGS. 15 to 17 are given in Non-Patent Document 3. In addition, S
A frame is defined as a frame encoded in the bitstream as follows: What is that frame?
It is a frame in which another bitstream can be switched there using the scheme proposed here (S2 in FIG. 12). In addition, SP frame, transition bit stream
It is defined as a frame for encoding (switching bitstream) (S12 in FIG. 12). A normal bitstream is a bitstream that is sent normally without switching.

It can be seen that the difference between S ₁ / S ₂ (S) frame coding and P picture coding is only the module in the small box with dashed lines, compared to the coding and decoding of P frames. S frame coding shares the same signal flow as during P frame coding. Therefore, the implementation of S frame encoding and decoding can be greatly simplified. Figure 1
5 and the dashed line with arrows in FIG. 16 shows an alternative path when the decoded / coded frame is a P frame. In the discussion below, these lines can be safely ignored.

The following description applies to inter macroblocks in a frame. For intra macroblocks, a simple "copy" operation can be used.

The decoding process will be described with reference to FIG. i. S frame S1 or S2 in a normal bitstream
(Using S1 as an example) 1) After entropy decoding of the bitstream S1, generate a prediction error coefficient level L _err1 and a motion vector for the _macroblock . Level L _{err 1} is dequantized using the dequantizer QP ₁ ^-1. K _qerr1 = QP ₁ ⁻¹ (L _err1 ). Inverse DCT transform is performed on K _qerr1 , and prediction error I _qerr1
To get 2) Perform motion compensation to obtain a prediction macroblock I _pred1 . The error image I _qerr1 is added to I _pred1 to obtain the reconstructed macroblock. I _rec1 = I _pred1 + I _{qerr1 The} reconstructed macroblock I _rec1 is passed through an optional _deblocking filter to smooth out certain uneven artifacts and output for display. 3) _Perform forward DCT transformation on I _rec1 to obtain the reconstructed coefficient K _rec1 . The reconstructed coefficient K _rec1 is quantized by Qs ₁ to obtain the reconstructed level L _rec1 . L _rec1 = Qs ₁ (K _rec1 ) 4) _Dequantize the level L _rec1 using Qs ₁ ^-1 , and inverse
A DCT transform is performed to obtain a quantized reconstructed image I _qrec1 . 5) The post-reconstruction image I _qrec1 passes through the loop filter to smooth out certain irregular artifacts and outputs it to the frame buffer for the next frame decoding.

If steps 3 and 4 are omitted, S (S
1 or S2) Frame decoding is exactly the same as P frame decoding. QP (QP ₁ or QP ₂ ) and Qs (Qs ₁ or Qs ₂ )
Information is encoded in the S (S1 or S2) bitstream.

Ii. Decoding the switching bitstream S12
(Switching from bitstream 1 to bitstream 2) -SP frame 1) After entropy decoding of bitstream S12,
For the macroblock, generate the error coefficient level L _err12 and the motion vector. 2) Perform motion compensation to obtain a prediction macroblock I _pred1 . _Performs forward DCT conversion on I _pred1 to _obtain prediction coefficients
Get K _{pred 1} . The prediction coefficient K _pred1 is quantized with Qs ₂ to obtain the prediction coefficient level L _pred12 . L _pred12 = Qs ₂ (K _pred1 ) 3) _Add level L _err12 to L _pred1 and reconstruct level L
_rec2 = L _pred1 + L _{er r12} is obtained. 4) Qs ₂ reconstructed after level L _rec2 using ^-1 and dequantized to obtain a reconstructed after image I _Qrec2 by performing an inverse DCT transformation. The reconstructed image I _qrec2 will be passed through a loop filter to smooth out certain uneven artifacts, output it to the display, and output it to the frame buffer for the next frame decoding. The picture obtained is exactly the same as the one decoded from S2. Thus, a drift-free switch from bitstream 1 to bitstream 2 is achieved. Qs ₂ is encoded in the S12 bitstream. S
Twelve decodings include many of the same modules as P frame decodings. Therefore, the module in P-frame decoding can be reused for S12 bitstream decoding.

The encoding process will be described with reference to FIG. 16 i. S frame S1 or S2 in a normal bitstream
Encoding (using S1 as an example) 1) Obtaining a macroblock from the original video image,
It is referred to as I _org1 . 2) Perform motion compensation to obtain a prediction macroblock I _pred1 . Obtaining a prediction error I _err1 by subtracting it from the original macroblocks _I orig1. I _err1 = I _orig1 -I _pred1 3) Prediction error DCT conversion is performed on I _err1 and the obtained coefficient is represented as K _err1 . Quantize K _err1 using QP ₁ to obtain error level L _err1 . L _err1 = QP ₁ (K _err1 ) 4) Entropy coding is performed on the error level L _err1 to obtain the bit stream S1. 5) Level L using the same S1 decoder described above
and _rec1, to reconstruct a reference for the next frame encoding (in this case, during reconstruction loop it should be noted that there are quantizers Qs ₁ and dequantizer Qs ₁ ^-1).
Again, it should be noted that this diagram is exactly the same as the P-frame encoder, except for the decoder part which pointed out the difference in the above description of the decoder.

Ii. Encoding of switching bitstream S12
(Switch from bitstream 1 to bitstream 2) -The encoding of SP frame S12 is bitstream 1
And based on the encoding of 2. When the switching point in the bit stream 1 is the P frame or the S frame, 1) The forward DCT transform is performed on the prediction macroblock I _pred1 by the S1 encoder to obtain the prediction coefficient K _pred1 . 2) quantizing the prediction coefficients K _pred1 using quantizer Qs _2. L _pred12 = Qs ₂ (K _pred1 ) 3) Reconstructed level L _rec2 to L in the S2 encoder
Subtract _pred12 . L _err12 = L _rec2 -L _pred12 4) _Performs entropy coding of L _err12 and the motion vector with the S1 encoder to generate a bitstream S12.

In the more general case, the encoding of the switching bitstream is as follows (FIG. 17). 1) Perform motion estimation and motion compensation on the macroblock with the S2 encoder, referring to the previously reconstructed frame with the S1 encoder, and predict the macroblock I
_{Get pred1} . 2) The forward DCT transform is performed on the prediction macroblock I _pred1 by the S1 encoder to obtain the prediction coefficient K _pred1 . 3) Quantize the prediction coefficient K _pred1 using the quantizer Qs ₂ . L _pred12 = Qs ₂ (K _pred1 ) 4) Reconstructed level L _rec2 to L in the S2 encoder
Subtract _pred12 . L _err12 = L _rec2 −L _pred12 5) _Perform entropy coding of the obtained L _err12 and the motion vector to generate a bitstream S12.

Conclusion A combination of the proposals in Non-Patent Document 2 and Non-Patent Document 3,
We proposed a further improved method for seamless switching of video bitstreams. But this is not trivial integration. This new scheme retains all the advantages of the previous proposals and eliminates all the drawbacks of the previous schemes that we have discovered so far.

1) This method uses a single switched bitstream to switch between different bitstreams.

2) This method uses two separate Qs for the upper and lower switched bitstreams, from the prediction loop and the reconstruction loop in the original bitstream, for switching bitstream coding. Separate Qs.

3) With this method, the contradiction between the reduction of the size of the switching bitstream and the improvement of the coding efficiency of the normal bitstream is resolved. This method
By separately optimizing the different Qs, the size of the switching bitstream can be significantly reduced while maintaining the high coding efficiency of a normal bitstream.

4) This method eliminates unnecessary quantization processes, which improves coding efficiency.

5) This method uses exactly the same criteria for prediction and reconstruction, which further helps improve the coding efficiency of normal bitstreams.

6) This method allows a lower switching point that is much more efficient than an upper switching point. This method allows a lower switching bitstream that is shorter than the upper switching bitstream. This is very suitable for TCP-friendly transport protocols.

7) This method uses the same processing flow as encoding and decoding in P frames, whereby
Many modules in P frame coding can be shared with S frame encoding / decoding.

8) Each DCT module is tightly coupled with quantization. Each IDCT module is tightly coupled with dequantization. In this case, the integrated DCT / quantization module and
The IDCT / dequantization module can be easily applied.

9) With this method, it is possible to output a high quality display image before starting the quantization in the reconstruction loop. This significantly improves the video quality when many switching points are inserted in the video bitstream.

10) This method relaxes the requirement that the switching point in each bitstream should be encoded as an S frame. The original bitstream now has the freedom to be any picture type and / or has freedom from different video sequences.

11) Since the loops of the encoder and the decoder are strictly matched, it is possible to easily remove the quantization premise required in Non-Patent Documents 1 and 2. In theory, it could be any quantizer and even a non-linear quantizer.

In general, the new scheme proposed here presents a more flexible, efficient, simple and natural way to seamlessly switch between video bitstreams.

[0132]

As described above, according to the present invention, it is possible to improve the coding efficiency and reduce the amount of data required to code the switching bit stream.

[Brief description of drawings]

FIG. 1 is a block diagram of an exemplary computing environment suitable for use with the practice of the invention.

FIG. 2 is a diagram illustrating switching between bitstreams according to an embodiment of the present invention.

FIG. 3 is a block diagram showing a conventional decoder.

FIG. 4 is a block diagram showing a conventional encoder.

FIG. 5 is a block diagram illustrating an improved decoder according to an exemplary implementation of the invention.

FIG. 6 is a block diagram illustrating an improved encoder according to an exemplary implementation of the invention.

FIG. 7 is a block diagram illustrating an improved decoder according to another exemplary implementation of the invention.

FIG. 8 is a block diagram illustrating an improved encoder according to another exemplary implementation of the invention.

FIG. 9 is a block diagram illustrating an improved decoder according to yet another example implementation of the invention.

FIG. 10 is a block diagram illustrating an improved decoder according to yet another exemplary implementation of the invention.

FIG. 11 is a block diagram illustrating an improved encoder according to yet another example implementation of the invention.

FIG. 12 is a high level diagram of a seamless switching scheme according to an implementation of the present invention.

FIG. 13 shows a normal P-picture decoding process.

FIG. 14 shows a normal P-picture encoding process.

FIG. 15 illustrates a decoding process for switching frames according to an exemplary implementation of the invention.

FIG. 16 illustrates an encoding process for switching frames according to an exemplary implementation of the invention.

FIG. 17 illustrates a general encoder using a switching scheme according to an exemplary implementation of the invention.

[Explanation of symbols]

130 computers 132 processors 134 System memory 136 bus 138 Read-only memory (ROM) 140 Random access memory (RAM) 142 Basic Input / Output System (BIOS) 144 hard disk drive 146 magnetic disk drive 148 Removable magnetic disk 150 optical disk drive 152 removable optical disk 154 interface 158a, 158b operating system 160a, 160b Application program 162a, 162b Other program modules 164a, 164b Program data 166 keyboard 168 pointing device 170 User input interface 172 monitor 174 video adapter 175 output peripheral interface 177 Local Area Network (LAN) 179 Universal Wide Area Network (WAN) 180 Internet 182 remote computer 186 network interface 189 Remote Application Program 200 switching process 300, 900, 1000 decoders 400, 1100 encoder 500, 700 decryption process 600, 800 encoding process 902 Box 1102 switch

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Ufen             People's Republic of China 100080 Peking Heide             Ian District Zizin Faze             (No address) (72) Inventor San Xiao Yang             People's Republic of China 100080 Peking Heide             Ian District Ji Chung Lo             Do Number 49 Pekin Sigma Center             -Five (72) Inventor Shen Gobin             People's Republic of China 100080 Peking Heide             Ian District Ji Chung Lo             Do Number 49 Pekin Sigma Center             ー 3 F F-term (reference) 5C059 MA23 MC11 ME01 ME11 PP06                       SS16 SS20 TA00 TA29 TA31                       TA45 TB07 TC03 TC04 TC37                       UA02 UA05 UA25 UA33                 5J064 AA00 BA04 BA09 BA16 BC01                       BC02 BC08 BC11 BC16 BC25                       BD02 BD03 BD04

Claims (54)

[Claims] 1. Encoding data into a first bitstream using a first quantization parameter, and using a second quantization parameter different from the first quantization parameter. Encoding the data into a second bitstream, the first to support upward switching between the first bitstream and the second bitstream.
Using the second quantization parameter to support down-switching between the first bitstream and the second bitstream. And a step of generating an encoding switching bitstream associated with the second bitstream. 2. The encoding method according to claim 1, wherein the first quantization parameter and the second quantization parameter are separated. 3. The encoding method of claim 1, wherein the encoded switching bitstream is configured to support a plurality of upper switching periods and a plurality of lower switching periods. 4. The encoding method according to claim 3, wherein the number of the lower switching periods is larger than the number of the upper switching periods over a certain period. 5. The encoding method according to claim 1, wherein the first bit stream and the second bit stream have different data bit rates. 6. Encoding data into a first bitstream using a first quantization parameter, and using a second quantization parameter different from the first quantization parameter. Encoding the data into a second bitstream, the first to support upward switching between the first bitstream and the second bitstream.
Using the second quantization parameter to support down-switching between the first bitstream and the second bitstream. And generating a coded switch bitstream associated with the second bitstream.
A computer-readable recording medium having computer-executable instructions to be executed by one processor. 7. The computer-readable recording medium according to claim 6, wherein the first quantization parameter and the second quantization parameter are separated. 8. The computer-readable recording medium of claim 6, wherein the coded switching bitstream is configured to support a plurality of upper switching periods and a plurality of lower switching periods. 9. The computer-readable recording medium of claim 8, wherein the number of the lower switching periods is greater than the number of the upper switching periods over a period of time. 10. The computer-readable recording medium according to claim 6, wherein the first bitstream and the second bitstream have different data bit rates. 11. Using the first quantization parameter,
A first bitstream encoder configured to encode data into an encoded first bitstream; and using a second quantization parameter different from the first quantization parameter, A second bitstream encoder configured to encode data into an encoded second bitstream; the first bitstream encoder and the second
Operably coupled to the Bitstream encoder of
And upward switching between the first coded bitstream and the second coded bitstream based on information processed using the first quantization parameter and the second quantization parameter. And a switched bitstream encoder configured to output a coded switched bitstream that supports downward switching. 12. The apparatus according to claim 11, wherein the first quantization parameter and the second quantization parameter are separated. 13. The apparatus of claim 11, wherein the encoded switching bitstream is configured to support a plurality of upper switching periods and a plurality of lower switching periods. 14. The apparatus of claim 13, wherein the number of the lower switching periods is greater than the number of the upper switching periods over a period of time. 15. The apparatus of claim 11, wherein the first bitstream and the second bitstream have different data bit rates. 16. A first bitstream generated using a first quantization parameter and a second bitstream generated using a second quantization parameter different from the first quantization parameter. Receiving at least one coded bitstream selected from the group comprising: the first bitstream and the second bitstream; The first to support upward switching to and from the stream
Of the first bitstream generated using the second quantization parameter to support down-switching between the first bitstream and the second bitstream. Receiving a bitstream and a coded switch bitstream associated with the second bitstream, the received coded bitstream using the first quantization parameter and the second quantization parameter And a step of decoding. 17. The decoding method according to claim 16, wherein the first quantization parameter and the second quantization parameter are separated. 18. The step of decoding the received encoded switching bitstream is selected from the group comprising at least one of a plurality of upper switching periods and at least one of a plurality of lower switching periods. 17. Decoding method according to claim 16, characterized in that it takes place during at least one period. 19. The decoding method according to claim 18, wherein the number of the lower switching periods is larger than the number of the upper switching periods over a certain period. 20. The decoding method according to claim 16, wherein the first bit stream and the second bit stream have different data bit rates. 21. A first bitstream generated using a first quantization parameter and a second bitstream generated using a second quantization parameter different from the first quantization parameter. Receiving at least one coded bitstream selected from the group comprising: a first bitstream, and a second bitstream, and decoding the received coded bitstream. The first to support upward switching between
Of the first bitstream generated using the second quantization parameter to support down-switching between the first bitstream and the second bitstream. Receiving a bitstream and a coded switch bitstream associated with the second bitstream, the received coded bitstream using the first quantization parameter and the second quantization parameter A computer-readable recording medium having computer-executable instructions for causing at least one processor to perform an operation including the step of: 22. The computer-readable recording medium according to claim 21, wherein the first quantization parameter and the second quantization parameter are separated. 23. The step of decoding the received encoded switching bitstream is selected from a group comprising at least one of a plurality of upper switching periods and at least one of a plurality of lower switching periods. 22. The computer-readable recording medium according to claim 21, wherein the recording medium is performed during at least one period. 24. The computer-readable recording medium according to claim 23, wherein the number of the lower switching periods is greater than the number of the upper switching periods over a period of time. 25. The computer-readable recording medium of claim 21, wherein the first bitstream and the second bitstream have different data bit rates. 26. A first decoder configured to decode a first encoded bitstream into a decoded first bitstream using a first quantization parameter, the first decoder. A second decoder configured to decode the second bitstream into a decoded second bitstream using a second quantization parameter different from the second quantization parameter of First decoded bitstream operatively coupled to the second decoder and the second decoder and based on information processed using the first quantization parameter and the second quantization parameter. And a second switching bitstream configured to output a decoding switching bitstream supporting upward switching and downward switching between the second decoding bitstream and the second decoding bitstream. Apparatus characterized by comprising a preparative stream decoder. 27. The apparatus of claim 26, wherein the first quantization parameter and the second quantization parameter are separate. 28. Decoding the received coded switching bitstream is selected from a group comprising at least one of a plurality of upper switching periods and at least one of a plurality of lower switching periods. 27. The device of claim 26, wherein the device is performed during at least one period. 29. The apparatus of claim 28, wherein over a period of time, the number of lower switching periods is greater than the number of upper switching periods. 30. The apparatus of claim 26, wherein the first bitstream and the second bitstream have different data bit rates. 31. Reconstructing DCT reference data, and not for decoded DCT residue and DCT prediction data,
Quantizing the post-reconstruction DCT reference data using a quantization step (Qs) for the post-reconstruction DCT reference. 32. The step of quantizing the post-reconstruction DCT reference data is represented by Y = [X * A (Qs) +2 ¹⁹ ] / 2 ²⁰ where X is the post-reconstruction DCT coefficient. Including,
32. Decoding method according to claim 31, characterized in that Y comprises quantized DCT coefficients and A (.) Is associated with a quantization table. 33. Dequantization QP and quantization QS are expressed as follows: _32. Decoding according to claim 31, further comprising the step of merging into one operation represented by, wherein L _err is the level of the prediction error coefficient and K _pred is the prediction coefficient. Method. 34. 34. The decoding method of claim 33, further comprising the step of generating high quality display data based at least in part on the. 35. The decoding method according to claim 31, further comprising the step of selectively reconstructing display images of different qualities. 36. The decoding method of claim 31, further comprising receiving decoder notification data associated with at least one switching event. 37. The decoding method of claim 36, wherein the decoder notification includes at least 1-bit syntax having a switching bitstream flag. 38. The decoding method according to claim 37, wherein the decoder notification data includes at least one quantization parameter. 39. Reconstructing the DCT reference data, and not for the decoded DCT residue and DCT prediction data,
A computer causing at least one processor to perform an operation including: quantizing the reconstructed DCT reference data using a quantization step (Qs) for the reconstructed DCT reference. A computer-readable recording medium having executable instructions. 40. The step of quantizing the post-reconstruction DCT reference data is represented by Y = [X * A (Qs) +2 ¹⁹ ] / 2 ²⁰ where X is the post-reconstruction DCT coefficient. Including,
40. The computer readable recording medium of claim 39, wherein Y comprises quantized DCT coefficients and A (.) Is associated with a quantization table. 41. Dequantization QP and quantization QS With computer-executable instructions that cause the at least one processor to perform a further operation, including merging into one operation represented by, where L _err is the level of the prediction error coefficient and K _pred is The computer-readable recording medium according to claim 39, which is a prediction coefficient. 42. 42. The computer-readable instructions of claim 41 having computer-executable instructions for causing the at least one processor to perform further operations including generating data for high quality display based at least in part on. Recording medium. 43. The computer read of claim 39 having computer-executable instructions for causing the at least one processor to perform further operations including the step of selectively reconstructing display images of different qualities. Possible recording medium. 44. The computer of claim 39, having computer-executable instructions that cause the at least one processor to perform further operations including receiving decoder notification data associated with at least one switching event. A readable recording medium. 45. The computer-readable recording medium of claim 44, wherein the decoder notification includes at least 1-bit syntax having a switching bitstream flag. 46. The computer-readable recording medium according to claim 45, wherein the decoder notification data includes at least one quantization parameter. 47. Reconstructing the DCT reference data and using the quantization step (Qs) for the post-reconstruction DCT reference rather than for the decoded DCT residue and DCT prediction data, the post-reconstruction DCT reference. A decoder comprising logic operably configured to quantize data. 48. The logic quantizes the reconstructed DCT reference data as Y = [X * A (Qs) +2 ¹⁹ ] / 2 ²⁰ where X is the reconstructed DCT coefficient. Including,
The decoder of claim 47, wherein Y comprises quantized DCT coefficients and A (.) Is associated with a quantization table. 49. The logic comprises dequantization QP and quantization QS _48. The decoder of claim 47, merged into one operation represented by, wherein _Lerr is the level of the prediction error coefficient and _Kpred is the prediction coefficient. 50. The logic is: 50. The decoder of claim 49, further configured to generate data for high quality display based at least in part on. 51. The decoder of claim 47, wherein the logic is further configured to selectively reconstruct different quality display images. 52. The decoder of claim 47, wherein the logic is further configured to receive decoder notification data associated with at least one switching event. 53. The decoder according to claim 52, wherein the decoder notification data includes at least 1-bit syntax having a switching bitstream flag. 54. The decoder of claim 53, wherein the decoder notification data includes at least one quantization parameter.