US20030007563A1 - Video signal coding method - Google Patents
Video signal coding method Download PDFInfo
- Publication number
- US20030007563A1 US20030007563A1 US09/179,861 US17986198A US2003007563A1 US 20030007563 A1 US20030007563 A1 US 20030007563A1 US 17986198 A US17986198 A US 17986198A US 2003007563 A1 US2003007563 A1 US 2003007563A1
- Authority
- US
- United States
- Prior art keywords
- video signal
- coding method
- signal coding
- buffer
- decision curve
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/115—Selection of the code volume for a coding unit prior to coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to a video signal coding method, and more particularly, to a video signal coding method which is able to select the optimal modes for macroblocks when a block based video signal is coded.
- MPEG-1 and H.261 which are the standard models in video coding deal with only quantization parameter based on the buffer fullness
- approaches proposed for MPEG-1 and H.261 use previous bit count values as predicted values of bit counts for current macroblock or sub-group-of-block, and each quantization level is controlled accordingly.
- Another approach uses an approximate value to predict the number of bit for the current macroblock from training sequences under the stationary supposition, controlling each quantization level.
- Another technique makes a formula of rate-constrained product code, to optimize the combination of quantization selection items.
- quantization parameters are determined based on the buffer fullness.
- the modes for macroblocks and quantizers are determined and macroblocks are coded accordingly through a series of decision procedures.
- the video coder transmits the video signals
- most of the conventional approaches for bitrate control focused on the decision step of determining whether the quantizer scale is satisfactory or required to be changed.
- the quantizer level is adjusted based on the buffer fullness.
- Proposed algorithm extends the coding decision options for rate control to motion/no-motion compensation as well as inter/intra decisions.
- Output bitrate coded is sensitive to the shapes of decision curves. Accordingly, when the bitrate is controlled using the quantizer, quantization error directly affects visual quality, producing various problems.
- the present invention is directed to a video signal coding method that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
- An object of the present invention is to provide a video signal coding method which finds proper decision curves according to characteristics of input frames and encodes the optimal macroblock by using the decision curves instead of a fixed motion/no-motion compensation curve and intra/inter coding curve.
- the optimal mode is selected for each macroblock of input frame when video signals are coded in a video coder.
- the optimal mode is determined through a step of judging whether the input frame is intra mode using a given function, a step of judging whether the input frame is inter mode when it is not intra mode using a given function, a step of controlling quantizer using a predetermined critical value when it is not inter mode, and step of performing skip when the quantizer controlling step is not carried out.
- the present invention it is possible to select the optimal modes to control bitrate so that a video coder whose visual quality variation is smaller compared with the conventional bitrate control by simple quantizer.
- the present invention presents an effective algorithm for selecting the optimal modes for macroblocks. Accordingly, the method proposed by the present invention is different from the previous ones in that it does not manipulate quantizer to meet target bitrate. Instead it finds the optimal macroblock modes which extract consistent visual quality to meet the target bitrate.
- FIG. 1 is a block diagram of a conventional AMS video coder
- FIG. 2 is a simplified structure of the video coder of FIG. 1;
- FIG. 3 is a decision curve which divides the region into A and B for motion and no-motion compensation
- FIG. 4 shows predetermined bounds to alert when buffer reaches underflow or overflow
- FIGS. 5A and 5B are graphs showing PSNR and buffer fullness for various forms of ⁇ ;
- FIG. 6 is a flow diagram showing AMS algorithm according to the present invention.
- FIGS. 7A and 7B are graphs showing the relationship between PSNR and decision curve orders (from 2 to 5) of motion/no-motion compensation;
- FIGS. 8A and 8B are graphs showing the relationship between PSNR and frame encoding rates in the case of encoding with HAMS and AMS in Claire;
- FIGS. 9A and 9B are graphs showing the relationship between PSNR and frame encoding rates in the case of encoding with HAMS in Claire;
- FIGS. 10A and 10B are graphs showing PSNR, buffer contents and effect of buffer overflow when frames are coded with 160 bps at 30 fps;
- FIGS. 11A and 11B are graphs showing decision curves for motion/no-motion compensation and intra/inter frame coding
- FIGS. 12A to 12 D are graphs showing the relationship between PSNR and encoded frame rates in the case of encoding with 160 kbps and 179 kbps using HAMS in Claire;
- FIGS. 13A to 13 D are graphs showing the relationship between PSNR and frame rates in the case of encoding with 128 kbps and 192 kbps using HAMS in Miss America sequences in CIF;
- FIGS. 14A to 14 D show PSNR, relation with frame rate, buffer fullness, and relation with MQIANT in the case of encoding with 339 kbps using HAMS in Salesperson sequences in CIF;
- FIGS. 15A to 15 D show PSNR, relation with frame rate, relation with buffer fullness, and relation with MQIANT in the case of encoding with 352 kbps using HAMS in Salesperson sequences in CIF.
- FIG. 1 is a block diagram of a conventional adaptive mode selection (AMS) video coder.
- the AMS video coder consists of a mode controller 100 and encoder 200 .
- Mode controller 100 includes a mode selector 11 for determining coding modes for macroblocks of input frame, a buffer 12 for storing video signals outputted from mode selector 11 to observe if overflow or underflow occurs, determining whether to skip corresponding block, and an annealing optimization module 13 for obtaining the optimal decision curve coefficient and ⁇ from the output of the buffer using a given mode.
- Encoder 200 includes a discrete cosine transform (DCT) module 21 for transforming a spacial domain value into a transmission domain value, a quantizer 22 for dividing the output of DCT module 21 into various levels to specify them as predetermined values, a variable length coder (VLC) 23 for coding the quantized values to meet a predetermined mode, a buffer 24 for observing the amount of signal outputted from VLC 23 to determine whether to transmit it or to control the quantization level and code it, an inverse quantizer 25 for inverse-quantizing signals outputted from quantizer 22 to predict it, an inverse discrete cosine transform (IDCT) module 26 for transforming a transmission domain value into a spatial domain value to predict signals outputted from inverse quantizer 25 , and a predictive module 27 for predicting the motion of signal outputted from IDCT module 26 to carry out motion compensation when the frame mode is inter mode.
- DCT discrete cosine transform
- VLC variable length coder
- IDCT inverse discrete cosine transform
- the present invention presents an effective algorithm for selecting the optimal modes for macroblocks in a block based video codec. These modes are selected by optimal decision curves which are sequentially determined by the relation of target bitrate and distortion. Characteristic decision curves for frames are applied to all macroblocks. These decision curves are determined by repeatedly comparing the target bitrate which minimizes encoding distortion with output bitrate. This repeated procedure having no relation with the channel buffer fullness ensures that the decision curves are optimal using simulated annealing optimization techniques. Upon determination of the optimal decision curves, the optimal modes for macroblocks are selected based on the decision curves which minimize the overall distortion for a given bitrate. An embodiment of the present invention optimizes two different decision curves, motion/no-motion compensation decision curve and intra/inter coding decision curve.
- F and ⁇ overscore (F) ⁇ can be partitioned into groups of macroblocks as shown in the following expression (1).
- ⁇ overscore (F) ⁇ ( ⁇ overscore (X) ⁇ 0 , ⁇ overscore (X) ⁇ 1 , ⁇ overscore (X) ⁇ 2 , . . . ⁇ overscore (X) ⁇ L ⁇ 1 ) (1)
- each macroblock X i,j in F j can be coded using only one of N possible modes given by the set S shown in the following expression (2).
- the number i and j represent the index of macroblock and frame, respectively.
- y 1,j be data of a macroblock to be coded, generated by processing of a selected mode M k i and ⁇ overscore (X) ⁇ i,j be the output after decoding of corresponding block.
- y i,j can be represented as follows depending on which mode it is assigned with.
- ,j ⁇ 1 represents a ith macroblock of (j ⁇ 1)th frame with motion vector ⁇ ,
- ⁇ x,y is the result of motion compensation algorithm.
- the equation (5) determines whether the macroblock is encoded using motion or no-motion compensation through a curve which divides the first quarter of a plane into two regions.
- FIG. 3 shows an example of decision curve which divides the region into A and B for motion and no-motion compensations.
- 3 is determined by the sum of absolute differences between pixel values in X i,j and X i,j ⁇ 1 blocks for a given i and j as follows x ⁇ ( X ) ⁇
- Variable y is defined as the sum of absolute differences between pixel values of X i,j and X i
- the motion vector A is determined through the curve g(X) as follows.
- the optimal solution may not exist when ⁇ >0 depending on R-D characteristic.
- the solution of the constraint problem is identical to that corresponding unconstraint problem.
- a k * becomes the solution for the constraint problem when R F j (a k *) is R for a given ⁇ .
- ⁇ can be obtained when R is given and the unconstraint problem can be solved accordingly, obtaining desired solution a k * which meets R F j ⁇ R.
- R can be obtained from the point on R-D curve whose slope is ⁇ .
- it is difficult to actually obtain the R-D curve for various decision curve parameters in a case that the quantizer takes a constant).
- N ⁇ M dimensional searching is needed for data point of R-D curve where the numbers of coefficients for two decision curves (for example, motion/no-motion compensation curve and intra/non-intra coding decision curve) are N and M.
- the values of ⁇ between 0 (minimum distortion, maximum rate) to infinity (maximum distortion, minimum rate) are searched and the optimal coefficient a k * is obtained.
- bitrate R F j (a k *) is calculated to confirm whether to approach a desired bitrate.
- the optimal decision curves are defined depending on parameter a k used during the computation.
- D and D ⁇ 1 denote DCT and IDCT
- Q and Q ⁇ 1 denote quantization and inverse quantization, respectively.
- equations (14), (6) and (7) are replaced by the following equations (15) and (16).
- x ⁇ ( X ) var ⁇ ( X i , j , X i , j - 1 ) ⁇ i , j ⁇ 1 256 ⁇ ⁇ m ⁇ ⁇ n ⁇ ( X i , j ⁇ ( m , n ) - X i , j - 1 ⁇ ( m , n ) ) 2 ( 15 )
- y ⁇ ( X ) var ⁇ ( X i , j , X i
- the Lagrangian multiplier of unconstraint function must be controlled to generate decision curves in order to create proper modes to satisfy the bit count value of current frame.
- the content of buffer is predicted using the bit count of previously coded frames according to input encoding parameters such as frame rate, channel rate and buffer size.
- Buffer control is required to adjust the average bitrate for desired one.
- the state of buffer is retransmitted to the encoder which selects quantizers to avoid the buffer being overflowed or underflowed.
- the conventional bitrate controlling method using mapping method between quantizers and buffer record does not produce satisfactory encoding result because the number of coded bits generated from frames (I, P and B) of various modes varies widely. Accordingly, there will be explained a method of setting the state of alerting overflow and underflow of the Ad buffer according to an embodiment of the present invention.
- R f and R c denote frame rate (fps) and channel rate (bps)
- B denote buffer size in k msec (kRc) of the channel rate, respectively.
- the upper part and lower part of equation (18) represent the state of buffer just after the nth frame is temporarily stored in the buffer and coded, and just before the (n+1)th frame is temporarily stored in the buffer, respectively.
- the buffer level, B L is controlled as follows
- the buffer can avoid overflow or underflow if terms in equation (18) meets equation (19). Accordingly, the following equation (20) is possible.
- FIG. 4 shows an example of the bounds, upper and lower bounds.
- Bitrate control according to quality coefficient ⁇ is explained below.
- the bitrate in the approach proposed by the present invention is controlled by quality coefficient ⁇ not by conventional quantizer, and Lagrangian multiplier is adjusted depending on the state of buffer.
- Lagrangian multiplier ⁇ in equation (14) is decreased to provide distortion constraint rather than bitrate constraint.
- ⁇ B, ⁇ is increased to allow the modes of macroblocks to use skipping mode.
- FIGS. 5A and 5B show Power Signal-to-Noise Ratio (PSNR) and buffer fullness for various forms of ⁇ . As shown in FIGS. 5A and 5B, the small increment in ⁇ reduces PSNR and buffer state. The decrement in ⁇ increases the buffer state but decreases PSNR before overflow occurs. Accordingly, coded bitrate can be controlled by adjusting ⁇ according to the buffer state not to allow overflow or underflow to occur.
- PSNR Power Signal-to-Noise Ratio
- Equation (22) means that the increment is small in ⁇ as long as the difference between a desired bitrate and coded bitrate of the jth frame is large, and the increment becomes large when the difference is small.
- ⁇ can be changed by decreasing its value as follows.
- FIG. 6 is a flow diagram showing AMS algorithm according to the present invention.
- mode controller 100 of the present invention judges if the input frame is new (S 1 ). Coefficient a, of decision curve is initialized (S 2 ) and then it is replaced by new value (S 3 ) when the input frame is new, and the coefficient is replaced without initialization when the input frame is not new one. Subsequently, it is judged if it is skip mode transmission according to alerting of a predetermined maximum value (overflow) or minimum value (underflow) (S 4 ). When it is not skip mode transmission in step S 4 , the procedure goes to group of next block (S 5 ). Here, GOB consists of thirty-three macroblocks. Then, the mode of macroblock is determined (S 6 ). Thereafter, it is determined whether the buffer level is within overflow level (S 7 ).
- step S 4 when it is skip mode transmission in step S 4 or buffer level is within overflow level in step S 7 , they are sequentially compared with predetermined critical values (S 8 , S 9 ), and skip mode is determined if they all correspond to the values (S 11 ).
- Virtual coding is performed (S 12 ) when the buffer level is not within overflow level in step S 7 or after determination of skip mode in step S 11 .
- S 12 it is judged that if the buffer level is in underflow state (S 12 ), and dummy bit is inserted when it is (S 13 ).
- S 14 When the buffer level is not in underflow state in step S 12 , it is judged that if it is the final GOB (S 14 ), feeding back to step S 5 when it is not the final GOB.
- step S 18 when the procedures are performed in step S 18 , the group of optimal blocks has been found and thus actual coding is carried out using those a k ad ⁇ (S 20 ). Otherwise, when the group of optimal blocks are found in step S 15 , it is judged if it is the final frame (S 21 ), and when it is not the procedure goes to the next frame, initializing the value of group of optimal blocks and a k (S 22 ) When it is the final frame in step S 21 , the procedure is completed.
- FIGS. 7A and 7B are graphs showing the relationship between PSNR and decision curve order (from 2 to 5) of motion/no-motion compensation.
- the frame rate is held constant at 30 fps and the average bitrate varies ranging from 128 kbps to 352 kbps.
- the modes are selected using the procedure described above, that is, process of obtaining the optimal motion/no-motion compensation coding curve and then obtaining the optimal intra/inter coding curve.
- the encoding results are compared with coded sequences generated by the video codec test model RM 8.
- FIGS. 8A and 8B are graphs which show the relationship between PSNR and encoding rates in the case of encoding with Hierarchical Search Based Adaptive Mode Selection (HAMS) and AMS in Claire.
- HAMS Hierarchical Search Based Adaptive Mode Selection
- the above optimal orders of polynomials are shown in FIGS. 7A and 7B.
- Hierarchical search motion estimation is employed in order to reduce the complexity in obtaining of motion vectors and increase encoding performance.
- Motion vectors affect the modes of macroblocks according to hierarchical search motion estimation.
- the modes obtained using HAMS are different from modes selected according to adaptive mode selection. This is because of absolute difference or deviation between corresponding macroblock and current macroblock due to motions of motion vectors, different from each other, from the previous frames.
- FIGS. 9A and 9B show that HAMS approach is performed better than AMS approach.
- FIGS. 9A and 9B are graphs showing the relationship between PSNR and encoding rates in the case of encoding with HAMS in Claire.
- Two different models of finding optimal decision curve using HAMS are compared with RM 8, i.e. finding optimal decision curve for motion/no-motion (HAMS-P) and inter/intra (HAMS-I). Proposed models provide consistent visual quality within and between frames as shown in FIGS. 8A and 8B.
- HAMS-P motion/no-motion
- HAMS-I inter/intra
- FIGS. 10A and 10B show corresponding PSNR, contents of buffer and effect of overflow and underflow of the buffer when frames are coded at 30 fps in 160 bps.
- Proposed approaches HAMS-P and HAMS-I
- RM 8 Proposed approaches
- HAMS-P and HAMS-I mean corresponding output PSNR and frame rates when optimal mode for motion/no-motion decision function is selected and intra/non-intra coding function is selected, respectively.
- FIGS. 10A and 10B are graphs showing the relationship between PSNR and buffer fullness in the case of encoding with HAMS in Claire. Referring to FIGS.
- the buffer begins to fill significantly after frame 13 which forces RM 8 to repeat the previous frames (shown in FIG. 10B).
- the buffer does not reach overflow state as macroblock modes are selected based on the optimal decision curves in all two models.
- visual quality of proposed approach until frame 13 before the buffer reaches overflow is better than a fixed model.
- proposed approach and visual quality of RM8 are similar to those in overflow state because the smallest quantization parameters are used.
- FIGS. 11A and 11B show motion/no-motion compensation curves and intra/inter coding decision curves. It can be seen in FIG. 11A that if the error magnitude is low (about 2.5 in HAMS case compared to 1.0 in FIGS. 11A and 11B) then no motion compensation is used. Obviously HAMS favors zero motion compensation and this can be explained by Lagrangian multiplier in the cost function. Intra/inter decision curves using HAMS favor intra frame encoding as shown in FIG. 11B. This explains why the optimal order of the polynomial in intra/inter decision curve is 2.
- FIGS. 11A and 11B are graphs showing motion/no-motion compensation and intra/inter coding decision curves which are found in the case of encoding with HAMS
- FIGS. 12A to 12 D are graphs showing the relationship between PSNR and encoded frame rates in the case of encoding using HAMS in Claire in 160 kbps and 179 kbps.
- FIGS. 12 to 15 show the results of lots of simulations using other bitrates and various sequences. It should be noted that difference in quality recognizable for conventional fixed encoding is quite large if buffer overflow occurs when bitrate is especially low due to repetition of frame. The visual quality is improved by 10 dB at lower bitrate when the proposed approach is used (shown in FIGS. 13A to 13 D).
- FIGS. 13A to 13 D are graphs showing the relationship between PSNR and frame rates in the case of coding using HAMS in 128 kbps and 192 kbps in Miss America sequences
- FIGS. 14A, 14B, 14 C and 14 D show PSNR, relation with frame rate, buffer fullness and relation with MQUANT in the case of encoding using HAMS in 339 kbps in Salesperson sequences, respectively
- FIGS. 15A, 15B, 15 C and 15 D show PSNR, relation with frame rate, buffer fullness and relation with MQUANT in the case of encoding using HAMS in 352 kbps in Salesperson sequences, respectively.
- the proposed method can obtain higher PSNR and better visual quality compared with standard quantizer feedback based bitrate control approach. Furthermore, the present invention can be applied to video compressor or video conference apparatus or communication terminal, to select optimal modes. Accordingly, it is possible to construct the video coder whose visual quality variation is smaller than that obtained in case of controlling by the conventional simple quantizer.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- 1. Field of the Invention
- The present invention relates to a video signal coding method, and more particularly, to a video signal coding method which is able to select the optimal modes for macroblocks when a block based video signal is coded.
- 2. Discussion of Related Art
- In transmitting of video signals outputted from a video coding application program through public switched telephone network (PST), it is preferable that a fixed bitrate is maintained for simple configuration or fixed band width of the network. Accordingly, in order to transmit the video signals in a fixed bitrate, variation in bitrate is reduced using a buffer placed between the output terminal of the encoder and channel. If the encoder controls the bitrate in transmission of video signals, the output bitrate of the encoder can be uniformly maintained to correspond to variable coding quality. However, to obtain the optimal bitrate requires simultaneous using of the two methods, that is, employing of buffer and controlling of the bitrate by the encoder. For this, there have been proposed various techniques which are compatible with the standard decoder and maximize the visual quality of channel bitrate set.
- MPEG-1 and H.261 which are the standard models in video coding deal with only quantization parameter based on the buffer fullness, approaches proposed for MPEG-1 and H.261 use previous bit count values as predicted values of bit counts for current macroblock or sub-group-of-block, and each quantization level is controlled accordingly. Another approach uses an approximate value to predict the number of bit for the current macroblock from training sequences under the stationary supposition, controlling each quantization level. Another technique makes a formula of rate-constrained product code, to optimize the combination of quantization selection items.
- In general, variation in the output bitrate is decreased and required buffer size is reduced as quantization is frequently controlled. On the other hand, a larger sized buffer is needed when controlling of quantization depends on the time required for extracting the predicted value of the buffer. A modelling technique is widely used when the quantization level is determined based on the buffer fullness. For example, buffer fullness and quantization related analysis models are employed to determine the magnitude of quantization level based on the buffer fullness. In MPEG-1 and H.261, there are fixed functions which determine modes for macroblocks. The encoder has to makes decisions for each macroblock: how to determine the best motion vectors to use, decide whether to code each macroblock as intra or predicted mode, and how to set the quantizer scale.
- As a conventional mode selection method, there has been proposed a simple suboptimal method which performs calculation more easily using a computer. This suboptimal method which makes a series of decision options for bitrate control is carried out through the following steps. First of all, motion compensation or no-motion compensation step is performed, which determines whether motion vector is transmitted or processed as ‘0’. The next step is to determine whether the mode of macroblock is intra or inter mode using motion vector which was found in the motion compensation or no-motion compensation step. In the case of inter coding, a step is implemented, which determines whether residual error is large enough to be coded using discrete cosine transform. The final step is to determine whether the quantizer scale is satisfactory or required to be changed. In each of the steps, functions or rules are used for effective decision. For example, decisions for motion/no-motion compensation and intra/inter coding use fixed functions but decision for code/no-code is determined according to the difference between the magnitudes of error signals.
- Furthermore, quantization parameters are determined based on the buffer fullness. The modes for macroblocks and quantizers are determined and macroblocks are coded accordingly through a series of decision procedures. However, when the video coder transmits the video signals, most of the conventional approaches for bitrate control focused on the decision step of determining whether the quantizer scale is satisfactory or required to be changed. For example, the quantizer level is adjusted based on the buffer fullness. Proposed algorithm extends the coding decision options for rate control to motion/no-motion compensation as well as inter/intra decisions. Output bitrate coded is sensitive to the shapes of decision curves. Accordingly, when the bitrate is controlled using the quantizer, quantization error directly affects visual quality, producing various problems.
- Accordingly, the present invention is directed to a video signal coding method that substantially obviates one or more of the problems due to limitations and disadvantages of the related art.
- An object of the present invention is to provide a video signal coding method which finds proper decision curves according to characteristics of input frames and encodes the optimal macroblock by using the decision curves instead of a fixed motion/no-motion compensation curve and intra/inter coding curve.
- According to an embodiment of the present invention to accomplish the object, the optimal mode is selected for each macroblock of input frame when video signals are coded in a video coder. Preferably, the optimal mode is determined through a step of judging whether the input frame is intra mode using a given function, a step of judging whether the input frame is inter mode when it is not intra mode using a given function, a step of controlling quantizer using a predetermined critical value when it is not inter mode, and step of performing skip when the quantizer controlling step is not carried out.
- According to the present invention, it is possible to select the optimal modes to control bitrate so that a video coder whose visual quality variation is smaller compared with the conventional bitrate control by simple quantizer. In a block based video codec, the present invention presents an effective algorithm for selecting the optimal modes for macroblocks. Accordingly, the method proposed by the present invention is different from the previous ones in that it does not manipulate quantizer to meet target bitrate. Instead it finds the optimal macroblock modes which extract consistent visual quality to meet the target bitrate.
- It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
- The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention:
- In the drawings:
- FIG. 1 is a block diagram of a conventional AMS video coder;
- FIG. 2 is a simplified structure of the video coder of FIG. 1;
- FIG. 3 is a decision curve which divides the region into A and B for motion and no-motion compensation;
- FIG. 4 shows predetermined bounds to alert when buffer reaches underflow or overflow;
- FIGS. 5A and 5B are graphs showing PSNR and buffer fullness for various forms of λ;
- FIG. 6 is a flow diagram showing AMS algorithm according to the present invention;
- FIGS. 7A and 7B are graphs showing the relationship between PSNR and decision curve orders (from 2 to 5) of motion/no-motion compensation;
- FIGS. 8A and 8B are graphs showing the relationship between PSNR and frame encoding rates in the case of encoding with HAMS and AMS in Claire;
- FIGS. 9A and 9B are graphs showing the relationship between PSNR and frame encoding rates in the case of encoding with HAMS in Claire;
- FIGS. 10A and 10B are graphs showing PSNR, buffer contents and effect of buffer overflow when frames are coded with 160 bps at 30 fps;
- FIGS. 11A and 11B are graphs showing decision curves for motion/no-motion compensation and intra/inter frame coding;
- FIGS. 12A to12D are graphs showing the relationship between PSNR and encoded frame rates in the case of encoding with 160 kbps and 179 kbps using HAMS in Claire;
- FIGS. 13A to13D are graphs showing the relationship between PSNR and frame rates in the case of encoding with 128 kbps and 192 kbps using HAMS in Miss America sequences in CIF;
- FIGS. 14A to14D show PSNR, relation with frame rate, buffer fullness, and relation with MQIANT in the case of encoding with 339 kbps using HAMS in Salesperson sequences in CIF; and
- FIGS. 15A to15D show PSNR, relation with frame rate, relation with buffer fullness, and relation with MQIANT in the case of encoding with 352 kbps using HAMS in Salesperson sequences in CIF.
- Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.
- FIG. 1 is a block diagram of a conventional adaptive mode selection (AMS) video coder. Referring to FIG. 1, the AMS video coder consists of a
mode controller 100 andencoder 200.Mode controller 100 includes amode selector 11 for determining coding modes for macroblocks of input frame, abuffer 12 for storing video signals outputted frommode selector 11 to observe if overflow or underflow occurs, determining whether to skip corresponding block, and anannealing optimization module 13 for obtaining the optimal decision curve coefficient and λ from the output of the buffer using a given mode.Encoder 200 includes a discrete cosine transform (DCT)module 21 for transforming a spacial domain value into a transmission domain value, aquantizer 22 for dividing the output ofDCT module 21 into various levels to specify them as predetermined values, a variable length coder (VLC) 23 for coding the quantized values to meet a predetermined mode, abuffer 24 for observing the amount of signal outputted fromVLC 23 to determine whether to transmit it or to control the quantization level and code it, aninverse quantizer 25 for inverse-quantizing signals outputted fromquantizer 22 to predict it, an inverse discrete cosine transform (IDCT)module 26 for transforming a transmission domain value into a spatial domain value to predict signals outputted frominverse quantizer 25, and apredictive module 27 for predicting the motion of signal outputted fromIDCT module 26 to carry out motion compensation when the frame mode is inter mode. - The present invention presents an effective algorithm for selecting the optimal modes for macroblocks in a block based video codec. These modes are selected by optimal decision curves which are sequentially determined by the relation of target bitrate and distortion. Characteristic decision curves for frames are applied to all macroblocks. These decision curves are determined by repeatedly comparing the target bitrate which minimizes encoding distortion with output bitrate. This repeated procedure having no relation with the channel buffer fullness ensures that the decision curves are optimal using simulated annealing optimization techniques. Upon determination of the optimal decision curves, the optimal modes for macroblocks are selected based on the decision curves which minimize the overall distortion for a given bitrate. An embodiment of the present invention optimizes two different decision curves, motion/no-motion compensation decision curve and intra/inter coding decision curve.
- When let F and {overscore (F)} be input and reconstructed frames of the video codec, F and {overscore (F)} can be partitioned into groups of macroblocks as shown in the following expression (1).
- F=(X 0 , X 1 , X 2 , . . . X L−1)
- {overscore (F)}=({overscore (X)} 0 , {overscore (X)} 1 , {overscore (X)} 2 , . . . {overscore (X)} L−1) (1)
- In this expression, each macroblock Xi,j in Fj can be coded using only one of N possible modes given by the set S shown in the following expression (2).
- S={M o , M 1 , M 2 , . . . , M N−1} (2)
- Let Mk iεS, where k=0,1, . . . , N−1, be the mode selected to code a macroblock Xi,j. The number i and j represent the index of macroblock and frame, respectively. Let y1,j be data of a macroblock to be coded, generated by processing of a selected mode Mk i and {overscore (X)}i,j be the output after decoding of corresponding block. In general, yi,j can be represented as follows depending on which mode it is assigned with.
- y i,j=0, if skipped
- y i,j =X i,j, if intra coded (3)
- y i,j =X i,j −
X i|,j−1, if inter coded - Xi|,j−1 represents a ith macroblock of (j−1)th frame with motion vector Δ, where
- Δ=0, if no MC
- Δ=δx,y, if MC (4)
-
- The equation (5) determines whether the macroblock is encoded using motion or no-motion compensation through a curve which divides the first quarter of a plane into two regions. FIG. 3 shows an example of decision curve which divides the region into A and B for motion and no-motion compensations. The X-axis in FIG. 3 is determined by the sum of absolute differences between pixel values in Xi,j and Xi,j−1 blocks for a given i and j as follows
-
- The motion vector A is determined through the curve g(X) as follows.
- Δ=0, if y≧g(x)
- Δ=δx,y, if y<g(x) (8)
-
-
-
-
- It is clear that the constraint problem of the equation (12) has the same solution as that of the unconstraint problem of the equation (9), and this is possible when R=RF
j (ak*) because λ is selected to satisfy the constraint optimal value. - It should be noted that the optimal solution may not exist when λ>0 depending on R-D characteristic. For all λ which is not negative value, the solution of the constraint problem is identical to that corresponding unconstraint problem. However, it important that ak* becomes the solution for the constraint problem when RF
j (ak*) is R for a given λ. In other words, λ can be obtained when R is given and the unconstraint problem can be solved accordingly, obtaining desired solution ak* which meets RFj ≦R. Here, R can be obtained from the point on R-D curve whose slope is λ. However, it is difficult to actually obtain the R-D curve for various decision curve parameters (in a case that the quantizer takes a constant). That is, it takes quite a long period of time to obtain the R-D curve. N×M dimensional searching is needed for data point of R-D curve where the numbers of coefficients for two decision curves (for example, motion/no-motion compensation curve and intra/non-intra coding decision curve) are N and M. To maximize the computation, the values of λ between 0 (minimum distortion, maximum rate) to infinity (maximum distortion, minimum rate) are searched and the optimal coefficient ak* is obtained. Subsequently, bitrate RFj (ak*) is calculated to confirm whether to approach a desired bitrate. When the bitrate obtained is sufficient approximate value, the optimal decision curves are defined depending on parameter ak used during the computation. - To solve the equation (11), λ as well as ak must be searched to obtain the constraint minimum cost function value. When consistency is maintained in variation in pictures between frames, it is considered that the variation in the value of λ is not severe between the frames. This suggests that the variation in coded bitrate in each frame is small between the frames. Therefore, instead of obtaining λ for each frame, using the previous λ for frames which are sequentially generated reduces the complexity in processing with a to computer.
- Since λ can be analyzed as the quality index of coded frames, it is important to obtain optimal λ for a period starting from encoding of the first frame until picture variation occurs (for example, in the case of cutting of picture). Distortion of the jth frame for macroblocks can be extended as follows
-
-
- These are the unconstraint optimal formulas for the proposed problem, and a stochastic annealing optimization algorithm is employed to find the coefficients ak in equation (5) which minimizes the distortion in equation (14).
- In applications for H.261 encoding, the Lagrangian multiplier of unconstraint function must be controlled to generate decision curves in order to create proper modes to satisfy the bit count value of current frame. The content of buffer is predicted using the bit count of previously coded frames according to input encoding parameters such as frame rate, channel rate and buffer size.
- Buffer control is required to adjust the average bitrate for desired one. In general, the state of buffer is retransmitted to the encoder which selects quantizers to avoid the buffer being overflowed or underflowed. However, the conventional bitrate controlling method using mapping method between quantizers and buffer record does not produce satisfactory encoding result because the number of coded bits generated from frames (I, P and B) of various modes varies widely. Accordingly, there will be explained a method of setting the state of alerting overflow and underflow of the Ad buffer according to an embodiment of the present invention. Let Rf and Rc denote frame rate (fps) and channel rate (bps), and B denote buffer size in k msec (kRc) of the channel rate, respectively. Let input rate to the buffer be represented as the following equation (17).
-
- The upper part and lower part of equation (18) represent the state of buffer just after the nth frame is temporarily stored in the buffer and coded, and just before the (n+1)th frame is temporarily stored in the buffer, respectively. In order to avoid the buffer being overflowed or underflowed, the buffer level, BL, is controlled as follows
- αB ≦B L <βB (19)
- where,
- 60 +β=1, α≧0, α≦β
-
-
- By varying α and β the bounds to alert when buffer reaches underflow or overflow can be preset. FIG. 4 shows an example of the bounds, upper and lower bounds.
- Bitrate control according to quality coefficient λ is explained below. The bitrate in the approach proposed by the present invention is controlled by quality coefficient λ not by conventional quantizer, and Lagrangian multiplier is adjusted depending on the state of buffer. When the content of buffer is in underflow alert state (for example, 0<BL<αB), Lagrangian multiplier λ in equation (14) is decreased to provide distortion constraint rather than bitrate constraint. When the content of buffer reaches upper bound, βB, λ is increased to allow the modes of macroblocks to use skipping mode.
- By increasing λ, algorithm gives more favor to bitrate constraint than distortion constraint. Once the optimal coefficients of the decision curve are found, each macroblock is coded according to the mode based on the decision curve found. FIGS. 5A and 5B show Power Signal-to-Noise Ratio (PSNR) and buffer fullness for various forms of λ. As shown in FIGS. 5A and 5B, the small increment in λ reduces PSNR and buffer state. The decrement in λ increases the buffer state but decreases PSNR before overflow occurs. Accordingly, coded bitrate can be controlled by adjusting λ according to the buffer state not to allow overflow or underflow to occur.
-
- where the increment in λ depends on the coded bitrate of previous frame. Equation (22) means that the increment is small in λ as long as the difference between a desired bitrate and coded bitrate of the jth frame is large, and the increment becomes large when the difference is small. When the content of buffer reaches underflow state, λ can be changed by decreasing its value as follows.
- FIG. 6 is a flow diagram showing AMS algorithm according to the present invention. Referring to FIG. 6, upon inputting of frame,
mode controller 100 of the present invention judges if the input frame is new (S1). Coefficient a, of decision curve is initialized (S2) and then it is replaced by new value (S3) when the input frame is new, and the coefficient is replaced without initialization when the input frame is not new one. Subsequently, it is judged if it is skip mode transmission according to alerting of a predetermined maximum value (overflow) or minimum value (underflow) (S4). When it is not skip mode transmission in step S4, the procedure goes to group of next block (S5). Here, GOB consists of thirty-three macroblocks. Then, the mode of macroblock is determined (S6). Thereafter, it is determined whether the buffer level is within overflow level (S7). - Subsequently, when it is skip mode transmission in step S4 or buffer level is within overflow level in step S7, they are sequentially compared with predetermined critical values (S8, S9), and skip mode is determined if they all correspond to the values (S11). Virtual coding is performed (S12) when the buffer level is not within overflow level in step S7 or after determination of skip mode in step S11. Here, it is judged that if the buffer level is in underflow state (S12), and dummy bit is inserted when it is (S13). When the buffer level is not in underflow state in step S12, it is judged that if it is the final GOB (S14), feeding back to step S5 when it is not the final GOB.
- Otherwise, it is judged if group of optimal blocks is found when it is the final GOB (S15). When the group of optimal blocks is not found, it is judged that if the mean square error obtained using ak given in step S3 is smaller than that for previous ak (S16). If smaller, the smaller value replaces the previous mean square error value (S17). It is judged if the group of optimal blocks are found or a given loop is repeated (S18) when the mean square error obtained using ak given in step S3 is not smaller than that for previous ak, or after execution of step S17. When the group of optimal blocks are not found and given loop is not repeated, new ak is generated (S19). On the other hand, when the procedures are performed in step S18, the group of optimal blocks has been found and thus actual coding is carried out using those ak ad λ (S20). Otherwise, when the group of optimal blocks are found in step S15, it is judged if it is the final frame (S21), and when it is not the procedure goes to the next frame, initializing the value of group of optimal blocks and ak (S22) When it is the final frame in step S21, the procedure is completed.
- FIGS. 7A and 7B are graphs showing the relationship between PSNR and decision curve order (from 2 to 5) of motion/no-motion compensation. Referring to FIGS. 7A and 7B, for testing of the proposed approach, simulations were performed using various sequences in CIF format (352 pixels×240 lines) for different degree of order from P=2 to 9 in equation (5). For this experiment, the frame rate is held constant at 30 fps and the average bitrate varies ranging from 128 kbps to 352 kbps. As a part of the encoding process, the modes are selected using the procedure described above, that is, process of obtaining the optimal motion/no-motion compensation coding curve and then obtaining the optimal intra/inter coding curve. The encoding results are compared with coded sequences generated by the video codec
test model RM 8. - FIGS. 8A and 8B are graphs which show the relationship between PSNR and encoding rates in the case of encoding with Hierarchical Search Based Adaptive Mode Selection (HAMS) and AMS in Claire. Referring to FIGS. 8A and 8B, during the experiment, it has been found that the order of polynomial was optimal when P=3 for motion/no-motion compensation decision curve. In case of order below 3, satisfactory supplementary optimal coefficients were not obtained. PSNR was increased small while the computation increased significantly in case of order above 3. Through the experiment, it is found that the order of polynomial was optimal when P=2 for intra/inter coding decision curve. The above optimal orders of polynomials are shown in FIGS. 7A and 7B. Hierarchical search motion estimation is employed in order to reduce the complexity in obtaining of motion vectors and increase encoding performance. Motion vectors affect the modes of macroblocks according to hierarchical search motion estimation. The modes obtained using HAMS are different from modes selected according to adaptive mode selection. This is because of absolute difference or deviation between corresponding macroblock and current macroblock due to motions of motion vectors, different from each other, from the previous frames. FIGS. 9A and 9B show that HAMS approach is performed better than AMS approach.
- FIGS. 9A and 9B are graphs showing the relationship between PSNR and encoding rates in the case of encoding with HAMS in Claire. Two different models of finding optimal decision curve using HAMS are compared with
RM 8, i.e. finding optimal decision curve for motion/no-motion (HAMS-P) and inter/intra (HAMS-I). Proposed models provide consistent visual quality within and between frames as shown in FIGS. 8A and 8B. In order to check the effect of overflow and underflow of the buffer, relatively fast and slow bitrates are applied while other parameters are identical. - FIGS. 10A and 10B show corresponding PSNR, contents of buffer and effect of overflow and underflow of the buffer when frames are coded at 30 fps in 160 bps. Proposed approaches (HAMS-P and HAMS-I) are compared with
RM 8. HAMS-P and HAMS-I mean corresponding output PSNR and frame rates when optimal mode for motion/no-motion decision function is selected and intra/non-intra coding function is selected, respectively. FIGS. 10A and 10B are graphs showing the relationship between PSNR and buffer fullness in the case of encoding with HAMS in Claire. Referring to FIGS. 10A and 10B, in experiment withRM 8, the buffer begins to fill significantly afterframe 13 which forcesRM 8 to repeat the previous frames (shown in FIG. 10B). However, with the proposed scheme, the buffer does not reach overflow state as macroblock modes are selected based on the optimal decision curves in all two models. Furthermore, visual quality of proposed approach untilframe 13 before the buffer reaches overflow is better than a fixed model. In case of underflow of the buffer, proposed approach and visual quality of RM8 are similar to those in overflow state because the smallest quantization parameters are used. - FIGS. 11A and 11B show motion/no-motion compensation curves and intra/inter coding decision curves. It can be seen in FIG. 11A that if the error magnitude is low (about 2.5 in HAMS case compared to 1.0 in FIGS. 11A and 11B) then no motion compensation is used. Obviously HAMS favors zero motion compensation and this can be explained by Lagrangian multiplier in the cost function. Intra/inter decision curves using HAMS favor intra frame encoding as shown in FIG. 11B. This explains why the optimal order of the polynomial in intra/inter decision curve is 2.
- FIGS. 11A and 11B are graphs showing motion/no-motion compensation and intra/inter coding decision curves which are found in the case of encoding with HAMS, and FIGS. 12A to12D are graphs showing the relationship between PSNR and encoded frame rates in the case of encoding using HAMS in Claire in 160 kbps and 179 kbps. Furthermore, FIGS. 12 to 15 show the results of lots of simulations using other bitrates and various sequences. It should be noted that difference in quality recognizable for conventional fixed encoding is quite large if buffer overflow occurs when bitrate is especially low due to repetition of frame. The visual quality is improved by 10 dB at lower bitrate when the proposed approach is used (shown in FIGS. 13A to 13D).
- Moreover, the proposed approach provides uniform visual quality through frames. FIGS. 13A to13D are graphs showing the relationship between PSNR and frame rates in the case of coding using HAMS in 128 kbps and 192 kbps in Miss America sequences, and FIGS. 14A, 14B, 14C and 14D show PSNR, relation with frame rate, buffer fullness and relation with MQUANT in the case of encoding using HAMS in 339 kbps in Salesperson sequences, respectively. FIGS. 15A, 15B, 15C and 15D show PSNR, relation with frame rate, buffer fullness and relation with MQUANT in the case of encoding using HAMS in 352 kbps in Salesperson sequences, respectively.
- According to the present invention, the proposed method can obtain higher PSNR and better visual quality compared with standard quantizer feedback based bitrate control approach. Furthermore, the present invention can be applied to video compressor or video conference apparatus or communication terminal, to select optimal modes. Accordingly, it is possible to construct the video coder whose visual quality variation is smaller than that obtained in case of controlling by the conventional simple quantizer.
- It will be apparent to those skilled in the art that various modifications and variations can be made in the video signal coding method of the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Claims (17)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/179,861 US6507616B1 (en) | 1998-10-28 | 1998-10-28 | Video signal coding method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/179,861 US6507616B1 (en) | 1998-10-28 | 1998-10-28 | Video signal coding method |
Publications (2)
Publication Number | Publication Date |
---|---|
US20030007563A1 true US20030007563A1 (en) | 2003-01-09 |
US6507616B1 US6507616B1 (en) | 2003-01-14 |
Family
ID=22658294
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/179,861 Expired - Lifetime US6507616B1 (en) | 1998-10-28 | 1998-10-28 | Video signal coding method |
Country Status (1)
Country | Link |
---|---|
US (1) | US6507616B1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030128756A1 (en) * | 2001-12-28 | 2003-07-10 | Nokia Corporation | Method and apparatus for selecting macroblock quantization parameters in a video encoder |
US6904094B1 (en) * | 2000-09-20 | 2005-06-07 | General Instrument Corporation | Processing mode selection for channels in a video multi-processor system |
US20060002478A1 (en) * | 2004-06-30 | 2006-01-05 | Lg Electronics Inc. | Coding image signal |
US20060227867A1 (en) * | 2005-04-12 | 2006-10-12 | Lsi Logic Corporation | Method for coefficient bitdepth limitation, encoder and bitstream generation apparatus |
US20080080616A1 (en) * | 2006-09-29 | 2008-04-03 | Jin Wuk Seok | Apparatus and method for compression-encoding moving picture |
US20100040150A1 (en) * | 2004-12-29 | 2010-02-18 | Koninklijke Philips Electronics, N.V. | Method and apparatus for encoding video data stream |
US20100266045A1 (en) * | 2009-04-17 | 2010-10-21 | Texas Instruments Incorporated | Adaptive real-time video prediction mode method and computer-readable medium and processor for storage and execution thereof |
US20180324427A1 (en) * | 2017-05-04 | 2018-11-08 | Sony Corporation | Quantization partitioning for enhanced image compression |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6735249B1 (en) * | 1999-08-11 | 2004-05-11 | Nokia Corporation | Apparatus, and associated method, for forming a compressed motion vector field utilizing predictive motion coding |
US6639942B1 (en) * | 1999-10-21 | 2003-10-28 | Toshiba America Electronic Components, Inc. | Method and apparatus for estimating and controlling the number of bits |
JP2005535155A (en) * | 2001-12-20 | 2005-11-17 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Coding method for compression of video sequences |
EP1345450A3 (en) * | 2002-01-16 | 2005-04-20 | Matsushita Electric Industrial Co., Ltd. | Image coding apparatus, image coding method, and image coding program |
US7317759B1 (en) * | 2002-02-28 | 2008-01-08 | Carnegie Mellon University | System and methods for video compression mode decisions |
CN1206864C (en) * | 2002-07-22 | 2005-06-15 | 中国科学院计算技术研究所 | Association rate distortion optimized code rate control method and apparatus thereof |
US6931350B2 (en) * | 2003-08-28 | 2005-08-16 | Hewlett-Packard Development Company, L.P. | Regression-clustering for complex real-world data |
US7697608B2 (en) * | 2004-02-03 | 2010-04-13 | Sony Corporation | Scalable MPEG video/macro block rate control |
JP4227067B2 (en) * | 2004-04-28 | 2009-02-18 | 株式会社東芝 | Moving picture coding method, apparatus and program |
WO2006005182A1 (en) | 2004-07-14 | 2006-01-19 | Slipstream Data Inc. | Method, system and computer program product for optimization of data compression |
US7570827B2 (en) | 2004-07-14 | 2009-08-04 | Slipstream Data Inc. | Method, system and computer program product for optimization of data compression with cost function |
CN101023674B (en) * | 2004-09-16 | 2010-08-18 | 汤姆森特许公司 | Method and apparatus for fast mode decision for interframes |
US20070274396A1 (en) * | 2006-05-26 | 2007-11-29 | Ximin Zhang | Complexity adaptive skip mode estimation for video encoding |
US8175150B1 (en) * | 2007-05-18 | 2012-05-08 | Maxim Integrated Products, Inc. | Methods and/or apparatus for implementing rate distortion optimization in video compression |
JP5489675B2 (en) * | 2009-11-27 | 2014-05-14 | 三菱電機株式会社 | Video information playback method and system, and video information content |
US8483272B2 (en) * | 2010-09-24 | 2013-07-09 | Intel Corporation | System and method for frame level bit rate control without pre-analysis |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5485214A (en) * | 1993-02-25 | 1996-01-16 | Industrial Technology Research Institute | Dual bus dual bank architecture for motion compensation |
US5563813A (en) * | 1994-06-01 | 1996-10-08 | Industrial Technology Research Institute | Area/time-efficient motion estimation micro core |
US5608458A (en) * | 1994-10-13 | 1997-03-04 | Lucent Technologies Inc. | Method and apparatus for a region-based approach to coding a sequence of video images |
US6192081B1 (en) * | 1995-10-26 | 2001-02-20 | Sarnoff Corporation | Apparatus and method for selecting a coding mode in a block-based coding system |
US5963673A (en) * | 1995-12-20 | 1999-10-05 | Sanyo Electric Co., Ltd. | Method and apparatus for adaptively selecting a coding mode for video encoding |
US6037987A (en) * | 1997-12-31 | 2000-03-14 | Sarnoff Corporation | Apparatus and method for selecting a rate and distortion based coding mode for a coding system |
-
1998
- 1998-10-28 US US09/179,861 patent/US6507616B1/en not_active Expired - Lifetime
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6904094B1 (en) * | 2000-09-20 | 2005-06-07 | General Instrument Corporation | Processing mode selection for channels in a video multi-processor system |
US20030128756A1 (en) * | 2001-12-28 | 2003-07-10 | Nokia Corporation | Method and apparatus for selecting macroblock quantization parameters in a video encoder |
US6763068B2 (en) * | 2001-12-28 | 2004-07-13 | Nokia Corporation | Method and apparatus for selecting macroblock quantization parameters in a video encoder |
EP1624683A3 (en) * | 2004-06-30 | 2010-06-30 | LG Electronics, Inc. | Method for coding image signals |
EP1624683A2 (en) | 2004-06-30 | 2006-02-08 | LG Electronics, Inc. | Method for coding image signals |
US20060002478A1 (en) * | 2004-06-30 | 2006-01-05 | Lg Electronics Inc. | Coding image signal |
US7801222B2 (en) | 2004-06-30 | 2010-09-21 | Lg Electronics Inc. | Coding image signal |
US20100040150A1 (en) * | 2004-12-29 | 2010-02-18 | Koninklijke Philips Electronics, N.V. | Method and apparatus for encoding video data stream |
US20060227867A1 (en) * | 2005-04-12 | 2006-10-12 | Lsi Logic Corporation | Method for coefficient bitdepth limitation, encoder and bitstream generation apparatus |
US7949044B2 (en) * | 2005-04-12 | 2011-05-24 | Lsi Corporation | Method for coefficient bitdepth limitation, encoder and bitstream generation apparatus |
US20080080616A1 (en) * | 2006-09-29 | 2008-04-03 | Jin Wuk Seok | Apparatus and method for compression-encoding moving picture |
US8194740B2 (en) * | 2006-09-29 | 2012-06-05 | Electronics And Telecommunications Research Institute | Apparatus and method for compression-encoding moving picture |
US20100266045A1 (en) * | 2009-04-17 | 2010-10-21 | Texas Instruments Incorporated | Adaptive real-time video prediction mode method and computer-readable medium and processor for storage and execution thereof |
US8379728B2 (en) * | 2009-04-17 | 2013-02-19 | Texas Instruments Incorporated | Adaptive real-time video prediction mode method and computer-readable medium and processor for storage and execution thereof |
US20180324427A1 (en) * | 2017-05-04 | 2018-11-08 | Sony Corporation | Quantization partitioning for enhanced image compression |
US10750175B2 (en) * | 2017-05-04 | 2020-08-18 | Sony Corporation | Quantization partitioning for enhanced image compression |
Also Published As
Publication number | Publication date |
---|---|
US6507616B1 (en) | 2003-01-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6507616B1 (en) | Video signal coding method | |
US7068718B2 (en) | Advanced method for rate control and apparatus thereof | |
EP0857394B1 (en) | Apparatus and method for optimizing the rate control in a coding system | |
KR100953152B1 (en) | Method and Apparatus for selecting macroblock quantization parameters in a video encoder | |
US6389072B1 (en) | Motion analysis based buffer regulation scheme | |
US7751476B2 (en) | Moving picture coding method and moving picture coding apparatus | |
US6396956B1 (en) | Method and apparatus for selecting image data to skip when encoding digital video | |
US6037987A (en) | Apparatus and method for selecting a rate and distortion based coding mode for a coding system | |
US5796435A (en) | Image coding system with adaptive spatial frequency and quantization step and method thereof | |
EP0804036B1 (en) | Image encoding apparatus | |
US7082163B2 (en) | Picture coding method, picture coding apparatus and image relaying apparatus | |
EP1010140A1 (en) | A perceptually motivated trellis based rate control method and apparatus for low bit rate video coding | |
US20030152151A1 (en) | Rate control method for real-time video communication by using a dynamic rate table | |
US7397855B2 (en) | Rate controlling method and apparatus for use in a transcoder | |
US7965768B2 (en) | Video signal encoding apparatus and computer readable medium with quantization control | |
US6385241B1 (en) | Method and apparatus for video rate control using a radial basis function rate estimator | |
US7106796B2 (en) | Apparatus and method for adaptively controlling bit rate of video transcoder | |
KR100601846B1 (en) | Apparatus and Method for Distortion Optimization of Moving Picture Compaction Encoder | |
WO1997016031A1 (en) | Apparatus and method for selecting a coding mode in a block-based coding system | |
CN1150771C (en) | Method and device for selection of transformation ratio in transformation coding system | |
US20020106021A1 (en) | Method and apparatus for reducing the amount of computation of the video images motion estimation | |
KR0185847B1 (en) | The quantization step control system in video encoder | |
EP1057344A1 (en) | Adaptive buffer and quantizer regulation scheme for bandwidth scalability of video data | |
Lu et al. | Efficient and low-cost video transcoding | |
CN116600122A (en) | Code rate control method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: LG INFORMATION & COMMUNICATIONS, LTD., KOREA, REPU Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RYU, CHUL;REEL/FRAME:009724/0296 Effective date: 19981208 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF Free format text: MERGER;ASSIGNOR:LG INFORMATION & COMMUNICATIONS, LTD.;REEL/FRAME:017155/0480 Effective date: 20000904 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: LG NORTEL CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LG ELECTRONICS INC.;REEL/FRAME:018279/0398 Effective date: 20060710 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: LG-ERICSSON CO., LTD., KOREA, REPUBLIC OF Free format text: CHANGE OF NAME;ASSIGNOR:LG-NORTEL CO., LTD.;REEL/FRAME:025948/0842 Effective date: 20100630 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 12 |