WO2010043140A1

WO2010043140A1 - Video compressing method

Info

Publication number: WO2010043140A1
Application number: PCT/CN2009/073587
Authority: WO
Inventors: 钟似玢; 曾嘉亮
Original assignee: 深圳市融创天下科技发展有限公司
Priority date: 2008-10-15
Filing date: 2009-08-28
Publication date: 2010-04-22
Also published as: CN101729902A; CN101729902B

Abstract

The invention provides a video compressing method, which includes steps as follows: (A) the scene whose changes between different video frames are less than or equal to a predetermined threshold value is defined as a background, while the scene whose changes are more than the predetermined threshold value is defined as a foreground; or the scene whose changes between different video frames are less than a predetermined threshold value is defined as a background, while the scene whose changes are more than or equal to the predetermined threshold value is defined as a foreground; (B) encoding or decoding backgrounds and foregrounds with different reference frames by using multiple reference frames mechanism of H.264/AVS video compressing standard; encoding or decoding backgrounds by using high quality reference frames, and encoding or decoding foregrounds by using normal reference frames.

Description

Instruction manual A video compression method

#细 or

[I] The present invention relates to a video processing method, and more particularly to a video compression method.

[2] Eliminating the inter-turn redundancy in image sequences is the main task of video compression algorithms. Daytime redundancy refers to the correlation between different images of a video sequence. There is generally a strong correlation between the images of the video sequence. Therefore, in video coding and communication, it is not necessary to transmit all the pixel information of each frame to the decoder; in most cases, only the change information between frames is transmitted, and the decoder will according to the already decoded image and the interframe. Change the information to correctly solve the current frame. It can be seen that the key to eliminating inter-turn redundancy is to determine which information does not change between frames and does not need to be transmitted.

[3] The basic idea of judging whether the inter-frame information changes or not is: Whether the pixel values of adjacent frames at the same position are very close. The following examples illustrate:

[4] Let f0 and fl be two adjacent frames, and the f0 frame is already encoded. Then, after encoding the fl frame, first check the change of fl relative to f0. The intensity of fl at the n=(nl,n2) pixel position is fl(n), and the intensity of f0 at n is f0(n), and the difference between them is defined as FD (Frame Difference):

[5] FD=fl(n)— f0(n) (l.l)

[6] If FD 0, it is considered that fl(n) is no different from f0 (n), and fl(n) is not required to be encoded.

[7] However, in practical applications, the decoder end can only obtain the result f0' after f0 has been "encoded to decoded". Since the quantization step of the encoding process is lossy, there is a certain difference between fO'(n) and f0(n), denoted as Q

D (Quantized Difference):

[8] QD=fO'(n)—f0(n) (l.l)

[9] In order to prevent error accumulation, both the editor and the decoder must use f0' as the basis for checking whether fl is changed. In this case, f0' is called the reference frame of fl.

[10] It can be seen that the reference frame refers to the video image that has undergone the "encoding to decoding" process. The codec and decoder must maintain the same reference frame for consistent prediction of subsequent images.

[II] The difference in intensity between fl and f0' at n is DD (Decision Difference), with: [12] DD=fl(n)— fO'(n) (1.3)

[13] Substituting equations (l.l) and (1.2) into equation (1.3) ' get:

[14] DD=FD-QD (1.4)

[15] Equation (1.4) Describes the encoder's judgment based on: If DD«0, there is no need to encode fl(n).

[16] However, QD always increases as the quantization step size increases, causing DD to be 0 when "should" be 0.

[17] The problem is here: Assume fl(n) and f0

(n) There is no difference, ie FD 0, then theoretically there is no need to encode fl(n); but due to the existence of quantization, DD=-QD is not close to 0, the encoder will actually still have fl(n) Encode. As a result, both the code rate overhead is increased and the quality of the decoded image is not improved.

[18] The technical problem to be solved by the present invention is to provide a video compression method with a small code rate overhead in view of the above-mentioned defect that the above-mentioned code rate overhead is large and the quality of the decoded image is not improved.

[19] The technical solution adopted by the present invention to solve the technical problem thereof is: Providing a video compression method, comprising the following steps:

[20] A. A scene in which a change in a different video frame is less than or equal to a predetermined threshold is defined as a background, and a scene in which a change is greater than a predetermined threshold is defined as a foreground; or a scene in which a change in a different video frame is less than a predetermined threshold is defined as Background, wherein a scene whose variation is greater than or equal to a predetermined threshold is defined as a foreground;

[21] B. Using the multi-reference frame mechanism of H.264 or AVS video compression standard, encode and decode different reference frames for background and foreground: encode and decode high-quality reference frames for background, and use common reference for foreground The frame is coded and decoded.

[22] In the video compression method of the present invention, the step B includes the following steps:

[23] B1. Determine the reference frame type with an encoder: a long-term reference frame or a short-term reference frame;

[24] B2. Synchronous decoding operation: notifying the decoder of the determined reference frame type, causing the decoder to reconstruct the reference frame list 0;

[25] B3. The encoder synchronizes the operation of the decoder by transmitting different adaptive memory control commands for the high quality reference frame and the low quality reference frame;

[26] B4. End. [27] In the video compression method of the present invention, in the step B3, the transmitting different adaptive memory control commands are as follows:

[28] For I frames:

[29] If it is a short-term reference, set the value of the long_term_reference_flag flag to 0;

[30] If it is a long-term reference, set the value of the long_term_reference_flag flag to 1;

[31] For P frames:

[32] If it is a short-term reference, set the value of the adaptive_ref_pic_marking_mode_flag flag to 0; [33] If it is a long-term reference, set the value of the adaptive_ref_pic_marking_mode_flag flag to 1.

[34] In the video compression method of the present invention, after the value of the adaptive_ref_pic_marking_mode_fkg flag is set to 1 for the P frame, the following steps are also performed:

[35] 1 Send the value of the memory-management-control-operation flag 6;

[36] 2 send the value of long_term_frame_idx 0;

[37] 3 send 0' to notify the corner coder memory-management-control-operation ends.

[38] In the video compression method of the present invention, the short-term reference frame predicts a subsequent five-frame video frame, and the long-term reference frame predicts a subsequent multi-frame video frame of five or more frames.

[39] The video compression method embodying the present invention has the following beneficial effects: The method is particularly suitable for a video application with a constant camera position. In a video application with a constant camera position, most of the image content belongs to the background, only a small part. Belongs to the future. Although the background frame is a high quality frame and the code stream is larger than the normal frame, since it is mainly used as a reference frame for a still scene, as long as a background frame is encoded in a long period, multiple predicted frames can be made. The background residual code rate is reduced. Therefore, in general, the compression method of the foreground background separation not only does not increase the code rate of the entire video stream due to the high quality of the background frame, but can achieve a reduced code rate through reasonable adjustment, or at the same target. In the case of bit rate, the image quality obtained by this method is higher; and, since only the existing encoder needs to be adjusted correspondingly, the existing decoder can be correctly decoded without any modification. The method compresses the video stream, so the method also has the advantage of being easy to implement.

difficult

[40] The main points of the video compression method of the present invention are: defining a scene with no change in the video as a background, and changing the scene as a foreground, using a multi-reference frame mechanism of H.264 or AVS video compression standard, opposite Scenes and foregrounds are coded with different reference frames—the background uses high-quality reference frames and the foreground uses normal reference frames. The term "quality" here refers to the fidelity of a compressed video image. The higher the fidelity, the closer the compressed image is to the original image, and the more generally the image is of high quality.

[41] Examples are as follows:

[42] Let f0, f7, and f8 be three frames of images with the same background in the same video stream, and f8 be adjacent to each other. The results of f0 and f7 after "encoding to decoding" are f0' and Π', respectively. , f0' quantization step size is small, is a high quality frame, f7' quantization step size is large, is a normal quality frame.

[43] Note that f8 has an intensity at the n=(nl, n2) pixel position of f8(n), f0' has an intensity at n of fO'(n), and Π' intensity at n is f7' (n) ).

[44] From equation (1.4), we can see that the difference between f8(n) and fO'(n) is DD80, and the difference between f7'(n) is DD87.

[45] DD80=FD80-QDO (2.1)

[46] DD87=FD87-QD7 (2.2)

[47] Among them, QD0 and QD7 are the quantization differences of f0' and fl', respectively. FD80 and FD87 are the frame differences between f8(n) and f0' (n) and f7' (n), respectively.

[48] Assuming that the image content at n belongs to the background of three frames, then FD80 =

FD87»0, generation (2.1), (2.2), get:

[49] DD80=-QD0 (2.3)

[50] DD87= -QD7 (2.4)

[51] As mentioned earlier, Π' is a normal quality frame with a large quantization step size. QD7 is also large, so the encoder will still encode the background f8(n) if it is judged according to equation (2.4).

[52] However, since f0' is a high quality frame, its quantization step size can be appropriately selected so that QD0 is small enough; this 吋, by equation (2.3) is: DD80 0. The encoder accordingly concludes that f8(n) is the background and may not be encoded.

[53] The decoder replaces the high quality f0' (n) directly where f8(n) is required.

[54] For video applications where the camera position is unchanged, in most cases, most of the frames belong to the background, and only a small part belongs to the foreground.

[55] Although the background is a high quality frame, the code stream is larger than the normal frame, but since it is mainly used as a reference frame for a still scene, as long as a background frame is encoded in a long period, multiple predictions can be made. Frame background The residual code rate is reduced.

[56] Therefore, in general, the compression method of the foreground background separation, not only does not increase the code rate of the entire video stream due to the high quality of the background frame, but can achieve a reduced code rate by reasonable adjustment, or Improve image quality at the same code rate.

[57] The key to implementing the video compression method of the present invention is that the H.264 and AVS standards support multiple reference frame mechanisms.

The following takes H.264 as an example to introduce the specific implementation steps of the method.

[58] H.264 reference frames are divided into two categories: short-term reference frames and long-term reference frames. The short-term reference frame can only predict the next 5 frames of video frames, and the long-term reference frame can predict the type of multi-frame video reference frames of more than 5 frames, which are set by the encoder and controlled by a set of adaptive memory. The command (Adapti memory control commands) is used to synchronize the reference frame operation of the decoder.

The multi-reference frame management mechanism of H.264 enables the method of the present invention to be implemented very conveniently. The method includes the following steps: A. defining a scene in a different video frame whose variation is less than or equal to a predetermined threshold as a background, and defining a scene whose change is greater than a predetermined threshold as a foreground; or, changing a scene in a different video frame that is less than a predetermined threshold Defined as the background, and the scene with the change greater than or equal to the predetermined threshold is defined as the foreground; B uses the multi-reference frame mechanism of H.264 or AVS video compression standard to encode and decode the background and foreground different reference frames: The codec is decoded with a high quality reference frame, and the foreground is coded and decoded with a normal reference frame. The following describes in more detail how the method is implemented in an H.264 encoder in conjunction with an example of unidirectional inter prediction.

[61] Referring to Figure 1, the unidirectionally predicted video stream contains only I and P frames, and only the reference frame list 0 (referred to as 1 istO) is used. It is assumed that two reference frames are used for predictive coding, and the video streams are I, Pl, P2, P3, ..., P120, where I, P10, and P60 need to be long-term reference frames, and the remaining images are short-term reference frames. In the encoding process, the change of listO with two reference frames is shown in Fig. 1.

[62] In FIG. 1, the symbol (L) indicates that the corresponding frame is set as the long-term reference frame, and (S) indicates that the corresponding frame is the short-term reference frame. After I is programmed, the reference frame list 0, ie, listO's indexO and indexl are set to I (L) and NULL respectively. After P1 is compiled, indexO and indexl are set to PI (S) and I (L), respectively. P2 to P9, indexO are sequentially set to P2 (S) to P9 (S), and indexl is set to I (L); P10 is compiled, indexO and inde xl are set to P9 (S) and P10 (L, respectively). ); after editing Pl l to P59, indexO is set to PI 1 in turn (S) to P59 (S), indexl is set to P10 (L); After finishing P60, indexO and indexl are set to P59 (S) and P60 (L), respectively.

[63] Referring to Figure 2, in the above step B, the following steps are included:

[64] S101: Start.

[65] S102: Determine an reference frame type by using an encoder: a long-term reference frame or a short-term reference frame;

[66] S103: Synchronous decoding operation: Notifying the decoder of the determined reference frame type, so that the encoder reconstructs the reference frame list 0.

[67] S104: The encoder synchronizes the operation of the decoder by transmitting different adaptive memory control commands, as follows:

[68] (1) For I frames:

[69] 1) If it is a short-term reference frame, set the value of the long_term_reference_flag flag to 0;

[70] 2) If it is a long-term reference frame, set the value of the long_term_reference_flag flag to 1.

[71] (2) For P frames:

[72] 1) If it is a short-term reference frame, set the value of the adaptive_ref_pic_marking_mode_flag flag to 0.

[73] 2) If it is a long-term reference frame, set adaptive_ref_pic_marking_mode_flag to 1, which means there is a corresponding memory management control operation (memory_management_control_operation), so the encoder has to perform the following three steps:

[74] 1 sends the value of the memory-management-control-operation flag 6;

[75] 2 sends the value of long_term_frame_idx 0;

[76] 3 sends 0, notifying the decoder that the memory_management_control_operation ends.

[77] S105: End.

[78] Due to space limitations, only the specific steps of implementing the video compression method of the present invention can be described herein. Each of the above-mentioned flags belongs to an adaptive memory control command, and its specific meaning is in the H.264 official standard ((Recommendation H.264: Advanced video). Section 8.2.5 of "coding for generic audiovisual services", "Decoded reference picture marking process" ^ ^ has a detailed description in the text, the implementer can refer to it.

[79] In summary, by setting the background as a long-term reference frame and setting the foreground as a short-term reference frame, you can The video compression method of the present invention is implemented within the framework of the H.264 standard. Obviously, those skilled in the art should understand that the video compression method of the present invention can also be implemented by the above method by using the multi-reference frame mechanism supported by the AVS standard.

[80] The video compression method of the present invention is particularly suitable for video applications where the camera position is unchanged. In such applications, compared with other one-pass encoding schemes, the image quality of the method is performed at the same target bit rate. higher.

[81] In the comparative experiment, the mainstream H.264 standard encoder X264 in the industry was selected to represent the conventional method, and the video compression method described in the present invention was named SWATAW.

[82] Table 1 gives the experimental results: Under different rate control conditions, the typical background-invariant video stream paris (1065 frames) is compressed with X264 and SWATAW, respectively, from the size of the compressed file. Peak Signal-to-Noise Ratio (PSNR) It is not difficult to see that the SWATAW compression ratio is higher after the image quality is similar. For example, the code rate is 20kbps吋, and the file sizes obtained by X264 and SWATAW compression are 84.9 and 84.1 respectively. The code rate is 80 kbps吋, and the file sizes obtained by X264 and SWATAW compression are 4111^8 and 4051^8 respectively. It can be seen that the compression ratio obtained by the SWATAW method is higher. At the same bit rate, SWATAW has higher image quality; and this advantage becomes more pronounced as the target bit rate decreases.

[83] Table 1 X264 I SWATAW algorithm comparison experimental data sheet

[84] Code rate

[85] Frame Rate Calculation

[86] File size after compression (KB

V component average PSNR (dB)

[87] 20kbps

[88] 5fps X264 84.9 25.08 33.66 34.17

[89] SWATAW 84.1 26.97 34.88 35.18

[90] 60kbps

[91] lOfps X264 254 28.57 35.14 35.37

[92] SWATAW 252 29.79 36.00 36.39

[93] 80kbps [94] 25fps X264 411 28.54 35.14 35.43

[95] SWATAW 405 28.23 34.95 35.32

[96] Another advantage of the video compression method of the present invention is that it is easy to implement, because the encoder only needs to be adjusted according to the foregoing, and the decoder can be correctly decoded without any modification. The video stream compressed by the method of the present invention.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., which are within the spirit and scope of the present invention, should be included in the present invention. Within the scope of protection.

Claims

Claim

[1] 1. A video compression method, comprising: the following steps:

A. defining a scene in which a change in a different video frame is less than or equal to a predetermined threshold as a background, and defining a scene whose change is greater than a predetermined threshold as a foreground; or defining a scene in a different video frame that is smaller than a predetermined threshold as a background, and A scene whose change is greater than or equal to a predetermined threshold is defined as a foreground;

B. Using the multi-reference frame mechanism of H.264 or AVS video compression standard, encode and decode different reference frames for background and foreground: encode and decode high-quality reference frames for the background, and edit the foreground with normal reference frames. decoding.

[2] The video compression method according to claim 1, wherein the step B includes the following steps:

B1. Determine the reference frame type with an encoder: a long-term reference frame or a short-term reference frame;

B2. Synchronous decoding operation: notifying the decoder of the determined reference frame type, causing the decoder to reconstruct the reference frame list 0;

B3. The encoder synchronizes the operation of the decoder by transmitting different adaptive memory control commands for the high quality reference frame and the low quality reference frame;

B4. End.

[3] The video compression method according to claim 2, wherein in the step B3, the different adaptive memory control commands are sent as follows:

For I frames:

If it is a short-term reference frame, set the value of the long_term_reference_flag flag to 0; if it is a long-term reference frame, set the value of the long_term_reference_flag flag to 1; for P frames:

If it is a short-term reference frame, set the value of the adaptive_ref_pic_marking_mode_flag flag to 0. If it is a long-term reference frame, set the value of the adaptive_ref_pic_marking_mode_flag flag to 1

[4] 4,

The video compression method according to claim 3, wherein for the P frame, adaptive_r After the value of the ef_pic_marking_mode_flag flag is set to 1, it also includes the following steps:

1 send the value of the memory-management-control-operation flag 6;

2 send the value of long_term_frame_idx 0;

3 send 0' notification corner military code memory-management-control-operation ends.

[5] The video compression method according to claim 2, wherein the short-term reference frame predicts a subsequent five-frame video frame, and the long-term reference frame predicts a subsequent multi-frame video frame of five or more frames.