CN103929641A - A Method of Intra-frame Coding Based on Virtual Reference Frame - Google Patents

Abstract

The present invention proposes an intra-frame coding method based on a virtual reference frame, which encodes the coding unit CU (Coding Unit) of the intra-frame video boundary using the HEVC standard prediction mode, and uses the virtual reference frame prediction and standard prediction mode for the remaining coding units. Coding is done in a predictive way. First, construct a virtual reference frame of the block to be coded. The scope of the virtual reference frame includes the reconstruction data of the coded area and the information of the uncoded area. The data of the uncoded area is filled by horizontal/vertical tiling. Secondly, the CU to be encoded is predicted based on the virtual reference frame. The prediction method adopts the fast diamond search method. If the prediction residual is small enough, mark the block as INTER mode and record the value of its motion vector; if the motion vector is 0, it is marked as a SKIP block; if the predicted residual is relatively large, the HEVC standard prediction method is used, and the block type is marked as an INTRA block. After testing, the prediction coding based on the virtual reference frame proposed by the present invention can effectively improve the prediction accuracy of intra-frame coding.

Description

A Method of Intra-frame Coding Based on Virtual Reference Frame

technical field

The invention relates to a new intra-frame encoding method, in particular to an intra-frame encoding based on a virtual reference frame, and belongs to the technical field of computer vision.

Background technique

The continuous improvement of computer performance, network bandwidth and mobile terminal processing capabilities has put forward new requirements for video processing in terms of speed and performance. In order to continuously improve compression performance, ITU and ISO organizations have introduced a series of video coding standards, including the H.26x series organized by ITU and the MPEG-x series standards organized by ISO, as well as the latest HEVC standard. The latest HEVC standard is committed to satisfying users' 1) high-definition, 2) 3D, 3) mobile wireless to meet the needs of new fields such as home theater, remote monitoring, digital broadcasting, mobile streaming media, portable video, medical imaging, etc. . There are multiple configuration modes in the HEVC standard, including HE (High Efficiency) high performance and LC (Low Complexity) low complexity configuration.

The basic coding framework adopted by these coding standards is the same, and the basic framework of motion compensation + DCT is mostly used. Under this framework, video frames are generally divided into three types: l (Intra-frame), P (Predictive-frame), and B (BidirectionaIIy predicted-frame): l frame is predicted without referring to other frame information, through transformation, Quantization and other processes complete the encoding; P frames refer to the reconstructed images of the previous encoded frames, and encode residuals after motion compensation; B frames refer to the reconstructed images of forward and backward bidirectional encoded frames, and encode residuals after motion compensation.

Among the three types of video frames, the I frame is a key frame, which plays a role in error tolerance of the code stream. Generally, only spatial correlation is used in the I frame encoding process, so the compression ratio is generally relatively low. Although in a video sequence, the number of l frames is relatively small, but the number of coding bits per frame is much higher than that of P and B frames. For example, in the H.264 standard, the encoding bit rate of I frames is 3 to 5 times that of P frames; in the HEVC standard, with the introduction of new inter-frame prediction technology, the compression ratio of I frames and P frames is further expanded , in some videos, it can reach 1:10. For a video stream with a constant output bit rate (CBR), a sudden increase in the bit rate of the I frame will directly reduce the number of coding bits for the subsequent P/B frames, thereby affecting the quality of the restored image and the predicted image. Improving the compression ratio of l frames not only plays a vital role in the stability and continuity of video quality.

Prediction is a commonly used method to improve the compression ratio in video, that is, according to the characteristics of the image, using the correlation between video frames and within them, using the encoded object as a reference, retaining less differential information and reducing the number of final encoding bits. How to find the best matching source of the object to be predicted in the encoded image, remove redundant information as much as possible, and minimize the amount of differential information is the key to prediction. Due to the role of I frames in the sequence, P/B frames cannot be used for inter-frame prediction, and only intra-frame prediction can be used, that is, to find similar objects within the current image. Intra-frame prediction mostly uses "spatial adjacent point prediction", that is, based on the spatial correlation of the image, it is considered that each point on the image has similarity with its spatial adjacent points, and after finding similar points through searching, differential coding is performed.

The intra-frame prediction adopted in the H.264 standard takes the 16×16 macroblock and the 4×4 block as the basic prediction unit, and predicts each point in the block in nine directions. Taking a 4×4 block as an example, as shown in Figure 1(a), a, b, ..., p are the current block to be predicted, and the surrounding 17 points Q, l, ..., P are coded points. For a-p point by point, along the nine directions of 0, 1, ..., 8 and direction 2 (DC prediction) shown in Fig. The sampling values are differentiated, and the mode with the smallest difference is the final prediction mode, and finally DCT coding is performed on the prediction residual coefficients.

In the latest HEVC standard, three new concepts, CU (Coding Unit), PU (Prediction Unit), and TU (Transform Unit), are introduced. The coding unit is similar to the macroblock concept in H.264/AVC, and its size can be up to 64×64; the prediction unit is the basic unit for prediction; the transformation unit is the basic unit for transformation and quantization. The separation of the three units makes the processing links of transformation, prediction and encoding more flexible and more in line with the texture characteristics of video images. The size of the prediction unit can be 4×4, 8×8, 16×16, 32×32, 64×64, the size of the block is different, and the prediction mode is also different, respectively 17, 34, 34, 34, and 3 Two modes are optional. The unified intra prediction angle is: +/-[0, 2, 5, 9, 13, 17, 21, 26, 32]/32.

When the H.264/HEVC standard performs intra-frame coding, multiple prediction modes are used for each pixel to be coded, but only the upper row and left column data of the block to be predicted are used. Within this data range, using The nearly exhaustive prediction point search method is used for prediction. If the correlation between adjacent points of the image is poor, this kind of prediction will not only reduce the amount of image data, but also introduce more redundant information due to factors such as recording mode information.

In other words, the core of intra-frame coding is: whether there are enough similarities to improve the accuracy of prediction, which directly determines the effect of prediction; how to find the position of similarity, which determines the computational complexity of the entire prediction process.

Contents of the invention

The purpose of the present invention is to provide an intra-frame predictive encoding method based on a virtual reference frame. This method analyzes the HEVC standard, changes the prediction method during intra-frame coding, and provides a coding structure suitable for intra-frame coding. In this structure, more reference data will be used to improve the accuracy of prediction and reduce data. redundancy.

Through the analysis of the HEVC standard, it is found that during intra-frame coding, intra-frame prediction only uses the data of the upper row and the left column of the current block, and the data of the wide area above and to the left are not used. The intra-frame encoding method based on the virtual reference frame of the present invention utilizes the reconstructed area, plus a fictitious area on the right and below, as the reference block of the current block to be encoded (or pixel to be encoded), and completes Intra prediction. This method, combined with the current intra-frame prediction method, can significantly improve the compression efficiency.

In order to achieve the above object, the present invention adopts the following technical solutions. It is characterized in that it comprises the following steps:

Step 1: Encode the coding unit CU (Coding Unit) located at the video boundary in the frame using the HEVC standard prediction mode;

Step 2: Judging the prediction mode of the remaining coding units, first search and match the data to be predicted in the virtual reference frame, the data of the virtual reference frame is obtained from known data or reconstruction data of the coded area, and fictitious The above two areas have a certain spatial correlation with the unit to be encoded. Secondly, judge the residual between the matching data obtained by searching and matching and the data to be predicted. If the residual is less than Predetermined threshold, the virtual reference frame is used for prediction, and the residual and motion vector are encoded; on the contrary, if the residual is greater than the predetermined threshold, the HEVC standard prediction mode will be further used for prediction. By comparing the two prediction methods Residual size, select a method with a smaller residual for intra-frame prediction, then quantize the predicted residual data, and entropy code;

Step 3: Before the coded stream is output, if the prediction mode of the HEVC standard is used for coding, the CU type is INTRA block; if the coding is based on a virtual reference frame, the CU type is INTER, and the motion vector value is given.

In order to save the computational load of decoding, a special INTER type CU is identified, that is, a SKIP block whose motion vector is 0.

When using uncoded data to construct a virtual reference frame, the method of horizontal tiling and vertical tiling is adopted. The horizontal tiling is to copy the data on the left side of the current block to the right and below the current block. The data above is copied to the right and below the current block.

The intra-frame prediction algorithm based on the virtual reference frame provided by the present invention can overcome the problem in the original standard that the intra-frame prediction only uses the data of the upper row and the left column of the current block, and the data of the wide area above and to the left are not utilized. The disadvantage is to use the reconstructed area, plus a fictitious area on the right and below, as the reference block of the current block to be encoded to complete intra-frame prediction. This method, combined with the current intra-frame prediction method, can significantly improve the compression efficiency.

Description of drawings

Figure 1 is the prediction of spatial adjacent points in the H.264 standard;

Figure 2 is intra-frame prediction in the HEVC standard;

Fig. 3 is the relative position of each block in the virtual reference frame;

Fig. 4 is the coding frame based on virtual reference frame;

FIG. 5 is an intra-frame coding process based on a virtual reference frame;

Detailed ways

As mentioned above, according to the intra-frame encoding method based on the virtual reference frame, the present invention improves the prediction effect and reduces the output code stream of the I frame without increasing the computational complexity.

The implementation of the present invention will be described below in conjunction with the accompanying drawings.

Step 1: Construction of virtual reference frame

The core of the present invention is to use coded and uncoded areas to complete the intra-frame prediction of the current block, which is different from the standard in which only coded areas are used for intra-frame prediction. It is equivalent to the introduction of the concept of motion estimation into the intra-frame coding algorithm, which essentially changes the prediction method of intra-frame prediction, that is, instead of using a fixed interpolation mode and a fixed angle for prediction, the current block to be predicted is regarded as It is a type of inter-frame block, which is predicted within a certain search range. If the best matching block is searched, then this block is its prediction reference block, and then the prediction is completed to improve the coding efficiency of the I frame. Thus the first step of the present invention is to construct a virtual reference frame.

To construct a virtual reference frame, it is necessary to use the reconstructed area, plus an imaginary area on the right and below, as the reference frame of the current block to be encoded. Figure 3 indicates the location of each block, and the maximum range shown in Figure 3 is defined as the virtual reference frame of the current block Current to be encoded.

The first row of blocks: UL is the UP Left block, that is, the upper left block; UP is the upper block; UR is the Up Right block, that is, the upper right block;

The second row of blocks: L is the Left block; Current is the current block to be encoded; R is the Right block;

Blocks in the third row: DL is the lower left block; Down is the lower block; DR is the lower right block.

The BM block is the Best Match block, which is the best matching block found for the current block. BM1 and BM2 in Figure 3 represent different possible positions of the best matching block. If it is the BM1 position, its data uses the encoded UL , UP, L block information, will also use the current information of the unencoded block; if it is BM2 position, in addition to the encoded UP, UR, will also use the unencoded Current block and R block information.

In addition to the illustrated two positions of BM1 and BM2, the BM block can also be in any position in FIG. 3 . The range shown in FIG. 3 is defined as the virtual reference frame of the current block Current to be coded, and also the range to be predicted by the current block to be coded. The search is performed within this range to achieve the best intra-frame prediction effect of the current block. The virtual reference frame refers to the size of the current block to be predicted. Generally, the range does not exceed ±64×64, and the smallest range can reach ±8×8. In order to improve the accuracy of block search, the concept of forward prediction + backward prediction is adopted.

Forward prediction: It means that the information used in the prediction is the block that can obtain the reconstructed content before. Take Figure 3 as an example, that is, the Current block finds information that can be used for prediction in UL, UP, UR, and L, and cannot be used Contents of R, Current, DL, Down, DR blocks.

Backward prediction refers to the use of information that has not yet been encoded during prediction, that is, the Current block can find the information used for prediction in the R, Current, DL, Down, and DR blocks.

The backward prediction here is different from the backward prediction concept of B frames. If backward prediction is used when encoding B frames, the order of decoding frames needs to be adjusted at the decoding end to decode correctly. In order not to cause unnecessary trouble to the decoding end, no adjustment will be made to the order of the coding blocks.

How to use the unencoded Current, R and other block position information to perform backward prediction of the Current block has become a focus of the present invention in constructing a virtual reference frame. The key to solving this problem is how to construct data that does not exist. There are three specific methods:

(1) In order to reduce the amount of calculation, the method of horizontal tiling and vertical tiling can be adopted. Horizontal tiling is to copy the L block to the Current, R blocks and the DL, Down, and DR blocks below; vertical tiling is to copy the UP block to the Current, Down blocks, and copy the information of the UR block to the R and DR blocks. Tile the information of the L block to the DL block.

(2) Select appropriate information for copying according to the texture statistical properties of the block.

(3) Directly fill in the information of the Current, R, DL, Down, and DR blocks as 128 or other values.

The above three methods have their own advantages and disadvantages. The (1) numerical structure method is simple and only involves data replication, but it is not accurate enough; the (2) numerical structure is complex and requires statistics of image features, but the data is relatively accurate; The numerical structure of type (3) is simple, but the correlation between the data is weaker. Considering the amount of calculation and the correlation of data comprehensively, the present invention will use (1) to construct unencoded data.

Step 2: Prediction process based on virtual reference frame

FIG. 4 is an intra-frame coding framework based on a virtual reference frame. In this figure, only I-frames are processed, and other frame types do not use this framework. For other frame types, the standard method is still used for encoding. For the input I frame, the coding unit CU is used as the unit to predict the combination of virtual reference frame prediction and HEVC standard prediction, and take the method with the lowest residual system as the final prediction mode. The residual coefficients are transformed, quantized, and entropy encoded for output, and the code stream is output.

Fig. 5 is an intra-frame encoding process based on a virtual reference frame, which includes the determination of the prediction type and the syntax annotation of the block type.

For collecting the input video sequence, determine the frame type. When the current frame is an I frame, it is divided into N×N small blocks during the encoding process, and the current block in the i-th row and j-th column is denoted as Cur _{i, j} .

first step: Judgment is made on the current block Cur _{i, j} , whether the judgment content is a boundary? Is it LCU level? If it is boundary or LCU level, the HEVC standard method will be used for intra prediction, and its Mode _{i, j} = INTRA will be recorded. Judgment criteria are: if one of the UL block, UP block, UR block, and L block of the current block does not exist, it means that Cur _{i, j} is a boundary block; otherwise, Cur _{i, j} is neither a boundary block nor an LCU level , then the search prediction can be carried out, and the next step is entered.

Step 2: Create a virtual reference frame, which is created by horizontal tiling in the present invention, and the current block and the R block are horizontally tiled. Set the search range. If the block size is 32×32, the search range is 64×64. For other block sizes, the search range is 32×32.

Step 3: Predict the current block Cur _{i, j} , the prediction method adopts the method of integer pixel and sub-pixel motion estimation, the matching criterion adopts the method with the smallest SAD value, and records its horizontal and vertical motion vectors VectX, VectY;

Step 4: Determine the prediction mode according to the type of the motion vector. There are three types of prediction modes Mode _{i, j} , namely Inter, Intra, and Skip. If the SAD value is less than the set threshold, VectX and VectY are recorded, and Mode _{i, j} =Inter is marked at the same time. If VectX=VectY=0, then it is a Mode _{i, j} = Skip block, and at the same time mark the block mode as Mode _{i, j} = Inter. If the SAD value is greater than the set threshold, it means that the best matching block is not obtained, Mode _{i, j} = Intra, and a standard prediction mode is used for prediction.

Step 5. Enter different encoding processes according to different types

Mode _{i, j} = Skip means that only the Mode _{i, j} information is recorded in the code stream without writing into the code stream.

Mode _{i, j} = Intra uses the HEVC standard method for prediction and direct conversion processing, and the code stream contains Mode _{i, j} information and processed data information.

Mode _{i, j} =Inter, adopt the method based on the virtual reference frame of the present invention to predict, and carry out transformation processing to the residual, the code stream contains Mode _{i, data information after j} information and residual processing and motion vector VectX, VectY information.

Step 6: Output compressed code stream

performance analysis

The method of the invention is used to realize and test the HM of the HEVC standard and compare it with the result of the standard algorithm HM6.0. For the restored image quality, the present invention adopts BDBR as a measure index. Configuration file: Low_delay_P_Main.cfg, low delay; encoded frame type: IPIPIP…; encoded frame number: 20 frames in total, 10 I frames, 10 P frames. The test sequences are CLASS A～E, among which CLASS A is the 2560x1600 sequence intercepted in the ultra-clear video sequence "Traffic" (4096x2048p30fps), "PeopleOnStreet" (3840x2160p30fps); CLASSB is 1920x1080p24fps: "ParkScene", "Kimono", 1920x108050-60fps: "Cactus", "BasketballDrive", "BQTerrace"; CLASS C is 832x480p30-60fps (WVGA): "BasketballDrill", "BQMall", "PartyScene", "RaceHorses"; CLASS D is 416x240p30-60fps (WQVGA ): "BasketballPaSS", "BQSquare", "BlowingBubbles", "RaceHorses"; CLASS E is 1280x720p60fps video conferencing scenes: "Vidyo1", "Vidyo3" and "Vidyo4" Different video types have different resolutions and different frame rates.

Tables 1 to 3 show the test results under different search ranges. The search ranges in Tables 1 to 3 are 64, 32, and 16 in turn. Taking Table 1 as an example, the first column is the video category CLASS A~E, and the second to fourth columns. On the basis of the same PSNR value as the standard HEVC HM6.0, the luminance component Y, chrominance component U, and chrominance component V , the percentage of bitrate saved by the three components.

Table 1 Search Range 64, BDBR=-0.85%

sequence type PSNR Y PSNR U PSNR V CLASS A -0.6% -0.9% -1.1% CLASS B -1.4% -2.0% -2.1% CLASS C -1.1% -1.6% -1.3% CLASS D -0.2% 0.4% 0.3% CLASS E -0.9% -1.6% -1.6%

Table 2 Search Range 32, BDBR=-0.78%

sequence type PSNR Y PSNR U PSNR V CLASS A -0.6% -0.9% -1.1%

CLASS B -1.4% -2.0% -2.1% CLASS C -1.1% -1.6% -1.3% CLASS D -0.1% 0.4% 0.3% CLASS E -0.9% -1.5% -1.6%

Table 3 Search Range 16, BDBR=-0.6%

sequence type PSNR Y PSNR U PSNR V CLASS A -0.6% -0.9% -1.1% CLASS B -1.4% -2.0% -2.1% CLASS C -1.1% -1.6% -1.3% CLASSSD -0.2% 0.4% 0.3% CLASS E -0.9% -1.6% -1.6%

Tables 1 to 3 show the BDBR values under different search ranges, indicating the bit rate savings of the two methods under the same objective quality. The larger the search range, the more BDBR values, which shows that compared with the standard prediction algorithm, the algorithm in this paper can save more than 0.5% of the code rate.

Claims (3)

1. the inner frame coding method based on virtual reference frame, is characterized in that:

Step 1: adopt the predictive mode of HEVC standard to encode to being positioned at the coding unit CU (Coding Unit) on video border in frame;

Step 2:, remaining coding unit is carried out to prediction mode judgement, first treat prediction data and in virtual reference frame, search for coupling, the data of described virtual reference frame are by given data or the data reconstruction of coding region, and fabricate the right-hand and lower zone of out one and form, there is certain spatial coherence above-mentioned two regions and unit to be encoded, secondly, residual error between matched data and the data to be predicted that search coupling is obtained judges, if residual error is less than predetermined threshold, predict with regard to the mode that adopts virtual reference frame, and residual sum motion vector is encoded, otherwise, if residual error is greater than predetermined threshold, the predictive mode that further adopts HEVC standard is predicted, by comparing the residual error size of two kinds of prediction mode, choose a kind of mode that residual error is less and carry out infra-frame prediction, then the residual error data of prediction is quantized, entropy coding,

Step 3: before encoding code stream output, if adopt the predictive mode of HEVC standard to encode, CU type is INTRA piece; If encode based on virtual reference frame, CU type is INTER, and provides motion vector value.

2. inner frame coding method as claimed in claim 1, is characterized in that: in order to save the operand of decoding, a kind of special INTER Type C U is identified to the SKIP piece that motion vector is 0.

3. inner frame coding method as claimed in claim 1, it is characterized in that: while utilizing not coded data constructing virtual reference frame, adopt the method for Horizontal Tile and vertical tile, Horizontal Tile is right side and the below to current block by the data Replica on the current block left side, and the mode of vertical tile is right side and the below to current block by the data Replica of current block top.