CN109889838B

CN109889838B - HEVC (high efficiency video coding) rapid coding method based on ROI (region of interest)

Info

Publication number: CN109889838B
Application number: CN201811629935.0A
Authority: CN
Inventors: 欧阳国胜
Original assignee: Beijing Jiaxun Feihong Electrical Co Ltd
Current assignee: Beijing Jiaxun Feihong Electrical Co Ltd
Priority date: 2018-12-29
Filing date: 2018-12-29
Publication date: 2021-02-12
Anticipated expiration: 2038-12-29
Also published as: CN109889838A

Abstract

The invention discloses an HEVC (high efficiency video coding) rapid coding method based on an ROI (region of interest), which specifically comprises the following steps of: determining a region of interest, determining a CU block depth, determining a CU block prediction mode and determining a candidate prediction mode; the region of interest is simply and effectively identified through a multi-frame image difference algorithm, the coding depth of the CU block is determined through the region of interest, the region of non-interest and the comparison of residual values, different candidate prediction modes are selected through three different positions of the CU block in the region of non-interest, the edge of the region of interest and the non-edge of the region of interest, unnecessary inter-frame prediction modes are eliminated, unnecessary mode judgment processes are reduced, the purpose of reducing motion complexity is achieved, and the coding efficiency is further improved.

Description

HEVC (high efficiency video coding) rapid coding method based on ROI (region of interest)

Technical Field

The invention relates to the technical field of information source coding communication, in particular to an HEVC (high efficiency video coding) rapid coding method based on an ROI (region of interest).

Background

The international standard organization ITU-T moving picture experts group VCEG and ISO moving picture experts group MPEG form a video coding cooperation group JCT-VC, and a new generation of high performance video coding standard HEVC is established. The main objective of the HEVC coding standard is to double the compression efficiency of high-resolution/high-fidelity video images on the basis of the h.264/AVC standard, that is, on the premise of ensuring the same video image quality, the code rate of a video stream is reduced by 50%, so as to better adapt to various different network environments, and the objective of improving the HEVC coding efficiency by 1 time has been achieved, but since a quadtree structure (quadtree structure) and a Larger Coding Unit (LCU) are used in a coding structure, the computational complexity of an encoder is significantly improved, the coding time cannot meet the real-time requirement, and the high-compression performance is obtained while the extremely high computational complexity is brought, which is obviously disadvantageous for the long-term development of the HEVC coding technology. Therefore, how to significantly reduce the amount of coding computation and increase the coding speed on the premise of preserving the compression efficiency and image quality of HEVC coding becomes especially important.

In order to improve compression efficiency, Coding Units (CUs) in HEVC employ quadtree recursive partitioning, and two major features of the CUs are characterized by the size and Depth of the CUs. Each frame image is first sequentially divided into LCUs of 64 × 64 size, each LCU coding depth is from 0 to 3, and the CU is recursively divided into CUs of 4 depths (sizes of 64 × 64, 32 × 32, 16 × 16, 8 × 8, respectively) to construct a quad-tree coding structure. In the HEVC inter-frame coding process, a CU at each coding depth has its corresponding PU partition mode, and motion estimation ME and motion compensation MC are performed. As shown in fig. 1, for CU at each depth d, the Inter prediction modes include SKIP/merge, Square split (Square, Inter 2N × 2N, Inter N × N), symmetric split (SMP, Inter 2N × N, Inter N × 2N), asymmetric split (AMP, Inter 2N × nU, Inter 2N nD, Inter nL 2N, Inter nR × 2N), and intra modes (intra 2N × 2N, intra N × N).

The HEVC encoder performs motion estimation and motion compensation on all partitions (SKIP/merge, Square, SMP, AMP, and intra modes) from top to bottom for CUs at different depths, and in reference software HM12.0, as shown in fig. 2, the inter prediction function call flow starts from LCU (coding depth d ═ 0) until the minimum coding unit SCU (coding depth d ═ 3) ends, the function call flow in fig. 2 is respectively executed in the CU at each layer depth, rate-Distortion cost RD-cost (rate distribution cost) is calculated for each inter prediction mode, and the PU partition with the smallest rate-Distortion cost is found as the best PU prediction mode of the current CU. Obviously, the calculation complexity of the encoding end is very high in the traversal calculation process, the encoding time consumed by video compression is long, and the real-time video compression requirement cannot be met. Therefore, a large amount of operation complexity is introduced in the inter-frame prediction process of HEVC, and how to effectively reduce the operation amount of an encoder becomes a problem to be solved urgently at present.

The prior art has the following disadvantages: in order to reduce the complexity of the HEVC coding process, several new algorithms have been proposed in recent two years. Judging whether the current CU needs to be further divided or not by setting a threshold value by using texture information and stable region detection information of the image; some utilization Rate Distortion (RD) cost determination threshold values terminate the continuous segmentation of the current CU in advance, and effective processing is not carried out on the selection of the inter-frame prediction mode; some current blocks are segmented by utilizing the distribution characteristic of gray level difference, but the distortion of gray level binary filtering is large, and the effect is not ideal; some methods calculate the eigenvalue of Pyramid Motion Divergence (PMD) by using an optical flow method to judge the segmentation condition of the CU, so that the complexity is effectively reduced, but the correlation among motion vectors is not fully considered, so that the performance of coding RD is poor; some JND models and ADD decisions are utilized to quickly select coding depth and inter-frame prediction modes, but for sequences with complex textures, the algorithm has a limited effect of reducing complexity; some depth value predictions are performed through weighting of adjacent CUs in a time-space domain, so that the depth traversal times of a Largest Coding Unit (LCU) are reduced, but the difference between sequences is not considered by a fixed weight algorithm, so that the problem that the predicted depth range has errors is caused.

Disclosure of Invention

The invention aims to provide an HEVC (high efficiency video coding) rapid coding method based on an ROI (region of interest) to solve the problems in the background technology.

In order to realize the purpose, the invention provides the following technical scheme: an HEVC (high efficiency video coding) fast coding method based on an ROI (region of interest) is characterized in that: the method specifically comprises the following steps:

s1, determination of the region of interest: calculating to obtain an interested area by using a multi-frame image difference algorithm: let f (x, y, i-1), f (x, y, i) and f (x, y, i +1) be three consecutive frames of the video image sequence, and perform difference operations on them two by two, respectively, where Df (x, y, i-1, i) is a binary difference image between the previous frame f (x, y, i-1) and the current frame f (x, y, i), and Df (x, y, i, i +1) is a binary difference image between the current frame f (x, y, i) and the next frame f (x, y, i +1), and then perform a sum operation on the binary difference image, and if Df (x, y, i) ═ Df (x, y, i-1, i) # Df (x, y, i, i +1), only if Df (x, y, i-1, i) ═ 1 and Df (x, y, i, i +1) simultaneously, if y, i) is 1, the image is regarded as a region of interest in the ith frame image;

s2, determining the CU block depth: the choice of CU block size depends on the complexity of the image and the severity of the motion. For a background static or stable area, namely a non-interested area, image information can be well expressed by adopting a larger CU block; whereas for complex motion regions, i.e. regions of interest, smaller CU blocks are better able to express the detailed information of the image. Because strong correlation exists between adjacent frames in a video sequence on a time domain, residual error numerical value distribution between two adjacent frames corresponding to the same position in a smooth area with smaller motion transformation is more uniform between two adjacent frames in the video, and the trend of continuously dividing the video into the depth of the next layer is weakened; in a region with severe motion, the residual value fluctuates greatly, and it is suitable to use a smaller CU size for processing, that is, the size of a CU has a close relationship with the pixel residual distribution of co-located CUs in adjacent frames, so that the FDD of a CU with depth d can be defined:

wherein:

represents the pixel value at the current frame CU block coordinate (x, y) position;

(x, y) pixel values representing the same position CU of the adjacent frame; delta_xAnd Δ_yThe components of the predicted motion vector PMV in the horizontal and vertical directions, M, respectively, for matching blocks in the coded reference frame_dRepresenting the current coded depth index, i.e. M_d＝2^d，d∈[0,3]；

Representing the side length of the CU block at the current depth; FDD_dReflecting the difference degree of the data values in the residual error, and for a CU at a certain depth, FDD_dThe smaller the residual error distribution is, the more uniform the residual error distribution is, and the smaller the trend of continuous depth segmentation to the next level is; if FDD of current CU_dThe value is small, the difference between the texture and motion changes of the current CU and the previous CU is considered to be small, and the similarity of the two CUs is large;

s3, determining CU block prediction mode: the interested areas are further distinguished to refine the prediction modes, and CU blocks at different positions in the interested areas correspond to different candidate prediction modes, so that unnecessary inter-frame prediction modes in the interested areas can be eliminated, and the aim of reducing the motion complexity is fulfilled; therefore, for the CU blocks at the edge of the region of interest, the candidate prediction modes are AMP and intra modes, and for the CU blocks at the middle position in the region of interest, the candidate prediction modes are Square and SMP;

s4, overall flow:

calculating to obtain a binary image of a region of interest (ROI) by utilizing a multi-frame image difference algorithm; if the current CU block is not in the region of interest, only using a large CU block for prediction, wherein the block size is 64 × 64 and 32 × 32; FDD if calculated at LCU with depth d ═ 0₀If the current LCU is equal to 0, the current LCU is basically consistent with the LCU of the adjacent frame, the subsequent sub-block division process can be directly skipped out, and the inter-frame prediction mode of the current LCU is judged to be SKIP/merge; if the calculation is carried out at depth d-0, FDD is obtained₀If not, the depth is increased to d to 1, namely the depth is divided into 4 32 blocks by 32 blocks to carry out the Square mode inter-frame prediction; thirdly, if the current CU block is in the interested region, thinning the image, and only adopting a small CU block for prediction, wherein the block size is 16 × 16, 8 × 8; FDD for each CU starting from depth d-2_dFDD of operation and 4 sub-blocks thereof_d+1Compare, FDD for 4 sub-blocks_d+1Sum greater than FDD of current CU_dIf the current CU is positioned at the edge of the region of interest, the candidate prediction modes are AMP and intra modes; if the current CU is in the middle position of the interested area, the candidate prediction modes are Square and SMP; FDD if four CU sub-blocks_d+1Value ratio parent block FDD_dIf the value of (1) is small, the CU is segmented, inter-frame prediction is carried out by adopting the depth d of the next level to be 3, and if the current CU is positioned at the edge of the interested area, the candidate prediction modes are AMP and intra modes; if the current CU is in a meta position in the region of interest, the candidate prediction modes are Square and SMP.

Compared with the prior art, the invention has the beneficial effects that: according to the HEVC fast coding method based on the ROI, the ROI is simply and effectively identified through a multi-frame image difference algorithm, the coding depth of a CU block is determined through the ROI, the ROI and the comparison residual value, different candidate prediction modes are selected through three different positions of the CU block, namely the ROI, the edge of the ROI and the non-edge of the ROI, unnecessary inter-frame prediction modes are eliminated, the unnecessary mode judgment process is reduced, the purpose of reducing the motion complexity is achieved, and the coding efficiency is further improved.

Drawings

Fig. 1 is a diagram illustrating HEVC inter-frame prediction partition modes in the prior art;

FIG. 2 is a diagram illustrating a flow of inter-frame prediction function call in the reference software HM12.0 in the prior art;

fig. 3 is a schematic diagram of an algorithm flow of an ROI-based HEVC fast coding method;

fig. 4 is a schematic view of a region of interest ROI of an HEVC fast coding method based on the ROI region.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1-4, the present invention provides a technical solution: an HEVC (high efficiency video coding) fast coding method based on an ROI (region of interest) is characterized in that: the method specifically comprises the following steps:

s1, determination of the region of interest: in the video playing process, generally not all information of an image is concerned with the same, a part which is most concerned by human eyes is often called an interested area, in the interested area coding, a rough area is usually defined or adopted by people in a regular shape, such as a rectangle or a circle, but the area which is really interested by human eyes cannot be accurately given, and if the area is wrong or distorted in a large range, the watching quality of the video is seriously influenced; calculating to obtain an interested area by using a multi-frame image difference algorithm: let f (x, y, i-1), f (x, y, i) and f (x, y, i +1) be three consecutive frames of the video image sequence, and perform difference operations on them two by two, respectively, where Df (x, y, i-1, i) is a binary difference image between the previous frame f (x, y, i-1) and the current frame f (x, y, i), and Df (x, y, i, i +1) is a binary difference image between the current frame f (x, y, i) and the next frame f (x, y, i +1), and then perform a sum operation on the binary difference image, and if Df (x, y, i) ═ Df (x, y, i-1, i) # Df (x, y, i, i +1), only if Df (x, y, i-1, i) ═ 1 and Df (x, y, i, i +1) simultaneously, if y, i) is 1, the image is regarded as a region of interest in the ith frame image;

as shown in fig. 4, the original image is on the left, and the binary image for identifying the region of interest ROI is on the right, so that the region of interest can be identified simply and effectively by using a multi-frame image difference algorithm, and conditions are provided for fast encoding of video encoding.

S2, determining the CU block depth: the choice of CU block size depends on the complexity of the image and the severity of the motion. For a background static or stable area, namely a non-interested area, image information can be well expressed by adopting a larger CU block; whereas for complex motion regions, i.e. regions of interest, smaller CU blocks are better able to express the detailed information of the image. Because strong correlation exists between adjacent frames in the video in the time domain, and the higher the frame rate of the video is, the stronger the correlation is, the more uniform the distribution of residual error values corresponding to smooth areas with smaller motion transformation is between CUs corresponding to the same positions of the two adjacent frames in the video, and the trend of the residual error values continuing to be divided into the depth of the next layer is weakened; in a region with severe motion, the residual value fluctuates greatly, and it is suitable to use a smaller CU size for processing, that is, the size of a CU has a close relationship with the pixel residual distribution of co-located CUs in adjacent frames, so that the FDD of a CU with depth d can be defined:

wherein:

s3, determining CU block prediction mode: the CU block prediction mode is determined, and the CU block prediction mode is relatively simple because the CU block in a uniform background region, namely a non-interested region, is generally small in coding depth in the inter-frame prediction process, such as SKIP/merge mode, inter 2N x 2N; the CU coding depth in a violent motion area, namely an interested area is larger, and corresponding prediction modes are more various, such as SMP, AMP and intra modes; the interested areas are further distinguished to refine the prediction modes, and CU blocks at different positions in the interested areas correspond to different candidate prediction modes, so that unnecessary inter-frame prediction modes in the interested areas can be eliminated, and the aim of reducing the motion complexity is fulfilled; therefore, for the CU blocks at the edge of the region of interest, the candidate prediction modes are AMP and intra modes, and for the CU blocks at the middle position in the region of interest, the candidate prediction modes are Square and SMP;

s4, overall flow:

calculating to obtain a binary image of a region of interest (ROI) by utilizing a multi-frame image difference algorithm; if the current CU block is not in the region of interest, only using a large CU block for prediction, wherein the block size is 64 × 64 and 32 × 32; FDD if calculated at LCU with depth d ═ 0₀If the current LCU is equal to 0, the current LCU is basically consistent with the LCU of the adjacent frame, the subsequent sub-block division process can be directly skipped out, and the inter-frame prediction mode of the current LCU is judged to be SKIP/merge; if the calculation is carried out at depth d-0, FDD is obtained₀If not, the depth is increased to d to 1, namely the depth is divided into 4 32 blocks by 32 blocks to carry out the Square mode inter-frame prediction; thirdly, if the current CU block is in the interested region, thinning the image, and only adopting a small CU block for prediction, wherein the block size is 16 × 16, 8 × 8; FDD for each CU starting from depth d-2_dFDD of operation and 4 sub-blocks thereof_d+1Compare, FDD for 4 sub-blocks_d+1Sum greater than FDD of current CU_dIf the current CU is positioned at the edge of the region of interest, the candidate prediction modes are AMP and intra modes; if the current CU is in the middle position of the interested area, the candidate prediction modes are Square and SMP; FDD if four CU sub-blocks_d+1Value ratio parent block FDD_dIf the value of (1) is small, the CU is segmented, inter-frame prediction is carried out by adopting the depth d of the next level to be 3, and if the current CU is positioned at the edge of the interested area, the candidate prediction modes are AMP and intra modes; if the current CU is in the middle position of the region of interest, the candidate prediction modes are Square and SMP, and the specific flow is shown in FIG. 3.

According to the HEVC fast coding method based on the ROI, the ROI is simply and effectively identified through a multi-frame image difference algorithm, the coding depth of a CU block is determined through the ROI, the ROI and the comparison residual value, different candidate prediction modes are selected through three different positions of the CU block, namely the ROI, the edge of the ROI and the non-edge of the ROI, unnecessary inter-frame prediction modes are eliminated, the unnecessary mode judgment process is reduced, the purpose of reducing the motion complexity is achieved, and the coding efficiency is further improved.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. An HEVC (high efficiency video coding) fast coding method based on an ROI (region of interest) is characterized in that: the method specifically comprises the following steps:

s1, determination of the region of interest: calculating to obtain an interested area by using a multi-frame image difference algorithm: let f (x, y, i-1), f (x, y, i) and f (x, y, i +1) be three consecutive frames of images in a video image sequence, and perform a difference operation on them two by two, respectively, where Df (x, y, i-1, i) is a binary difference image between a previous frame f (x, y, i-1) and a current frame f (x, y, i), and Df (x, y, i, i +1) is a binary difference image between the current frame f (x, y, i) and a next frame f (x, y, i +1), and then perform a sum operation on the binary difference image, and if Df (x, y, i) is Df (x, y, i-1, i) and Df (x, y, i-1, i +1) are equal to 1 and if Df (x, y, i, i +1) is equal to 1, then simultaneously if Df (x, y, i, i +1) is equal to 1, if y, i) is 1, the image is regarded as the interested region in the ith frame image;

s2, determining the CU block depth: the smaller CU size is used for processing, i.e. the size of the CU has a close relationship with the pixel residual distribution of co-located CUs in neighboring frames, thereby defining the FDD of a CU with depth d:

wherein:

Representing the side length of the CU block at the current depth; FDD_dReflecting the degree of difference of data values in residual error, and for a CU at a certain depth, FDD_dThe smaller the residual error distribution is, the more uniform the residual error distribution is, and the smaller the trend of continuous depth segmentation to the next level is; if FDD of current CU_dThe value is small, the difference between the texture and the motion change of the current CU and the previous frame CU is considered to be small, and the similarity of the two CUs is large;

s3, determining CU block prediction mode: further distinguishing the interested areas to refine the prediction modes, wherein different candidate prediction modes correspond to the CU blocks at different positions in the interested areas, and unnecessary inter prediction modes in the interested areas are eliminated, so that the candidate prediction modes are AMP and intra modes for the CU blocks at the edges of the interested areas, and the candidate prediction modes are Square and SMP for the CU blocks at the middle positions in the interested areas;

s4, overall flow:

calculating to obtain a binary image of a region of interest (ROI) by utilizing a multi-frame image difference algorithm; if the current CU block is not in the region of interest, only using a large CU block for prediction, wherein the block size is 64 × 64 and 32 × 32; FDD if calculated at LCU with depth d ═ 0₀If equal to 0, the current LCU is basically consistent with the LCU of the adjacent frame, the subsequent sub-block division process is directly skipped out, and the inter-frame prediction mode of the current LCU is determined to be SKIP/merge; if the calculation is carried out at depth d-0, FDD is obtained₀If not, the depth is increased to d to 1, namely the depth is divided into 4 32 blocks by 32 blocks to carry out the Square mode inter-frame prediction; thirdly, if the current CU block is in the interested region, thinning the image, and only adopting a small CU block for prediction, wherein the block size is 16 × 16, 8 × 8; FDD for each CU starting from depth d-2_dFDD of operation and 4 sub-blocks thereof_d+1Compare, FDD for 4 sub-blocks_d+1Sum greater than FDD of current CU_dIf the current CU is positioned at the edge of the region of interest, the candidate prediction modes are AMP and intra modes; if the current CU is in the middle position of the interested area, the candidate prediction modes are Square and SMP; FDD if four CU sub-blocks_d+1Value ratio parent block FDD_dIf the value of (1) is small, the CU is segmented, inter-frame prediction is carried out by adopting the depth d of the next level to be 3, and if the current CU is positioned at the edge of the interested area, the candidate prediction modes are AMP and intra modes; if the current CU is in a meta position in the region of interest, the candidate prediction modes are Square and SMP.