CN104104955B - The decoding method and device of a kind of image block - Google Patents

Abstract

The invention provides a codec method and device for an image block. The encoding method includes that the encoder determines the reference block of the target block and the motion information of the reference block; the sampling points in the reference block correspond to the sampling points in the target block; the encoder adopts the motion information of the reference block information to perform motion compensation on the reference block to obtain a residual block; the encoder compares the value of each sampling point in the residual block with the division threshold value determined by the encoder, and according to each The comparison result of the sampling points performs motion division on the target block; the encoder encodes the target block according to the result of the motion division. Compared with the current video codec, the embodiment of the present invention can improve the compression performance under the premise of ensuring the same video quality.

Description

Method and device for encoding and decoding image blocks

technical field

The present invention relates to video image signal encoding and decoding technology, in particular to an image block encoding and decoding method and device.

Background technique

The inherent redundant information of the video signal itself is divided into spatial redundant information and temporal redundant information. Spatial redundant information can be removed by orthogonal transform coding, and temporal redundant information can be removed by motion compensation. In the current common motion compensation algorithm, one or several division methods are predefined for a video block.

The division method in the prior art is fixed. For example, the division method defined in the H.264/AVC standard is symmetrical. For example, a 16x16 video macroblock can be divided into 16x16, 8x16, 16x8 or 8x8, etc.; HEVC In addition to the symmetric division of 2Nx2N, Nx2N, 2NxN and NxN, the division methods defined in the standard also include asymmetric division. For example, in addition to the above four division methods, it can also include 2N×nU, 2N×nD, nL× 2N or nR×2N etc.

With a fixed division method, the boundaries of moving objects cannot be accurately described, and the encoder or decoder cannot obtain accurate motion information, so that accurate motion compensation prediction cannot be performed, resulting in impaired compression performance.

Contents of the invention

In view of this, the embodiments of the present invention provide a method and device for encoding and decoding image blocks, which are used to solve the problem in the prior art that the compression performance is damaged due to the fact that the division of image blocks cannot accurately reflect the movement of objects.

In a first aspect, an image block encoding method is provided, the method comprising:

The encoder determines a reference block of the target block and motion information of the reference block, and the sampling points in the reference block correspond to the sampling points in the target block;

The encoder uses the motion information of the reference block to perform motion compensation on the reference block to obtain a residual block;

The encoder compares the value of each sampling point in the residual block with the division threshold value determined by the encoder, and performs motion division on the target block according to the comparison result of each sampling point ;

The encoder encodes the target block according to the result of the motion division.

According to the image block coding method of the embodiment of the first aspect of the present invention, the precision and performance of motion compensation can be effectively improved, thereby improving the compression effect of video signals.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the division threshold is an average value of all residual values in the residual block; or, the division threshold is all The median of all residual values in the residual block described above.

In the second possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the encoder compares the value of each sampling point in the residual block with the value determined by the encoder Comparing the division threshold value, and performing motion division on the target block according to the comparison result includes:

If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

In a third possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the encoder compares the value of each sampling point in the residual block with the value determined by the encoder Comparing the division threshold value, and performing motion division on the target block according to the comparison result includes:

If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

In a fourth possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the encoder determines the motion information of the reference block, including:

When the encoder encodes the original video block in a non-hierarchical encoding manner, at least one of the following items is acquired as the reference block motion information:

Motion information of video blocks spatially adjacent to the original video block;

Motion information of video blocks temporally adjacent to the original video block.

In a fifth possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the determination of the reference block of the target block includes:

When the encoder encodes the original video block according to the layered encoding method, the reference block is determined according to the corresponding image block of the target block in the low-quality layered reconstruction image:

Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than that of the reconstructed image of the layer where the target block is located.

In the sixth possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the determination of the motion information of the reference block includes:

When the encoder encodes the original video block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block:

Motion information of image blocks spatially adjacent to the target block;

Motion information of image blocks temporally adjacent to the target block;

Motion information of the reference block;

Motion information of image blocks spatially adjacent to the reference block;

Motion information of image blocks temporally adjacent to the reference block.

In the seventh possible implementation manner of the embodiment of the first aspect of the present invention combined with any one of the above embodiments, the determination of the reference block of the target block includes:

When the encoder encodes the target block in a non-hierarchical encoding manner, the third image block in the stored reference frame is determined as a reference block, and the third image block refers to the set distortion metric Next, the image block in the stored reference frame has the smallest distortion metric with the target block.

According to an image block coding method according to an embodiment of the present invention, the accuracy and performance of motion compensation can be effectively improved, thereby improving the compression effect of video signals; at the same time, the efficiency of motion compensation can be improved, and the resulting computational complexity can be reduced degree requirements.

In a second aspect, a method for decoding an image block is provided, the method comprising:

The decoder determines the reference block of the target block and the motion information of the reference block, and the sampling points in the reference block are in one-to-one correspondence with the sampling points in the target block;

The decoder uses the motion information of the reference block to perform motion compensation on the reference block to obtain a residual block;

The decoder compares the value of each sampling point in the residual block with the division threshold value determined by the decoder, and performs motion division on the target block according to the comparison result of each sampling point ;

The decoder decodes the target block according to the result of the motion division.

With reference to the second aspect, in a first possible implementation manner of the second aspect, the division threshold is an average value of all residual values in the residual block; or, the division threshold is all The median of all residual values in the residual block described above.

With reference to the second aspect or the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the decoder converts the value of each sampling point in the residual block to Comparing with the division threshold value determined by the decoder, and performing motion division on the target block according to the comparison result includes:

If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

With reference to the second aspect or any of the first to second possible implementation manners of the second aspect, in a third possible implementation manner of the second aspect, the decoder converts the residual block The value of each sampling point is compared with the division threshold value determined by the decoder, and performing motion division on the target block according to the comparison result includes:

If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

In the fourth possible implementation manner of the embodiment of the second aspect of the present invention combined with any one of the above embodiments, the determination of the reference block of the target block includes:

When the encoder encodes the original video block in a layered encoding manner, the decoder determines the reference block according to the image block corresponding to the target block in the low-quality layered reconstruction image:

Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than that of the reconstructed image of the layer where the target block is located.

In a fifth possible implementation manner of the embodiment of the second aspect of the present invention combined with any one of the above embodiments, the determination of the motion information of the reference block includes:

When the encoder encodes the original video block in a layered encoding manner, the decoder acquires at least one of the following items as the reference block motion information:

Motion information of image blocks spatially adjacent to the target block;

Motion information of image blocks temporally adjacent to the target block;

Motion information of the reference block;

Motion information of image blocks spatially adjacent to the reference block;

Motion information of image blocks temporally adjacent to the reference block.

In the sixth possible implementation manner of the embodiment of the second aspect of the present invention combined with any one of the above embodiments, the determination of the reference block of the target block includes:

When the encoder encodes the target block in a non-hierarchical encoding manner, the decoder determines the third image block in the stored reference frame as a reference block, and the third image block refers to the set Under a certain distortion metric, the image block in the stored reference frame has the smallest distortion metric with respect to the target block.

The embodiment of the third aspect of the present invention provides a device for dividing an image block, the device is located in an encoder, and the device includes:

An acquisition module, configured to acquire a reference block of the target block and motion information of the reference block, where sampling points in the reference block correspond to sampling points in the target block;

a motion compensation module, configured to perform motion compensation on the reference block acquired by the acquisition module by using the motion information acquired by the acquisition module to obtain a residual block;

a motion division module, configured to compare the value of each sampling point in the residual block with the division threshold value determined by the encoder, and perform a process on the target block according to the comparison result of each sampling point sport division;

An encoding module, configured to encode the target block according to the result of the motion division.

With reference to the third aspect, in a first possible implementation manner of the third aspect, the acquiring module is specifically configured to:

Obtain a reference block of the original video block;

When the encoder encodes the original video block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block:

Motion information of image blocks spatially adjacent to the target block;

Motion information of image blocks temporally adjacent to the target block;

Motion information of the reference block;

Motion information of image blocks spatially adjacent to the reference block;

Motion information of image blocks temporally adjacent to the reference block.

With reference to the third aspect, in a second possible implementation manner of the third aspect, the acquiring module is specifically configured to:

When the encoder encodes the original video block according to the layered encoding method, the reference block is determined according to the corresponding image block of the target block in the low-quality layered reconstruction image:

Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than the reconstructed image of the layer where the target block is located;

Acquire motion information of the reference block.

With reference to the third aspect or any of the first to second possible implementation manners of the third aspect, in a third possible implementation manner of the third aspect, the motion division module is specifically configured to:

Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder;

If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

With reference to the third aspect or any one of the first to second possible implementations of the third aspect, in a fourth possible implementation of the third aspect, the motion division module is specifically configured to:

Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder;

If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

In a fourth aspect, a device for dividing an image block is provided, wherein the device is located in a decoder, and the device includes:

An acquisition module, configured to acquire a reference block of the target block and motion information of the reference block, where sampling points in the reference block correspond to sampling points in the target block;

a motion compensation module, configured to perform motion compensation on the reference block acquired by the acquisition module by using the motion information acquired by the acquisition module to obtain a residual block;

a motion division module, configured to compare the value of each sampling point in the residual block with the division threshold value determined by the decoder, and perform a process on the target block according to the comparison result of each sampling point sport division;

A decoding module, configured to decode the target block according to the result of the motion division.

With reference to the fourth aspect, in a first possible implementation manner of the fourth aspect, the acquiring module is specifically configured to:

Obtain a reference block of the original video block;

When the encoder encodes the original video block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block:

Motion information of image blocks spatially adjacent to the target block;

Motion information of image blocks temporally adjacent to the target block;

Motion information of the reference block;

Motion information of image blocks spatially adjacent to the reference block;

Motion information of image blocks temporally adjacent to the reference block.

With reference to the fourth aspect or the first possible implementation manner of the fourth aspect, in the second possible implementation manner of the fourth aspect, the acquiring module is specifically configured to:

When the encoder encodes the original video block according to the layered encoding method, the reference block is determined according to the corresponding image block of the target block in the low-quality layered reconstruction image:

Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than the reconstructed image of the layer where the target block is located;

Acquire motion information of the reference block.

With reference to the fourth aspect or any of the first to second possible implementations of the fourth aspect, in a third possible implementation of the fourth aspect, the motion division module is specifically configured to:

Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder;

If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

In a fourth possible implementation manner of the fourth aspect embodiment of the present invention combined with any one of the above embodiments, the motion division module is specifically used for:

Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder;

If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block;

If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.

Through the above technical solution, the available motion information is used to perform motion compensation on the reference block, and the residual block obtained by motion compensation is statistically analyzed, and the reference block is divided according to the result of the statistical analysis, so that the adaptive division of the reference block can be completed. More accurate motion information can be obtained through adaptive division, and then accurate motion compensation prediction can be performed to improve compression performance.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic flowchart of an image block encoding method provided by an embodiment of the present invention;

FIG. 2 is a schematic flowchart of another image block encoding method provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of generation of adaptive division in an embodiment of the present invention;

Fig. 4 is a schematic flow chart of obtaining the rate-distortion cost value according to the division method in the embodiment of the present invention;

FIG. 5 is a schematic diagram of comparison between video division completed by HEVC in the prior art and video division completed by adaptive division in an embodiment of the present invention;

FIG. 6 is a schematic flowchart of an encoding method provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of the encoder corresponding to FIG. 6;

FIG. 8 is a schematic flowchart of a decoding method provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of the decoder corresponding to FIG. 8;

FIG. 10 is a schematic structural diagram of an apparatus for dividing an image block of an encoder according to an embodiment of the present invention.

FIG. 11 is a schematic structural diagram of an apparatus for dividing an image block of a decoder according to an embodiment of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Fig. 1 is a schematic flow chart of a video block division method provided by an embodiment of the present invention, including:

11: The encoder determines the reference block of the target block and the motion information of the reference block, and the sampling points in the reference block are in one-to-one correspondence with the sampling points in the target block.

In one embodiment of the present invention, for example, the available motion information can be determined for the current coding unit (which can be regular shape or irregular shape) in the enhancement layer

For the current coding unit in the enhancement layer, find the corresponding base layer coding unit and obtain related coding information of the base layer coding unit, including but not limited to motion information.

Identify the workout in the list of available workouts. The motion information in the motion information list mainly comes from three aspects: the spatially adjacent motion information of the current coding unit in the enhancement layer; the temporally adjacent motion information of the current coding unit in the enhancement layer; the basic Motion information of layer coding units. The available motion information includes but is not limited to the above three aspects, for example, the spatial and temporal motion information of the corresponding base layer unit can also be added to the list of available motion information. The number of motion information in the motion information list can be a pre-agreed number or obtained from the code stream, and the motion information therein can be the same or must be different.

In one embodiment of the present invention, when the resolution of the corresponding block is the same as that of the target block, the corresponding block is used as the reference block; otherwise, the corresponding block is resampled, and the resampled image block is used as the reference block. The purpose of resampling is to change the resolution of the corresponding block to be the same as the target block.

In an embodiment of the present invention, there is a corresponding relationship between the coordinates and the size of the corresponding block and the coordinates and size of the target block.

In one embodiment of the present invention, the input video can be divided. Without loss of generality, assuming that a plurality of non-overlapping video blocks with a size of 64×64 are obtained, a 64×64 video block to be encoded can be Called the original video block.

At the encoding end, the encoder can encode the original video block to obtain a bit stream, and at the decoding end, the decoder can decode the received bit stream to restore the original video block.

In addition, the video block with a size of 64x64 is only an example, and the current video block in an actual application may also be of other sizes.

The available motion information refers to the currently available motion information. For example, on the encoding side, the available motion information includes but not limited to the motion information of the adjacent coded video blocks of the original video block.

Specifically, for an encoder, in a non-hierarchical video coding application, the available motion information includes but is not limited to at least one of the following items:

Motion information of video blocks spatially adjacent to the original video block;

Motion information of video blocks temporally adjacent to the original video block.

In layered video coding applications, available motion information includes, but is not limited to, at least one of the following:

Motion information of video blocks spatially adjacent to the original video block;

Motion information of video blocks temporally adjacent to the original video block;

motion information of the first video block;

motion information for video blocks spatially adjacent to the first video block;

motion information for video blocks temporally adjacent to the first video block;

Wherein, the first video block refers to the video block corresponding to the original video block in the first layer, and the first layer refers to that the resolution of at least one of time, space, and signal-to-noise ratio is lower than that of the original video block. layering of layers. For example, the layer where the original video block is located is the enhancement layer, and the first layer refers to the base layer.

In the embodiment of the present invention, when temporal layering and/or signal-to-noise ratio layering are performed, the "corresponding" video blocks in the base layer and the enhancement layer refer to video blocks with the same position. For example, when temporal layering, the base layer and the enhancement layer The time intervals of frames used by the enhancement layer are different when encoding, but the video blocks used are located at the same position. For example, the frame numbers of the base layer encoding are 0, 2, 4, 6..., and the frame numbers of the enhancement layer encoding They are 0, 1, 2, 3..., the first video block is located in the second frame of the base layer, and its position is (x0~xn, y0~ym), and the original video block is located in the first frame of the enhancement layer and its position It is also (x0~xn, y0~ym). In spatial layering, the "corresponding" video blocks in the base layer and enhancement layer are video blocks whose number of pixels and their positions correspond to the spatial resolution, e.g., the ratio of the spatial resolution of the enhancement layer to the base layer is 2:1, then assuming that the position of the first video block is (x0-xn, y0-ym), then the position of the original video block is (x0/2-xn/2, y0/2-ym/2).

It can be understood that the above-mentioned available motion information is only an example, and it can also be obtained in other ways, for example, it can be obtained by weighting the motion information of temporally and/or spatially adjacent video blocks of the original video block, etc. The motion information of the original video block may even be independent of the motion information of temporally and/or spatially adjacent video blocks of the original video block.

The above-mentioned motion information includes but not limited to at least one of the following items: motion vector, such as component values of the motion vector in the x direction and y direction; prediction mode, such as unidirectional prediction or bidirectional prediction; and, the reference image frame in the reference Indexes in image lists, etc.

For the decoder, the available motion information is available in the received bitstream. For example, the encoder and the decoder can form the above available motion information into an available motion information list, each available motion information has an index in the available motion information list, and the bit stream output from the encoder to the decoder can include the encoder The index of the available motion information is used, and the decoder obtains the available motion information by searching the available motion information list according to the index of the available motion information. Alternatively, the bit stream received by the decoder may also directly include bits encoded with the available motion information, and the available motion information may be obtained by decoding the bits.

That is, optionally, for the decoder, the acquisition of the available motion information of the original video block includes:

the decoder obtains available motion information from the bitstream;

Wherein, the bit stream includes the available motion information index, so as to search and determine the available motion information according to the available motion information index and the pre-established list of available motion information same as the encoder; or,

The bit stream includes encoded bits of the available motion information, and the available motion information is obtained through decoding.

The aforementioned reference blocks are available at both the encoding end and the decoding end, so that the reference blocks used by the encoder and the reference blocks used by the decoder are the same.

During layered coding, a reference block can be predefined in the encoder and decoder, specifically, a video block with the same spatial resolution as the original video block can be determined as the reference block.

In non-hierarchical coding, reference blocks can be obtained from stored reference frames. For the encoder, the original video block is available, so a video block in the stored reference frame that is closest to the division of the original video block can be determined as the reference block. Afterwards, relative displacement information between the original video block and the reference block may be included in the encoded bit stream, so that the decoder obtains the reference block according to the relative displacement information. For example, in non-hierarchical coding applications, since the original video block at the encoding end is available, the original video block is directly used for motion compensation division to obtain division 1, and a certain video block in the frame memory is used as a reference block for motion compensation division to obtain Division 2, the encoder selects the best reference block is under a certain distortion criterion, the distortion of the division 2 of the reference block is the smallest relative to the division 1, the relative displacement between the best reference block and the current video block and the reference block in The positions in the frame registers are defined together as a partition vector or a shape vector; in layered coding applications, due to the strong correlation between the base layer and the enhancement layer, the second video block (second The definition of the video block can refer to the subsequent description) can be a selection of a reference block, of course, the selection of the reference block at this time can also be similar to the non-hierarchical coding application, find a reference block in the frame memory.

That is, for the encoder, optionally, obtain the reference block of the original video block, including:

When the encoder is performing layered encoding, the second video block is determined as a reference block, and the second video block has the same spatial resolution as the original video block, wherein the second video block refers to A video block corresponding to the original video block in the second layer, where the second layer refers to a layer whose time and/or signal-to-noise ratio resolution is lower than the layer where the original video block is located, or, the The second video block refers to the video block corresponding to the original video block upsampled in the third layer, and the third layer refers to a layer whose spatial resolution is lower than the layer where the original video block is located; or,

When the encoder is performing non-hierarchical encoding, the third video block in the stored reference frame is determined as a reference block, and the third video block refers to a set of division distortion metrics that have the same The division of the video block is closest to the divided video block.

For the decoder, optionally, obtain the reference block of the original video block, including:

During layered encoding, the decoder determines the reference block agreed upon by the encoder and decoder in advance as the reference block used by the decoder (the reference block at this time is inferred by the codec and does not require additional bit cost );or,

In non-hierarchical encoding, the bit stream contains indication information for indicating the reference block used by the encoder, and the decoder determines the reference block used by the decoder according to the indication information, wherein if the The reference block is composed of a video block, and the indication information includes the position index of the reference frame where the reference block is located in the frame memory, and the relative displacement between the reference block and the original video block; or, if the reference block It is formed by weighting multiple video blocks, and the indication information includes the position index of the reference frame where each video block is located in the frame memory, the relative displacement between each video block and the original video block, and the weight of each video block.

After obtaining the available motion information and the reference block, the first device may adaptively divide the reference block according to the available motion information.

Wherein, the adaptive partitioning refers to not being limited to the fixed partitioning method in the prior art, for example, not limited to the symmetric partitioning method of H.264/AVC, nor limited to the asymmetrical partitioning method of HEVC. It is not limited to rectangular division, and may be any division shape.

The above adaptive division may be performed based on motion compensation, that is, optionally, the adaptive division of the reference block according to the available motion information includes:

Adaptively divide the reference block in a motion compensation manner according to the available motion information.

Optionally, the adaptive division of the reference block by using motion compensation according to the available motion information includes:

12: The encoder uses the motion information of the reference block to perform motion compensation on the reference block to obtain a residual block.

13: The encoder compares the value of each sampling point in the residual block with the value determined by the encoder

dividing the threshold value for comparison, and according to the comparison result of each of the sampling points for the target

blocks for motion division.

Among them, the median value of the residual block refers to sorting the values of all residual blocks, and then taking the middle value

data.

In one embodiment of the present invention, the pixels whose value of the residual block is smaller than the median value of the residual block are composed of

is partition 0, otherwise partition 1.

In one embodiment of the present invention, pixels whose value is less than or equal to the median value of the residual block

A composed partition is partition 0, otherwise partition 1.

14: The encoder encodes the target block according to the result of the motion division.

Wherein, each or each group of available motion information may correspond to a division manner. It can be understood that each group of available motion information includes at least two pieces of available motion information; a division manner may divide to obtain two or at least three video blocks.

In one embodiment of the present invention, encoding the target block according to the motion division includes:

determining motion information for the two motion partitions by motion estimation;

performing motion compensation on the two motion divisions according to the determined motion information to obtain a prediction of the target block, and calculating a residual of the target block according to the prediction;

Writing the motion information of the two motion divisions and the residual of the target block into a code stream.

For example, when each available motion information corresponds to a division method, a residual block may be obtained according to the available motion information, and two video blocks, or at least three video blocks. or,

When each group of available motion information corresponds to a division method, at least two residual blocks may be obtained according to at least two pieces of motion information, and at least three video blocks may be obtained through statistical analysis of the at least two residual blocks. When performing statistical analysis on each residual block during the above division, one or at least two thresholds can be set.

Since the motion compensation module is a commonly used module in existing video codecs, on the one hand, the implementation methods proposed by the embodiments of the present invention can make full use of existing technologies to enhance the application breadth and maturity of the embodiments of the present invention; on the other hand Using the motion compensation module to implement adaptive division can reduce the computational complexity and limit the motion complexity to an acceptable range.

Optionally, when it is an encoder, after adaptively dividing the reference block according to the available motion information, it may also include:

The encoder determines the division method of the original video block according to the adaptive division of the reference block;

The encoder divides and encodes the original video block according to the division manner of the original video block, obtains an encoded bit stream and outputs it to a decoder.

According to the above-mentioned adaptive division of the reference block, determining the division method of the original video may refer to: since each available motion information or its combination can correspond to an adaptive division of the reference block, after determining an adaptive division , the adaptive division can be applied to the original video block, and then the original video block can be divided and encoded by this division to obtain an evaluation value, and the division method with the best evaluation value can be determined as the division method of the original video block . The evaluation value includes but not limited to the rate-distortion penalty value.

After determining the division method of the original video block, the division method can be used to divide and encode the original video block, and the coded bit stream can include: bits corresponding to the division method, which indicates adaptive division; The bit corresponding to the information, the bit can be the index of the motion information in the available motion information list, for example, if the index of MV0 in the available motion information list is 0, then the bit corresponding to the motion information is 0, or the motion information corresponds to The bits of may be the bits directly encoded on MV0. It can be understood that, here, one available motion information (MVO) corresponds to one division mode as an example. Optionally, when the division mode corresponds to at least two available motion information, bits corresponding to at least two available motion information are included. , for example, including at least two indexes of the available running information in the available motion information list, or directly encoded bits of the available motion information.

The above gives the division method when the first device is an encoder. At the encoder side, it is necessary to determine a final division method from multiple possible division methods, so there is the above-mentioned cost value comparison process. For the decoder, since the encoder will include the division method and motion information in the output bit stream, the decoder can directly determine the division mode and motion information from the bit stream, and use these information to decode to restore the original video block .

That is, optionally, for the decoder, the method further includes:

the decoder receives a bitstream;

The decoder decodes the bit stream according to the adaptive division of the reference block, so as to restore the original video block.

In this embodiment, the adaptive division of the reference block is completed by using the available motion information, and the adaptive division of the reference block at the encoding end can be applied to the original video block, so that the original video block can be adaptively divided, and the decoding end can be based on the The adaptive division of the reference block restores the original video block, and more accurate motion information can be obtained through adaptive division, and then accurate motion compensation prediction can be performed to improve compression performance.

Referring to FIG. 2 , an embodiment of the present invention provides a schematic flow chart of an encoder determining a division method of an original video block. Without loss of generality, in this embodiment, it is assumed that the original video block has two divisions. The value of each sampling point in the residual block is compared with the division threshold value determined by the encoder, and motion division is performed on the target block according to the comparison result of each sampling point.

Among them, the median value of the residual block refers to sorting the values of all residual blocks, and then taking the middle data;.

Or perform iterative quadtree segmentation on the original video block, and then make two divisions on the divided video block. This embodiment can include:

21: Corresponding to each available motion information, use the available motion information to perform motion compensation on the reference block to obtain a residual block;

For example, if the available motion information used is MV0, then the MV0 may be firstly used to perform motion compensation on the reference block to obtain the corresponding residual block.

Referring to FIG. 3 , taking layered coding as an example, the above-mentioned motion compensation for the reference block using the available motion information may include:

The 0th step, determine the available motion information of the former video block of enhancement layer (show 3 motion information among Fig. 3, MV0, MV1 and MV2);

Step 1, assuming that the reference block is the first video block of the base layer, the first video block refers to the video block corresponding to the original video block in the base layer;

The image content contained in the first video block is strongly correlated with the image content contained in the original video, and this strong correlation is based on the strong correlation of video coding signals in layered coding.

In one embodiment of the present invention, the content of the first video block is very similar to the content of the original video. For example, they all encode the same part of an object.

Step 2, using an available motion information (such as MV0), perform motion compensation on the reference block to obtain the prediction block;

In step 3, the prediction block is subtracted from the reference block to obtain a residual block.

22: Perform statistical analysis on the residual block to obtain a division method corresponding to the available motion information of the reference block.

In this embodiment, each available motion information corresponds to a division manner as an example. It can be understood that a group of available motion information may also correspond to a division manner.

Referring to Figure 3, the specific content of 22 may include:

Step 4, perform statistical analysis on the residual block;

Step 5: According to the result of the statistical analysis, the division method corresponding to the available motion information is obtained.

The statistical analysis is based on the image content of the video block, according to the principle that the image content with different motion information belongs to different division areas, for example, the image content includes A and B, for a good division algorithm, when A and B have the same motion When A and B have different motion information, they belong to different division blocks.

Since the above statistical analysis is based on the motion information of the image content, when the image content has different motion information, there will be different division methods, that is, the division has the feature of self-adaptation.

Specifically, the statistical analysis includes but is not limited to: calculating the mean value of the absolute value of the pixels of the residual block, using this mean value as a threshold, and determining the division method by comparing the absolute value of the pixels of the residual block with this threshold; or, by calculating The gradient value of the pixel determines the division method; calculate the median value of the pixels of the residual block, or calculate the average value of the pixels of the residual block.

For example, when the absolute value of a residual block pixel exceeds the calculated average threshold, then the pixel belongs to a different division from the pixels that do not exceed the threshold. Or, if the gradient value of a residual block pixel exceeds a preset threshold, then the pixels on both sides of the gradient value belong to different divisions.

Referring to FIG. 3 , taking the divided image block as two blocks as an example, a specific example of the division result is given in the lower left corner of FIG. 3 . This division is a division on a reference block.

23: Apply the division of the reference block to the original video block.

Referring to step 6 in FIG. 3 , the same division method as the reference block is used to divide the original video block, as shown in the upper right corner of FIG. 3 .

The above describes the division method of MV0, similarly, the division methods of MV1 and MV2 can be obtained. There is a one-to-one correspondence between the available motion information and the division method.

It can be understood that this embodiment takes three motion information as an example, the method in the embodiment of the present invention is not limited to the three motion information, the motion information may be multiple, and the multiple motion information may form a list, called is a motion information list, and each motion information may correspond to a unique index in the motion information list, therefore, each motion information may have a motion information index.

24: Encoding the original video block by using each division method, obtaining the evaluation value corresponding to each division method, and determining the division method with the best evaluation value as the division method of the original video block.

Wherein, the evaluation value may refer to the rate-distortion cost value, and in this case, the optimal evaluation value refers to the smallest rate-distortion cost value. It can be understood that the evaluation value is not limited to the rate-distortion cost value. For example, the residual value corresponding to each division method may be determined according to the residual value, and the division method with the smallest residual value is determined as the final division method. In this embodiment of the present invention, the rate-distortion cost is used as an example.

As shown above, each type of motion information will correspond to a division method, and a rate-distortion cost value can be obtained according to each division method. After calculating the rate-distortion cost values corresponding to all motion information, the final division can be determined. Way. For example, the rate-distortion cost value obtained by using the division method corresponding to MV0 is cost0, the rate-distortion cost value obtained by using the division method corresponding to MV1 is cost1, and the rate-distortion cost value obtained by using the division method corresponding to MV2 is cost2, assuming that cost0 is the smallest , then the division method corresponding to MV0 can be determined as the final division method.

In this embodiment, a motion compensation module is used in the adaptive division. Since the motion compensation module is a general-purpose module, motion complexity can be reduced and applicability can be improved.

Optionally, referring to FIG. 4, each division method is used to encode the original video block, and the rate-distortion cost value corresponding to each division method is obtained, which may include:

41: Corresponding to each division method, the divided video blocks are obtained, and the motion information of each divided video block is determined;

Optionally, motion estimation may be performed on each divided video block to obtain corresponding motion information. For example, assuming that two video blocks are obtained after dividing the original video block, which are respectively the first video block and the second video block, then the motion estimation can be performed on the first video block to obtain the corresponding motion information of the first video block, and , performing motion estimation on the second video block to obtain motion information corresponding to the second video block. or,

Optionally, the available motion information may also be used as the motion information of the divided video blocks adjacent to the available motion information. For example, the motion information used is MV0, assuming that the video block adjacent to the MV0 after division is the first video block (such as the video block on the left side after division as shown in Figure 3), then the motion information of the first video block can be determined is MV0, and similarly, the motion information of the video block on the right side after division as shown in FIG. 3 can be determined as MV1.

42: Perform motion compensation on each divided video block according to the motion information of each divided video block, to obtain a prediction block of each divided video block, and each divided video block a prediction block of the video block and a residual block of each divided video block;

For example, the divided video blocks are the first video block and the second video block, and the motion information corresponding to the two video blocks is the first motion information and the second motion information, then the first motion information can be used to perform Motion compensation, obtain the first prediction block, the first prediction block is the prediction block of the first video block, and obtain the residual block between the first prediction block and the first video block, the residual block may be called the first residual block ; Similarly, the second motion information can be used to perform motion compensation on the second video block to obtain the second prediction block, the second prediction block is the prediction block of the second video block, and the relationship between the second prediction block and the second video block is obtained A residual block, which may be called a second residual block.

It can be understood that, in the embodiment of the present invention, the specific content of the prediction block and the residual block obtained during motion compensation, and the specific content of the motion information obtained by motion estimation can refer to the specific content of motion compensation and motion estimation in the prior art, here No longer.

43: Obtain a reconstructed residual block of the original video block after performing orthogonal transformation, quantization, inverse quantization, and inverse orthogonal transformation according to the residual block corresponding to each divided video block.

Optionally, the orthogonal transformation may be a regular shape transformation, for example, a rectangular transformation. When a regular shape transformation is used, the residual blocks obtained after each division may be first combined to obtain a regular residual block corresponding to the original video block , such as combining the first residual block and the second residual block, and then performing an orthogonal transformation on the combined residual block. or,

The orthogonal transformation may also be an irregular shape transformation, and in this case, the orthogonal transformation may be performed on the first residual block and the second residual block respectively.

The residual block is subjected to orthogonal transformation, quantization, and scanning to obtain one-dimensional quantized transformation coefficients; corresponding inverse scanning, inverse quantization, and inverse orthogonal transformation are performed on the quantized transformation coefficients to obtain the reconstructed residual block of the original video block.

44: Obtain the predicted block of the original video block according to the predicted block of each divided video block, and obtain the reconstructed residual block according to the predicted block of the original video block and the reconstructed residual block of the original video block raw video chunks.

Wherein, the first predicted block and the second predicted block can be combined to obtain the predicted block of the source video block, and then the predicted block of the source video block is added to the reconstructed residual block of the original video block to obtain Reconstructed raw video chunks.

In addition, after post-processing (including but not limited to deblocking, etc.), the reconstructed original video block/frame can be stored as a reference frame in the frame memory for use in motion estimation of subsequent coded image frames.

45: Comparing the reconstructed original video block with the original video block to obtain encoding distortion, and encoding the original video block through the encoding mode corresponding to the adaptive division to obtain encoding bits, according to encoding distortion and encoding bits, Get the rate-distortion cost value.

The above coding distortion can be obtained by comparing the reconstructed original video block with the original video block.

The above coding bits may include: bits indicating that the division mode is adaptive division, and bits indicating available motion information corresponding to the division mode of the original video block.

The available motion information may form an available motion information list, and the motion information contained in the coded bits may be an index of the available motion information, or directly encode the adopted motion information. The motion information of the divided video block may be the information obtained by direct encoding, or may be the difference information of the available motion information corresponding to the final division manner.

That is, optionally, the coded bits include:

Bits corresponding to the division mode, the division mode indicates that the division is adaptive division; and,

A bit corresponding to the first available motion information, where the bit corresponding to the first available motion information is an index of the first available motion information in the available motion information list, or, the bit corresponding to the first available motion information are bits after encoding the first available motion information.

For example, the motion information for generating the final partition (that is, the determined division method of the original video block) is MV0, assuming that the index of MV0 in the motion information list is 0, then the motion information corresponding to the division method contained in the coded bits is 0 ; It can be understood that when the division mode corresponds to a group of available motion information, the group of available motion information needs to be encoded during encoding.

Further, if motion estimation is performed on each divided video block, the coded bits may also include bits corresponding to the motion information of each divided video block, and the bits corresponding to the motion information include but are not limited to: bits, or encoding bits used to generate a difference value between divided motion information and motion information corresponding to each divided image block. Taking the difference value as an example, for example, the divided video blocks are the first video block and the second video block, the motion information corresponding to the first video block is the first motion information, and the motion information corresponding to the second video block is the second motion information, the motion information set used to generate division is {MV0, MV1, MV2}, assuming that the first motion information is the closest to MV0 in the motion information set, and the second motion information is the closest to MV1 in the motion information set, then it also includes The difference between MV0 and the first motion information and the difference between MV1 and the second motion information. The closest means that when using a certain motion information in the motion information set to encode the first or second motion information, the corresponding index and the difference values in the x and y directions require the least number of bits. For example, assuming that the first One motion information is NMV, corresponding to the above three motion information MV0, MV1 and MV3 can be obtained: (1) index value 0, NMV(x)-MV0(x), NMV(y)-MV0(y); (2 ) index value 1, NMV(x)-MV1(x), NMV(y)-MV1(y); (3) index value 2, NMV(x)-MV2(x), NMV(y)-MV2(y ); respectively encode the above three kinds of information to obtain three kinds of encoded bits, assuming that the encoding bit overhead of (1) is the smallest, then MV0 is the closest motion information of NMV. The above (x) and (y) respectively represent the values of the corresponding motion information in the x and y directions, for example, NMV(x) represents the value of the NMV in the x direction, and NMV(y) represents the value of the NMV in the y direction. In addition, the coded bits may also include: structure information of the orthogonal transformation, all-zero coefficient indication of the substructure, quantized orthogonal transformation coefficients, and the like.

After obtaining the coding distortion and coding bits, the rate-distortion cost value can be obtained. Wherein, the rate-distortion cost value includes not only the distortion metric but also the number of coded bits corresponding to the distortion. For example, the coding distortion obtained by using the division method corresponding to MV0 has the same distortion metric as the coding distortion obtained by using the division method corresponding to MV1 , but the number of coded bits obtained by the division method corresponding to MV0 is less than the number of coded bits obtained by the division method corresponding to MV1, then it can be determined that the rate-distortion cost value corresponding to the division method corresponding to MV0 is less than that corresponding to the division method corresponding to MV1 Rate-distortion cost value.

In this embodiment, the bit overhead can be reduced by using the index of the motion information during encoding. When there is a new motion information encoding method that can further improve the encoding efficiency of the motion information, the idea proposed by the present invention is still applicable.

The adaptive division of the original video block can be completed through the above description. For example, referring to FIG. 5, the left side of FIG. 5 is an example of existing HEVC division, and the right side of FIG. In each example, the range on the right side of the curve is the moving object, and the left side is the static background. It can be seen from FIG. 5 that although the existing HEVC division has carried out deep divisions on the original video block, it still cannot accurately describe the moving object, thus reducing the prediction accuracy and affecting the compression performance. However, using the method of the embodiment of the present invention, It can effectively divide moving objects, improve prediction accuracy, and then improve compression performance.

After the division method of the original video block is determined, encoding can be completed according to the division method. For example, referring to FIG. 6 and the corresponding structural diagram of FIG. 7, it is a schematic flowchart of an encoding method proposed by an embodiment of the present invention, including:

61: Divide the original video block by using the division method of the original video block to obtain divided video blocks;

For the specific content of determining the division manner of the original video block, refer to the above embodiment shown in FIG. 2 .

This step can be completed by the division module shown in FIG. 7 .

62: Perform motion compensation on each divided video block to obtain a prediction block corresponding to each video block, and a residual block between the prediction block corresponding to each video block and the video block;

This step can be completed by the motion compensation module shown in Figure 7, and the residual block can be obtained. In addition, header information can also be obtained during prediction. The header information includes the prediction mode, the division size of each divided video block, sports information, etc.

63: Obtain residual coding after performing orthogonal transformation, quantization and coefficient scanning on the residual block;

Residual coding includes: structure information of orthogonal transformation, all-zero coefficient indication of substructure, and quantized orthogonal transformation coefficients, etc.

This step can be completed by the orthogonal transformation module, the quantization module and the coefficient scanning module shown in FIG. 7 respectively.

64: Perform entropy coding on the residual coding and the header information to obtain the coded bit stream, and then output it to the decoder.

This step can be completed by the entropy encoding module shown in FIG. 7 .

Additionally, the reconstructed video blocks may be stored in a frame store for encoding reference of subsequent video frames. As shown in FIG. 7 , an inverse quantization module, an inverse orthogonal transformation module, and a frame storage unit may also be included.

The above describes the behavior of the encoding side, and the corresponding behavior of the decoding side can be implemented as follows.

Referring to FIG. 8 and the corresponding structural schematic diagram in FIG. 9, the process executed by the decoder may include:

81: The decoder performs entropy decoding on the received bit stream.

This step can be completed by the entropy decoding module shown in FIG. 9 to obtain header information and 1-dimensional coefficients.

82: The decoder divides the reference block.

This step can specifically include:

The available motion information is obtained from the received bitstream (such as the header information described above). Assuming that the available motion information index is included, the available motion information can be determined by searching in the pre-established list of available motion information that is the same as that of the encoder according to the available motion information index;

performing motion compensation on the reference block by using the available motion information to obtain a residual block;

Perform statistical analysis on the residual block to obtain a division method corresponding to the available motion information;

The reference block is divided by using a division manner corresponding to the available motion information to obtain divided video blocks.

This step can be completed by the division module shown in FIG. 9 . The acquisition of the reference block used by the division module may be as described in the above embodiments, such as determining a predefined video block as a reference block, or determining a video block in a stored reference frame as a reference block.

83: The decoder predicts the divided video block according to the header information to obtain the predicted block.

This step can be completed by the motion compensation module shown in FIG. 9 to obtain a prediction block.

84: The decoder performs coefficient scanning, inverse quantization, and inverse orthogonal transformation on the one-dimensional coefficients to obtain a reconstructed residual block.

This step can be completed by the coefficient scanning module, the inverse quantization module, and the inverse orthogonal transformation module shown in FIG. 9 to obtain a reconstructed residual block.

85: Obtain a reconstructed block according to the predicted block and the reconstructed residual block.

This step can be completed by the reconstruction module in FIG. 9 . The reconstructed block is also the restored original video block. It can be understood that, due to reasons such as encoding loss, the restored original video block is not exactly the same as the original video block encoded by the encoder.

Afterwards, after all the video blocks of the current decoded frame have completed the above-mentioned 81-85 processing, the reconstructed decoded frame can be obtained, and then the reconstructed decoded frame is post-processed and put into the reference frame buffer to remove the coding brought image damage and improve subjective image quality.

It should be noted that, in the foregoing embodiments, layered coding is used as an example. It should be understood that the embodiments of the present invention are not limited to layered coding, and may also be applied to single-layer coding. Different from layered coding, the reference block is determined by dividing vectors during single-layer coding.

The above-mentioned prediction unit may adopt unidirectional prediction or bidirectional prediction, and for bidirectional prediction, generation of prediction pixels is weighted by two prediction signals. Further, there may be at least three prediction signals used by the prediction unit, and the prediction pixels are obtained by weighting the at least three prediction signals.

In an embodiment of the present invention, in video coding technology, using the spatial correlation of pixels to remove redundant information is an effective technology. This redundancy is reflected in motion information, which is the strong correlation between motion vectors. Specifically, due to this redundancy, the motion vector values between different adjacent motion blocks are very close.

Utilizing the strong correlation between the motion vectors of adjacent blocks, the present invention uses the motion vectors of adjacent blocks to perform motion compensation on the current encoding/decoding unit and analyzes the statistical characteristics of the generated motion compensation residuals. Since different moving objects have different motion characteristics in the current encoding/decoding unit, using the same motion vector for motion compensation will inevitably produce residuals with different statistical characteristics. Different moving objects can be effectively divided by using the statistical properties of the residual.

On the other hand, since the motion estimation/compensation module is a common module in current video codecs, its working performance and implementation have been extensively researched and optimized, and the computational complexity can fully meet the requirements of video codecs.

In the following description, we will give a specific embodiment based on the present invention. For ease of understanding, this embodiment is based on a layered coding structure. But as far as the present invention itself is concerned, this is a general coding technique and is not necessarily limited to a layered coding structure.

In a layered coding structure, a video signal is divided into a base layer and an enhancement layer. Wherein the temporal, spatial or quality resolution of the base layer is smaller than that of the enhancement layer.

In one embodiment of the present invention, on the encoder side,

In layered video coding, it is assumed that the base layer has the same resolution as the enhancement layer. In spatial layered coding, the above assumption does not seem to hold true, but at this time the previously assumed base layer is defined as the up-sampled base layer, which still has the same spatial resolution as the enhancement layer.

The present invention can divide the current coding unit in the enhancement layer into multiple partitions by adjusting the statistical characteristic threshold value of the motion compensation residual. Without loss of generality, the following description assumes that the current coding unit is divided into two partitions, and the case of multiple partitions can be deduced by analogy. The specific number of divisions can be pre-agreed by the codec end, or can be written by the encoder to the code stream and obtained by the decoder by parsing the code stream.

The operation of the encoder can be summarized as the following steps:

Step 1: For the current coding unit (which can be regular or irregular shape) in the enhancement layer, determine the available motion information

For the current coding unit in the enhancement layer, find the corresponding base layer coding unit and obtain related coding information of the base layer coding unit, including but not limited to motion information.

Identify the workout in the list of available workouts. The motion information in the motion information list mainly comes from three aspects: the spatially adjacent motion information of the current coding unit in the enhancement layer; the temporally adjacent motion information of the current coding unit in the enhancement layer; the basic Motion information of layer coding units. The available motion information includes but is not limited to the above three aspects, for example, the spatial and temporal motion information of the corresponding base layer unit can also be added to the list of available motion information. The number of motion information in the motion information list can be a pre-agreed number or obtained from the code stream, and the motion information therein can be the same or must be different.

Step 2: For a motion information in the motion information list, generate a partition

Use this motion information to perform motion compensation on the entire current enhancement layer coding unit to obtain a corresponding residual.

Perform a statistical analysis on the residuals from the previous step, e.g. the median of the residuals can be calculated.

Using the statistical properties generated from the previous step (such as the median value as the threshold value of the partition), generate a partition for the residual obtained in step 1 (such as defining a partition composed of pixels whose corresponding residual is less than the median value as partition 0, otherwise it is partition 1).

The third step: For the division generated in the second step

Motion estimation can be further performed: Motion estimation can use various fast algorithms to reduce computational complexity.

Estimation of the rate-distortion function in motion estimation requires a specific encoding method of the motion vector, and an encoding algorithm that fully utilizes the correlation of motion information can be used. For example, it may be by encoding the index in the motion vector list generated in the first step. The number of bits required by a specific motion vector coding method will be included in the calculation of the rate-distortion function.

The motion vector with the smallest rate-distortion cost value is the final motion vector. Use this motion vector to perform motion compensation on the current coding unit to generate a motion compensation residual.

Step 4: For the motion compensation residual generated in Step 3, an orthogonal transformation is applied to remove spatial redundancy. Since the shape of the current coding unit is not necessarily a regular shape, the orthogonal transformation does not necessarily adopt a regular shape (current video coding standards all use regular-shaped orthogonal transformation).

Step 5: Perform inverse quantization and inverse orthogonal transformation on the orthogonal transformation coefficients in the fourth step to obtain the reconstructed residual (if all the orthogonal transformation coefficients obtained in the fourth step are 0, define the reconstruction residual at this time The difference is 0), add the motion compensation value obtained from the third step, and perform the clipping operation to obtain the coding reconstruction of the current coding unit. Coding distortion can be obtained by comparing with the original current coding unit pixel value before encoding. Estimate the bit coding of the current coding unit, including: the index used to generate the divided motion vector in the motion vector list in the first step; if further motion estimation is performed, the index of the reference motion vector, and the final motion vector and The differential value of the reference motion vector; the structure information of the orthogonal transform, the all-zero coefficient indication of the substructure, and the quantized orthogonal transform coefficients. Combining coding distortion and bit consumption, the rate-distortion cost value of this motion information division can be obtained.

Step 6: If all the motion information and its divisions in the motion information list in the second step have been estimated, select the division with the smallest rate-distortion cost as the final division of the current coding unit. Otherwise, repeat steps two to five.

Step 7: For the final division of the current coding unit, write the coded bits of the current coding unit into the output bit stream through the entropy coding module, including: the motion vector used to generate the division is in the motion vector list generated in the first step Index; if further motion estimation is performed, it includes the index of the reference motion vector, and the difference value between the final motion vector and the reference motion vector; the structure information of the orthogonal transformation, the all-zero coefficient indication of the substructure, and the quantized orthogonal transformation coefficient Wait.

Step 8: After all the coding units of the current coded frame are completed, post-processing operations can be performed on the reconstructed coded frame to remove image damage caused by coding and improve subjective image quality.

The above algorithm describes the algorithm flow of the present invention through an example of encoding enhancement layer frames in layered encoding. As mentioned above, the present invention is a general coding algorithm, which can be applied in both layered coding and single-layer video coding. In single-layer video coding, a reference coding unit similar to the corresponding up-sampled coding unit of the base layer in layered video coding is needed, and the division of the current coding unit is estimated based on the reference coding unit. Using the temporal correlation of the video signal, the reference CU can be found in the reconstructed reference frame. The encoding of the reference CU can adopt a method similar to motion vector encoding, that is, encoding the reference picture list where the reference CU is located, the position index in the list, and the displacement of the reference CU relative to the current CU. To distinguish from motion vectors, this information can be defined as shape vectors or partition vectors.

The above algorithm is applicable to the case of one-way and two-way prediction. For bidirectional prediction, the process of motion compensation is more complicated than that of unidirectional prediction: the generation of predicted pixels is weighted by two prediction signals; and the process of motion estimation is much more complicated than that of unidirectional prediction, so the required operation The complexity is also higher than unidirectional prediction.

Further extension, the present invention can be applied to a situation where there are multiple prediction signals: in this application scenario, a prediction pixel is generated by weighting multiple prediction signals.

Supplementary Note In the example where the present invention is applied to layered coding, the above-mentioned coding algorithm also has a broad meaning. Among layered coding techniques, there can be two types of techniques. One is based on the coding unit technology, that is, the information reuse of the coding unit of the enhancement layer to the base layer is based on the coding unit level of the base layer; the other is a technology based on the reference frame level generated by the base layer, that is, the reconstructed base layer After processing, it is placed into the reference frame buffer of the enhancement layer, and the information reuse of the coding unit of the enhancement layer for the base layer is based on the reference frame of the base layer in the reference frame buffer. The example of the application of the present invention to layered coding can be generally used in the above two types of technologies, the only difference is in the acquisition of the basic layer motion vector information: in the technology based on coding units, the acquisition of the basic layer motion vector information is relatively straightforward, The motion information has a direct correspondence; in the technology based on the reference frame level, the motion vector information of the base layer needs a mapping, and after being converted into the motion information in the reference frame buffer of the enhancement layer, it can also be conveniently referred to.

Finally, the above descriptions of the embodiments of the present invention are general descriptions, and do not specifically refer to a specific implementation manner. For example, there is no specific limitation on the specific implementation of entropy coding: it can be variable-length coding; it can also be arithmetic coding. For arithmetic coding, it can be either binary arithmetic coding or universal arithmetic coding.

In one embodiment of the present invention, on the decoding side,

After receiving the encoded bit stream, the decoder parses the bit stream and performs corresponding decoding and reconstruction operations according to the parsed syntax elements. The specific steps and process are as follows.

Step 1: For the current coding unit (which can be regular or irregular shape) in the enhancement layer, determine the available motion information

For the current coding unit in the enhancement layer, find the corresponding base layer coding unit and obtain related coding information of the base layer coding unit, including but not limited to motion information.

Identify the workout in the list of available workouts. The motion information in the motion information list mainly comes from three aspects: the spatially adjacent motion information of the current coding unit in the enhancement layer; the temporally adjacent motion information of the current coding unit in the enhancement layer; the basic Motion information of layer coding units. The available motion information includes but is not limited to the above three aspects, for example, the spatial and temporal motion information of the corresponding base layer unit can also be added to the list of available motion information. The number of motion information in the motion information list can be a pre-agreed number or obtained from the code stream, and the motion information therein can be the same or must be different.

Step 2: Analyze the motion information index in the code stream, obtain the corresponding motion information from the motion information list, and use this motion information to generate partitions

Use this motion information to perform motion compensation on the entire current enhancement layer coding unit to obtain a corresponding residual.

Perform a statistical analysis on the residuals from the previous step, e.g. the median of the residuals can be calculated.

Using the statistical properties generated from the previous step (e.g., the median value as a threshold value), generate partitions for the residuals obtained from step 1 (e.g., define a partition consisting of pixels whose corresponding residuals are less than the median value as partition 0, Otherwise, partition 1).

Step 3: For the divisions generated in the second step, further analyze the code stream to determine the precise motion information of each division; and use the obtained motion information to obtain the motion compensation of each division of the current coding unit

Step 4: Analyze the code stream to obtain the residual of the current coding unit, including the structure of the orthogonal transformation, the all-zero coefficient indication of the substructure, the quantized orthogonal transformation coefficient, and the like.

Step 5: Perform inverse quantization and inverse orthogonal transformation on the orthogonal transformation coefficients in the fourth step to obtain the reconstruction residual (if all the orthogonal transformation coefficients obtained in the fourth step are 0, define the reconstruction residual at this time is 0), add the motion compensation value obtained from the third step, and perform the clipping operation to obtain the reconstruction of the current coding unit.

Step 6: After all the coding units of the current coded frame are completed, post-processing operations can be performed on the reconstructed coded frame to remove image damage caused by coding and improve subjective image quality.

The method flow is described above, and correspondingly, the embodiment of the present invention also provides an apparatus for executing the above method.

Referring to FIG. 10 , an embodiment of the present invention provides an image block division device 10, which can be located in an encoder, and the device 10 includes an acquisition module 101, a motion compensation module 102, a motion division module 103, and an encoding module 104; The module 101 is used to obtain the reference block of the target block and the motion information of the reference block, and the sampling points in the reference block correspond to the sampling points in the target block; the motion compensation module 102 is used to use the motion information obtained by the obtaining module 101 to obtain The reference block obtained by module 101 is motion compensated to obtain a residual block; the motion division module 103 is used to compare the value of each sampling point in the residual block obtained in the motion compensation module 102 with the division threshold value determined by the encoder , and perform motion division on the target block according to the comparison result of each sampling point; the encoding module 104 is configured to encode the target block according to the motion division result obtained by the motion division module 103 .

Referring to Fig. 11, an embodiment of the present invention provides an image block division device 11, which can be located in an encoder, and the device 10 includes an acquisition module 111, a motion compensation module 112, a motion division module 113 and an encoding module 114; Module 111 is used to obtain the reference block of the target block and the motion information of the reference block, and the sampling points in the reference block correspond to the sampling points in the target block; the motion compensation module 112 is used to obtain The reference block obtained by the module 111 is motion compensated to obtain a residual block; the motion division module 113 is used to compare the value of each sampling point in the residual block obtained in the motion compensation module 112 with the division threshold value determined by the decoder , and perform motion division on the target block according to the comparison result of each sampling point; the decoding module 114 is configured to decode the target block according to the motion division result obtained by the motion division module 113 .

In terms of hardware implementation, the above acquisition module, motion division module, motion compensation module, encoding module and decoding module can be specifically completed by a processor, and the processor can be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, etc. The above acquisition module may specifically be a receiver or a transceiver.

Further, the embodiment of the present invention may also provide an encoder, including the above-mentioned division device located in the encoder, and may also include a motion compensation module, an orthogonal transformation unit, a quantization unit, a coefficient scanning unit, and an entropy encoding unit. See Figure 7 for details.

Further, the embodiment of the present invention may also provide a decoder, including the above-mentioned dividing device located in the decoder, and may also include an entropy decoding unit, a coefficient scanning unit, an inverse quantization unit, an inverse orthogonal transformation unit, a prediction unit, a weighting unit, structural unit, etc. See Figure 9 for details.

In this embodiment, motion compensation is performed on the reference block by using the available motion information, statistical analysis is performed on the residual block obtained by motion compensation, and the reference block is divided according to the result of the statistical analysis, so that the adaptive division of the reference block can be completed. Adaptive division can obtain more accurate motion information, and then can perform accurate motion compensation prediction and improve compression performance.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the above-described system, device, and unit, reference may be made to the corresponding process in the foregoing method embodiments, and details are not repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions described in each embodiment are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application.

Claims (23)

1. An image block coding method, characterized in that the method comprises: The encoder determines a reference block of the target block and motion information of the reference block, and the sampling points in the reference block correspond to the sampling points in the target block; The encoder uses the motion information of the reference block to perform motion compensation on the reference block to obtain a residual block; The encoder compares the value of each sampling point in the residual block with the division threshold value determined by the encoder, and performs motion division on the target block according to the comparison result of each sampling point , the division threshold value is the median value of all residual values in the residual block; The encoder encodes the target block according to the result of the motion division. 2. The method according to claim 1, wherein the encoder compares the value of each sampling point in the residual block with the division threshold value determined by the encoder, and according to each The comparison result of the sampling points performing motion division on the target block includes: If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 3. The method according to claim 1, wherein the encoder compares the value of each sampling point in the residual block with the division threshold value determined by the encoder, and according to the comparison result Performing motion division on the target block includes: If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 4. The method according to any one of claims 1-3, wherein the encoder determines the motion information of the reference block, comprising: When the encoder encodes the target block in a non-hierarchical encoding manner, at least one of the following items is acquired as the reference block motion information: motion information of video blocks spatially adjacent to the target block; Motion information of video blocks temporally adjacent to the target block. 5. The method according to any one of claims 1-3, wherein said determining the reference block of the target block comprises: When the encoder encodes the target block in a layered encoding manner, the reference block is determined according to a corresponding image block of the target block in a low-quality layered reconstruction image: Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than that of the reconstructed image of the layer where the target block is located. 6. The method according to any one of claims 1-3, wherein the determining the motion information of the reference block comprises: When the encoder encodes the target block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block: Motion information of image blocks spatially adjacent to the target block; Motion information of image blocks temporally adjacent to the target block; Motion information of the reference block; Motion information of image blocks spatially adjacent to the reference block; Motion information of image blocks temporally adjacent to the reference block. 7. The method according to any one of claims 1-3, wherein the determining the reference block of the target block comprises: When the encoder encodes the target block in a non-hierarchical encoding manner, the third image block in the stored reference frame is determined as a reference block, and the third image block refers to the set distortion metric Next, the image block in the stored reference frame has the smallest distortion metric with the target block. 8. An image block decoding method, characterized in that the method comprises: The decoder determines the reference block of the target block and the motion information of the reference block, and the sampling points in the reference block are in one-to-one correspondence with the sampling points in the target block; The decoder uses the motion information of the reference block to perform motion compensation on the reference block to obtain a residual block; The decoder compares the value of each sampling point in the residual block with the division threshold value determined by the decoder, and performs motion division on the target block according to the comparison result of each sampling point , the division threshold value is the median value of all residual values in the residual block; The decoder decodes the target block according to the result of the motion division. 9. The method according to claim 8, wherein the decoder compares the value of each sampling point in the residual block with the division threshold value determined by the decoder, and according to each The comparison result of the sampling points performing motion division on the target block includes: If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 10. The method according to claim 8, wherein the decoder compares the value of each sampling point in the residual block with the division threshold value determined by the decoder, and according to each The comparison result of the sampling points performing motion division on the target block includes: If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 11. The method according to any one of claims 8-10, wherein the determining the reference block of the target block comprises: When the encoder encodes the target block in a layered encoding manner, the decoder determines the reference block according to a corresponding image block of the target block in a low-quality layered reconstruction image: Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than that of the reconstructed image of the layer where the target block is located. 12. The method according to any one of claims 8-10, wherein said determining the motion information of the reference block comprises: When the encoder encodes the target block in a layered encoding manner, the decoder acquires at least one of the following items as the reference block motion information: Motion information of image blocks spatially adjacent to the target block; Motion information of image blocks temporally adjacent to the target block; Motion information of the reference block; Motion information of image blocks spatially adjacent to the reference block; Motion information of image blocks temporally adjacent to the reference block. 13. The method according to any one of claims 8-10, wherein the determining the reference block of the target block comprises: When the encoder encodes the target block in a non-hierarchical encoding manner, the decoder determines the third image block in the stored reference frame as the reference block, and the third image block refers to the set Under the distortion metric, the image block in the stored reference frame has the smallest distortion metric with respect to the target block. 14. A device for dividing an image block, characterized in that the device is located in an encoder, and the device includes: An acquisition module, configured to acquire a reference block of the target block and motion information of the reference block, where sampling points in the reference block correspond to sampling points in the target block; a motion compensation module, configured to perform motion compensation on the reference block acquired by the acquisition module by using the motion information acquired by the acquisition module to obtain a residual block; a motion division module, configured to compare the value of each sampling point in the residual block with the division threshold value determined by the encoder, and perform a process on the target block according to the comparison result of each sampling point motion division, the division threshold value is the median value of all residual values in the residual block; An encoding module, configured to encode the target block according to the result of the motion division. 15. The device according to claim 14, wherein the acquiring module is specifically used for: Obtain a reference block of the target block; When the encoder encodes the target block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block: Motion information of image blocks spatially adjacent to the target block; Motion information of image blocks temporally adjacent to the target block; Motion information of the reference block; Motion information of image blocks spatially adjacent to the reference block; Motion information of image blocks temporally adjacent to the reference block. 16. The device according to claim 14, wherein the acquiring module is specifically used for: When the encoder encodes the target block in a layered encoding manner, the reference block is determined according to a corresponding image block of the target block in a low-quality layered reconstruction image: Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than the reconstructed image of the layer where the target block is located; Acquire motion information of the reference block. 17. The device according to any one of claims 14-16, wherein the motion division module is specifically used for: Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder; If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 18. The device according to any one of claims 14-16, wherein the motion division module is specifically used for: Comparing the value of each sampling point in the residual block with the division threshold value determined by the encoder; If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 19. A device for dividing an image block, characterized in that the device is located in a decoder, and the device comprises: An acquisition module, configured to acquire a reference block of the target block and motion information of the reference block, where sampling points in the reference block correspond to sampling points in the target block; a motion compensation module, configured to perform motion compensation on the reference block acquired by the acquisition module by using the motion information acquired by the acquisition module to obtain a residual block; a motion division module, configured to compare the value of each sampling point in the residual block with the division threshold value determined by the decoder, and perform a process on the target block according to the comparison result of each sampling point motion division, the division threshold value is the median value of all residual values in the residual block; A decoding module, configured to decode the target block according to the result of the motion division. 20. The device according to claim 19, wherein the acquiring module is specifically configured to: Obtain a reference block of the target block; When the encoder encodes the target block in a layered encoding manner, at least one of the following items is acquired as the motion information of the reference block: Motion information of image blocks spatially adjacent to the target block; Motion information of image blocks temporally adjacent to the target block; Motion information of the reference block; Motion information of image blocks spatially adjacent to the reference block; Motion information of image blocks temporally adjacent to the reference block. 21. The device according to claim 19, wherein the acquiring module is specifically used for: When the encoder encodes the target block according to the layered coding method, the reference block is determined according to the corresponding image block of the target block in the low-quality layered reconstruction image: Wherein, at least one index of spatial resolution and signal-to-noise ratio of the low-quality layered reconstructed image is lower than the reconstructed image of the layer where the target block is located; Acquire motion information of the reference block. 22. The device according to any one of claims 19-21, wherein the motion division module is specifically used for: comparing the value of each sampling point in the residual block with the division threshold value determined by the decoder; If the value of the sampling point in the residual block is greater than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is not greater than the division threshold, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block. 23. The device according to any one of claims 19-21, wherein the motion division module is specifically used for: comparing the value of each sampling point in the residual block with the division threshold value determined by the decoder; If the value of the sampling point in the residual block is not less than the division threshold value, then determine that the corresponding sampling point in the target block belongs to the first motion division of the target block; If the value of the sampling point in the residual block is smaller than the division threshold value, it is determined that the corresponding sampling point in the target block belongs to the second motion division of the target block.