WO2019157718A1

WO2019157718A1 - Motion compensation method, device and computer system

Info

Publication number: WO2019157718A1
Application number: PCT/CN2018/076853
Authority: WO
Inventors: 王钊; 马思伟; 郑萧桢
Original assignee: 北京大学; 深圳市大疆创新科技有限公司
Priority date: 2018-02-14
Filing date: 2018-02-14
Publication date: 2019-08-22
Also published as: CN110720221A

Abstract

Provided are a motion compensation method, device and a computer system. The method comprises: determining, according to a motion vector of the current image block, a predicted value of a pixel to be processed, wherein the pixel to be processed is a pixel in a first boundary pixel block of the current image block; determining, according to pixel information of a pixel of an adjacent image block of the current image block, whether to perform overlapping block motion compensation on the predicted value of the pixel to be processed, wherein the first boundary pixel block is adjacent to the adjacent image block. The technical solution of the embodiment of the invention can improve motion compensation performance.

Description

Motion compensation method, device and computer system

Copyright statement

The disclosure of this patent document contains material that is subject to copyright protection. This copyright is the property of the copyright holder. The copyright owner has no objection to the reproduction of the patent document or the patent disclosure in the official records and files of the Patent and Trademark Office.

Technical field

The present invention relates to the field of information technology and, more particularly, to a method, apparatus and computer system for motion compensation.

Background technique

Prediction is an important module of the video coding framework, which is realized by motion compensation. For a frame of image, it is first divided into equal-sized Coding Tree Units (CTUs), such as 64x64, 128x128 size coding regions. Each CTU may be further divided into square or rectangular coding units (CUs), and the CU may be further divided into a prediction unit (PU) or directly as a PU. In order to facilitate uniformity, this article is collectively referred to as an image block. That is to say, the image block can be a CU or a PU. Each image block finds the most similar image block in the reference frame (typically a reconstructed frame near the time domain) as the predicted value of the current image block. The relative position between the current image block and the similar image block is a motion vector (Motion Vector, MV). The process of finding a similar image block as a predicted value of the current image block in the reference frame is motion compensation.

A general motion compensation is to acquire a predicted image block from the motion vector of the image block for each image block. Based on this, an Overlapped Block Motion Compensation (OBMC) technique has appeared. That is, for the pixels of the current image block boundary portion, the motion vector of the current image block and the motion vector weighted prediction of the adjacent image block are used to obtain the predicted value.

However, in the existing OBMC technology, as long as the motion vector of the adjacent image block is different from the current image block, the overlapping block motion compensation must be performed on the boundary pixel block, which limits the performance of the OBMC. Therefore, there is a need for an improved motion compensation method to improve the performance of motion compensation.

Summary of the invention

Embodiments of the present invention provide a method, apparatus, and computer system for motion compensation, which can improve performance of motion compensation.

In a first aspect, a method for motion compensation is provided, including: determining a predicted value of a pixel to be processed according to a motion vector of a current image block, wherein the pixel to be processed is a first boundary pixel block of the current image block Determining whether to perform overlap block motion compensation on the predicted value of the pixel to be processed according to pixel information of a pixel of a neighboring image block of the current image block, wherein the first boundary pixel block is adjacent to the adjacent block The image blocks are adjacent.

In a second aspect, a method for motion compensation is provided, including: determining, according to pixel information of a pixel of a neighboring image block of a current image block, a weighting coefficient of a predicted value of a pixel to be processed, wherein the pixel to be processed is the a pixel in a first boundary pixel block of a current image block, the first boundary pixel block being adjacent to the adjacent image block; determining a predicted value of the pixel to be processed according to the weighting coefficient.

In a third aspect, a device for motion compensation is provided, comprising: a prediction value determining unit, configured to determine a predicted value of a pixel to be processed according to a motion vector of a current image block, wherein the pixel to be processed is the current image a pixel in the first boundary pixel block of the block; a processing unit, configured to determine, according to pixel information of a pixel of the adjacent image block of the current image block, whether to perform overlap block motion compensation on the predicted value of the pixel to be processed, where The first boundary pixel block is adjacent to the adjacent image block.

A fourth aspect provides a motion compensation apparatus, including: a weighting coefficient determining unit, configured to determine, according to pixel information of a pixel of a neighboring image block of a current image block, a weighting coefficient of a predicted value of a pixel to be processed, where The processing pixel is a pixel in a first boundary pixel block of the current image block, the first boundary pixel block is adjacent to the adjacent image block, and a prediction value determining unit is configured to determine according to the weighting coefficient The predicted value of the pixel to be processed.

In a fifth aspect, a computer system is provided, comprising: a memory for storing computer executable instructions; a processor for accessing the memory, and executing the computer executable instructions to perform the first or second The operation in the aspect of the method.

In a sixth aspect, a computer storage medium is provided having stored therein program code, the program code being operative to indicate a method of performing the first or second aspect described above.

The technical solution of the embodiment of the present invention determines whether to perform overlapping block motion compensation on the predicted values of the pixels in the boundary pixel block of the current image block according to the pixel information of the pixels of the adjacent image block of the current image block, and can reasonably utilize the phase The motion vector of the adjacent image block performs overlap block motion compensation, thereby improving the performance of motion compensation.

DRAWINGS

FIG. 1 is an architectural diagram of a technical solution to which an embodiment of the present invention is applied.

2 is a processing architecture diagram of an encoder according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of data to be encoded according to an embodiment of the present invention.

4 is a schematic diagram of a boundary pixel block in accordance with an embodiment of the present invention.

FIG. 5 is a schematic flow chart of a method of motion compensation according to an embodiment of the present invention.

6 is a schematic diagram of a current image block and adjacent image blocks in an embodiment of the present invention.

FIG. 7 is a schematic flow chart of a method of motion compensation according to another embodiment of the present invention.

Figure 8 is a schematic block diagram of an apparatus for motion compensation in accordance with one embodiment of the present invention.

9 is a schematic block diagram of an apparatus for motion compensation according to another embodiment of the present invention.

Figure 10 is a schematic block diagram of a computer system in accordance with an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings.

It should be understood that the specific examples herein are merely intended to provide a better understanding of the embodiments of the invention.

It should be understood that the formulas in the embodiments of the present invention are only examples, and are not intended to limit the scope of the embodiments of the present invention, and the formulas may be modified, and such modifications are also within the scope of the present invention.

It should also be understood that, in various embodiments of the present invention, the size of the sequence numbers of the processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be construed as an embodiment of the present invention. The implementation process constitutes any limitation.

It should be understood that the various embodiments described in the specification may be implemented separately or in combination, and the embodiments of the present invention are not limited thereto.

Unless otherwise indicated, all technical and scientific terms used in the embodiments of the present invention have the same meaning The terminology used in the present application is for the purpose of describing particular embodiments and is not intended to limit the scope of the application. The term "and/or" used in this application includes any and all combinations of one or more of the associated listed.

As shown in FIG. 1, system 100 can receive data to be processed 102 and process data to be processed 102 to produce processed data 108. For example, system 100 can receive data to be encoded, encode the data to be encoded to produce encoded data, or system 100 can receive the data to be decoded and decode the data to be decoded to produce decoded data. In some embodiments, components in system 100 may be implemented by one or more processors, which may be processors in a computing device or processors in a mobile device (eg, a drone). The processor may be any type of processor, which is not limited in this embodiment of the present invention. In some possible designs, the processor may include an encoder or decoder or the like. One or more memories may also be included in system 100. The memory can be used to store instructions and data, such as computer executable instructions to implement the technical solution of the embodiments of the present invention, data to be processed 102, processed data 108, and the like. The memory may be any kind of memory, which is not limited in this embodiment of the present invention.

The data to be encoded may include text, images, graphic objects, animation sequences, audio, video, or any other data that needs to be encoded. In some cases, the data to be encoded may include sensory data from sensors, which may be vision sensors (eg, cameras, infrared sensors), microphones, near field sensors (eg, ultrasonic sensors, radar), position sensors, temperature Sensors, touch sensors, etc. In some cases, the data to be encoded may include information from a user, such as biometric information, which may include facial features, fingerprint scans, retinal scans, voice recordings, DNA sampling, and the like.

Encoding is necessary for efficient and/or secure transmission or storage of data. The encoding of the encoded data may include data compression, encryption, error correction coding, format conversion, and the like. For example, compression of multimedia data, such as video or audio, can reduce the number of bits transmitted in the network. Sensitive data, such as financial information and personally identifiable information, can be encrypted to protect confidentiality and/or privacy prior to transmission and storage. In order to reduce the bandwidth occupied by video storage and transmission, it is necessary to encode and compress the video data.

Any suitable coding technique can be used to encode the data to be encoded. The type of encoding depends on the data being encoded and the specific coding requirements.

In some embodiments, the encoder can implement one or more different codecs. Each codec may include code, instructions or a computer program that implements different encoding algorithms. Based on various factors, including the type and/or source of the data to be encoded, the receiving entity of the encoded data, the available computing resources, the network environment, the business environment, the rules and standards, etc., a suitable encoding algorithm can be selected to encode the given Data to be encoded.

For example, the encoder can be configured to encode a series of video frames. Encoding the data in each frame can take a series of steps. In some embodiments, the encoding step can include processing steps such as prediction, transforming, quantizing, entropy encoding, and the like.

The prediction includes two types of intra prediction and inter prediction, and the purpose is to use the prediction block information to remove redundant information of the current image block to be encoded. Intra prediction uses the information of the current frame image to obtain prediction block data. Inter prediction uses the information of the reference frame to obtain prediction block data, the process comprising dividing the image block to be encoded into a plurality of sub image blocks; and then, for each sub image block, searching the reference image for the image that best matches the current sub image block. The block is used as a prediction block; thereafter, the sub-image block is subtracted from the corresponding pixel value of the prediction block to obtain a residual, and the obtained residuals of the respective sub-image blocks are combined to obtain a residual of the image block.

Transforming the residual block of the image using the transformation matrix can remove the correlation of the residual of the image block, that is, remove the redundant information of the image block, so as to improve the coding efficiency, the transformation of the data block in the image block usually adopts a two-dimensional transformation. That is, the residual information of the data block is multiplied by an NxM transform matrix and its transposed matrix at the encoding end, and the transform coefficients are obtained after multiplication. The transform coefficients are quantized to obtain quantized coefficients, and finally the quantized coefficients are entropy encoded, and finally the entropy-encoded bit stream and the encoded coding mode information, such as intra prediction mode and motion vector information, are performed. Store or send to the decoder. At the decoding end of the image, the entropy-encoded bit stream is first obtained and entropy decoded to obtain a corresponding residual, according to the predicted image block corresponding to the information image block such as the motion vector or the intra prediction obtained by decoding, according to the predicted image block and the image block. The residual is obtained by the value of each pixel in the current sub-image block.

FIG. 2 is a block diagram showing the processing architecture of an encoder according to an embodiment of the present invention. As shown in FIG. 2, the prediction process may include intra prediction and inter prediction. Through the prediction process, a residual corresponding to the data unit (for example, a pixel point) can be obtained, wherein when the prediction is performed on a certain pixel point, the pixel obtained by reconstructing the reference pixel point can be obtained from the stored context, according to the reference pixel point. The pixel obtained after the reconstruction and the pixel of the pixel point obtain the pixel residual corresponding to the pixel point. The pixel residual is subjected to entropy coding by transforming and quantization. In the quantization process, the control of the quantization rate can be achieved by controlling the quantization parameter. The quantized pixel residual corresponding to a certain pixel point may also be subjected to inverse quantization inverse transform processing, and then reconstructed to obtain a pixel reconstructed by the pixel, and the reconstructed pixel of the pixel is stored, so that When the pixel is used as the reference pixel, the pixel reconstructed by the pixel acquires the pixel residual corresponding to the other pixel.

The quantization parameter may include a quantization step size indicating a quantization step size or a value related to the quantization step size, for example, a quantization parameter (QP) in an H.264, H.265, or similar encoder, or a quantization matrix. Or its reference matrix, etc.

For the decoding end, an operation corresponding to the encoding end is performed to decode the encoded data to obtain original data, that is, data to be encoded.

FIG. 3 shows a schematic diagram of data to be encoded in an embodiment of the present invention.

As shown in FIG. 3, the data 302 to be encoded may include a plurality of frames 304. For example, multiple frames 304 may represent consecutive image frames in a video stream. Each frame 304 can include one or more stripes or tiles 306. Each strip or tile 306 may include one or more macroblocks or encoding units 308. Each macroblock or coding unit 308 can include one or more blocks 310. Each block 310 can include one or more pixels 312. Each pixel 312 can include one or more data sets corresponding to one or more data portions, such as a luminance data portion and a chrominance data portion. The data unit can be a frame, a stripe, a tile, a coding unit, a macroblock, a block, a pixel, or a group of any of the above. In various embodiments, the size of the data unit can vary. By way of example, one frame 304 may include 100 stripes 306, each strip 306 may include 10 macroblocks 308, each macroblock 308 may include 4 (eg, 2x2) blocks 310, each block 310 may include 64 (eg, 8x8) pixels 312.

The technical solution of the embodiment of the present invention may be used for motion compensation of pixels in a boundary pixel block of an image block in a prediction process of encoding or decoding.

The existing OBMC technique is to divide the current coded image block into individual 4x4 pixel blocks. According to the prediction mode of the current image block, the existing OBMC is also divided into two types: normal mode and sub-block mode: when the current image block has only one motion vector (for example, normal inter prediction, normal merge mode), OBMC uses normal mode; current image Each 4x4 pixel block of a block has its own motion vector (eg, sub-block merge mode, affine mode and decoder-side motion vector derivation mode), and OBMC also uses sub-block mode. The existing OBMC technology is shown in Figure 4. The normal OBMC mode processes the boundary 4x4 pixel block of the current block, and the predicted value of the 4 row/column pixel of each 4x4 boundary pixel block is changed according to the motion vector of the adjacent 4x4 pixel block. The sub-block OBMC mode processes each 4x4 pixel block of the current image block, and the predicted value of the 2 row/column pixels of each 4x4 block is changed according to the motion vector of the adjacent 4x4 pixel block.

The predicted value after the change is obtained by:

P=a·P _cur +b·P _ner (1)

P _cur and P _ner represent the predicted values obtained from the motion vector of the current 4×4 block (the motion vector of the current image block) and the motion vector of the adjacent 4×4 block (the motion vector of the adjacent image block), respectively, and a and b correspond to each other. The weighting factor, the sum of a and b is 1. P is the final predicted value. Existing OBMC techniques use fixed weighting coefficients. For example, a and b can take the following values:

First line: 3/4, 1/4;

The second line: 7/8, 1/8;

Third line: 15/16, 1/16;

The fourth line: 31/32, 1/32.

The existing OBMC technology ignores the content correlation between the current image block and the adjacent image block, and limits the improvement of coding efficiency. For example, the current image block and the adjacent image block belong to the same object at some proximity, and the 4×4 pixel block at the position should be processed by OBMC; however, at other interfaces, the current image block does not belong to the adjacent image block. When an object is used, it is not appropriate to use the motion vector of the adjacent image block to perform weighted prediction on the boundary pixels of the current image block.

In view of this, embodiments of the present invention provide a method for motion compensation, which considers pixel information of pixels of an image block to improve performance of motion compensation.

FIG. 5 shows a schematic flow chart of a method 500 of motion compensation in accordance with an embodiment of the present invention. The method 500 can be performed by the system 100 shown in FIG.

510. Determine a predicted value of a pixel to be processed according to a motion vector of a current image block, where the pixel to be processed is a pixel in a first boundary pixel block of the current image block.

Specifically, for the pixels in the boundary pixel block of the image block, the corresponding similar block is first determined according to the motion vector of the current image block, and then the predicted value of each pixel in the current boundary pixel block is the pixel corresponding to the position of the similar block. value.

520. Determine, according to the prime information of the pixels of the adjacent image block of the current image block, whether to perform overlap block motion compensation on the predicted value of the pixel to be processed, where the first boundary pixel block and the adjacent image block Adjacent. For example, the first boundary pixel block is an NxM pixel block adjacent to an adjacent image block in the current image block. In one embodiment of the invention, NxM is set to 4x4.

In the embodiment of the present invention, for the pixels in the boundary pixel block of the image block, the overlapping block motion compensation is no longer fixed, but whether the overlapping block motion is performed according to the pixel information of the pixels of the adjacent image block of the current image block. make up. If it is determined that the overlap block motion compensation is performed, the overlap block motion compensation is performed on the predicted value of the pixel to be processed determined according to the motion vector of the current image block in the foregoing step according to the motion vector of the adjacent image block; if it is determined that the overlapping block motion is not performed For compensation, the overlap block motion compensation process can be skipped.

Optionally, in an embodiment of the present invention, determining whether to wait for the pixel according to the pixel information of the pixel of the adjacent image block and the pixel information of the pixel of the second boundary pixel block of the adjacent image block The predicted values of the processed pixels are subjected to overlap block motion compensation, wherein pixels of the adjacent image blocks include pixels of the second boundary pixel block, and the second boundary pixel block is adjacent to the first boundary pixel block. For example, the second boundary pixel block may be an N'xM' pixel block adjacent to the first boundary pixel block in the adjacent image block.

The second boundary pixel block may be a square block, for example, an adjacent 4x4 block in FIG. 6, or may be another block, for example, several rows or columns of pixels adjacent to the current image block in the adjacent image block, The application embodiment is not limited thereto. However, the width M' of the second boundary pixel block does not exceed the width of the adjacent image block, and the height N' of the second boundary pixel block does not exceed the height of the adjacent image block. Alternatively, in one embodiment of the present invention, N'xM' is set to 2x4 (when the neighboring block is adjacent to the current block) or 4x2 (when the neighboring block is adjacent to the left and right of the current block).

In the present embodiment, the difference between the pixel information of the pixels of the second boundary pixel block of the adjacent image block and the pixel information of the pixels of the adjacent image block is compared to determine whether to perform the overlap block motion compensation.

Optionally, the pixel information includes at least one of a pixel average value, a pixel gradient, and a pixel distribution. The pixel information of the pixel reflects the content of the pixel block. In the embodiment of the present invention, at least one of the pixel average value, the pixel gradient, and the pixel distribution may be used to compare the content difference between the second boundary pixel block and the adjacent image block. However, the embodiments of the present invention are not limited thereto.

Optionally, in an embodiment of the present application, if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, the pending The predicted values of the pixels are subjected to overlapping block motion compensation.

Specifically, when the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is larger, that is, greater than the first threshold, the content of the second boundary pixel block may be adjacent to the adjacent image. The body content of the block does not belong to the same object. The first boundary pixel block of the current image block is adjacent to the second boundary pixel block, and therefore, the first boundary pixel block and the two boundary pixel block may belong to the same object. Thus, the first boundary pixel block and the body content of the adjacent image block may not belong to the same object. In this case, it is not necessary to use the motion vector of the adjacent image block to correct the prediction value obtained by the motion vector of the current image block, that is, the prediction value of the pixel to be processed is not subjected to the overlap block motion compensation.

Optionally, in an embodiment of the present application, if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is not greater than a first threshold, The predicted values of the pixels are processed for overlap block motion compensation.

When the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is small, that is, when the value is not greater than the first threshold, it indicates that the content of the second boundary pixel block may be related to the main content of the adjacent image block. Belong to the same object. The first boundary pixel block of the current image block is adjacent to the second boundary pixel block, and therefore, the first boundary pixel block and the two boundary pixel block may belong to the same object. Thus, the first boundary pixel block and the body content of the adjacent image block may belong to the same object. In this case, it is necessary to use the motion vector of the adjacent image block to correct the prediction value obtained by the motion vector of the current image block, that is, it is necessary to perform the overlap block motion compensation on the predicted value of the pixel to be processed.

It should be understood that all pixels in the first boundary pixel block are determined based on the second boundary pixel block, and therefore, whether or not to perform overlapping block motion compensation may be determined according to the granularity of the boundary pixel block. That is to say, for the first boundary pixel block, only one determination can be made, and the determination result is applied to all the pixels in the first boundary pixel block.

Pixel average value as a pixel average value of the pixel information, the second boundary pixel blocks 4x4 blocks, for example, as shown in FIG. 6 first calculates the second block boundary pixel (4x4 neighboring blocks in FIG. 6), denoted V ₁ of each Then, calculate the pixel average of the entire adjacent image block, denoted as V ₂ . Whether or not the first boundary pixel block to be processed (the 4x4 boundary pixel block adjacent to the adjacent 4x4 block in FIG. 6) performs OBMC processing is determined by the following formula:

Where T _h is a configurable threshold. When the difference between V ₁ and V ₂ is not greater than the threshold value T _h , it indicates that the content difference between the adjacent 4×4 block and the adjacent image block is not large, so the motion vector of the adjacent image block may represent the adjacent 4×4 block. The real movement situation. In this case, OBMC is performed on the boundary pixel block. When the difference between V ₁ and V ₂ is greater than the threshold value T _h , it indicates that there is a large content difference between the adjacent 4×4 block and the adjacent image block, so the motion vector of the adjacent image block may not conform to the adjacent 4×4 block. The real movement situation. In this case, the OBMC processing of the predicted values of the boundary pixel blocks should not be performed again using the motion vectors of the adjacent image blocks, and can be skipped directly.

Alternatively, as an embodiment of the present invention, the threshold T _h may be set to 15.

Optionally, the threshold in the present invention may be used as a preset value of the encoding end and the decoding end, or the threshold information may be written into the code stream at the encoding end, and the decoding end obtains the threshold information from the code stream. At the encoding end, it can be written in the video parameter set (VPS, video parameter set), sequence parameter set (SPS, sequence parameter set), picture parameter set (PPS, picture parameter set), sequence header, image header, strip header, etc. The threshold information.

It should be understood that, for the case where the overlap block motion compensation is not performed, the weighting coefficient corresponding to the adjacent image block is 0. Based on this, the embodiment of the present invention further provides a motion compensation method, which will be described below.

FIG. 7 shows a schematic flow chart of a method 700 of motion compensation in accordance with another embodiment of the present invention. The method 700 can be performed by the system 100 shown in FIG.

710. Determine a weighting coefficient of a pixel prediction value to be processed according to pixel information of a pixel of a neighboring image block of the current image block, where the to-be-processed pixel is a pixel in a first boundary pixel block of the current image block, The first boundary pixel block is adjacent to the adjacent image block. For example, the first boundary pixel block is a 4x4 pixel block adjacent to an adjacent image block in the current image block.

In the embodiment of the present invention, for the pixels in the boundary pixel block of the image block, the weighting coefficients are determined according to the pixel information of the pixels of the adjacent image blocks of the current image block.

The weighting coefficient may include a first coefficient and a second coefficient, the first coefficient is used to weight a first predicted value of the pixel to be processed determined according to a motion vector of the current image block (ie, a formula (1) a), the second coefficient is used to weight a second predicted value of the pixel to be processed determined according to a motion vector of the adjacent image block (ie, b in the formula (1)).

Optionally, in an embodiment of the present invention, the weighting coefficient may be determined according to pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block, The pixel of the adjacent image block includes a pixel of the second boundary pixel block, and the second boundary pixel block is adjacent to the first boundary pixel block. For example, the second boundary pixel block may be an MxN pixel block adjacent to the first boundary pixel block in the adjacent image block.

In this embodiment, the difference between the pixel information of the pixels of the second boundary pixel block of the adjacent image block and the pixel information of the pixels of the adjacent image block is compared, and the weighting coefficient is determined.

Optionally, the pixel information includes at least one of a pixel average value, a pixel gradient, and a pixel distribution. The pixel information reflects the content of the pixel block. In the embodiment of the present invention, at least one of the pixel average value, the pixel gradient, and the pixel distribution may be used to compare the content difference between the second boundary pixel block and the adjacent image block, but The embodiment of the invention is not limited thereto.

Optionally, in an embodiment of the present application, if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining the first The coefficient is 1, and the second coefficient is zero.

Specifically, when the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is larger, that is, greater than the first threshold, the content of the second boundary pixel block may be adjacent to the adjacent image. The body content of the block does not belong to the same object. The first boundary pixel block of the current image block is adjacent to the second boundary pixel block, and therefore, the first boundary pixel block and the two boundary pixel block may belong to the same object. Thus, the first boundary pixel block and the body content of the adjacent image block may not belong to the same object. In this case, it is not necessary to use the motion vector of the adjacent image block to correct the predicted value obtained by the motion vector of the current image block, that is, the first coefficient is 1, and the second coefficient is zero.

Optionally, in an embodiment of the present application, if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is not greater than a first threshold, determining the A coefficient is a first predetermined value and the second coefficient is a second predetermined value, wherein the first predetermined value is less than one and the second predetermined value is greater than zero.

When the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is small, that is, when the value is not greater than the first threshold, it indicates that the content of the second boundary pixel block may be related to the main content of the adjacent image block. Belong to the same object. The first boundary pixel block of the current image block is adjacent to the second boundary pixel block, and therefore, the first boundary pixel block and the two boundary pixel block may belong to the same object. Thus, the first boundary pixel block and the body content of the adjacent image block may belong to the same object. In this case, it is necessary to use the motion vector of the adjacent image block to correct the predicted value obtained by the motion vector of the current image block, that is, the second coefficient is not zero. For example, preset values of the first coefficient and the second coefficient may be employed.

It should be understood that all pixels in the first boundary pixel block are determined based on the second boundary pixel block, and therefore, whether the second coefficient is zero or not can be determined according to the granularity of the boundary pixel block. That is to say, for the first boundary pixel block, only one determination can be made, and the determination result is applied to all the pixels in the first boundary pixel block.

Pixel average value as a pixel average value of the pixel information, the second boundary pixel blocks 4x4 blocks, for example, as shown in FIG. 6 first calculates the second block boundary pixel (4x4 neighboring blocks in FIG. 6), denoted V ₁ of each Then, calculate the pixel average of the entire adjacent image block, denoted as V ₂ . If |V ₁ -V ₂ |>T _h , the second coefficient b is zero; if |V ₁ -V ₂ | ≤T _h , then b is not zero, for example, a preset value may be employed.

Alternatively, as an embodiment of the present invention, the threshold T _h may be set to 15. The preset values of the first coefficient a and the second coefficient b may be: first row/column: 3/4, 1/4; second row/column: 6/7, 1/7; third row/column: 11/12, 1/12; fourth row/column: 19/20, 1/20.

720. Determine a predicted value of the pixel to be processed according to the weighting coefficient.

After the weighting coefficient is obtained in the foregoing manner, the weighting coefficient is used to determine the predicted value of the pixel to be processed.

Specifically, a first predicted value of the pixel to be processed may be determined according to a motion vector of the current image block; and a second predicted value of the pixel to be processed is determined according to a motion vector of the adjacent image block; And weighting the first predicted value and the second predicted value according to the weighting coefficient to obtain a predicted value of the pixel to be processed.

For example, for one pixel in the boundary pixel block of the current image block, the corresponding similar block may be determined according to the motion vector of the current image block, and the value of the corresponding pixel of the pixel in the similar block is the first predicted value P _cur ; Similarly, the second predicted value P _ner can be obtained according to the motion vector of the adjacent image block; and the weighting coefficients a and b are obtained by the foregoing manner, and the predicted value P of the pixel is obtained according to the formula (1).

After obtaining the predicted value of the pixel in the above manner, the encoding end may perform encoding based on the predicted value, and for the decoding end, decoding may be performed based on the predicted value.

The technical solution of the embodiment of the present invention determines the weighting coefficient of the pixel in the boundary pixel block of the current image block according to the pixel information of the pixel of the adjacent image block of the current image block, and can reasonably utilize the motion vector of the adjacent image block. Overlap block motion compensation, which improves the performance of motion compensation.

When the technical solution of the embodiment of the present invention is used, the encoding end and the decoding end perform similar operations, and no additional information needs to be written in the code stream, so that no additional overhead is incurred.

The method of motion compensation of the embodiment of the present invention is described in detail above, and the motion compensation apparatus and computer system of the embodiment of the present invention will be described below.

FIG. 8 shows a schematic block diagram of a motion compensated device 800 in accordance with one embodiment of the present invention. The apparatus 800 can perform the motion compensation method 500 of the embodiments of the present invention described above.

As shown in FIG. 8, the apparatus 800 can include:

a predicted value determining unit 810, configured to determine a predicted value of a pixel to be processed according to a motion vector of a current image block, where the pixel to be processed is a pixel in a first boundary pixel block of the current image block;

The processing unit 820 is configured to determine, according to pixel information of a pixel of a neighboring image block of the current image block, whether to perform overlap block motion compensation on the predicted value of the pixel to be processed, where the first boundary pixel block is Adjacent image blocks are adjacent.

Optionally, in an embodiment of the present invention, the processing unit 820 is specifically configured to:

Determining whether to perform overlapping block motion compensation on the predicted value of the pixel to be processed, according to the pixel information of the pixel of the adjacent image block and the pixel information of the pixel of the second boundary pixel block of the adjacent image block, where The pixels of the adjacent image block include pixels of the second boundary pixel block, and the second boundary pixel block is adjacent to the first boundary pixel block.

Optionally, in an embodiment of the present invention, the pixel information includes at least one of a pixel average value, a pixel gradient, and a pixel distribution.

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than the first threshold, the predicted value of the pixel to be processed is not subjected to overlap block motion compensation.

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is not greater than the first threshold, the predicted value of the pixel to be processed is subjected to overlap block motion compensation.

FIG. 9 shows a schematic block diagram of a motion compensated device 900 in accordance with another embodiment of the present invention. The apparatus 900 can perform the method 700 of motion compensation of the embodiments of the present invention described above.

As shown in FIG. 9, the apparatus 900 can include:

The weighting coefficient determining unit 910 is configured to determine a weighting coefficient of the pixel predicted value of the pixel to be processed according to the pixel information of the pixel of the adjacent image block of the current image block, where the pixel to be processed is the first boundary of the current image block a pixel in a pixel block, the first boundary pixel block being adjacent to the adjacent image block;

The predicted value determining unit 920 is configured to determine a predicted value of the pixel to be processed according to the weighting coefficient.

Optionally, in an embodiment of the present invention, the predicted value determining unit 920 is specifically configured to:

Determining, according to a motion vector of the current image block, a first predicted value of the pixel to be processed;

Determining, according to a motion vector of the adjacent image block, a second predicted value of the pixel to be processed;

And performing weighted summation on the first predicted value and the second predicted value according to the weighting coefficient to obtain a predicted value of the pixel to be processed, wherein the predicted value weighting coefficient includes a first coefficient and a second a coefficient, the first coefficient is used to weight the first predicted value, and the second coefficient is used to weight the second predicted value.

Optionally, in an embodiment of the present invention, the weighting coefficient determining unit 910 is specifically configured to:

Determining the weighting coefficient according to pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block, wherein a pixel of the adjacent image block includes the a pixel of a second boundary pixel block adjacent to the first boundary pixel block.

And if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is 1, and the second coefficient is zero.

Determining, when the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is a first predetermined value, the second The coefficient is a second predetermined value, wherein the first predetermined value is less than one and the second predetermined value is greater than zero.

It should be understood that the apparatus for motion compensation according to the embodiment of the present invention may be a chip, which may be specifically implemented by a circuit, but the specific implementation manner of the embodiment of the present invention is not limited.

An embodiment of the present invention also provides an encoder comprising the motion compensation apparatus of the various embodiments of the present invention described above.

Embodiments of the present invention also provide a decoder including the motion compensation apparatus of the various embodiments of the present invention described above.

FIG. 10 shows a schematic block diagram of a computer system 1000 in accordance with an embodiment of the present invention.

As shown in FIG. 10, the computer system 1000 can include a processor 1010 and a memory 1020.

It should be understood that the computer system 1000 may also include components that are generally included in other computer systems, such as input and output devices, communication interfaces, and the like, which are not limited by the embodiments of the present invention.

Memory 1020 is for storing computer executable instructions.

The memory 1020 may be various kinds of memories, for example, may include a high speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, which is implemented by the present invention. This example is not limited to this.

The processor 1010 is configured to access the memory 1020 and execute the computer executable instructions to perform the operations in the motion compensation method of the embodiment of the present invention described above.

The processor 1010 may include a microprocessor, a Field-Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), etc., and is implemented by the present invention. This example is not limited to this.

The motion compensated apparatus and computer system of an embodiment of the present invention may correspond to an execution subject of the motion compensation method of the embodiment of the present invention, and the above and other operations and/or functions of the motion compensation apparatus and each module in the computer system respectively In order to implement the corresponding processes of the foregoing various methods, for brevity, no further details are provided herein.

Embodiments of the present invention also provide an electronic device that can include the motion compensated device or computer system of the various embodiments of the present invention described above.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program code, and the program code can be used to indicate a method for performing motion compensation according to the embodiment of the invention.

It should be understood that in the embodiment of the present invention, the term "and/or" is merely an association relationship describing an associated object, indicating that there may be three relationships. For example, A and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent person can be easily conceived within the technical scope of the present invention by any person skilled in the art. Modifications or substitutions are intended to be included within the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

A method of motion compensation, comprising:

Determining, according to a motion vector of the current image block, a predicted value of the pixel to be processed, wherein the pixel to be processed is a pixel in a first boundary pixel block of the current image block;

Determining whether to perform overlap block motion compensation on the predicted value of the pixel to be processed according to pixel information of a pixel of a neighboring image block of the current image block, wherein the first boundary pixel block is adjacent to the adjacent image block .
The method according to claim 1, wherein the determining whether to perform the overlap block motion compensation on the predicted value of the pixel to be processed according to the pixel information of the pixel of the adjacent image block of the current image block comprises:

Determining whether to perform overlapping block motion compensation on the predicted value of the pixel to be processed, according to the pixel information of the pixel of the adjacent image block and the pixel information of the pixel of the second boundary pixel block of the adjacent image block, where The pixels of the adjacent image block include pixels of the second boundary pixel block, and the second boundary pixel block is adjacent to the first boundary pixel block.
The method of claim 2, wherein the pixel information comprises at least one of a pixel average, a pixel gradient, and a pixel distribution.
The method according to claim 2 or 3, wherein the determining is based on pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block Performing overlapping block motion compensation on the predicted value of the pixel to be processed, including:

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than the first threshold, the predicted value of the pixel to be processed is not subjected to overlap block motion compensation.
The method according to claim 2 or 3, wherein the determining is based on pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block Performing overlapping block motion compensation on the predicted value of the pixel to be processed, including:

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is not greater than the first threshold, the predicted value of the pixel to be processed is subjected to overlap block motion compensation.
A method of motion compensation, comprising:

Determining a weighting coefficient of a pixel prediction value to be processed according to pixel information of a pixel of a neighboring image block of the current image block, wherein the pixel to be processed is a pixel in a first boundary pixel block of the current image block, a first boundary pixel block adjacent to the adjacent image block;

A predicted value of the pixel to be processed is determined according to the weighting coefficient.
The method according to claim 6, wherein the determining the predicted value of the pixel to be processed according to the weighting coefficient comprises:

Determining, according to a motion vector of the current image block, a first predicted value of the pixel to be processed;

Determining, according to a motion vector of the adjacent image block, a second predicted value of the pixel to be processed;

And performing weighted summation on the first predicted value and the second predicted value according to the weighting coefficient to obtain a predicted value of the pixel to be processed, wherein the predicted value weighting coefficient includes a first coefficient and a second a coefficient, the first coefficient is used to weight the first predicted value, and the second coefficient is used to weight the second predicted value.
The method according to claim 7, wherein the determining the weighting coefficient of the predicted value of the pixel to be processed comprises:

Determining the weighting coefficient according to pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block, wherein a pixel of the adjacent image block includes the a pixel of a second boundary pixel block adjacent to the first boundary pixel block.
The method of claim 8, wherein the pixel information comprises at least one of a pixel average, a pixel gradient, and a pixel distribution.
The method according to claim 8 or 9, wherein said determining said weighting coefficient comprises:

And if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is 1, and the second coefficient is zero.
The method according to claim 8 or 9, wherein said determining said weighting coefficient comprises:

Determining, when the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is a first predetermined value, the second The coefficient is a second predetermined value, wherein the first predetermined value is less than one and the second predetermined value is greater than zero.
A motion compensation device, comprising:

a prediction value determining unit, configured to determine a predicted value of a pixel to be processed according to a motion vector of the current image block, wherein the pixel to be processed is a pixel in a first boundary pixel block of the current image block;

a processing unit, configured to determine, according to pixel information of a pixel of a neighboring image block of the current image block, whether to perform overlap block motion compensation on the predicted value of the pixel to be processed, wherein the first boundary pixel block and the phase The adjacent image blocks are adjacent.
The device according to claim 12, wherein the processing unit is specifically configured to:

Determining whether to perform overlapping block motion compensation on the predicted value of the pixel to be processed, according to the pixel information of the pixel of the adjacent image block and the pixel information of the pixel of the second boundary pixel block of the adjacent image block, where The pixels of the adjacent image block include pixels of the second boundary pixel block, and the second boundary pixel block is adjacent to the first boundary pixel block.
The apparatus of claim 13, wherein the pixel information comprises at least one of a pixel average value, a pixel gradient, and a pixel distribution.
The device according to claim 13 or 14, wherein the processing unit is specifically configured to:

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than the first threshold, the predicted value of the pixel to be processed is not subjected to overlap block motion compensation.
The device according to claim 13 or 14, wherein the processing unit is specifically configured to:

If the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is not greater than the first threshold, the predicted value of the pixel to be processed is subjected to overlap block motion compensation.
A motion compensation device, comprising:

a weighting coefficient determining unit, configured to determine a weighting coefficient of a pixel prediction value to be processed according to pixel information of a pixel of a neighboring image block of the current image block, wherein the pixel to be processed is a first boundary pixel of the current image block a pixel in the block, the first boundary pixel block being adjacent to the adjacent image block;

a predicted value determining unit configured to determine a predicted value of the pixel to be processed according to the weighting coefficient.
The apparatus according to claim 17, wherein the predicted value determining unit is specifically configured to:

Determining, according to a motion vector of the current image block, a first predicted value of the pixel to be processed;

Determining, according to a motion vector of the adjacent image block, a second predicted value of the pixel to be processed;

And performing weighted summation on the first predicted value and the second predicted value according to the weighting coefficient to obtain a predicted value of the pixel to be processed, wherein the predicted value weighting coefficient includes a first coefficient and a second a coefficient, the first coefficient is used to weight the first predicted value, and the second coefficient is used to weight the second predicted value.
The apparatus according to claim 18, wherein the weighting coefficient determining unit is specifically configured to:

Determining the weighting coefficient according to pixel information of a pixel of the adjacent image block and pixel information of a pixel of a second boundary pixel block of the adjacent image block, wherein a pixel of the adjacent image block includes the a pixel of a second boundary pixel block adjacent to the first boundary pixel block.
The apparatus of claim 19, wherein the pixel information comprises at least one of a pixel average, a pixel gradient, and a pixel distribution.
The apparatus according to claim 19 or 20, wherein the weighting coefficient determining unit is specifically configured to:

And if the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is 1, and the second coefficient is zero.
The apparatus according to claim 19 or 20, wherein the weighting coefficient determining unit is specifically configured to:

Determining, when the difference between the pixel information of the pixel of the second boundary pixel block and the pixel information of the pixel of the adjacent image block is greater than a first threshold, determining that the first coefficient is a first predetermined value, the second The coefficient is a second predetermined value, wherein the first predetermined value is less than one and the second predetermined value is greater than zero.
A computer system, comprising:

a memory for storing computer executable instructions;

A processor for accessing the memory and executing the computer executable instructions to perform the operations in the method of any one of claims 1 to 11.