WO2018120290A1

WO2018120290A1 - Prediction method and device based on template matching

Info

Publication number: WO2018120290A1
Application number: PCT/CN2017/070735
Authority: WO
Inventors: 林永兵
Original assignee: 华为技术有限公司
Priority date: 2016-12-26
Filing date: 2017-01-10
Publication date: 2018-07-05

Abstract

Provided are a prediction method and device based on template matching. The prediction method comprises: performing match-searching with respect to a current template in a first reference image of a to-be-processed unit, to obtain first motion information, a first matching template, and a first template distortion value of the to-be-processed unit; updating a pixel value of the current template according to a pixel value of the first matching template; performing match-searching with respect to the updated current template in a second reference image of the to-be-processed unit, to obtain second motion information, a second matching template, and a second template distortion value of the to-be-processed unit; and when the second template distortion value is less than the first template distortion value, determining a weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, such that prediction accuracy is improved, further improving a coding and decoding effect.

Description

Prediction method and device based on template matching

Technical field

The present application relates to the field of video image technologies, and in particular, to a prediction method and apparatus based on template matching.

Background technique

The most important performance metric of video coding compression technology is the compression ratio, which enables the transmission of the highest quality video content with the least bandwidth. The improvement of the compression ratio is achieved by removing the redundant information of the video content itself. The mainstream technical framework of video coding compression standard adopts image block-based hybrid video coding scheme. The main technologies of video coding include: prediction, transform and quantization, entropy coding, spatial and temporal correlation through prediction technology, and transformation and quantization. The technology removes the frequency domain correlation and further removes information redundancy between codewords by entropy coding techniques.

As the compression ratio of video coding continues to increase, the proportion of motion information in the code stream is increasing. Based on the template matching (TM) motion information prediction technology, motion information can be derived at the decoding end, and motion information does not need to be transmitted, which greatly saves coding bits and improves compression ratio. TM-based motion information prediction technology has become one of the candidate technologies for next-generation video coding standards.

In general, "L" type templates are used in template matching techniques. As shown in FIG. 1, the left adjacent block and the upper adjacent block of the current block (curBlock) constitute an "L" type template, and the template is used to perform a matching search in the reference image Ref0 to obtain the The motion information of the template. Since the spatial position of the template and the current block are in close proximity, their motion information should be consistent, so the motion information of the template can be used as the motion information of the current block.

The above "L" type template, including the left neighboring block and the upper neighboring block of the current block, is generated by decoding and reconstruction, and is available at both the encoding end and the decoding end. Therefore, the method based on template matching can be performed at the decoding end to obtain motion information of the current block without transmitting the motion information.

However, when the template matching search is performed, only the forward or backward reference image is used, which is a one-way search, and the subsequent prediction uses the bidirectional prediction of the forward and backward reference images to make the template matching search and motion prediction. Mismatch. Affects prediction accuracy and prediction results.

Summary of the invention

The embodiment of the present application provides a prediction method and device based on template matching, so as to solve the problem that the template matching search does not match the motion prediction, and improve the prediction accuracy.

In a first aspect, a prediction method based on template matching is provided, including: performing a matching search of a current template in a first reference image of a to-be-processed unit, obtaining first motion information, a first matching template, and a first template distortion value, where the current template includes a plurality of reconstructed pixels of a preset position and number in a neighborhood of the to-be-processed unit, the first template distortion value representing the first matching template and the current a difference of the template; updating the pixel value of the current template according to the pixel value of the first matching template; performing a matching search of the updated current template in the second reference image of the to-be-processed unit, obtaining a second motion information, a second matching template, and a second template distortion value of the to-be-processed unit, where the second template distortion value represents a difference between the second matching template and the updated current template; When the second template distortion value is smaller than the first template distortion value, according to the first motion information and the Determining, by the second motion information, a weighted prediction value of the to-be-processed unit.

The beneficial effect is that when the codec device acquires the predicted value of the unit to be processed, when searching and predicting the to-be-processed unit by using the template matching method, the problem of mismatch between search and prediction can be avoided, thereby improving prediction accuracy and prediction. The effect is to further improve the quality of the codec.

With reference to the first aspect, in a possible design, the updating, according to the pixel value of the first matching template, the pixel value of the current template, including updating the current pixel value in the following manner: T1 = (T0 - ω0 × P1) / (1 - ω0), where T1 represents the pixel value of the updated current template, T0 represents the pixel value of the current template, and P1 represents the pixel of the first matching template. The value, ω0 represents a weighting coefficient corresponding to the first matching template, and ω0 is a positive number less than one.

The beneficial effect is that compared with the prior art, the update operation of the current template is added, and the template matching operation performed after the template update is equivalent to the two-way matching search, and the two-way search process of template matching can be implemented relatively simply, and the two-way search process is further combined. Forecast, improve forecasting accuracy and forecasting results.

In conjunction with the first aspect, in one possible design, the ω0 is 0.5.

The beneficial effect is that the updated template obtained by the first matching template with the weighting coefficient of 0.5 is set, and the template matching operation performed after the template updating is equivalent to the standard two-way matching search process, and the prediction precision and the prediction effect are combined with the bidirectional prediction. better.

With reference to the first aspect, in a possible design, the matching search of the current template is performed in the first reference image of the to-be-processed unit, and the first motion information, the first matching template, and the first The template distortion value includes: traversing a plurality of first candidate reconstructed pixel combinations having the same size and shape in the reconstructed pixel of the preset range of the first reference image, and calculating the plurality of a first candidate reconstructed pixel combination and a plurality of first pixel difference values of the current template; determining the first template distortion value, the first matching template according to the smallest first pixel difference value; Determining, by the first matching template, a position vector difference of the current template, and an image of the first matching template, determining the first motion information, where the first matching template includes the first template distortion value The first candidate reconstructs the pixel combination.

With reference to the first aspect, in a possible design, the matching search of the updated current template is performed in the second reference image of the to-be-processed unit, and the second motion information of the to-be-processed unit is obtained, The second matching template and the second template distortion value, including: traversing a plurality of second candidate reconstructed pixels having the same size and shape as the current template in the reconstructed pixels of the preset range of the second reference image Combining, calculating a plurality of second candidate pixel combinations and a plurality of second pixel difference values of the updated current template; determining the second template distortion value according to the smallest second pixel difference value And the second matching template; determining the second motion information according to the position vector difference between the second matching template and the updated current template, and the image of the second matching template, The second matching template includes a second candidate reconstructed pixel combination corresponding to the second template distortion value.

With reference to the first aspect, in a possible design, the determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, includes: according to the first motion information, Obtaining, in the first reference image, a first prediction value of the to-be-processed unit; and obtaining, in the second reference image, a second-direction prediction of the to-be-processed unit according to the second motion information And performing weighting calculation on the first predicted value and the second predicted value to obtain a weighted predicted value of the to-be-processed unit.

The beneficial effect is that the weighted prediction value of the to-be-processed unit is directly used as the predicted value of the to-be-processed unit. At this time, the template matching search process is actually equivalent to the two-way search, and the first reference image is utilized in essence at the same time. And the second reference image, the implementation is simple, and at the same time, the prediction accuracy and the prediction effect can be improved.

With reference to the first aspect, in a possible design, a matching search of the current template is performed in the first reference image of the to-be-processed unit, and after obtaining the first motion information of the to-be-processed unit and the first matching template, The method includes: traversing, in a reconstructed pixel of a preset range of the third reference image, traversing a plurality of third candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of third candidate reconstructed pixels Combining a plurality of third pixel difference values of the current template; determining the third template distortion value, the third matching template according to the smallest third pixel difference value; according to the third matching template and The third vector information is determined by the position vector difference of the current template, and the image of the third matching template, where the third matching template includes the third candidate reconstructed pixel corresponding to the third template distortion value. combination.

With reference to the first aspect, in a possible design, after determining the third motion information according to the position vector difference of the third matching template and the current template, and the image of the third matching template, When the third template distortion value is less than or equal to the first template distortion value, further comprising: using the third reference image as the first reference image; using the third motion information as the first The motion information; the third template distortion value is used as the first template distortion value; and the third matching template is used as the first matching template.

With reference to the first aspect, in a possible design, when the second template distortion value is not less than the first template distortion value, the method includes: according to the first motion information, in the first reference image, Obtaining a predicted value of the unit to be processed.

The beneficial effect is that the adaptive selection can be performed in the one-way search one-way prediction mode and the two-way search two-way prediction mode. At this time, the prediction accuracy and the prediction effect are improved in the case where the implementation complexity is low.

With reference to the first aspect, in a possible design, when the second template distortion value is smaller than the first template distortion value, determining the according to the first motion information and the second motion information The weighted prediction value of the to-be-processed unit includes: when the second template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, the weighting is performed The predicted value is used as a predicted value of the unit to be processed.

With reference to the first aspect, in a possible design, when the first template distortion value, the second template distortion value, and the third template distortion value are the first template distortion value is the smallest, according to the Determining, in the first reference image, a predicted value of the to-be-processed unit; or, when the first template distortion value, the second template distortion value, and the third template distortion In the value, when the third template distortion value is the smallest, the predicted value of the to-be-processed unit is obtained in the third reference image according to the third motion information.

The beneficial effect is that adaptive selection can be performed in two different one-way search unidirectional prediction modes and two-way search bidirectional prediction mode. At this time, although the implementation complexity is high, there is better adaptation. Selectivity to improve the predictive effect and accuracy.

In conjunction with the first aspect, in a possible design, when the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward direction of the to-be-processed unit a reference image list; or, when the first reference image is from a backward reference image list of the to-be-processed unit, the third reference image is from a forward reference image list of the to-be-processed unit.

With reference to the first aspect, in a possible design, the matching search of the updated current template is performed in the second reference image of the to-be-processed unit, and the second motion information of the to-be-processed unit is obtained. After the second matching template and the second template distortion value, the method further includes: updating the second template distortion value according to the weighting coefficient, where the updated second template distortion value is (1-ω0) times And the second template distortion value; correspondingly, when the second template distortion value is smaller than the first template distortion value, determining the to-before according to the first motion information and the second motion information The weighted prediction value of the processing unit includes: when the updated second template distortion value is smaller than the first When the template distortion value is obtained, the weighted prediction value of the to-be-processed unit is determined according to the first motion information and the second motion information.

The beneficial effect is that the second template distortion value is updated according to the weighting coefficient at the time of template update, so that the comparison result of the distortion value is more accurate.

With reference to the first aspect, in a possible design, when the updated second template distortion value is not less than the first template distortion value, the method further includes: according to the first motion information, In the first reference image, the predicted value of the to-be-processed unit is obtained.

With reference to the first aspect, in a possible design, when the updated second template distortion value is smaller than the first template distortion value, determining, according to the first motion information and the second motion information, The weighted prediction value of the processing unit includes: the updated second template distortion value in the first template distortion value, the updated second template distortion value, and the third template distortion value At the very least, the weighted prediction value is taken as the predicted value of the unit to be processed.

With reference to the first aspect, in a possible design, the method further includes: when the first template distortion value, the updated second template distortion value, and the third template distortion value, the first template distortion When the value is minimum, obtaining a predicted value of the to-be-processed unit in the first reference image according to the first motion information; or, when the first template distortion value, the updated second template In the distortion value and the third template distortion value, when the third template distortion value is the smallest, the predicted value of the to-be-processed unit is obtained in the third reference image according to the third motion information.

In a second aspect, a template matching-based prediction apparatus is provided, including: a search unit, configured to perform a matching search of a current template in a first reference image of a to-be-processed unit, to obtain first motion information of the to-be-processed unit, a first matching template and a first template distortion value, the current template including a plurality of reconstructed pixels of a preset position and number in a neighborhood of the to-be-processed unit, the first template distortion value representing the first matching a difference between the template and the current template; an update unit, configured to update a pixel value of the current template according to a pixel value of the first matching template; the searching unit is further configured to be used in the to-be-processed unit Performing a matching search of the updated current template in the second reference image, obtaining second motion information, a second matching template, and a second template distortion value of the to-be-processed unit, where the second template distortion value represents the a difference between the second matching template and the updated current template; a determining unit, configured to: when the second template distortion value is smaller than the first template distortion value, according to the First motion information and the second motion information is determined to be the weighted prediction value processing unit.

With reference to the second aspect, in a possible design, the updating unit, when updating the pixel value of the current template according to the pixel value of the first matching template, includes using the following manner to the current pixel value. Update: T1=(T0-ω0×P1)/(1-ω0), where T1 represents the pixel value of the updated current template, T0 represents the pixel value of the current template, and P1 represents the first Matching the pixel value of the template, ω0 represents the weighting coefficient corresponding to the first matching template, and ω0 is a positive number less than 1.

In conjunction with the second aspect, in one possible design, the ω0 is 0.5.

With reference to the second aspect, in a possible design, the searching unit performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first motion information, the first matching template, and the to-be-processed unit. The first template distortion value is specifically used to: traverse the plurality of first candidate reconstructed pixel combinations having the same size and shape in the reconstructed pixel of the preset range of the first reference image, Calculating a plurality of first candidate reconstructed pixel combinations and a plurality of first pixel difference values of the current template; determining the first template distortion value, the first according to the smallest first pixel difference value Matching a template; according to a position vector difference between the first matching template and the current template, And determining, by the first matching template, the first motion template, where the first matching template includes a first candidate reconstructed pixel combination corresponding to the first template distortion value.

With reference to the second aspect, in a possible design, the searching unit performs a matching search of the updated current template in a second reference image of the to-be-processed unit to obtain a second motion of the to-be-processed unit. The information, the second matching template, and the second template distortion value are specifically used to: traverse the plurality of the same size and shape of the current template in the reconstructed pixels of the preset range of the second reference image a second candidate reconstructed pixel combination, calculating a plurality of second candidate reconstructed pixel combinations and a plurality of second pixel difference values of the updated current template; determining, according to the smallest second pixel difference value a second template distortion value, the second matching template; determining the second motion according to a position vector difference between the second matching template and the updated current template, and an image of the second matching template Information, the second matching template includes a second candidate reconstructed pixel combination corresponding to the second template distortion value.

With reference to the second aspect, in a possible design, the determining unit, when determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, is specifically used to: Determining a first motion information, in the first reference image, obtaining a first prediction value of the to-be-processed unit; and obtaining, in the second reference image, the to-be-processed according to the second motion information a second direction prediction value of the unit; performing weighting calculation on the first direction prediction value and the second direction prediction value to obtain a weighted prediction value of the to-be-processed unit.

With reference to the second aspect, in a possible design, the search unit performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first motion information and the first match of the to-be-processed unit. After the template is further configured to: traverse the plurality of third candidate reconstructed pixel combinations having the same size and shape as the current template in the reconstructed pixels of the preset range of the third reference image, and calculate the plurality of a third candidate reconstructed pixel combination and a plurality of third pixel difference values of the current template; determining the third template distortion value and the third matching template according to the smallest third pixel difference value; And determining, by the third matching template, a position vector difference of the current template, and the image of the third matching template, determining the third motion information, where the third matching template includes the third template distortion value corresponding to the third Three candidate reconstructed pixel combinations.

With reference to the second aspect, in a possible design, the searching unit determines the third in the image according to a position vector difference between the third matching template and the current template, and an image in which the third matching template is located. After the motion information, when the third template distortion value is less than or equal to the first template distortion value, further used to: use the third reference image as the first reference image; and the third motion information As the first motion information; using the third template distortion value as the first template distortion value; and using the third matching template as the first matching template.

With reference to the second aspect, in a possible design, when the second template distortion value is not less than the first template distortion value, the determining unit is specifically configured to: according to the first motion information, in the In the first reference image, a predicted value of the unit to be processed is obtained.

With reference to the second aspect, in a possible design, the determining unit, when the second template distortion value is smaller than the first template distortion value, according to the first motion information and the second motion information, When determining the weighted prediction value of the to-be-processed unit, specifically, when the first template distortion value, the second template distortion value, and the third template distortion value, the second template distortion value is the most The weighted predicted value is used as the predicted value of the unit to be processed.

With reference to the second aspect, in a possible design, the determining unit is specifically configured to: when the first template distortion value, the second template distortion value, and the third template distortion value, the first When the template distortion value is the smallest, the predicted value of the to-be-processed unit is obtained in the first reference image according to the first motion information; or, when the first template distortion value and the second template are distorted And a value of the third template distortion value, when the third template distortion value is the smallest, according to the third motion information, obtaining a predicted value of the to-be-processed unit in the third reference image.

With reference to the second aspect, in a possible design, when the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward direction of the to-be-processed unit a reference image list; or, when the first reference image is from a backward reference image list of the to-be-processed unit, the third reference image is from a forward reference image list of the to-be-processed unit.

With reference to the second aspect, in a possible design, the updating unit is further configured to: update the second template distortion value according to the weighting coefficient, and the updated second template distortion value is (1 Ω0) times the second template distortion value; correspondingly, the determining unit is configured to: when the updated second template distortion value is smaller than the first template distortion value, according to the first motion information and The second motion information determines a weighted prediction value of the to-be-processed unit.

With reference to the second aspect, in a possible design, when the updated second template distortion value is not less than the first template distortion value, the determining unit is further configured to: according to the first motion information, In the first reference image, a predicted value of the unit to be processed is obtained.

With reference to the second aspect, in a possible design, when the updated second template distortion value is smaller than the first template distortion value, the determining unit is specifically configured to: when the first template distortion value, In the updated second template distortion value and the third template distortion value, when the updated second template distortion value is the smallest, the weighted prediction value is used as the predicted value of the to-be-processed unit.

With reference to the second aspect, in a possible design, the method further includes: when the first template distortion value, the updated second template distortion value, and the third template distortion value, the first template distortion When the value is minimum, obtaining a predicted value of the to-be-processed unit in the first reference image according to the first motion information; or, when the first template distortion value, the updated second template In the distortion value and the third template distortion value, when the third template distortion value is the smallest, the predicted value of the to-be-processed unit is obtained in the third reference image according to the third motion information.

In a third aspect, a prediction device based on template matching is provided, the device includes a processor and a memory, wherein the memory stores a computer readable program, and the processor implements a program by running a program in the memory A prediction method based on template matching involved on the one hand.

In a fourth aspect, a computer storage medium is provided for storing the computer software instructions of the first aspect described above, comprising a program designed to perform the above aspects.

It should be understood that the second to fourth aspects of the embodiments of the present application are consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects obtained by the aspects and the corresponding implementable design manners are similar, and are not described again.

DRAWINGS

1 is a schematic diagram of an "L" type template in a template matching technique;

2 is a schematic diagram of a template matching implementation process;

3 is a flowchart of a prediction method based on template matching in an embodiment of the present application;

4A and FIG. 4B are flowcharts of obtaining a predicted value of a current block according to an embodiment of the present application;

5 and FIG. 6 are schematic block diagrams of a video codec device or an electronic device;

7 is a schematic block diagram of a video codec system;

8 is a structural diagram of a prediction device based on template matching in an embodiment of the present application;

FIG. 9 is a structural diagram of a prediction device based on template matching according to an embodiment of the present application;

FIG. 10 is a flowchart of another method for predicting a template matching according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described in the following with reference to the accompanying drawings in the embodiments.

As shown in Figure 2, in the Joint Exploration Model (JEM), the existing template matching implementation method steps are as follows:

1. In the forward reference image ref0, a template matching search is performed to obtain forward motion information MV0.

2. In the backward reference image ref1, a template matching search is performed to obtain backward motion information MV1.

3. Using MV0 and MV1, perform bidirectional motion prediction on the current block and obtain predicted values.

4. Encoding or decoding of the image block based on the predicted value.

It can be seen that when the template matching search, only the forward or backward reference image is used, which is a one-way search, and the subsequent prediction uses the bidirectional prediction of the forward and backward reference images to make the template match the search and the motion. The forecast does not match. Affect prediction accuracy and prediction

The embodiment of the present application provides a prediction method and device based on template matching, so as to solve the problem that the template matching search does not match the motion prediction, and improve the prediction accuracy. The method and the device are based on the same application concept. Since the principles of the method and the device for solving the problem are similar, the implementation of the device and the method can be referred to each other, and the repeated description is not repeated.

As shown in FIG. 3, the embodiment of the present application provides a prediction method based on template matching, and the specific process is as follows:

Step 31: Perform a matching search of the current template in the first reference image of the to-be-processed unit, and obtain first motion information, a first matching template, and a first template distortion value of the to-be-processed unit, where the current template includes the A plurality of reconstructed pixels of a preset position and number are preset in a neighborhood of the unit to be processed, and the first template distortion value represents a difference between the first matching template and the current template.

The to-be-processed unit in the present application refers to the current block; optionally, the current template may be an "L"-type template, including the left neighboring block and the upper neighboring block of the current block, and the current template may be adjacent to the current block. There may also be a certain distance from the current block, which is not made in this application.

Specifically, performing a matching search of the current template in the first reference image of the to-be-processed unit, obtaining the first motion information, the first matching template, and the first template distortion value of the to-be-processed unit, by using the following process:

S310: traverse the plurality of first candidate reconstructed pixel combinations of the same size and shape in the reconstructed pixels of the preset range of the first reference image, and calculate the plurality of first candidate weights Forming a pixel combination and a plurality of first pixel difference values of the current template.

S311: Determine, according to the smallest first pixel difference value, the first template distortion value, the first matching template.

Here, the smallest first pixel difference value is determined as the first template distortion value, and the predicted block corresponding to the smallest first pixel difference value is determined as the first matching template.

S312: Determine the first motion information according to a position vector difference between the first matching template and the current template, and an image where the first matching template is located, where the first matching template includes the first template. The first candidate reconstructed pixel combination corresponding to the distortion value.

It should be noted that the matching search process of the current template in the present application refers to finding a template that best matches the current template, and the matching degree is measured by the distortion value; the current template points to the motion information of the matching template is the current mode. The first motion information of the board. Therefore, after the matching search is completed, the first matching template that best matches the current template is obtained, and the distortion value between the current template and the first matching template is the first template distortion value, and the motion information of the current template is the first motion. Information, they are one-to-one correspondence.

Step 32: Update pixel values of the current template according to pixel values of the first matching template.

Specifically, when the pixel value of the current template is updated according to the pixel value of the first matching template, the following formula may be implemented:

T1=(T0-ω0×P1)/(1-ω0)

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, P1 represents a pixel value of the first matching template, and ω0 represents a weighting coefficient corresponding to the first matching template, Ω0 is a positive number less than one.

In a possible implementation, the above ω0 is set to 0.5. In this case, when the pixel value of the current template is updated according to the pixel value of the first matching template, the following formula may be implemented:

T1=2T0-P1

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, and P1 represents a pixel value of the first matching template.

It should be understood that ω0 has a value ranging from 0 to 1, and ω0 is not 1.

Step 33: Perform a matching search of the updated current template in the second reference image of the to-be-processed unit, and obtain second motion information, a second matching template, and a second template distortion value of the to-be-processed unit, The second template distortion value represents a difference between the second matching template and the updated current template.

It is worth mentioning that the first reference image and the second reference image may be from the same reference image list, or may be from different reference image lists, and the first reference image and the second reference image may be the same or different, in the present application. No restrictions.

Optionally, when the first reference image is from a forward reference image list of the to-be-processed unit, the second reference image is from a backward reference image list of the to-be-processed unit; or When the first reference image is from the backward reference image list of the to-be-processed unit, the second reference image is from the forward reference image list of the to-be-processed unit.

Specifically, the matching search of the updated current template is performed in the second reference image of the to-be-processed unit, and the second motion information, the second matching template, and the second template distortion value of the to-be-processed unit are obtained. Through the following process:

S320: traverse the plurality of second candidate reconstructed pixel combinations of the same size and shape in the reconstructed pixels of the preset range of the second reference image, and calculate the plurality of second candidate weights Forming a pixel combination and a plurality of second pixel difference values of the updated current template.

S321: Determine the second template distortion value and the second matching template according to the smallest second pixel difference value.

Here, the smallest second pixel difference value is determined as the second template distortion value, and the predicted block corresponding to the smallest second pixel difference value is determined as the second matching template.

S322: Determine the second motion information according to a position vector difference between the second matching template and the updated current template, and an image of the second matching template, where the second matching template includes the A second candidate reconstructed pixel combination corresponding to the second template distortion value.

In some possible implementations, step 33 further includes:

S323: Update the second template distortion value according to the foregoing weighting coefficient ω0. Specifically, for example, the updated second template distortion value is (1-ω0) times the second template distortion value before the update.

When ω0 is 0.5, the updated second template distortion value is 0.5 times the second template distortion value before the update.

It should be understood that since the same weighting coefficient ω0 is used as when the template is updated, the update of the second template distortion value and the template matching are performed, and the template is updated.

It should also be understood that when S323 is executed, the second template distortion value in the subsequent step refers to the updated second template distortion value, and will not be described again.

Since the second template distortion value is updated according to the weighting coefficient when the template is updated, the prediction error when using the second template is characterized, so that the result of the subsequent distortion value comparison is more in line with the real distortion condition, thereby selecting a more appropriate prediction. Values improve coding efficiency.

Step 34: When the second template distortion value is smaller than the first template distortion value, determine a weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information.

Specifically, determining, according to the first motion information and the second motion information, a weighted prediction value of the to-be-processed unit, by using the following process:

S340: Obtain, according to the first motion information, a first predicted value of the to-be-processed unit in the first reference image.

S341: Obtain, according to the second motion information, a second prediction value of the to-be-processed unit in the second reference image.

S342: Perform weighting calculation on the first predicted value and the second predicted value to obtain a weighted predicted value of the to-be-processed unit.

It should be noted that, when the first prediction value and the second prediction value are weighted, the first prediction value and the second prediction value may be weighted and averaged to obtain a The weighted prediction value of the processing unit is mentioned. Optionally, the first predicted value and the second predicted value are averaged to obtain a weighted predicted value of the to-be-processed unit.

In a possible implementation, the weighted prediction value of the to-be-processed unit is directly used as the predicted value of the to-be-processed unit. In the above step 32, the update operation of the current template is implemented, and in the subsequent step 33, The template matching search process is actually equivalent to the two-way search, and the first reference image and the second reference image are utilized in essence at the same time, and the implementation method is simple, and the prediction precision and the prediction effect can be improved at the same time.

Further, in a possible implementation manner, performing a matching search of the current template in the first reference image of the to-be-processed unit, and obtaining the first motion information of the to-be-processed unit and the first matching template, Perform the following process:

Composing, in a reconstructed pixel of a preset range of the third reference image, a plurality of third candidate reconstructed pixel combinations having the same size and shape as the current template, calculating the plurality of third candidate reconstructed pixel combinations and a plurality of third pixel difference values of the current template;

Determining, according to the smallest third pixel difference value, the third template distortion value, the third matching template; here, determining the smallest third pixel difference value as the third template distortion value, and the smallest third The prediction block corresponding to the pixel difference value is determined as the third matching template.

Determining, according to the position vector difference of the third matching template and the current template, and the image of the third matching template, the third motion template, where the third matching template includes the third template distortion value Corresponding third candidate reconstructed pixel combination.

It should be noted that, when the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward reference image list of the to-be-processed unit; or, when When the first reference image is from the backward reference image list of the to-be-processed unit, the third reference image is from the forward reference image list of the to-be-processed unit.

Further, after the third motion information is determined according to the position vector difference of the third matching template and the current template, and the image of the third matching template, when the third template distortion value is smaller than Or equal to the first template distortion value, using the third reference image as the first reference image; using the third motion information as the first motion information; using the third template distortion value as The first template distortion value; the third matching template is used as the first matching template.

It is worth mentioning that the first reference image and the third reference image may be from the same reference image list, or may be from different reference image lists, and the first reference image and the third reference image may be the same or different, in the present application. No restrictions.

Further, in a possible implementation, before determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, the method further includes:

Comparing the first template distortion value and the second template distortion value;

Based on the comparison result, a predicted value of the unit to be processed is determined.

Specifically, the determining, according to the comparison result, the predicted value of the to-be-processed unit includes the following two situations:

In the first case, when the first template distortion value is less than or equal to the second template distortion value, the predicted value of the to-be-processed unit is obtained in the first reference image according to the first motion information.

In the second case, when the first template distortion value is greater than or equal to the second template distortion value, the weighted prediction value is used as the predicted value of the to-be-processed unit.

Therefore, in this embodiment, the foregoing prediction method can perform adaptive selection in a unidirectional search unidirectional prediction manner and a bidirectional search bidirectional prediction manner. At this time, the prediction is improved in a case where the implementation complexity is low. The effect and precision.

Further, in another possible implementation, before the determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, the method further includes:

Comparing the first template distortion value, the second template distortion value, and the third template distortion value;

Specifically, the determining, according to the comparison result, the predicted value of the to-be-processed unit includes the following three situations:

Case 1, in the first template distortion value, the second template distortion value, and the third template distortion value, when the first template distortion value is the smallest, according to the first motion information, in the In the first reference image, a predicted value of the unit to be processed is obtained.

Case 2: when the third template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, according to the third motion information, In the third reference image, the predicted value of the to-be-processed unit is obtained.

Case 3: when the second template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, the weighted prediction value is used as the to-be-processed The predicted value of the unit.

In this implementation manner, adaptive selection can be performed in two different unidirectional search unidirectional prediction modes and two-way search bidirectional prediction mode. At this time, although the implementation complexity is high, there is a better self. Adapt to selectivity to improve the predictive effect and accuracy.

The method in FIG. 3 will be described in detail below through actual application scenarios.

scene one

In scenario 1, the predicted value of the current block is obtained by bidirectional search and bidirectional prediction of template matching, and the specific process is as follows:

1. Perform a matching search of the current module in the forward reference image ref0 of the current block, and obtain a forward motion information MV0 of the current block and a forward prediction value of the current template, that is, a pixel value P1 of the first matching template;

2. Use the following formula to update the current template.

T1=2T0-P1

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, and P1 represents a forward predicted value of the current template.

3. In the backward reference image of the current block, the matching search of the template is performed by using the updated current template T1 to obtain the backward motion information MV1 of the current block.

4. Using MV0 and MV1, bidirectional prediction is performed on the current block, and the forward prediction value pred0 and the backward prediction value pred1 of the current block are obtained.

5. Calculate an average value for the forward predicted value pred0 and the backward predicted value pred1 of the current block to obtain a predicted value of the current block.

In this way, compared with the prior art, since the update operation of the current template is added, after the current template update, the template matching search process in the subsequent step is equivalent to the two-way search, essentially utilizing both the forward reference image and the backward reference. Image, the two-way search process of template matching is realized in a simple way to improve prediction accuracy and prediction effect.

It should be noted that Steps 1 - 3 in Scene 1 can be performed iteratively multiple times in order to obtain more accurate motion information.

It should be noted that if the bidirectional prediction in step 5 adopts a weighted prediction manner, such as: predBi=w0*pred0+ω1*pred1), the formula used in the template update process in the corresponding step 2 is changed to the following form:

T1=(T0-ω0×P1)/ω1

In the formula, ω0, ω1 are weighting coefficients corresponding to the forward predicted value pred0 and the backward predicted value pred1, and satisfy ω0 + ω1 = 1.

If the bidirectional prediction adopts the method of weighted prediction, the weighting coefficients involved may be calculated according to the distortion values corresponding to the forward prediction and the backward prediction. In principle, the more accurate the allocation is, the larger the weight, that is, the smaller the prediction distortion, the larger the weight, such as:

Ω0=cost1/(cost0+cost1); ω1=cost0/(cost0+cost1),

Cost0 is the distortion value of the current template corresponding to the forward prediction, and cost1 is the distortion value of the current template corresponding to the backward prediction.

Among them, the weighted prediction is particularly suitable for the scene of brightness gradation. In this application, the prediction method based on template matching can adaptively calculate the weighting coefficient, and different weighting coefficients are allowed in different regions of the image, so that there is better local adaptability. Helps make more accurate predictions.

The reason for the template matching search in step 3 of scene one is equivalent to two-way search is as follows:

The template matching of step 3 is to find a backward prediction value Pred1 in the backward reference image Ref1 so that it matches the template most, and the matching process is to minimize T1-Pred1. due to

T1-Pred1=(2×T0–Pred0)–Pred1

=2×[T0–(Pred0+Pred1)/2]

Where (Pred0+Pred1)/2 is the weighted prediction value of the current block, and the minimum (T0–(Pred0+Pred1)/2) is the standard two-way matching search process.

It can be seen that the process of minimizing (T1-Pred1) is equivalent to minimizing the process of (T0–(Pred0+Pred1)/2). Therefore, the matching search process of step 3 is actually a two-way matching search. At the same time, as seen from the above equation, the template difference (T1-Pred1) is equivalent to twice the difference between the current template and the bidirectional prediction value (T0–(Pred0+Pred1)/2); therefore, after normalization, 1/2*(T1-Pred1) is equivalent to characterizing the difference between the current template and the bidirectional prediction value for comparison of subsequent costs.

It is worth mentioning that in the scenario 1, the implementation process of the prediction value of the current block is obtained by the bidirectional search and the bidirectional prediction of the template matching, and the matching search of the current module may be performed in the backward reference image ref1 first, and the current block is obtained. The current template is updated to the motion information MV1, the backward prediction value P1 of the current template, and the backward prediction value P1, so that the bidirectional matching search process of the template is performed in the forward reference image, thereby obtaining the prediction value of the current block. Other processes are similar and will not be described here.

Scene two

In scenario 2, the two different one-way search and one-way prediction modes include forward search forward prediction and backward search backward prediction, two-way search and bidirectional prediction modes, and adaptively select prediction modes to obtain prediction values of current blocks. The specific process is as follows:

1. Perform a matching search of the current module in the forward reference image ref0 of the current block, and obtain the forward motion information MV0 of the current block, the forward prediction value of the current template, that is, the pixel value of the matching template, and the current template and the current template. The first distortion value cost0 between the forward predicted values.

2. Perform a matching search of the current module in the backward reference image ref1 of the current block, and obtain a backward predicted value of the current template of the backward motion information MV1 of the current block, that is, a pixel value of the matching template, and a current template and a current template. The second distortion value cost2 between the backward predicted values.

3. Compare the sizes of cost0 and cost2, and select a prediction mode corresponding to the smaller distortion value to obtain the bidirectional motion information of the current block and the first distortion value cost1 between the current template and the weighted prediction value of the current template.

Specifically, in the first embodiment, if cost0 is less than cost2, the current template is updated by using the following formula.

T1=2T0-P1

Then, in the backward reference image of the current block, the matching search of the template is performed by using the updated current template T1 to obtain the backward motion information MV1′ of the current block, the weighted prediction value of the updated current template, and the updated The first distortion value of the current template is cost1.

At this time, the bidirectional motion information of the current block is MV0 and MV1'.

Specifically, in the second implementation manner, if cost1 is not less than cost2, the current template is updated by using the following formula.

T1=2T0-P1'

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, and P1′ represents a backward predicted value of the current template.

Then, in the forward reference image of the current block, the matching search of the template is performed by using the updated current template T1, and the forward motion information MV0' of the current block, the weighted prediction value of the updated current template, and the updated Current The first distortion value of the template is cost1.

At this time, the bidirectional motion information of the current block is MV0' and MV1.

4. The smaller one of cost0 and cost2 is compared with cost1, and the prediction mode corresponding to the minimum distortion value is selected to obtain the predicted value of the current block.

Specifically, in the case of the first embodiment in the above step 3:

If cost0 is less than cost1, the forward prediction value of the current block is obtained in the forward reference image of the current block by using the forward motion information MV0 as the predicted value of the current block.

If cost0 is not less than cost1, the forward motion information MV0 and the backward motion information MV1' are used to obtain the forward prediction value of the current block in the forward reference image of the current block, and the current block is obtained in the backward reference image of the previous block. The backward prediction value, weighting the forward prediction value of the current block and the backward prediction value of the current block to obtain a prediction value of the current block.

Specifically, in the case of the second embodiment in the above step 3:

If cost2 is less than cost1, the backward prediction value of the current block is obtained in the backward reference image of the current block by using the backward motion information MV1 as the predicted value of the current block.

If cost2 is not less than cost1, the forward motion information MV0' and the backward motion information MV1 are used to obtain the forward prediction value of the current block in the forward reference image of the current block, and the current block is obtained in the backward reference image of the previous block. The backward prediction value, weighting the forward prediction value of the current block and the backward prediction value of the current block to obtain a prediction value of the current block.

It should be noted that in the second scenario, the three-way template matching search process needs to be performed three times, the adaptability is the best, the prediction effect is the best, but the implementation complexity is high.

Scene three

In scenario 3, adaptively selecting a prediction mode in a determined one-way search and one-way prediction mode, a two-way search mode, and a bidirectional prediction mode to obtain a prediction value of a current block, the specific process is as follows:

Specifically, when determining the direction of the one-way search and the one-way prediction mode, the determination may be performed according to some indicators in advance. For example, (1) before performing the template matching search, performing matching search on the current template according to the initial value of the forward motion vector and the initial value of the backward motion vector, respectively obtaining the forward prediction value corresponding to the current template and the forward distortion of the current template. The value cost0'; and the backward prediction value corresponding to the current template and the backward distortion value cost2', cost0' and cost2' of the current template are selected, and the corresponding one-way prediction mode with a small distortion value is selected; for example, (2) And determining, according to the time domain distance between the current block and the forward reference image and the backward reference image, a unidirectional prediction manner in which the time domain distance is small.

If the determined one-way prediction mode is forward search and forward prediction, the specific implementation flowchart can be seen in FIG. 4A:

2. Update the current template with the following formula:

T1=2T0-P1

3. In the backward reference image of the current block, the matching search of the template is performed by using the updated current template T1, and the updated weighted prediction value of the current module and the second distortion value between the updated current template and the updated current template are obtained. Cost1.

4. Compare the size of cost0 and cost1.

5. According to the comparison result of cost0 and cost1, the prediction mode corresponding to the smaller distortion value is selected to obtain the predicted value of the current block.

Specifically, if cost0 is less than cost1, the forward prediction value of the current block is obtained in the forward reference image of the current block by using the forward motion information MV0 as the predicted value of the current block.

If the determined one-way prediction mode is backward search and backward prediction, the specific implementation flowchart can be seen in FIG. 4B:

1. Perform a matching search of the current module in the backward reference image ref1 of the current block, and obtain the backward motion information MV1' of the current block, the backward prediction value of the current template, that is, the pixel value of the matching template, and the current template and the current template. The third distortion value cost2 between the backward predicted values of the template.

2. Update the current template with the following formula:

T1=2T0-P1'

4. Compare the size of cost1 and cost2.

5. According to the comparison result of cost1 and cost2, the prediction mode corresponding to the smaller distortion value is selected to obtain the predicted value of the current block.

It should be noted that, compared with the second scenario, the scenario 3 reduces the template matching search process, reduces the complexity, and improves the prediction accuracy.

The predicted value is obtained according to the above prediction method, and encoding or decoding of the image block is performed.

5 is a schematic block diagram of a video codec device or electronic device 50 that may incorporate a predictive device in accordance with an embodiment of the present application. FIG. 6 is a schematic diagram of an apparatus for video encoding in accordance with an embodiment of the present application. The units in Figs. 5 and 6 will be explained below.

The electronic device 50 can be, for example, a mobile terminal or user equipment of a wireless communication system. It should be understood that embodiments of the present application may be implemented in any electronic device or device that may require encoding and decoding, or encoding, or decoding of a video image.

Device 50 can include a housing 30 for incorporating and protecting the device. Device 50 may also include display 32 in the form of a liquid crystal display. In other embodiments of the present application, the display may be any suitable display technology suitable for displaying images or video. Device 50 may also include a keypad 34. In other embodiments of the present application, any suitable data or user interface mechanism can be utilized. For example, the user interface can be implemented as a virtual keyboard or data entry system As part of a touch-sensitive display. The device may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input. The device 50 may also include an audio output device, which in the embodiment of the present application may be any of the following: an earphone 38, a speaker, or an analog audio or digital audio output connection. Device 50 may also include battery 40, and in other embodiments of the present application, the device may be powered by any suitable mobile energy device, such as a solar cell, fuel cell, or clock mechanism generator. The device may also include an infrared port 42 for short-range line of sight communication with other devices. In other embodiments, device 50 may also include any suitable short range communication solution, such as a Bluetooth wireless connection or a USB/Firewire wired connection.

Device 50 may include a controller 56 or processor for controlling device 50. The controller 56 can be coupled to a memory 58, which in the embodiments of the present application can store data in the form of data and audio in the form of images, and/or can also store instructions for execution on the controller 56. The controller 56 can also be coupled to a codec 54 suitable for implementing encoding and decoding of audio and/or video data or assisted encoding and decoding by the controller 56.

The apparatus 50 may also include a card reader 48 and a smart card 46, such as a UICC and a UICC reader, for providing user information and for providing authentication information for authenticating and authorizing users on the network.

Apparatus 50 may also include a radio interface circuit 52 coupled to the controller and adapted to generate, for example, a wireless communication signal for communicating with a cellular communication network, a wireless communication system, or a wireless local area network. Apparatus 50 may also include an antenna 44 coupled to radio interface circuitry 52 for transmitting radio frequency signals generated at radio interface circuitry 52 to other apparatus(s) and for receiving radio frequency signals from other apparatus(s).

In some embodiments of the present application, device 50 includes a camera capable of recording or detecting a single frame, and codec 54 or controller receives these single frames and processes them. In some embodiments of the present application, the device may receive video image data to be processed from another device prior to transmission and/or storage. In some embodiments of the present application, device 50 may receive images for encoding/decoding via a wireless or wired connection.

FIG. 7 is a schematic block diagram of another video codec system 10 in accordance with an embodiment of the present application. As shown in FIG. 7, video codec system 10 includes source device 12 and destination device 14. Source device 12 produces encoded video data. Thus, source device 12 may be referred to as a video encoding device or a video encoding device. Destination device 14 may decode the encoded video data produced by source device 12. Thus, destination device 14 may be referred to as a video decoding device or a video decoding device. Source device 12 and destination device 14 may be examples of video codec devices or video codec devices. Source device 12 and destination device 14 may include a wide range of devices including desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set top boxes, smart phones, etc., televisions, cameras, display devices , digital media player, video game console, on-board computer, or the like.

Destination device 14 may receive the encoded video data from source device 12 via channel 16. Channel 16 may include one or more media and/or devices capable of moving encoded video data from source device 12 to destination device 14. In one example, channel 16 may include one or more communication media that enable source device 12 to transmit encoded video data directly to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data in accordance with a communication standard (eg, a wireless communication protocol) and may transmit the modulated video data to destination device 14. The one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network (eg, a local area network, a wide area network, or a global network (eg, the Internet)). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from source device 12 to destination device 14.

In another example, channel 16 can include a storage medium that stores encoded video data generated by source device 12. In this example, destination device 14 can access the storage medium via disk access or card access. Storage media can contain A variety of local access data storage media, such as Blu-ray Disc, DVD, CD-ROM, flash memory, or other suitable digital storage medium for storing encoded video data.

In another example, channel 16 can include a file server or another intermediate storage device that stores encoded video data generated by source device 12. In this example, destination device 14 may access the encoded video data stored at a file server or other intermediate storage device via streaming or download. The file server may be a server type capable of storing encoded video data and transmitting the encoded video data to the destination device 14. The instance file server includes a web server (eg, for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, and a local disk drive.

Destination device 14 can access the encoded video data via a standard data connection (e.g., an internet connection). An instance type of a data connection includes a wireless channel (eg, a Wi-Fi connection), a wired connection (eg, DSL, cable modem, etc.), or both, suitable for accessing encoded video data stored on a file server. combination. The transmission of the encoded video data from the file server may be streaming, downloading, or a combination of both.

The technology of the present application is not limited to a wireless application scenario. Illustratively, the technology can be applied to video codecs supporting various multimedia applications such as aerial television broadcasting, cable television transmission, satellite television transmission, and streaming video. Transmission (eg, via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other application. In some examples, video codec system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

In the example of FIG. 7, source device 12 includes video source 18, video encoder 20, and output interface 22. In some examples, output interface 22 can include a modulator/demodulator (modem) and/or a transmitter. Video source 18 may include a video capture device (eg, a video camera), a video archive containing previously captured video data, a video input interface to receive video data from a video content provider, and/or a computer for generating video data. A graphics system, or a combination of the above video data sources.

Video encoder 20 may encode video data from video source 18. In some examples, source device 12 transmits the encoded video data directly to destination device 14 via output interface 22. The encoded video data may also be stored on a storage medium or file server for later access by the destination device 14 for decoding and/or playback.

In the example of FIG. 7, destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some examples, input interface 28 includes a receiver and/or a modem. Input interface 28 can receive the encoded video data via channel 16. Display device 32 may be integral with destination device 14 or may be external to destination device 14. In general, display device 32 displays the decoded video data. Display device 32 may include a variety of display devices such as liquid crystal displays (LCDs), plasma displays, organic light emitting diode (OLED) displays, or other types of display devices.

Video encoder 20 and video decoder 30 may operate in accordance with a video compression standard (eg, the High Efficiency Video Codec H.265 standard) and may conform to the HEVC Test Model (HM). A textual description of the H.265 standard is published on April 29, 2015, ITU-T.265(V3) (04/2015), available for download from http://handle.itu.int/11.1002/1000/12455 The entire contents of the document are incorporated herein by reference.

Based on the foregoing method embodiment, as shown in FIG. 8, the embodiment of the present application provides a template matching based prediction apparatus 800. As shown in FIG. 8, the apparatus 800 includes a searching unit 801, an updating unit 802, and a determining unit 803. :

The searching unit 801 is configured to perform a matching search of the current template in the first reference image of the to-be-processed unit, and obtain the first motion information, the first matching template, and the first template distortion value of the to-be-processed unit, where the current template is Including Determining a plurality of reconstructed pixels of a preset position and number in a neighborhood of the processing unit, where the first template distortion value represents a difference between the first matching template and the current template;

The updating unit 802 is configured to update, according to a pixel value of the first matching template, a pixel value of the current template.

The searching unit 801 is further configured to perform a matching search of the updated current template in the second reference image of the to-be-processed unit, to obtain second motion information, a second matching template, and a second template distortion value, the second template distortion value representing a difference between the second matching template and the updated current template;

The determining unit 803 is configured to determine, according to the first motion information and the second motion information, a weighted prediction value of the to-be-processed unit when the second template distortion value is smaller than the first template distortion value.

Optionally, the updating unit 802 meets the following expression when updating the pixel value of the current template according to the pixel value of the first matching template:

T1=(T0-ω0×P1)/(1-ω0),

Optionally, the ω0 is 0.5.

Optionally, the searching unit 801 performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first motion information, the first matching template, and the first template distortion value of the to-be-processed unit, Specifically used for:

Calculating, in the reconstructed pixels of the preset range of the first reference image, a plurality of first candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of first candidate reconstructed pixels Combining the sum of the current template and the plurality of first pixel difference values;

Determining, according to the smallest first pixel difference value, the first template distortion value, the first matching template;

Determining the first motion information according to a position vector difference between the first matching template and the current template, and an image in which the first matching template is located, where the first matching template includes the first template distortion value Corresponding first candidate reconstructed pixel combination.

Optionally, the searching unit 801 performs a matching search of the updated current template in the second reference image of the to-be-processed unit, and obtains second motion information, a second matching template, and When the second template distortion value is used, it is specifically used to:

Composing, in the reconstructed pixels of the preset range of the second reference image, a plurality of second candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of second candidate reconstructed pixels Combining and a plurality of second pixel difference values of the updated current template;

Determining, according to the smallest second pixel difference value, the second template distortion value and the second matching template;

Determining the second motion information according to a position vector difference between the second matching template and the updated current template, and an image of the second matching template, where the second matching template includes the second A second candidate reconstructed pixel combination corresponding to the template distortion value.

Optionally, the determining unit 803 is configured to: when determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, specifically:

Obtaining, in the first reference image, a first predicted value of the to-be-processed unit according to the first motion information;

Obtaining, in the second reference image, a second prediction value of the to-be-processed unit according to the second motion information;

Weighting the first predicted value and the second predicted value to obtain a weighted predicted value of the to-be-processed unit.

Optionally, the searching unit 801 performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first motion information of the to-be-processed unit and the first matching template, and is further used to:

Determining, according to the smallest third pixel difference value, the third template distortion value and the third matching template;

Optionally, after the determining, by the searching unit 801, the location information difference according to the third matching template and the current template, and the image of the third matching template, determining the third motion information, when When the third template distortion value is less than or equal to the first template distortion value, it is also used to:

Using the third reference image as the first reference image;

Using the third motion information as the first motion information;

Using the third template distortion value as the first template distortion value;

The third matching template is used as the first matching template.

Optionally, the determining unit 803 is further configured to: before determining, according to the first motion information and the second motion information, the weighted prediction value of the to-be-processed unit:

Comparing the first template distortion value and the second template distortion value.

Optionally, when the result of the comparison is that the second template distortion value is not less than the first template distortion value, the determining unit 803 is specifically configured to:

And obtaining, according to the first motion information, a predicted value of the to-be-processed unit in the first reference image.

Comparing the first template distortion value, the second template distortion value, and the third template distortion value.

Optionally, the determining unit 803 determines, according to the first motion information and the second motion information, that the to-be-processed unit is when the second template distortion value is smaller than the first template distortion value. When weighting the predicted value, it is specifically used to:

When the second template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, the weighted prediction value is used as a prediction of the to-be-processed unit value.

Optionally, the determining unit 803 is configured to:

When the result of the comparison is the first template distortion value, the second template distortion value, and the third template distortion value, when the first template distortion value is the smallest, according to the first motion information, Obtaining a predicted value of the to-be-processed unit in the first reference image; or

When the result of the comparison is the first template distortion value, the second template distortion value, and the third template distortion value, when the third template distortion value is the smallest, according to the third motion information, In the third reference image, obtained The predicted value of the unit to be processed is obtained.

Optionally, when the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward reference image list of the to-be-processed unit; or When the first reference image is from the backward reference image list of the to-be-processed unit, the third reference image is from the forward reference image list of the to-be-processed unit.

Optionally, the updating unit 802 is further configured to update the second template distortion value according to the foregoing weighting coefficient ω0, specifically, for example, the updated second template distortion value is (1-ω0) times the update. The second template distortion value before.

It should also be understood that when the updating unit 802 is further configured to update the second template distortion value according to the foregoing weighting coefficient ω0, the second template distortion value when the other unit performs the subsequent step refers to the updated second template. Distortion value, no longer repeat.

For the function implementation and the interaction mode of the respective units of the device 800 in the embodiment of the present application, reference may be made to the description of the related method embodiments, and details are not described herein again.

It should be understood that the division of each unit in the above device 800 is only a division of a logical function, and the actual implementation may be integrated into one physical entity in whole or in part, or may be physically separated. For example, each of the above units may be a separately set processing element, or may be integrated in one chip of the codec device, or may be stored in a storage element of the codec device in the form of program code, by the codec device. One of the processing elements invokes and performs the functions of each of the above units. In addition, the individual units can be integrated or implemented independently. The processing elements described herein can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method or each of the above units may be completed by an integrated logic circuit of hardware in the processor element or an instruction in a form of software. The processing element may be a general purpose processor, such as a central processing unit (CPU), or may be one or more integrated circuits configured to implement the above method, for example: one or more specific integrated circuits ( English: application-specific integrated circuit (abbreviation: ASIC), or one or more microprocessors (English: digital signal processor, referred to as: DSP), or one or more field programmable gate arrays (English: field-programmablegatearray, referred to as: FPGA) and so on.

According to the same inventive concept, the embodiment of the present application further provides a template matching based prediction device 900. As shown in FIG. 9, the device 900 includes a processor 901 and a memory 902. The program code for executing the solution of the present application is stored in the memory 902. The instruction processor 901 executes the prediction method shown in FIG. 3, FIG. 4A, and FIG. 4B.

The application can also perform programming programming on the processor to solidify the code corresponding to the method shown in FIG. 3, FIG. 4A and FIG. 4B into the chip, so that the chip can execute FIG. 3, FIG. 4A and FIG. 4B during operation. The method shown.

It is to be understood that the processor involved in the foregoing apparatus 900 may be a CPU, a DSP, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application. One or more memories included in the computer system, which may be read-only memory (ROM: ROM) or other types of static storage devices that can store static information and instructions, random access memory (English: random accessmemory, Abbreviation: RAM) or other types of dynamic storage devices that can store information and instructions, or disk storage. This These memories are connected to the processor via a bus or can also be connected to the processor via a dedicated cable.

It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be performed by a program, and the program may be stored in a computer readable storage medium, which is non-transitory ( English: non-transitory) media, such as random access memory, read-only memory, flash memory, hard disk, solid state disk, magnetic tape (English: magnetic tape), floppy disk (English: floppy disk), CD (English: optical disc) And any combination thereof.

A specific embodiment of the present invention implements frame rate up-conversion (FRUC) in JVET-D1001, "Algorithm Description of Joint Exploration Test Model 4" (available from http://phenix.int-evry.fr/jvet/ The entire contents of this reference are incorporated herein by reference, and are modified as shown in FIG. 10, and the specific possible implementations are described in detail above and will not be described again.

The present application is described with reference to the respective flowcharts and block diagrams of the method and apparatus of the embodiments of the present application. It will be understood that each flow and block of the flowchart illustrations. FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. A device that implements the functions specified in one or more blocks of a flowchart or a plurality of flows and block diagrams.

Claims

A prediction method based on template matching, which is characterized in that it comprises:

Performing a matching search of the current template in the first reference image of the to-be-processed unit, obtaining first motion information, a first matching template, and a first template distortion value of the to-be-processed unit, where the current template includes the to-be-processed unit a plurality of reconstructed pixels of a preset position and number in a neighborhood, the first template distortion value representing a difference between the first matching template and the current template;

Updating a pixel value of the current template according to a pixel value of the first matching template;

Performing a matching search of the updated current template in the second reference image of the to-be-processed unit, obtaining second motion information, a second matching template, and a second template distortion value of the to-be-processed unit, where The second template distortion value represents a difference between the second matching template and the updated current template;

And when the second template distortion value is smaller than the first template distortion value, determining a weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information.
The method according to claim 1, wherein the updating the pixel value of the current template according to the pixel value of the first matching template comprises: updating the current pixel value in the following manner:

T1=(T0-ω0×P1)/(1-ω0),

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, P1 represents a pixel value of the first matching template, and ω0 represents a weighting coefficient corresponding to the first matching template, Ω0 is a positive number less than one.
The method of claim 2 wherein said ω0 is 0.5.
The method according to any one of claims 1 to 3, wherein the matching search of the current template is performed in the first reference image of the to-be-processed unit, and the first motion information of the to-be-processed unit is obtained. A matching template and first template distortion values, including:

Calculating, in the reconstructed pixels of the preset range of the first reference image, a plurality of first candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of first candidate reconstructed pixels Combining and a plurality of first pixel difference values of the current template;

Determining, according to the smallest first pixel difference value, the first template distortion value, the first matching template;

Determining the first motion information according to a position vector difference between the first matching template and the current template, and an image in which the first matching template is located, where the first matching template includes the first template distortion value Corresponding first candidate reconstructed pixel combination.
The method according to any one of claims 1 to 4, wherein the matching search of the updated current template is performed in a second reference image of the to-be-processed unit, and the to-be-processed unit is obtained. The second motion information, the second matching template, and the second template distortion value, including:

Composing, in the reconstructed pixels of the preset range of the second reference image, a plurality of second candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of second candidate reconstructed pixels Combining and a plurality of second pixel difference values of the updated current template;

Determining, according to the smallest second pixel difference value, the second template distortion value and the second matching template;

Determining the second motion information according to a position vector difference between the second matching template and the updated current template, and an image of the second matching template, where the second matching template includes the second A second candidate reconstructed pixel combination corresponding to the template distortion value.
The method according to any one of claims 1 to 5, wherein the determining the weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information comprises:

Obtaining, in the first reference image, a first predicted value of the to-be-processed unit according to the first motion information;

Obtaining, in the second reference image, a second prediction value of the to-be-processed unit according to the second motion information;

Weighting the first predicted value and the second predicted value to obtain a weighted predicted value of the to-be-processed unit.
The method according to any one of claims 1 to 6, wherein a matching search of the current template is performed in the first reference image of the to-be-processed unit, and the first motion information and the first After matching the template, it also includes:

Composing, in a reconstructed pixel of a preset range of the third reference image, a plurality of third candidate reconstructed pixel combinations having the same size and shape as the current template, calculating the plurality of third candidate reconstructed pixel combinations and a plurality of third pixel difference values of the current template;

Determining, according to the smallest third pixel difference value, the third template distortion value and the third matching template;

Determining, according to the position vector difference of the third matching template and the current template, and the image of the third matching template, the third motion template, where the third matching template includes the third template distortion value Corresponding third candidate reconstructed pixel combination.
The method according to claim 7, wherein after the third motion information is determined according to the position vector difference of the third matching template and the current template, and the image of the third matching template And when the third template distortion value is less than or equal to the first template distortion value, the method further includes:

Using the third reference image as the first reference image;

Using the third motion information as the first motion information;

Using the third template distortion value as the first template distortion value;

The third matching template is used as the first matching template.
The method according to any one of claims 5 to 8, wherein when the second template distortion value is not less than the first template distortion value, the method further comprises:

And obtaining, according to the first motion information, a predicted value of the to-be-processed unit in the first reference image.
The method according to any one of claims 7 to 8, wherein when the second template distortion value is smaller than the first template distortion value, according to the first motion information and the second The motion information determines a weighted prediction value of the to-be-processed unit, including:

When the second template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, the weighted prediction value is used as a prediction of the to-be-processed unit value.
The method of claim 10, further comprising:

When the first template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, according to the first motion information, in the first reference Obtaining a predicted value of the to-be-processed unit in an image; or

And when the third template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, according to the third motion information, in the third reference In the image, the predicted value of the unit to be processed is obtained.
The method according to any one of claims 7 to 11, comprising:

When the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward reference image list of the to-be-processed unit; or, when the first reference When the image is from the backward reference image list of the to-be-processed unit, the third reference image is from the forward reference image list of the to-be-processed unit.
The method according to any one of claims 2 to 12, wherein the matching search of the updated current template is performed in the second reference image of the to-be-processed unit, and the to-be-processed is obtained. After the second motion information, the second matching template, and the second template distortion value of the unit, the method further includes:

And updating the second template distortion value according to the weighting coefficient, where the updated second template distortion value is (1-ω0) times the second template distortion value;

Correspondingly, when the second template distortion value is smaller than the first template distortion value, determining, according to the first motion information and the second motion information, a weighted prediction value of the to-be-processed unit, including :

And when the updated second template distortion value is smaller than the first template distortion value, determining a weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information.
The method according to claim 13, wherein when the updated second template distortion value is not less than the first template distortion value, the method further comprises:

And obtaining, according to the first motion information, a predicted value of the to-be-processed unit in the first reference image.
The method according to claim 13 or 14, wherein when the updated second template distortion value is smaller than the first template distortion value, according to the first motion information and the second motion information Determining the weighted prediction value of the to-be-processed unit, including:

When the updated second template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, the weighted prediction value is taken as The predicted value of the processing unit is mentioned.
The method of claim 15 further comprising:

And when the first template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, according to the first motion information, Obtaining a predicted value of the to-be-processed unit in the first reference image; or

And when the third template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, according to the third motion information, In the third reference image, the predicted value of the to-be-processed unit is obtained.
A prediction device based on template matching, comprising:

a search unit, configured to perform a matching search of the current template in the first reference image of the to-be-processed unit, to obtain first motion information, a first matching template, and a first template distortion value of the to-be-processed unit, where the current template includes a plurality of reconstructed pixels of a preset position and number in a neighborhood of the to-be-processed unit, where the first template distortion value represents a difference between the first matching template and the current template;

An updating unit, configured to update, according to a pixel value of the first matching template, a pixel value of the current template;

The searching unit is further configured to perform a matching search of the updated current template in a second reference image of the to-be-processed unit, to obtain second motion information, a second matching template, and a second a second template distortion value, the second template distortion value representing a difference between the second matching template and the updated current template;

a determining unit, configured to determine, according to the first motion information and the second motion information, a weighted prediction value of the to-be-processed unit when the second template distortion value is smaller than the first template distortion value.
The device according to claim 17, wherein the updating unit, when updating the pixel value of the current template according to the pixel value of the first matching template, comprises: using the following manner to the current pixel The value is updated:

T1=(T0-ω0×P1)/(1-ω0),

Wherein, T1 represents a pixel value of the updated current template, T0 represents a pixel value of the current template, P1 represents a pixel value of the first matching template, and ω0 represents a weighting coefficient corresponding to the first matching template, Ω0 is a positive number less than one.
The apparatus of claim 18 wherein said ω0 is 0.5.
The device according to any one of claims 17 to 19, wherein the search unit performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first motion information of the to-be-processed unit. When the first matching template and the first template distortion value are used, specifically:

Calculating, in the reconstructed pixels of the preset range of the first reference image, a plurality of first candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of first candidate reconstructed pixels Combining and a plurality of first pixel difference values of the current template;

Determining, according to the smallest first pixel difference value, the first template distortion value, the first matching template;

Determining the first motion information according to a position vector difference between the first matching template and the current template, and an image in which the first matching template is located, where the first matching template includes the first template distortion value Corresponding first candidate reconstructed pixel combination.
The device according to any one of claims 17 to 20, wherein the searching unit performs a matching search of the updated current template in a second reference image of the to-be-processed unit, and obtains the waiting When the second motion information, the second matching template, and the second template distortion value of the processing unit are used, specifically:

Composing, in the reconstructed pixels of the preset range of the second reference image, a plurality of second candidate reconstructed pixel combinations having the same size and shape as the current template, and calculating the plurality of second candidate reconstructed pixels Combining and a plurality of second pixel difference values of the updated current template;

Determining, according to the smallest second pixel difference value, the second template distortion value and the second matching template;

Determining the second motion information according to a position vector difference between the second matching template and the updated current template, and an image of the second matching template, where the second matching template includes the second A second candidate reconstructed pixel combination corresponding to the template distortion value.
The apparatus according to any one of claims 17 to 21, wherein the determining unit determines a weighted prediction value of the to-be-processed unit according to the first motion information and the second motion information, Specifically used for:

Obtaining, in the first reference image, a first predicted value of the to-be-processed unit according to the first motion information;

Obtaining, in the second reference image, a second prediction value of the to-be-processed unit according to the second motion information;

Weighting the first predicted value and the second predicted value to obtain a weighted predicted value of the to-be-processed unit.
The device according to any one of claims 17 to 22, wherein the search unit performs a matching search of the current template in the first reference image of the to-be-processed unit, and obtains the first of the to-be-processed unit. After the motion information and the first matching template, it is also used to:

In the reconstructed pixels of the preset range of the third reference image, the traversal and the current template have the same size and shape And calculating, by the plurality of third candidate reconstructed pixel combinations, the plurality of third candidate reconstructed pixel combinations and the plurality of third pixel difference values of the current template;

Determining, according to the smallest third pixel difference value, the third template distortion value and the third matching template;

Determining, according to the position vector difference of the third matching template and the current template, and the image of the third matching template, the third motion template, where the third matching template includes the third template distortion value Corresponding third candidate reconstructed pixel combination.
The apparatus according to claim 23, wherein said searching unit determines said number in said image according to a position vector difference of said third matching template and said current template, and said image of said third matching template After the three motion information, when the third template distortion value is less than or equal to the first template distortion value, it is further used for:

Using the third reference image as the first reference image;

Using the third motion information as the first motion information;

Using the third template distortion value as the first template distortion value;

The third matching template is used as the first matching template.
The device according to any one of claims 21 to 24, wherein when the second template distortion value is not less than the first template distortion value, the determining unit is specifically configured to:

And obtaining, according to the first motion information, a predicted value of the to-be-processed unit in the first reference image.
The device according to any one of claims 23 to 25, wherein when the second template distortion value is smaller than the first template distortion value, the determining unit is specifically configured to:

When the second template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, the weighted prediction value is used as a prediction of the to-be-processed unit value.
The device according to claim 26, wherein the determining unit is specifically configured to:

When the first template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, according to the first motion information, in the first reference Obtaining a predicted value of the to-be-processed unit in an image; or

And when the third template distortion value is the smallest among the first template distortion value, the second template distortion value, and the third template distortion value, according to the third motion information, in the third reference In the image, the predicted value of the unit to be processed is obtained.
The device according to any one of claims 23 to 27, comprising:

When the first reference image is from a forward reference image list of the to-be-processed unit, the third reference image is from a backward reference image list of the to-be-processed unit; or, when the first reference When the image is from the backward reference image list of the to-be-processed unit, the third reference image is from the forward reference image list of the to-be-processed unit.
The device according to any one of claims 18 to 28, wherein the updating unit is further configured to:

And updating the second template distortion value according to the weighting coefficient, where the updated second template distortion value is (1-ω0) times the second template distortion value;

Correspondingly, the determining unit is configured to determine, according to the first motion information and the second motion information, that the to-be-processed when the updated second template distortion value is smaller than the first template distortion value The weighted prediction of the unit.
The apparatus according to claim 29, wherein when the updated second template distortion value is not less than the first template distortion value, the determining unit is further configured to:

And obtaining, according to the first motion information, a predicted value of the to-be-processed unit in the first reference image.
The device according to claim 29 or 30, wherein, when the updated second template distortion value is smaller than the first template distortion value, the determining unit is specifically configured to:

When the updated second template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, the weighted prediction value is taken as The predicted value of the processing unit is mentioned.
The device according to claim 31, further comprising:

And when the first template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, according to the first motion information, Obtaining a predicted value of the to-be-processed unit in the first reference image; or

And when the third template distortion value is the smallest among the first template distortion value, the updated second template distortion value, and the third template distortion value, according to the third motion information, In the third reference image, the predicted value of the to-be-processed unit is obtained.