WO2021208588A1

WO2021208588A1 - Video encoding and decoding methods and apparatus, and electronic device

Info

Publication number: WO2021208588A1
Application number: PCT/CN2021/076735
Authority: WO
Inventors: 张弓
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-04-17
Filing date: 2021-02-18
Publication date: 2021-10-21
Also published as: CN111526370A; CN111526370B

Abstract

The present disclosure relates to the technical field of image processing, and specifically relates to a video encoding method, a video decoding method, a video encoding apparatus, and an electronic device. The method comprises: according to a preset selection rule, determining N pairs of original bidirectional reference frames in a reference frame list corresponding to a bidirectional predictive encoded frame; performing frame interpolation processing on the N pairs of original bidirectional reference frames according to a preset frame interpolation rule to obtain T frame interpolation reference frames, and updating the reference frame list according to the T frame interpolation reference frames so as to obtain a target reference frame list; encoding the bidirectional predictive encoded frame according to the target reference frame list to obtain encoded data; acquiring key parameters for generating the frame interpolation reference frames, and adding the key parameters to the encoded data to generate bit stream data corresponding to the bidirectional predictive encoded frame. The present disclosure may overcome the problem of large encoded data caused by large motion vectors or motion vector differences.

Description

Video encoding and decoding method and device and electronic equipment

cross reference

This disclosure claims the priority of a Chinese patent application filed on April 17, 2020 with the application number 202010307909.7 titled "Video encoding and decoding method and device and electronic equipment", and the entire content of the Chinese patent application is incorporated by reference in its entirety. This article.

Technical field

The present disclosure relates to the field of image processing technology, and in particular to a video encoding method, a video decoding method, a video encoding device, and electronic equipment.

Background technique

Video encoding is a way to convert files in the original video format into another video format file through compression technology, which can reduce the size of video data to a certain extent. Inter-frame prediction is a type of video coding. It can use the correlation of pixels between adjacent frames to predict the predicted value, and then achieve the purpose of compressing the video. There are three types of inter-frame prediction, one-way prediction, two-way prediction and overlap compensation prediction. Among them, the bidirectional predictive coding technology is a commonly used coding technology, which can use multiple frames before and after the current frame in the playback sequence as reference frames to perform motion estimation and inter-frame prediction to obtain the final prediction value.

Public content

The purpose of the present disclosure is to provide a video encoding method, a video decoding method, a video encoding device, and an electronic device, so as to overcome at least to a certain extent the problem of large motion vectors or large encoded data caused by large motion vector differences.

Other characteristics and advantages of the present disclosure will become apparent through the following detailed description, or partly learned through the practice of the present disclosure.

According to a first aspect of the present disclosure, a video encoding method is provided, which includes: determining N pairs of original bidirectional reference frames in a reference frame list corresponding to a bidirectional predictive coding frame according to a preset selection rule; wherein, N takes a positive integer; Set the interpolation frame rule to perform interpolation processing on the original two-way reference frame of N, obtain T interpolation frame reference frames, and update the reference frame list according to the T interpolation frame reference frames to obtain the target reference frame list; where T is positive Integer; encode the bidirectional predictive coded frame according to the target reference frame list to obtain coded data; obtain the key parameters for generating the interpolated frame reference frame, append the key parameters to the coded data, and generate the code stream data corresponding to the bidirectional predictive coded frame.

According to a second aspect of the present disclosure, a video decoding method is provided, which includes: obtaining code stream data corresponding to a bidirectional predictive coded frame to be decoded; code stream data including coded data and key parameters; parsing the coded data and analyzing the coded data according to the key parameters Determine the target reference frame list corresponding to the encoded data; perform decoding and reconstruction based on the target reference frame list, and obtain the decoded bidirectional prediction reconstruction frame.

According to a third aspect of the present disclosure, there is provided a video encoding device, including: a reference acquisition module, configured to determine N pairs of original bidirectional reference frames in a reference frame list corresponding to a bidirectional predictive coding frame according to a preset selection rule; where N Take a positive integer; the first list determination module is used to perform frame interpolation processing on N pairs of original bidirectional reference frames according to preset frame interpolation rules, to obtain T frame interpolation reference frames, and to list the reference frames according to the T frame interpolation reference frames Update to obtain the target reference frame list; where T is a positive integer; the data encoding module is used to encode the two-way predictive encoding frame according to the target reference frame list to obtain the encoded data; the code stream generation module is used to obtain the generated frame insertion reference The key parameter of the frame, the key parameter is appended to the coded data, and the code stream data corresponding to the bidirectional predictive coded frame is generated.

According to a fourth aspect of the present disclosure, a video decoding device is provided, which includes: a data acquisition module for acquiring code stream data corresponding to a bidirectional predictive coding frame to be decoded; code stream data includes coded data and key parameters; the second list is determined The module is used to analyze the encoded data and determine the target reference frame list corresponding to the encoded data according to the key parameters; the data reconstruction module is used to decode and reconstruct based on the target reference frame list to obtain the decoded bidirectional prediction reconstruction frame.

According to a fifth aspect of the present disclosure, there is provided a computer-readable medium having a computer program stored thereon, and the computer program is executed by a processor to implement the above-mentioned video encoding method or video decoding method.

According to a sixth aspect of the present disclosure, there is provided an electronic device, which is characterized by comprising: a processor; and a memory, configured to store one or more programs, when the one or more programs are executed by one or more processors , Enabling one or more processors to implement the above-mentioned video encoding method or video decoding method.

In the video encoding method provided by an embodiment of the present disclosure, the original bidirectional reference frame can be determined by determining the original bidirectional reference frame in the reference frame list corresponding to the bidirectional predictive coding frame according to a preset selection rule, and then the determined original bidirectional reference frame can be obtained by interpolation T number of interpolated frame reference frames with relatively small relative motion between the bidirectional predictive coding frame, and finally a target reference frame list is formed according to the reference frame list and the T interpolated frame reference frames. Since there are T interpolated reference frames with relatively small relative motion between the target reference frame list and the bidirectional predictive coding frame, there will be no large motion vector or motion vector difference, and there will be no relatively large amount of coded data. Big problem.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments consistent with the disclosure, and are used together with the specification to explain the principle of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the attached picture:

Fig. 1 schematically shows a flowchart of a video encoding method in an exemplary embodiment of the present disclosure;

Fig. 2 schematically shows a flowchart of a method for determining N pairs of original bidirectional reference frames according to a preset selection rule in an exemplary embodiment of the present disclosure;

Fig. 3 schematically shows a flowchart of a method for obtaining T frame interpolation reference frames by performing frame interpolation processing according to a preset frame interpolation rule in an exemplary embodiment of the present disclosure;

FIG. 4 schematically shows a flowchart of a method for determining the time phase of an interpolation frame reference frame corresponding to an original bidirectional reference frame in an exemplary embodiment of the present disclosure;

FIG. 5 schematically shows a flowchart of a method for determining a third time point in an exemplary embodiment of the present disclosure;

Fig. 6 schematically shows a flowchart of a video decoding method in an exemplary embodiment of the present disclosure;

FIG. 7 schematically shows a flowchart of a method for determining a target reference frame list corresponding to encoded data according to key parameters in an exemplary embodiment of the present disclosure;

Fig. 8 schematically shows a flowchart of a video encoding method in an exemplary embodiment of the present disclosure;

Fig. 9 schematically shows a flowchart of a video decoding method in an exemplary embodiment of the present disclosure;

FIG. 10 schematically shows a schematic diagram of determining a motion vector based on motion estimation in an exemplary embodiment of the present disclosure;

FIG. 11 schematically shows a schematic diagram of a corrected motion vector in an exemplary embodiment of the present disclosure;

FIG. 12 schematically shows a schematic diagram of frame interpolation based on motion compensation in an exemplary embodiment of the present disclosure;

FIG. 13 schematically shows a schematic diagram of the implementation principle of one of the exemplary embodiments of the present disclosure;

FIG. 14 schematically shows a schematic diagram of the implementation principle of another embodiment in the exemplary embodiment of the present disclosure;

FIG. 15 schematically shows a schematic diagram of the composition of a video encoding device in an exemplary embodiment of the present disclosure;

FIG. 16 schematically shows a schematic diagram of the composition of a video decoding device in an exemplary embodiment of the present disclosure;

FIG. 17 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed To those skilled in the art. The described features, structures or characteristics can be combined in one or more embodiments in any suitable way.

In addition, the drawings are only schematic illustrations of the present disclosure, and are not necessarily drawn to scale. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in the form of software, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

In the related art, when bidirectional predictive coding is performed on a video with relatively large frame-to-frame relative motion, since the relative motion between frames is relatively large, a large motion vector or motion vector difference will be generated. When encoding these large running vectors or motion vector differences, a larger amount of data may be formed, which will lead to the problem of a larger amount of encoded data; at the same time, the relative motion between frames is relatively large. At this time, it may also cause the search range of motion estimation to be too large, which in turn leads to the problem that the encoding process takes a long time.

In view of the above-mentioned shortcomings and deficiencies in the prior art, this exemplary embodiment provides a video encoding method, which can be applied to terminal devices with image processing functions, such as mobile phones, tablet computers, and the like.

Referring to FIG. 1, the foregoing video encoding method may include the following steps S110 to S140:

Step S110: Determine N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to a preset selection rule.

Among them, N takes a positive integer. It should be noted that, for videos with different attributes, the logarithm N of the original bidirectional reference frame can be set according to different reference frame lists, and the present disclosure does not specifically limit the logarithm of the selected original bidirectional reference frame. For example, in the reference frame list, there are only two frames of storage space, that is, when storing one frame forward original reference frame and one frame backward original reference frame, you can set N to be 1; for another example, there are multiple frames in the reference frame list When the frame forward original reference frame and the multi-frame backward original reference frame, the value of N can be set according to the magnitude of the relative motion between frames.

In this example embodiment, the preset selection rules may include pairing rules and selection rules. At this time, referring to FIG. 2, determining N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to the preset selection rule may include the following steps S210 to S220:

Step S210: Pair the original forward reference frame and the original backward reference frame in the reference frame list corresponding to the bidirectional predictive coding frame according to the pairing rule to generate at least one pair of candidate original bidirectional reference frames.

In this example embodiment, the pairing rule may include any rule for pairing one frame of original forward reference frame and one frame of original backward reference frame in the reference frame list, and the present disclosure does not specifically limit the pairing rule . For example, when the frame number corresponding to the bidirectional predictive coding frame is A, you can set the original forward reference frame with the frame number Ak and the original backward reference frame with the frame number A+k as a pair; for another example, you can also set the frame The original forward reference frame numbered Ak and the original backward reference frame numbered A+(k+1) are a pair.

It should be noted that since the reference frame of bidirectional predictive coding must include a forward reference frame and a backward reference frame at the same time, at least one pair of candidate original bidirectional reference frames generated according to the pairing rule must also include one Frame original forward reference frame and one frame original backward reference frame.

Step S220: Determine N pairs as the original bidirectional reference frames from at least one pair of candidate original bidirectional reference frames according to the selection rule.

In this example embodiment, the selection rule may include any rule for determining N pairs as the original bidirectional reference frame among at least one pair of candidate original bidirectional reference frames, and the present disclosure does not specifically limit the pairing rule. For example, the selection rule can be to determine the N pairs of candidate original bidirectional reference frames that are closest to the current bidirectional predictive coding frame as the original bidirectional reference frame; N pairs of bidirectional reference frames are randomly selected and determined as the original bidirectional reference frames.

Step S120: Perform frame interpolation processing on N original bidirectional reference frames according to preset frame interpolation rules to obtain T frame interpolation reference frames, and update the reference frame list according to the T frame interpolation reference frames to obtain a target reference frame list.

In this example embodiment, the preset frame interpolation rule may be any frame interpolation rule for interpolating frames on the original reference frame pair, and may include frame interpolation method rules, time phase rules, etc., or may be both frame interpolation method rules and time phase rules. And other rules, this disclosure does not make any special restrictions on this.

For example, the preset frame interpolation rules may include the following frame interpolation rules: perform one frame interpolation on N original bidirectional reference frames to obtain N primary frame interpolation reference frames, and then perform secondary frame interpolation on N primary frame interpolation reference frames, Obtain M secondary interpolation frame reference frames, and use N primary interpolation frame reference frames and M secondary interpolation frame reference frames together as T interpolation frame reference frames; for another example, the preset interpolation frame rule may include A pair of original bidirectional reference frames forms the time phase rule of the interpolated frame reference frame; for another example, the preset interpolated frame rule may include the above-mentioned interpolated frame method rule and the time phase rule set for the primary interpolated frame and the secondary interpolated frame at the same time.

It should be noted that since the frame interpolation reference frame can be implemented according to various preset frame interpolation rules, the value of T may not necessarily be the same as the value of N, and T may be greater than N, less than or equal to N. This is the case in this disclosure. There are no special restrictions.

For example, when the preset frame interpolation rule includes the time phase rule, referring to FIG. 3, the frame interpolation process is performed on N original bidirectional reference frames according to the preset frame interpolation rule to obtain T frame interpolation reference frames, which may include The following steps S310 to S320:

In step S310, the time phases of the interpolated frame reference frames corresponding to the N pairs of original bidirectional reference frames are respectively determined according to the time phase rule.

Wherein, the time phase of the interpolated frame reference frame can be used to determine the position of the interpolated frame reference frame between the original bidirectional reference frames. For example, the time phase of the interpolated frame reference frame may be the time that the interpolated frame reference frame is later than the original forward reference frame in the original bidirectional reference frame; for another example, the time phase of the interpolated frame reference frame may be the time that the interpolated frame reference frame is earlier than the original bidirectional reference frame. The time of the original backward reference frame in the reference frame.

Specifically, when the time phase of the interpolated frame reference frame is the time of the interpolated frame reference frame later than the original forward reference frame in the original bidirectional reference frame, assuming that the time phase of a certain interpolated frame reference frame is 0.5, the corresponding original forward The time corresponding to the reference frame is the first second, and the time corresponding to the inserted frame reference frame is the 1.5 second.

In this exemplary embodiment, referring to FIG. 4, determining the time phase of the interpolated frame reference frame corresponding to the original bidirectional reference frame through the time phase rule may include the following steps S410 to S420:

Step S410: Obtain the first frame number corresponding to the original forward reference frame in the original bidirectional reference frame, the first time point and the second frame number and second time point corresponding to the original backward reference frame, and the corresponding bidirectional predictive coding frame The third frame number.

Step S420: Determine a third time point between the first time point and the second time point according to the first frame number, the second frame number, and the third frame number, and determine the third time point as the time phase of the interpolated frame reference frame .

In this exemplary embodiment, in order to make the relative motion between the interpolated frame reference frame and the bidirectional predictive coded frame smaller, it is necessary to make the interpolated frame reference frame closer to the bidirectional predictive coded frame. At this time, the relationship between the frame number and the time point of each frame image in the video can be used to adaptively determine the time phase of the interpolated frame reference frame. Specifically, after acquiring the corresponding first frame number, the first time point of the original forward reference frame, the second frame number corresponding to the original backward reference frame, the second time point, and the third frame corresponding to the bidirectional predictive coding frame According to the first frame number, the second frame number, and the third frame number, the third time point is determined between the first time point and the second time point, and the third time point is determined as the time phase of the interpolated frame reference frame.

In this exemplary embodiment, the third time point is determined between the first time point and the second time point according to the first frame number, the second frame number, and the third frame number. As shown in FIG. 5, the following steps S510 to S540:

Step S510: Calculate the first frame number difference corresponding to the first frame number and the second frame number, and calculate the time difference between the first time point and the second time point.

Step S520: Calculate the quotient of the time difference and the first frame number difference as the unit duration.

In this exemplary embodiment, since the frame number of each frame in the video is inherently associated with the time point at which each frame is located, the unit duration between each frame can be determined according to the frame number and time point. Specifically, the frame number difference between the frame numbers corresponding to the original forward reference frame and the original backward reference frame can be calculated as the first frame number difference, and then the video corresponding to the original forward reference frame and the original backward reference frame can be calculated. The time difference between the first time point and the second time point, and finally the quotient of the time difference and the first frame number difference is calculated to obtain the unit duration between each frame.

Step S530: Calculate the product of the difference between the second frame number corresponding to the first frame number and the third frame number and the unit duration as the first duration, and determine the time point later than the first time point by the first duration as the third time point .

Step S540: Calculate the product of the third frame number difference corresponding to the second frame number and the third frame number and the unit duration as the second time length, and determine the time point earlier than the second time point by the second time as the third time point .

In this exemplary embodiment, after determining the unit duration between each frame, the frame number difference between the original forward reference frame, the original backward reference frame, and the current bidirectional predictive coding frame can be used to determine the corresponding interpolated frame reference frame. The third point in time. Specifically, the second frame number difference corresponding to the first frame number and the third frame number can be calculated first, and then the product of the second frame number difference and the unit duration can be calculated as the first duration, and finally the first time will be later than the first time point. The time point of the duration is determined as the third time point; for example, the first time point is 10, and the calculated first time length is 0.5 units, then the third time point is 10.5; or, the second frame number and the first time point can be calculated first. The third frame number difference corresponding to the three frame numbers is calculated, and the product of the third frame number difference and the unit duration is calculated as the second time length, and finally the time point earlier than the second time point by the second time point is determined as the third time point.

It should be noted that since the frame number of each frame in the video is inherently associated with the time point where each frame is located, the third time point calculated by the above method is usually very close to the time point corresponding to the current bidirectional predictive coding frame Therefore, the relative motion between the final interpolated frame reference frame and the current bidirectional predictive coding frame will be relatively small, which can avoid the problem of large coding data caused by large motion vectors or motion vector differences; at the same time, it can also reduce the motion Estimated search range, thereby reducing coding time.

In this exemplary embodiment, the aforementioned frame interpolation processing may use motion estimation motion compensation method, optical flow method, neural network frame interpolation, or other arbitrary frame rate conversion technology.

For example, the aforementioned motion estimation motion compensation method may include the following two steps:

1. Motion estimation

According to at least two images, the motion estimation method is used to determine the motion vectors of all objects or regions in the two images. Specifically, the two images can be recorded as the current image and the reference image, the two images can be divided into blocks according to the preset size, and the divided images can be traversed to search for each block in the current image in the reference image. Match the block, determine the motion vector (forward MV) of each block of the current image relative to the reference image. Similarly, use the above method to determine the motion vector (backward MV) of each block of the reference image relative to the current image, as shown in Figure 10. Shown.

Subsequently, a correction operation is performed on the forward and backward MV, where the correction operation includes at least one or a combination of multiple operations such as filtering and weighting, and finally determines the forward or backward MV of each block, as shown in FIG. 11

2. Motion compensation

Correct the final forward or backward MV of each block through the interpolated frame time phase, and then generate a mapping MV of each interpolated block relative to the current image and the reference image in the interpolated frame image. Find the corresponding block in the image and the current image, perform weight interpolation of the two blocks, generate all the pixels of the interpolation block, and finally get the interpolated frame image, as shown in Figure 12.

In step S320, frame interpolation is performed on N original bidirectional reference frames, respectively, to generate T frame interpolation reference frames in time phase.

In this example embodiment, after determining the time phase of the interpolated frame reference frame corresponding to the N pairs of original bidirectional reference frames, the time phase can be determined according to the time phase of the interpolated frame reference frame corresponding to each pair of original bidirectional reference frames. Interpolate the frame to obtain the interpolated frame reference frame at the time phase. By setting the time and phase rules, the time difference between the generated interpolated frame reference frame and the bidirectional predictive coding frame can be controlled, and then the relative motion between the interpolated frame reference frame and the bidirectional predictive coding frame can be controlled, which can reduce the motion vector and the motion vector more effectively The size of the difference can more effectively reduce the search range of motion estimation.

In this exemplary embodiment, after T interpolated frame reference frames are obtained, the relative motion of the T interpolated frame reference frames to the current bidirectional coded frame is smaller than the original bidirectional reference frame. Therefore, it can be based on the T interpolated frames. The frame reference frame updates the reference frame list to obtain the target reference frame list, and then encodes according to the target reference frame list to obtain encoded data.

In this exemplary embodiment, updating the reference frame list according to the T interpolated frame reference frames to obtain the target reference frame list may include: adding the T interpolated frame reference frames to the reference frame list to generate the target reference frame list. At this time, the generated T interpolated frame reference frames can be inserted into the reference frame list according to the previously determined time phase to form a list of target reference frames arranged in chronological order. In addition, the sort order and the position of the inserted reference frame list can also be set for the inserted frame reference frame, which is not particularly limited in the present disclosure.

In this exemplary embodiment, updating the reference frame list according to the T insertion frame reference frames to obtain the target reference frame list may also include: replacing the reference frame list with the list formed by the arrangement of the T insertion frame reference frames to generate the target reference frame List. At this time, you can directly set the sorting method and frame number for the T interpolated frame reference frames, and arrange the T interpolated frame reference frames according to the sorting method and frame number to form a list, and replace the reference frame list according to the list to generate the target Reference frame list. In addition, the T interpolated frame reference frames can also be converted into a list in other ways, and the present disclosure does not impose special restrictions on this.

It should be noted that, in other exemplary embodiments, the target reference frame list can also be obtained by performing other processing on the reference frame list and the T interpolated frame reference frames. The process of obtaining the target reference frame list in the present disclosure is also No special restrictions. For example, the reference frame list and T interpolated frame reference frames can be filtered under certain conditions, and the target reference frame list can be formed according to the filtered reference frame list and the T interpolated frame reference frames; for another example, the target reference frame list can be formed by comparing the T interpolated reference frames. The insertion frame reference frame is screened under certain conditions, and a target reference frame list is formed according to the filtered insertion frame reference frame.

Step S130, encoding the bidirectionally predictive encoded frame according to the target reference frame list to obtain encoded data.

In this exemplary embodiment, after the target reference frame list is obtained, encoding can be performed according to the target reference frame list to obtain encoded data. Since there are interpolated frame reference frames in the target reference frame list, there will be no large motion vector or motion vector difference when the bidirectional predictive coding frame is coded. At the same time, the search range of coding can be reduced and the coding speed can be increased.

Step S140: Obtain key parameters for generating the interpolated frame reference frame, append the key parameters to the coded data, and generate code stream data corresponding to the bidirectional predictive coded frame.

In this example embodiment, since the interpolated frame is used in the encoding process, the relevant data of the interpolated frame may also be needed in the decoding process. Therefore, some key parameters for generating the interpolated reference frame need to be transmitted together with the encoded data. To the decoding end for decoding. Specifically, the key parameter for generating the interpolated frame reference frame may be used as the accessory parameter and the coded data to jointly generate the bitstream data corresponding to the bidirectional predictive coded frame.

In this exemplary embodiment, the key parameters may include N pairs of frame numbers corresponding to the original forward reference frame and the original backward reference frame in the N pairs of original bidirectional reference frames, and the number T of the interpolated frame reference frames. Specifically, the frame numbers corresponding to the original forward reference frame and the original backward reference frame in the N pairs of original bidirectional reference frames are stored in pairs, so that the basis for generating each interpolated frame reference frame can be determined.

In addition, in an exemplary embodiment, the encoding end and the decoding end can also be directly set as a set, that is, a certain encoding method is used on the encoding end, and the corresponding encoding method can be directly used in the decoding end. Decoding method. In this case, if a certain encoding method is used, only the used encoding method enable flag can be written in the key parameters, which means that the bidirectional predictive encoding frame uses the encoding method, and the corresponding decoding end can be based on The enable flag determines the corresponding decoding mode.

It should be noted that the foregoing encoding may be performed based on the H.264 video encoding standard. In this case, N pairs of frame numbers and the number of interpolated reference frames T can be written into the Supplementary Enhancement Information Unit (SEI), and the enable flag and the reference selected by each coding block in the bidirectional predictive coding frame The frame number complies with the standard to write the code stream. In addition, other video coding standards can also be used for coding, and the code stream data can be written in different ways according to the coding standard, which is not specifically limited in the present disclosure.

In another exemplary embodiment of the present disclosure, a video decoding method is also provided, which can be applied to terminal devices with image processing functions, such as mobile phones and tablet computers. As shown in FIG. 6, the above-mentioned video decoding method may include step S610 to step S630, which will be described in detail below:

Step S610: Obtain code stream data corresponding to the bidirectional predictive coded frame to be decoded.

In this exemplary embodiment, the acquired bitstream data corresponding to the bidirectional predictive encoded frame to be decoded includes encoded data and key parameters. Among them, the encoded data refers to the encoded data obtained based on the foregoing encoding method, and the key parameter is the key parameter for generating the interpolated frame reference frame in the foregoing encoding method. Specifically, the key parameter includes a key parameter used to determine the interpolated frame reference frame, for example, may be the frame number of the original bidirectional reference frame on which the interpolated frame reference frame is generated.

It should be noted that, in an exemplary embodiment, the encoding end and the decoding end can also be directly set up, that is, a certain encoding method is used on the encoding end, and the corresponding encoding can be directly used in the decoding end. The corresponding decoding method. In this case, the key parameters may also include only the enable flag corresponding to the encoding method used during encoding, and during decoding, the corresponding decoding method can be determined according to the enable flag for decoding.

Step S620: Analyze the encoded data, and determine the target reference frame list corresponding to the encoded data according to the key parameters.

In this example embodiment, the key parameter may include N pairs of frame numbers and the number T, where N and T are both positive integers. At this time, referring to FIG. 7, parsing the encoded data and determining the target reference frame list corresponding to the encoded data according to the key parameters may include the following steps S710 to S740:

Step S710: Analyze and reconstruct the encoded data to obtain a reference frame list corresponding to the bidirectionally predictive encoded frame to be decoded.

In this exemplary embodiment, the reference frame list corresponding to the bidirectional predictive coded frame to be decoded is the original reference frame list when the bidirectional predictive coded frame to be decoded is decoded, and each reference frame in the reference frame list is a reference frame that can be used as a reference frame. A video frame obtained by reconstructing the encoded data corresponding to the video frame. Before decoding the bidirectional predictive coded frame to be decoded, it is necessary to obtain a reference frame list corresponding to the bidirectional predictive coded frame to be decoded.

Step S720: Determine N pairs of original bidirectional reference frames in the reference frame list according to the N pairs of frame numbers.

In this example embodiment, the key parameters include the frame numbers of the N pairs of original bidirectional reference frames that generate the interpolated frame reference frame. Therefore, the N pairs of original bidirectional reference frames can be determined in the reference frame list through the N pairs of frame numbers, and then the interpolated frame can be determined. The basis for frame reference frame generation.

In step S730, the original bidirectional reference frame, the number T and the coded data are interpolated and reconstructed based on N to obtain T interpolated reference frames.

In this exemplary embodiment, the number of original bidirectional reference frames, the number of generated interpolated reference frames T, and the prediction residuals in the encoded data can be interpolated and reconstructed based on N to reproduce T interpolated reference frames.

In step S740, the reference frame list is updated based on the T interpolated frame reference frames to obtain the target reference frame list.

In this exemplary embodiment, after T number of interpolated frame reference frames are reproduced, the reference frame list can be updated according to the T interpolated frame reference frames, so as to obtain the target reference frame list. In order to reproduce the target reference frame list used in encoding, the same method of constructing the target reference frame list as in the encoding process can be adopted.

For example, updating the reference frame list according to the T interpolated frame reference frames to obtain the target reference frame list may include: adding the T interpolated frame reference frames to the reference frame list to generate the target reference frame list. At this time, the generated T interpolated frame reference frames can be inserted into the reference frame list according to the previously determined time phase to form a list of target reference frames arranged in chronological order. In addition, the sort order and the position of the inserted reference frame list can also be set for the inserted frame reference frame, which is not particularly limited in the present disclosure.

For another example, updating the reference frame list according to the T interpolated frame reference frames to obtain the target reference frame list may also include: replacing the reference frame list with the list formed by the arrangement of the T interpolated frame reference frames to generate the target reference frame list. At this point, you can directly set the sorting method and frame number for the T interpolated frame reference frames, and arrange the T interpolated frame reference frames according to the sorting method and frame number to form a list, and replace the reference frame list according to the list to generate the target Reference frame list. In addition, the T interpolated frame reference frames can also be converted into a list in other ways, and the present disclosure does not impose special restrictions on this.

Step S630: Perform decoding and reconstruction based on the target reference frame list to obtain a decoded bidirectional prediction reconstruction frame.

In this exemplary embodiment, after the target reference frame list during encoding is reproduced, the encoded data can be decoded and reconstructed according to the target reference frame list to obtain the bidirectional predictive encoded frame corresponding to the bidirectional predictive encoded frame to be decoded. For example, the predicted value corresponding to the video to be decoded can be obtained according to the target reference frame list corresponding to the bidirectional predictive coded frame to be decoded, and the video frame can be reconstructed according to the prediction residual and the predicted value in the coded data to obtain the bidirectional predictive coded frame.

For the foregoing encoding and decoding solutions, the embodiments of the present disclosure provide the following five specific embodiments:

Example 1:

Encoding process:

Referring to Figure 8, step S810, if the current coded frame is a bidirectional predictive coded frame, obtain its original reference frame list; step S820, determine N pairs of forward and backward original reference frames in the original reference frame list; step S830, input interpolation The frame module performs frame insertion to form T frame insertion reference frames; step S840, accumulate the newly formed T frame insertion reference frames and the original reference frame list to form a target reference frame list; step S850, perform according to the target reference frame list The predictive coding of the fixed video coding standard forms the coded data. The number T of interpolated frame reference frames used in the current bidirectional predictive coding frame and the frame numbers of the original forward and backward reference frames selected by these interpolated frame reference frames are used as additional parameters and encoded data Commonly used as code stream data.

Among them, the rules for selecting the original two-way reference frame can be corresponded in the order of playback. For example, if the current frame is the k-th frame, then k-1 and k+1 form a pair, and then k-2 and k+2 form a pair. In addition, any pair of original reference frames can also be selected as a pair; when inserting frames based on the original two-way reference frame, the time phase of the inserted frame reference frame can be manually set; or according to the time phase of the current frame in the original two-way reference frame Adaptive setting, for example, if the reference frame pair is k-1 and k+1, and the total length of the two reference frames is set to 1, then the adaptive time phase is 0.5; if the reference frame pair is k-1 and k+2, suppose the total length of the two reference frames is 1, then the adaptive time phase is 0.33; in the frame insertion process, each pair of original bidirectional reference frames forms an inserted frame reference frame, so T=N.

It should be noted that during the encoding process and the decoding and reconstruction process, the frame number of the reference frame selected by each encoding block in the current bidirectional predictive encoding frame may be required. In this case, the data can also be used as an additional parameter Add code stream data; in addition, when this encoding method is used, the corresponding enable flag can be written into additional parameters to identify that the method is enabled during the encoding process.

Decoding process:

Referring to Figure 9, step S910, when the current bitstream data is a bidirectional predictive coded frame, determine N pairs of original bidirectional reference frames in the original reference frame list according to additional parameters; step S920, according to the obtained N pairs of original bidirectional reference frames Perform interpolation frame reconstruction to obtain T interpolation frame reference frames; step S930, a target reference frame list formed based on the original reference frame list and T interpolation frame reference frames; step S940, parse the encoded data to obtain prediction residuals and predictions Value; Step S950, decoding and reconstruction based on the target reference frame list, prediction residuals, and prediction values, to obtain a decoded bidirectional prediction reconstruction frame.

Example 2:

Encoding process:

As shown in Figure 13, the previous frame and the next frame in the playback sequence of the current bidirectional predictive coding frame are determined as the original bidirectional reference frame, that is, N=1, and the intermediate phase interpolation is performed on this pair of original bidirectional reference frames. An interpolated frame reference frame is formed, and the encoder only uses the formed interpolated frame reference frame as a reference for encoding during encoding. In addition, since the previous frame and the next frame are set as the original bidirectional reference frame by default, there is no need to form the key parameters of the interpolated frame reference frame, and only the enable flag can be written into the code stream information to indicate that the encoding method is used .

Decoding process:

As shown in Figure 13, when decoding, the corresponding strategy is used for decoding. For each decoded block in the bidirectional predictive coding frame that needs to be reconstructed, the previous frame and the following frame of the currently reconstructed bidirectional predictive coding frame are used as the original two-way predictive coding frame. For reference frame, the original bidirectional reference frame is interpolated to obtain an interpolated frame reference frame, and the interpolated frame reference frame is used for reference decoding to obtain the decoded bidirectional predictive reconstructed frame.

Example 3:

Encoding process:

As shown in FIG. 14, for only the current bidirectional predictive coding frame, based on N pairs of original bidirectional reference frames, T interpolated frame reference frames with the same time phase as the current bidirectional predictive coding frame are formed. Among them, in the frame insertion process, each pair of original bidirectional reference frames forms an insertion frame reference frame, so T=N. When encoding, the encoder only uses the formed T interpolated frame reference frames as a reference for encoding, and writes the frame number of the original bidirectional reference frame and the number of interpolated frame reference frames T as additional parameters into the code stream.

Decoding process:

As shown in Figure 14, when decoding, the corresponding strategy is adopted for decoding. For each decoded block in the bidirectional predictive coding frame that needs to be reconstructed, the number T of the original bidirectional reference frame and the number of interpolated reference frames are added. The parameters determine N pairs of original bidirectional reference frames, and then based on N pairs of original bidirectional reference frames, T interpolated frame reference frames with the same time phase as the current bidirectional predictive coding frame are formed, and the last T interpolated reference frames are decoded based on The decoded bidirectional prediction reconstructs the frame.

Example 4:

Encoding process:

During encoding, for the current bidirectional predictive coded frame, first interpolate the original bidirectional reference frame according to N to obtain A one-time interpolated frame reference frames with the same time phase as the current bidirectional predictive coded frame, and then according to A one time Interpolation frame reference frames perform two-stage interpolation frames in pairs to form B two-stage interpolation frame reference frames. Subsequently, a target reference frame list including T reference frames is formed through A primary interpolation frame reference frames and B secondary interpolation frame reference frames, where T=A+B.

Specifically, when sorting the A primary interpolation frame reference frames and the B secondary interpolation frame reference frames, the B secondary interpolation frame reference frames may be sequentially numbered as the frame of the secondary interpolation frame reference frame. For example, the frame numbers are 0, 1, ..., B-2, B-1; then, the A one-time interpolated frame reference frames are sequentially numbered as the frame number of the one-time interpolated frame reference frame, for example, the frame numbers are respectively B, B+1,..., B+A-2, B+A-1.

When encoding, the encoder only refers to the target reference frame list formed by sorting A primary interpolated frame reference frames and B secondary interpolated frame reference frames. At the same time, it encodes the frame number of the original bidirectional reference frame and the primary interpolated frame reference. The number of frames A and the reference frame number B of the secondary interpolation frame are written into the code stream as additional parameters.

Decoding process:

When decoding, the corresponding strategy is adopted for decoding. For each decoding block in the bidirectional predictive coding frame that needs to be reconstructed, according to the frame number of the original bidirectional reference frame and the number of primary interpolated frame reference frames A and the number of secondary interpolated frame reference frames B determines N pairs of original bidirectional reference frames, and based on N pairs of original bidirectional reference frames, forms A one-time interpolated frame reference frames with the same time phase as the current bidirectional predictive coding frame, and then performs two-by-two based on the A one-time interpolated frame reference frames Re-interpolation frame, obtain B secondary-interpolation frame reference frames, and perform encoding and sorting with A primary-interpolation frame reference frames and B secondary-interpolation frame reference frames to form a target reference frame list containing T inter-frame reference frames, Finally, the T interpolated frame reference frames are used as references for decoding, and the decoded bidirectional prediction reconstructed frame is obtained.

Example 5:

Encoding process:

In an embodiment, there may be only two frames of storage space in the original reference frame list, that is, only one frame of original forward reference frame and one frame of original backward reference frame can be stored. At this time, according to this frame forward reference frame and a frame backward reference frame, an interpolated frame reference frame with the same time phase as the current bidirectional predictive coding frame can be formed, and then this interpolated frame reference frame and the original front frame can be used respectively. Perform secondary interpolation to the reference frame and the original backward reference frame. The time phase of the interpolation frame can be 0.5 to obtain a forward interpolation frame reference frame and a backward interpolation frame reference frame, and then pass the forward interpolation frame reference frame and The backward interpolation frame reference frame replaces the original forward reference frame and the original backward reference frame in the original reference frame list to form the target reference frame list.

When the encoder is encoding, it only refers to one frame of forward interpolated frame reference frame and one frame of backward interpolated frame reference frame in the target reference frame list. At the same time, it encodes the frame number of the original bidirectional reference frame and the number of interpolated frame reference frames 2 Write the code stream as an additional parameter.

Decoding process:

When decoding, the corresponding strategy is adopted for decoding. For each decoding block in the bidirectional predictive coding frame that needs to be reconstructed, a pair of original bidirectional reference frames is determined according to the frame number of the original bidirectional reference frame and the number of interpolated reference frames 2, and then Based on this pair of original bidirectional reference frames, one interpolated frame is obtained to obtain an interpolated frame reference frame with the same time phase as the current bidirectional predictive encoded frame, and then the interpolated frame reference frame, the original forward reference frame, and the original back Perform secondary interpolation to the reference frame, and the time phase of the interpolation frame can be 0.5 to obtain a forward interpolation frame reference frame and a backward interpolation frame reference frame to form a target reference frame list, and finally use two of the target reference frame lists. Two interpolated frame reference frames are used as a reference for decoding, and a decoded bidirectional prediction reconstructed frame is obtained.

It should be noted that the embodiments of the present disclosure can reduce the relative motion between the current bidirectional predictive encoding frame and the reference frame in the video by inserting the frame before encoding, thereby reducing the motion vector or the motion vector difference, and avoiding the motion vector or the motion vector. The problem of too large coded data caused by a large difference; at the same time, because the motion vector or the motion vector difference is small, the search range is reduced when encoding, so the coding efficiency can be improved and the coding time can be shortened.

It should be noted that the above-mentioned drawings are only schematic illustrations of the processing included in the method according to the exemplary embodiment of the present invention, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

Further, referring to FIG. 15, the embodiment of this example also provides a video encoding device 1500, including: a reference acquiring module 1510, a first list determining module 1520, a data encoding module 1530 and a code stream generating module 1540. in:

The reference acquisition module 1510 may be configured to determine N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to a preset selection rule; where N is a positive integer.

The first list determining module 1520 may be configured to perform frame interpolation processing on N pairs of original bidirectional reference frames according to preset frame interpolation rules, obtain T frame interpolation reference frames, and update the reference frame list according to the T frame interpolation reference frames, Obtain a list of target reference frames; where T is a positive integer.

The data encoding module 1530 may be used to encode the bidirectionally predictive encoded frame according to the target reference frame list to obtain encoded data.

The code stream generating module 1540 may be used to obtain key parameters for generating the interpolated frame reference frame, append the key parameters to the coded data, and generate code stream data corresponding to the bidirectional predictive coded frame.

In this example embodiment, the reference acquisition module 1510 may be used to pair the original forward reference frame and the original backward reference frame in the reference frame list corresponding to the bidirectional predictive coded frame according to the pairing rule to generate at least one pair of candidate original bidirectional Reference frame: Determine N pairs as the original bidirectional reference frame from at least one pair of candidate original bidirectional reference frames according to the selection rule.

In this example embodiment, the first list determining module 1520 may be used to determine the time phases of the interpolation frame reference frames corresponding to the N pairs of original bidirectional reference frames respectively according to the time phase rule; wherein, the time phase of the interpolation frame reference frame is used to determine the time phase of the interpolation frame. The position of the frame reference frame between the original bidirectional reference frames; for N pairs of original bidirectional reference frames, frame interpolation is performed to generate T interpolated frame reference frames in time phase.

In this example embodiment, the first list determining module 1520 may be used to obtain the first frame number, the first time point, and the second frame number corresponding to the original forward reference frame in the original bidirectional reference frame, and the second frame number corresponding to the original backward reference frame. The second time point, and the third frame number corresponding to the bidirectional predictive coding frame; the third time point is determined between the first time point and the second time point according to the first frame number, the second frame number, and the third frame number, and The third time point is determined as the time phase of the interpolated frame reference frame.

In this example embodiment, the first list determining module 1520 may be used to calculate the first frame number difference corresponding to the first frame number and the second frame number, and calculate the time difference between the first time point and the second time point; calculate the time difference and The quotient of the first frame number difference is used as the unit duration; the product of the second frame number difference corresponding to the first frame number and the third frame number and the unit duration is calculated as the first duration, and the first duration will be later than the first time point by the first duration The time point is determined as the third time point; or the product of the third frame number difference corresponding to the second frame number and the third frame number and the unit duration is calculated as the second time length, which will be a second time earlier than the second time point The point is determined as the third time point.

In this example embodiment, the first list determining module 1520 may be used to add T interpolated frame reference frames to the reference frame list to generate a target reference frame list.

In this example embodiment, the first list determining module 1520 may be configured to replace the reference frame list with a list formed by the arrangement of T interpolated frame reference frames to generate a target reference frame list.

In this exemplary embodiment, the key parameters include N pairs of frame numbers corresponding to the original forward reference frame and the original backward reference frame in the N pairs of original bidirectional reference frames, and the number T of interpolated reference frames.

In addition, as shown in FIG. 16, in an exemplary embodiment of the present disclosure, a video decoding device 1600 is also provided, which is characterized in that it includes: a data acquisition module 1610, a second list determination module 1620, and a data reconstruction module 1630 . in:

The data acquisition module 1610 can be used to acquire code stream data corresponding to the bidirectional predictive coding frame to be decoded; the code stream data includes coded data and key parameters.

The second list determining module 1620 may be used to analyze the encoded data and determine the target reference frame list corresponding to the encoded data according to key parameters.

The data reconstruction module 1630 may be configured to perform decoding and reconstruction based on the target reference frame list, and obtain the bidirectional predictive coding frame corresponding to the bidirectional predictive coding frame to be decoded.

In this example embodiment, the second list determining module 1620 can be used to parse and reconstruct the encoded data to obtain the reference frame list corresponding to the bidirectional predictive encoded frame to be decoded; determine the N pairs of original bidirectional frames in the reference frame list according to the N pairs of frame numbers. Reference frame: Based on N, the original two-way reference frame, the number T and the coded data are inserted into the frame to obtain T interpolation frame reference frames; the reference frame list is updated based on the T interpolation frame reference frames, and the target reference frame list is obtained.

In this example embodiment, the second list determining module 1620 may be used to add T interpolated frame reference frames to the reference frame list to generate a target reference frame list.

In this example embodiment, the second list determining module 1620 may be configured to replace the reference frame list with a list formed by the arrangement of T interpolated frame reference frames to generate a target reference frame list.

The specific details of each module in the above-mentioned video encoding device and video decoding device have been described in detail in the corresponding video encoding method and video decoding method, and therefore will not be repeated here.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units to be embodied.

Fig. 17 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present invention.

It should be noted that the computer system 1700 of the electronic device shown in FIG. 17 is only an example, and should not bring any limitation to the function and scope of use of the embodiment of the present invention.

As shown in FIG. 17, the computer system 1700 includes a central processing unit (CPU) 1701, which can be based on a program stored in a read only memory (ROM) 1702 or a program loaded from a storage portion 1708 into a random access memory (RAM) 1703 And perform various appropriate actions and processing. In RAM 1703, various programs and data required for system operation are also stored. The CPU 1701, the ROM 1702, and the RAM 1703 are connected to each other through a bus 1704. An input/output (I/O) interface 1705 is also connected to the bus 1704.

The following components are connected to the I/O interface 1705: an input part 1706 including a keyboard, a mouse, etc.; an output part 1707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and speakers, etc.; a storage part 1708 including a hard disk, etc. ; And a communication section 1709 including a network interface card such as a LAN card, a modem, and the like. The communication section 1709 performs communication processing via a network such as the Internet. The driver 1710 is also connected to the I/O interface 1705 as needed. A removable medium 1711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1710 as needed, so that the computer program read therefrom is installed into the storage portion 1708 as needed.

In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 1709, and/or installed from the removable medium 1711. When the computer program is executed by the central processing unit (CPU) 1701, various functions defined in the system of the present application are executed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable removable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device . The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

The flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more for realizing the specified logic function. Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by It is realized by a combination of dedicated hardware and computer instructions.

The units described in the embodiments of the present invention may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.

As another aspect, this application also provides a computer-readable medium. The computer-readable medium may be included in the electronic device described in the above-mentioned embodiment; or it may exist alone without being assembled into the electronic device. middle. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by an electronic device, the electronic device realizes the method in the following embodiments. For example, the electronic device of FIG. 1 can implement the various steps shown in FIG. 1 to FIG. 7.

In addition, the above-mentioned drawings are merely schematic illustrations of the processing included in the method according to the exemplary embodiment of the present invention, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

Those skilled in the art will easily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field that are not disclosed in the present disclosure. . The description and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

It should be understood that the present disclosure is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

A video coding method, characterized in that it comprises:

Determine N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to the preset selection rule; wherein, N is a positive integer;

Perform frame interpolation processing on the original bidirectional reference frame for N according to the preset frame interpolation rule to obtain T frame interpolation reference frames, and update the reference frame list according to the T frame interpolation reference frames to obtain the target reference Frame list; where T is a positive integer;

Encoding the bidirectionally predictive encoded frame according to the target reference frame list to obtain encoded data;

Obtain key parameters for generating the interpolated frame reference frame, append the key parameters to the coded data, and generate code stream data corresponding to the bidirectionally predictive coded frame.
The method according to claim 1, wherein the preset selection rule includes a pairing rule and a selection rule;

The determining N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to the preset selection rule includes:

Pairing the original forward reference frame and the original backward reference frame in the reference frame list corresponding to the bidirectional predictive coding frame according to the pairing rule to generate at least one pair of candidate original bidirectional reference frames;

According to the selection rule, N pairs of at least one pair of candidate original bidirectional reference frames are determined as the original bidirectional reference frames.
The method according to claim 1, wherein the preset frame interpolation rule comprises a time phase rule;

The performing frame insertion processing on N of the original bidirectional reference frames according to a preset frame insertion rule to obtain T frame insertion reference frames includes:

The time phases of the interpolated frame reference frames corresponding to the original bidirectional reference frames of N are respectively determined according to the time phase rule; wherein, the time phase of the interpolated frame reference frame is used to determine that the interpolated frame reference frame is in the original The position between two-way reference frames;

Frame interpolation is performed on the original bidirectional reference frames for N respectively to generate T frame interpolation reference frames in the time phase.
The method according to claim 3, wherein determining the time phase of the interpolated frame reference frame corresponding to the original bidirectional reference frame according to the time phase rule comprises:

Acquire the first frame number corresponding to the original forward reference frame, the first time point and the second frame number and second time point corresponding to the original backward reference frame in the original bidirectional reference frame, and the corresponding bidirectional predictive coding frame The third frame number;

A third time point is determined between the first time point and the second time point according to the first frame number, the second frame number, and the third frame number, and the third time point Determined as the time phase of the interpolated frame reference frame.
The method according to claim 4, wherein the first frame number, the second frame number, and the third frame number are set at the first time point and the second time point according to the first frame number, the second frame number, and the third frame number. Determine the third time point in the middle, including:

Calculating a first frame number difference corresponding to the first frame number and the second frame number, and calculating a time difference between the first time point and the second time point;

Calculating the quotient of the time difference and the first frame number difference as the unit duration;

Calculate the product of the difference between the second frame number corresponding to the first frame number and the third frame number and the unit duration as the first duration, and set it to be later than the first time point by the first duration Point is determined as the third time point; or

Calculate the product of the difference between the third frame number corresponding to the second frame number and the third frame number and the unit duration as the second duration, which will be earlier than the second time point by the second duration The point is determined as the third time point.
The method according to claim 1, wherein the updating the reference frame list according to the T interpolated frame reference frames to obtain a target reference frame list comprises:

Adding the T interpolated frame reference frames to the reference frame list to generate a target reference frame list.
The method according to claim 1, wherein the updating the reference frame list according to the T interpolated frame reference frames to obtain a target reference frame list comprises:

Replace the reference frame list with a list formed by arranging the T interpolated frame reference frames to generate a target reference frame list.
The method according to claim 1, wherein the key parameters include N pairs of frame numbers corresponding to the original forward reference frame and the original backward reference frame in the original two-way reference frame, and the interpolation frame The number of reference frames T.
A video decoding method, characterized in that it comprises:

Obtain code stream data corresponding to the bidirectional predictive coding frame to be decoded; the code stream data includes coded data and key parameters;

Parse the encoded data, and determine a target reference frame list corresponding to the encoded data according to the key parameters;

Perform decoding and reconstruction based on the target reference frame list, and obtain a decoded bidirectional prediction reconstruction frame.
The method according to claim 9, wherein the key parameters include N pairs of frame numbers and the number T; wherein, N and T are both positive integers;

The parsing the encoded data and determining the target reference frame list corresponding to the encoded data according to the key parameters includes:

Analyzing and reconstructing the encoded data to obtain a reference frame list corresponding to the bidirectional predictive encoded frame to be decoded;

Determining N pairs of original bidirectional reference frames in the reference frame list according to N pairs of the frame numbers;

Performing frame interpolation reconstruction on the original bidirectional reference frames, the number T, and the encoded data based on N, to obtain T frame interpolation reference frames;

The reference frame list is updated based on the T interpolated frame reference frames to obtain a target reference frame list.
The method according to claim 10, wherein the updating the reference frame list based on the T interpolated frame reference frames to obtain a target reference frame list comprises:

Adding the T interpolated frame reference frames to the reference frame list to generate a target reference frame list.
The method according to claim 10, wherein the updating the reference frame list based on the T interpolated frame reference frames to obtain a target reference frame list comprises:

Replace the reference frame list with a list formed by arranging the T interpolated frame reference frames to generate a target reference frame list.
A video encoding device, characterized in that it comprises:

The reference acquisition module is configured to determine N pairs of original bidirectional reference frames in the reference frame list corresponding to the bidirectional predictive coding frame according to a preset selection rule; wherein, N is a positive integer;

The first list determining module is configured to perform frame interpolation processing on N of the original bidirectional reference frames according to preset frame interpolation rules to obtain T frame interpolation reference frames, and compare the reference frames according to the T frame interpolation reference frames The frame list is updated to obtain the target reference frame list; where T is a positive integer;

A data encoding module, configured to encode the bidirectionally predictive encoded frame according to the target reference frame list to obtain encoded data;

The code stream generation module is used to obtain key parameters for generating the interpolated frame reference frame, append the key parameters to the coded data, and generate code stream data corresponding to the bidirectionally predictive coded frame.
The device according to claim 13, wherein the preset selection rule comprises a pairing rule and a selection rule;

The reference module is further configured to pair the original forward reference frame and the original backward reference frame in the reference frame list corresponding to the bidirectional predictive coding frame according to the pairing rule to generate at least one pair of candidate original bidirectional reference frames ; Determine N pairs as the original bidirectional reference frame in at least one pair of the candidate original bidirectional reference frames according to the selection rule.
The device according to claim 13, wherein the preset frame interpolation rule comprises a time phase rule;

The first list determining module is further configured to determine the time phases of the interpolated frame reference frames corresponding to the original bidirectional reference frames according to the time phase rule; wherein, the time phase of the interpolated frame reference frames is used to determine The position of the interpolated frame reference frame between the original bidirectional reference frames; interpolating the original bidirectional reference frames for N respectively to generate T interpolated frame reference frames in the time phase.
The apparatus according to claim 15, wherein the determining the time phase of the interpolated frame reference frame corresponding to the original bidirectional reference frame according to the time phase rule comprises:

Acquire the first frame number corresponding to the original forward reference frame, the first time point and the second frame number and second time point corresponding to the original backward reference frame in the original bidirectional reference frame, and the corresponding bidirectional predictive coding frame The third frame number;

A third time point is determined between the first time point and the second time point according to the first frame number, the second frame number, and the third frame number, and the third time point Determined as the time phase of the interpolated frame reference frame.
The apparatus according to claim 16, wherein the first frame number, the second frame number, and the third frame number are set at the first time point and the second time point according to the first frame number, the second frame number, and the third frame number. Determine the third time point in the middle, including:

Calculating a first frame number difference corresponding to the first frame number and the second frame number, and calculating a time difference between the first time point and the second time point;

Calculating the quotient of the time difference and the first frame number difference as the unit duration;

Calculate the product of the difference between the second frame number corresponding to the first frame number and the third frame number and the unit duration as the first duration, and set it to be later than the first time point by the first duration Point is determined as the third time point; or

Calculate the product of the difference between the third frame number corresponding to the second frame number and the third frame number and the unit duration as the second duration, which will be earlier than the second time point by the second duration The point is determined as the third time point.
A video decoding device, characterized in that it comprises:

A data acquisition module for acquiring code stream data corresponding to the bidirectional predictive coding frame to be decoded; the code stream data includes coded data and key parameters;

The second list determining module is configured to analyze the encoded data and determine the target reference frame list corresponding to the encoded data according to the key parameters;

The data reconstruction module is used to perform decoding and reconstruction based on the target reference frame list to obtain a decoded bidirectional prediction reconstruction frame.
The device according to claim 18, wherein the key parameter includes N pairs of frame numbers and a number T; wherein, N and T are both positive integers;

The second list determining module is further configured to analyze and reconstruct the encoded data to obtain a reference frame list corresponding to the bidirectional predictive encoded frame to be decoded; determine N pairs of original frames in the reference frame list according to the N pairs of the frame numbers. Two-way reference frame; based on N, the original two-way reference frame, the number T and the encoded data are interpolated and reconstructed to obtain T interpolated frame reference frames; the reference frame list is updated based on the T interpolated frame reference frames, Obtain a list of target reference frames.
An electronic device, characterized in that it comprises:

Processor; and

The memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors can implement any one of claims 1 to 8 The video encoding method described in clause 1 or the video decoding method described in any one of claims 9 to 12.