CN108989799B - Method and device for selecting reference frame of coding unit and electronic equipment - Google Patents
Method and device for selecting reference frame of coding unit and electronic equipment Download PDFInfo
- Publication number
- CN108989799B CN108989799B CN201710407439.XA CN201710407439A CN108989799B CN 108989799 B CN108989799 B CN 108989799B CN 201710407439 A CN201710407439 A CN 201710407439A CN 108989799 B CN108989799 B CN 108989799B
- Authority
- CN
- China
- Prior art keywords
- reference frame
- coding
- coding unit
- unit
- frame set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses a method and a device for selecting a reference frame of a coding unit and electronic equipment. The method for selecting the reference frame of the coding unit comprises the following steps: acquiring a coding unit; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit; and taking the selected preferred reference frame as the preferred reference frame of the coding unit. The technical scheme does not need to go through each reference frame in the original reference frame set, avoids the calculation of irrelevant reference frames, and reduces the calculation complexity of the selection process of multiple reference frames in the video coding process, thereby obviously improving the calculation efficiency of coding, transcoding and compressing under the condition of not reducing the quality basically and not increasing the code rate, and also enhancing the real-time applicability of the video coder.
Description
Technical Field
The present application relates to the field of video coding technologies, and in particular, to a method for selecting a reference frame of a coding unit. The application also relates to a device for selecting the reference frame of the coding unit, an electronic device and another method for selecting the reference frame of the coding unit.
Background
The HEVC coding standard can be used for improving the relation among code stream, coding quality, time delay and algorithm complexity in video compression and video coding, so that the video coding and the video compression achieve a relatively optimized effect.
Generally, HEVC coding uses flexible data modes, for example, a coding tree unit with a size of 64 × 64 and a depth of 4 may include at most 85 coding units with different depths, and these coding units respectively correspond to different numbers of prediction units with different partition modes at different depths. Wherein, each coding unit with the depth less than 3 has 7 kinds of division modes on the same depth and corresponds to 13 prediction units; each coding unit with depth equal to 3 has 3 partition modes and corresponds to 5 prediction units.
Because the multi-reference frame technology adopted by the HEVC coding standard can fully utilize the correlation between frames, the efficiency of inter-frame coding can be effectively improved. However, there is a problem in HEVC coding that each coding unit needs to be inter-predicted (and intra-predicted) in units of prediction units. If 13 prediction units (or 5 prediction units) in one coding unit are subjected to inter prediction such as motion search and motion compensation, the maximum number of coding tree units with the size of 64 × 64 and the depth divided into 4 levels needs to be subjected to calculation of inter prediction such as motion search and motion compensation up to 1105 times. This would result in a considerable amount of computation, resulting in a low efficiency of HEVC coding.
It can be seen that a drawback of the HEVC coding standard is that, when performing inter-frame prediction, since each coding unit traverses each reference frame from the original reference frame set to each prediction unit, as the number of reference frames increases, the amount of calculation for coding increases by a factor of two, and the coding efficiency becomes very low. Similar drawbacks exist for other similar video coding standards, such as H264 coding.
Disclosure of Invention
The application provides a method for selecting a reference frame of a coding unit, which aims to solve the problems of high complexity and low coding efficiency of the existing coding technical scheme. The application also relates to a device for selecting the reference frame of the coding unit, an electronic device and another method for selecting the reference frame of the coding unit.
The application provides a method for selecting a reference frame of a coding unit, which comprises the following steps:
acquiring a coding unit;
acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit;
and taking the selected preferred reference frame as the preferred reference frame of the coding unit.
Optionally, the obtaining, according to the coded depth of the coding unit, a reference frame set of a prediction unit corresponding to the coding unit includes:
if the coding depth of the coding unit is the lowest level, acquiring an original reference frame set;
and if the coding depth of the coding unit is not the lowest level, acquiring a reference frame set of a prediction unit corresponding to the coding unit.
Optionally, the reference frame set of the prediction unit includes:
and taking the number of levels lower than the coding depth of the coding unit as splitting strength to split the preferred reference frame of each sub-coding unit.
Optionally, the inter-frame partition mode of the reference frame set of the prediction unit includes: 2N × 2N.
Optionally, the performing coding optimization calculation on each reference frame in the reference frame set, and selecting a preferred reference frame includes:
performing rate distortion optimization calculation on one or more reference frames in the reference frame set to obtain a rate distortion cost corresponding to each reference frame;
and selecting rate distortion cost lower than a preset threshold value from each rate distortion cost obtained by calculation, and taking the reference frame corresponding to the selected rate distortion cost as a preferred reference frame of the reference frame set.
Optionally, performing rate distortion optimization calculation on one or more reference frames in the reference frame set; the method comprises the following steps:
and if the reference frames in the reference frame set have the same reference frame, performing the rate distortion optimization calculation on the same reference frame only once.
Optionally, after the step of using the selected preferred reference frame as the preferred reference frame of the coding unit, the method includes:
and taking the preferred reference frame of the coding unit as the preferred reference frame of the prediction unit of the other inter partition mode corresponding to the coding unit.
Optionally, the other inter-frame division modes at least include: n × 2N and 2N × N.
Optionally, the obtaining and encoding unit includes:
the coding units are obtained from the original coding sequence.
Correspondingly, an embodiment of the present application further provides a device for selecting a reference frame of a coding unit, where the device for selecting a reference frame of a coding unit includes:
an encoding unit acquisition unit for acquiring an encoding unit;
a reference frame set obtaining unit, configured to obtain a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
the preferred reference frame selecting unit is used for carrying out coding optimization calculation on each reference frame in the reference frame set and selecting a preferred reference frame;
and the preferred reference frame determining unit is used for taking the selected preferred reference frame as the preferred reference frame of the coding unit.
Optionally, the reference frame set obtaining unit includes:
a coded depth judgment subunit, configured to judge a level of a coded depth of the coding unit;
an original reference frame set obtaining subunit, configured to receive a determination result of the coding depth determination subunit, and if the coding depth of the coding unit is a lowest level, obtain an original reference frame set;
and the prediction unit reference frame set acquisition subunit is used for receiving the judgment result of the coding depth judgment subunit, and acquiring the reference frame set of the prediction unit corresponding to the coding unit if the coding depth of the coding unit is not the lowest level.
Optionally, the prediction unit reference frame set obtaining sub-unit is specifically configured to receive a determination result of the coding depth determination sub-unit, and if the coding depth of the coding unit is not the lowest level, obtain a preferred reference frame of each sub-coding unit, which is formed by splitting the preferred reference frame, by using the number of levels, which is lower than the coding depth of the coding unit, corresponding to the coding unit as a splitting strength.
Optionally, the preferred reference frame selecting unit includes:
the optimization calculation subunit is configured to perform rate distortion optimization calculation on one or more reference frames in the reference frame set to obtain a rate distortion cost corresponding to each reference frame;
and the preferred reference frame selecting subunit is used for selecting the rate distortion cost lower than a preset threshold value from each rate distortion cost obtained by calculation, and taking the reference frame corresponding to the selected rate distortion cost as the preferred reference frame of the reference frame set.
In addition, an embodiment of the present application further provides an electronic device, including:
a processor;
a memory for storing a reference frame selection program which, when read and executed by the processor, performs the operations of acquiring an encoding unit; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit; and taking the selected preferred reference frame as the preferred reference frame of the coding unit.
In addition, another method for selecting a reference frame of a coding unit is provided in an embodiment of the present application, where the method for selecting a reference frame of a coding unit includes:
acquiring a coding unit;
acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
selecting a preferred reference frame from the reference frame set;
and taking the selected preferred reference frame as the preferred reference frame of the coding unit.
Compared with the prior art, the scheme has the following advantages:
the method, the device and the electronic equipment for selecting the reference frame of the coding unit provided by the application are characterized in that the coding unit is obtained; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit; and taking the selected preferred reference frame as the preferred reference frame of the coding unit.
According to the scheme, the reference frame set of the prediction unit with the inter-frame partition mode is formed by directly using the correlation of the reference frames among the four sub-coding units with the next coding depth contained in the coding unit and the preferred reference frames of the four sub-coding units with the next coding depth contained in the coding unit, and the preferred reference frame of the prediction unit with the inter-frame partition mode is selected and used as the preferred reference frame of the coding unit to perform subsequent coding calculation.
When the method and the device execute the calculation of selecting the optimal reference frame, each reference frame in the original reference frame set does not need to be traversed, the calculation of irrelevant reference frames is avoided, the calculation complexity of a multi-reference frame selection process in the video coding process is reduced, the calculation efficiency of coding, transcoding and compressing is obviously improved under the condition of not reducing the quality basically and not increasing the code rate, and the real-time applicability of the video coder is also enhanced.
The technical scheme of the scheme is not only suitable for the HEVC coding standard, but also suitable for the H264 coding standard.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 illustrates a schematic diagram of a current frame and reference frame motion search and motion compensation process provided in accordance with the present application;
FIG. 2 illustrates another schematic diagram of the current and reference frame motion search and motion compensation processes provided in accordance with the present application;
FIG. 3 is a diagram illustrating a recursive partitioning process of a coding unit and specific modes of prediction units corresponding to different depths provided in accordance with the present application;
FIG. 4 is a flow chart illustrating a method of selecting a coding unit reference frame provided in accordance with an embodiment of the present application;
FIG. 5 is a diagram illustrating an apparatus for selecting a reference frame of a coding unit according to an embodiment of the present application;
FIG. 6 shows a schematic view of an electronic device provided in accordance with an embodiment of the present application;
fig. 7 is a flowchart illustrating another method for selecting a reference frame of a coding unit according to an embodiment of the present application.
Detailed Description
In order that the above objects, features and advantages of the present application can be more clearly understood, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit and scope of this application, and it is therefore not limited to the specific implementations disclosed below.
The embodiment of the application provides a method for selecting a reference frame of a coding unit. The application also relates to a device for selecting the reference frame of the coding unit, an electronic device and another method for selecting the reference frame of the coding unit. Details are described in the following examples one by one.
For better understanding of the technical solutions of the present application, the more detailed technical background related to the present application and the reasons for proposing the present application will be further described first.
Video coding generally achieves compression of a video sequence by removing temporal redundancy between pictures (frames). For example, to encode a current image, a temporal prediction of the image to be encoded is generated based on previously encoded images. Such prediction may be accomplished by using widely used block-based motion estimation and compression methods (e.g., MPEG standard, H264, HEVC, etc.), and the prediction result may be compared to the current actual image to determine a prediction error and encode the prediction error.
The method of motion estimation and compensation is generally applied to the case where a reference block is found in a reference frame set (one or more reference images) to predict the position of a corresponding target block in a target image, so that only the prediction residual of the target block needs to be encoded. In general, the prediction residual is specifically performed by obtaining a prediction error and a motion vector, and then performing block matching to identify a reference pixel block in a reference image that is most similar to a corresponding target block in the target image.
The pixel distance between the reference block and the corresponding target block is the motion vector of the target block. For example, the motion vector mv [ i ] [ j ] represents a motion vector of the i, j-th block in an image, and the motion vector includes displacement in the x horizontal direction and displacement in the y vertical direction, which are denoted as mv [ i ] [ j ]. x and mv [ i ] [ j ]. y; the motion vector is used for video compression along with a reference block to generate a prediction value for a target block.
Referring to fig. 1, a schematic diagram of a current frame and reference frame motion search and motion compensation process provided in accordance with the present application is shown.
Assume that the current frame is P and the reference frame is PrWhen the current coding block is B, searching a block B with the minimum residual difference between the current coding block and the block B in Prr,BrCalled the best matching block of B, this process is motion estimation. In FIG. 1, B*The coordinate position of B in the image is the same; the motion vector MV is the Br coordinate (x)r,yr) Minus B*Coordinates (x, y) equal to (x)r-x,yr-y)；BrThe block is the reference block of the B block, BrThe pixel value of the block can be used as the prediction value of the B block pixel.
Referring to fig. 2, another schematic diagram of the motion search and motion compensation process for the current frame and the reference frame provided in accordance with the present application is shown.
To identify a reference block in a reference image, a candidate block in the reference image needs to be compared to a target block. Each candidate block is the same size as the target block and is located within a specified search range in the reference image. The candidate block that most closely matches the target block is identified as the reference block. Searching for a reference block is an important part of motion estimation, and generally such searching requires a large amount of arithmetic computation.
The computational complexity of motion estimation for a target block is proportional to the size of the search range. If there are multiple reference frames in the reference frame set, the motion estimation will go through all the reference frames in the reference frame set, and perform analysis and comparison to find out a certain reference block on a certain reference frame. It can be seen that the amount of motion estimation computation is proportional to the search space of multiple reference frames (the size of the reference frame set, the number of reference frames in the reference frame set).
The data Unit partitioning mode in HEVC Coding is flexible, for example, a Coding Tree Unit (Coding Tree Unit) with a size of 64 × 64 and a depth of 4 may contain 85 Coding units (Coding units) at most. Each coding Unit needs to perform inter Prediction (and intra Prediction), i.e., perform temporal Prediction, in units of Prediction units (Prediction units). The purpose of performing temporal prediction is to eliminate temporal redundant information so that a previously coded picture can be used to predict a picture that is now to be coded.
The inter-frame division mode, or the inter-frame division method of the prediction unit, refers to: in block-based hybrid video encoders, each video frame is divided into fixed-size blocks. Coding tree units adopted in HEVC coding generally may include 85 coding units with different depths at most, and these coding units respectively correspond to different numbers of prediction units with different partition modes at different depths.
Referring to fig. 3, a schematic diagram of a coding unit recursive partitioning process and specific modes of prediction units corresponding to different depths is shown according to the present application.
Taking a coding tree unit with a size of 64 × 64 and a depth of 4 as an example, when the depth is less than 3, each of the 64 × 64, 32 × 32, or 16 × 16 coding units corresponds to prediction units of seven inter partition modes, i.e., 2N × 2N, 2N × N, N × 2N, 2N × nU, 2N × nD, nL × 2N, and nR × 2N modes, and includes 13 prediction units; the 2N × 2N mode has only one prediction unit, and the remaining six partition modes include two prediction units. When the coding depth is equal to 3, each 8 × 8 coding unit has three inter partition modes, namely 2 nx 2N, 2 nxn and nx 2N modes, and comprises 5 prediction units; similarly, 2N × 2N has only one prediction unit, and the remaining two partition modes each include two prediction units.
Each prediction unit of HEVC needs to perform inter-frame prediction, and find a reference frame corresponding to the prediction unit with the smallest difference from the coding unit, that is, a preferred reference frame of the prediction unit, as a basis for performing the next coding. A total of 13 prediction units (or 5 prediction units) in one coding unit of each coded depth need to perform inter prediction such as motion search and motion compensation, and therefore a coding tree unit with a size of 64 × 64 and a coded depth of 4 needs to perform inter prediction such as motion search and motion compensation 1105 times at most.
HEVC coding doubles the coding complexity as the number of reference frames increases if each reference frame is traversed from the original set of reference frames for each prediction unit when performing inter-frame prediction. If there is a method to determine the reference frame set of the prediction unit, and this reference frame set can adaptively contain only valid reference frames, skip or avoid traversing each original reference frame, and reduce the search space of the reference frame, the encoding complexity and the calculation amount can be greatly reduced. The same problem exists with other similar video coding standards, such as H264 coding.
The following describes embodiments of the present application in detail with reference to the drawings in turn.
The embodiment of the application provides a method for selecting a reference frame of a coding unit, which can reduce the search space of the reference frame and improve the coding efficiency under the condition of basically not influencing the video quality and the code rate. The embodiment of the method for selecting the reference frame of the coding unit comprises the following steps:
referring to fig. 4, a flowchart of a method for selecting a reference frame of a coding unit according to an embodiment of the present application is shown.
The method for selecting the reference frame of the coding unit comprises the following steps:
in step S101, a coding unit is acquired.
In this embodiment, the obtaining and encoding unit may be implemented as follows: from the original coding sequence as an input source, a coding unit is obtained.
In general, the original image may be segmented to form a coded sequence having a particular order, and the coded sequences may be combined in the particular order to form the complete original image. In this step, the coding unit is obtained from each frame of original image in the coding sequence.
It should be noted that the image of each frame is composed of a plurality of coding units.
Step S103, acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit.
In this embodiment, the obtaining of the reference frame set corresponding to the coding unit according to the coded depth of the coding unit may be implemented as follows: determining the level of the coding depth of the coding unit, and acquiring a reference frame set corresponding to the level of the coding depth of the coding unit according to the level of the coding depth of the coding unit.
It should be noted that the present application can be applied to coding units of any size and corresponding coded depth, for example, the size of a coding unit can be 128 × 128 or 256 × 256 or more, and the size of a coding unit can also be 4 × 4 or less; if the coding unit size is 128 × 128 max and 4 × 4 min, the coding depth corresponding to the coding unit is 6. For simplicity of description, the coding unit according to this embodiment uniformly sets a maximum size of 64 × 64 and a coding depth of 4, that is, in this embodiment, the coding unit includes four types, i.e., a 64 × 64 coding unit, a 32 × 32 coding unit, a 16 × 16 coding unit, and an 8 × 8 coding unit. Since a frame of image is composed of several coding units, in order to perform inter-frame prediction during encoding, calculation needs to be performed for each coding unit of the frame of image.
It can be understood that when the coded depths of the coding units are totally divided into 4 levels, the 8 × 8 coding unit is the coded depth of the deepest level (i.e., coded depth ═ 3); the 16 × 16 coding unit is a coded depth of a next deep level (i.e., depth ═ 2); the 32 × 32 coding unit is a coding depth of a level higher than the next deep level (i.e., depth ═ 1); the 64 × 64 coding unit is the coded depth of the highest level (i.e., depth ═ 0). If the coding depth of a coding unit is 3, the coding depth is the lowest level, i.e. this particular coding unit is an 8 × 8 coding unit.
In a specific implementation, when the reference frame set corresponding to the coding unit is obtained according to the coded depth of the coding unit, the following steps need to be performed:
if the coding depth of the coding unit is the lowest level, acquiring an original reference frame set;
and if the coding depth of the coding unit is not the lowest level, acquiring a reference frame set of a prediction unit corresponding to the coding unit.
It should be noted that, it is necessary to determine the coding depth of the coding unit according to the coding size of the coding unit, for example: if the coding size of the coding unit is 32X32, the coding depth of the coding unit is assumed to be 1.
And when the coding depth of the coding unit is judged, acquiring a corresponding reference frame set according to the coding depth of the coding unit.
The following describes a process of acquiring a corresponding reference frame set when the coding unit is at each coding depth.
When the coding depth of the coding unit is 3, it indicates that the coding unit does not include any coding unit of the next coding depth, and the original reference frame set is used as the reference frame set of the coding unit.
For example, in the present embodiment, assuming that the current coding unit is an 8 × 8 coding unit and that the inter partition mode is specifically set to be a 2N × 2N mode, since the 8 × 8 coding unit is already the smallest coding unit, i.e., the coding depth is the lowest level, the original reference frame set, i.e., the reference frame set { ref1, ref2, ref3, ref4, ref5} may be selected as the reference frame set of the 2N × 2N mode prediction unit corresponding to the 8 × 8 coding unit.
The interframe partition mode comprises 7 specific modes: the 64 × 64 coding unit, the 32 × 32 coding unit, and the 16 × 16 coding unit have 7 modes, respectively: 2 nx 2N, 2 nxn, nx2N, 2 nxnu, 2 nxnd, nlx 2N and nR x 2N; the 8 × 8 coding unit has 3 modes, which are: 2N × 2N, 2N × N, and N × 2N.
The inter-frame division mode of the prediction unit is set to 2N × 2N in this embodiment; of course, it is not excluded to set other specific inter division modes as the setting mode, for example, the 2N × N mode. The selection of the set inter-frame division mode may be determined by comprehensive consideration according to the characteristics of the coding unit and the effect of the coding.
It should be noted that, the reference frame, i.e. the previously coded picture; a set of reference frames, i.e. a set of a plurality of previously encoded pictures; the original reference frame set, which is composed of a plurality of original reference frames, i.e., an original reference frame search space, includes several reference frames used for inter-frame prediction in encoding. The number of these original reference frames is preset, either manually or automatically generated during the encoding process according to set rules.
For example, in the present embodiment, assuming no other limitation or constraint, the number of original reference frames may be set to 5, that is, the original reference frame set is { ref1, ref2, ref3, ref4, ref5}, and ref1 to ref5 represent a specific original reference frame of a frame respectively.
When the coded depth of the coding unit is 0, 1 or 2, that is, the coded depth of the coding unit is not the lowest level, the four preferred reference frames of the four sub-coding units of the next coded depth of the current coding unit are used to form a reference frame set corresponding to the prediction unit of the current coding unit.
The reference frame set of the prediction unit is a preferred reference frame of each sub-coding unit, which is split by using the number of layers lower than the coding depth of the coding unit as a splitting strength.
When the coding depth of the coding unit is higher than the lowest level, each coding unit consists of four sub-coding units of the next coding depth. The coding unit comprises four sub-coding units of the next coding depth, namely 4 sub-coding units divided by four equal parts; the coding unit of the previous coded depth is also called a parent coding unit, and after being combined, 4 sub-coding units of the next coded depth can form 1 complete parent coding unit. Specifically, in the present embodiment, the 64 × 64 coding unit includes 4 32 × 32 coding units of the next coded depth, the 32 × 32 coding unit includes 4 16 × 16 coding units of the next coded depth, and the 16 × 16 coding unit includes 4 8 × 8 coding units of the next coded depth.
When the coding depth of the coding unit is 0, 1 or 2, that is, for a selected coding unit of 64 × 64, 32 × 32 or 16 × 16, a reference frame set composed of preferred reference frames of four coding units of the next coding depth included in the coding unit is selected as a reference frame set of a prediction unit corresponding to the coding unit and setting an inter-frame partition mode.
For example, in this embodiment, if the selected coding unit is a 64 × 64 coding unit, a reference frame set composed of preferred reference frames of four 32 × 32 coding units of the next coded depth included in the selected 64 × 64 coding unit may be used as a reference frame set of a prediction unit corresponding to the selected 64 × 64 coding unit; in this reference frame set, i.e., there are 4 reference frames, which are respectively the preferred reference frames of 4 32 × 32 coding units, which are assumed to be reference frames ref1, ref1, ref2, and ref 4; assuming that the set mode is specifically 2N × 2N mode, the reference frame set of the 2N × 2N mode PU corresponding to the 64 × 64 CU is { ref1, ref1, ref2, ref4 }.
It can be understood that, if the coding depth of the coding unit is 0, a reference frame set of a prediction unit of a next coding depth 1 of the coding unit is obtained as the reference frame set of the coding unit, and if the prediction unit of the next coding depth of the coding unit does not have a reference frame set, a reference frame set of a prediction unit of a further next coding depth is obtained again until a reference frame set of a prediction unit of a coding depth of a corresponding level is obtained (when a coding unit of a deepest level is obtained, an original reference frame set is obtained).
Step S105, performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit.
In this embodiment, the performing coding optimization calculation on one or more reference frames in the reference frame set to select a preferred reference frame of a prediction unit corresponding to the coding unit may be implemented as follows: and performing rate distortion optimization calculation on one or more acquired reference frames in the reference frame set corresponding to the coding unit, and selecting a preferred reference frame of the prediction unit corresponding to the coding unit.
In specific implementation, when performing the encoding optimization calculation on one or more reference frames in the reference frame set and selecting a preferred reference frame of a prediction unit corresponding to the coding unit, the following steps need to be performed:
performing rate distortion optimization calculation on one or more reference frames in the reference frame set to obtain a rate distortion cost corresponding to each reference frame;
and selecting rate distortion cost lower than a preset threshold value from each rate distortion cost obtained by calculation, and taking the reference frame corresponding to the selected rate distortion cost as a preferred reference frame of the reference frame set.
In performing the above steps, it is not excluded to use methods other than rate-distortion optimization calculation to select the corresponding preferred reference frame. As long as the method can select the preferred reference frame according to a certain standard, the preferred reference frame can be calculated. The present embodiment only describes the rate-distortion optimization calculation in detail, and details of other calculation methods are not described.
Where the computational effort for rate distortion optimization is large. Assuming that the set mode is a 2N × 2N mode, when the reference frame set of the 2N × 2N mode prediction unit is the original reference frame set (the current coding unit is an 8 × 8 coding unit), the rate-distortion optimization calculation needs to go through virtually all the original reference frame sets; and when the reference frame set of the 2N × 2N mode prediction unit is the reference frame set composed of the preferred reference frames of the four coding units of the next coded depth (the current coding unit is a 64 × 64, 32 × 32 or 16 × 16 coding unit), the rate-distortion optimization calculation does not need to go through all the original reference frame sets, and is only performed on the maximum 4 reference frames (the preferred reference frames of the 4 next coded depth coding units).
For example, in the embodiment, the current coding unit is an 8 × 8 coding unit, and rate distortion optimization calculation is performed on each reference frame in the reference frame set { ref1, ref2, ref3, ref4, ref5} of the prediction unit for setting the inter partition mode, so that a reference frame with a rate distortion cost lower than a preset threshold can be selected as a preferred reference frame of the 8 × 8 coding unit (for example, ref2 is used as the preferred reference frame if the rate distortion cost of ref2 is lower than the preset threshold); for another example, the reference frame set composed of the preferred reference frames of the four 32 × 32 coding units with the next coding depth included in the selected 64 × 64 coding unit is { ref1, ref1, ref2, ref4}, and rate distortion optimization calculation is performed on each of the reference frames to select ref1 as the preferred reference frame of the 64 × 64 coding unit.
It should be noted that, in order to obtain a more accurate preferred reference frame, the preset threshold may be set to a minimum value, that is: in the step of selecting the preferred reference frame, the reference frame with the smallest value of the rate distortion cost is selected as the preferred reference frame of the reference frame set.
In addition, when performing the rate-distortion optimization calculation, if the same reference frame exists in the reference frame set of the prediction unit in which the inter-frame division mode is set, the rate-distortion optimization calculation may be performed only once for the same reference frame. The preferred reference frames for 4 coding units of the next coded depth are likely to have the same reference frame, and in this case, it may not be necessary to repeat the rate-distortion optimization calculation.
For example, in the present embodiment, when performing the rate-distortion optimization calculation on the reference frame set { ref1, ref1, ref2, ref4} composed of the preferred reference frames of the four 32 × 32 coding units of the next coding depth included in the selected 64 × 64 coding unit, only the calculation of ref1, ref2, and ref4 is required, and the calculation of ref1 need not be repeated twice.
Next, the encoding optimization calculation performed in step 105 on one or more reference frames in the reference frame set to select a preferred reference frame of the prediction unit corresponding to the coding unit will be described as an example.
If the inter-frame partition mode is specifically a 2N × 2N mode, and the coding depth of the coding unit is the deepest layer, performing rate distortion optimization calculation on one or more reference frames in an original reference frame set { ref1, ref2, ref3, ref4, ref5} corresponding to the coding unit 8 × 8 to obtain 5 specific rate distortion costs of ref1, ref2, ref3, ref4 and ref5 reference frames corresponding to a 2N × 2N mode prediction unit corresponding to the 8 × 8 coding unit, selecting a reference frame ref2 with a rate distortion cost lower than a preset threshold (or the lowest) from the 5 reference frames as a preferred reference frame ref _8x8_0 of the current coding unit, and determining preferred reference frames ref _8x8_1, ref _8x8_2 and ref _8x8_3 of other 3 prediction modes of the current coding unit by the same process; for another example, if the coding depth of the coding unit is the next deep layer 16 × 16, the preferred reference frame { ref _8x8_0, ref _8x8_1, ref _8x8_2, ref _8x8_3} of the coding unit (the deepest layer) of the next level coding depth 3 of the current coding unit is obtained as the reference frame set of the current coding unit, then rate distortion optimization calculation is performed on one or more reference frames in { ref _8x8_0, ref _8x8_1, ref _8x8_2, ref _8x8_3} to obtain 4 specific rate distortion costs of 16 × 16, and a reference frame _8x8_1 with a rate distortion cost lower than a preset threshold (or the lowest) is selected from the 4 reference frames to be the preferred reference frame ref _16x16_0 of the current coding unit.
It should be noted that, if the inter-frame partition mode is specifically a 2N × 2N mode, and the coded depth of the coding unit is the highest layer 64X64, a preferred reference frame of a coding unit of the next-level coded depth 1 of the current coding unit is obtained, if the coding unit of the coded depth 1 does not have the preferred reference frame, a preferred reference frame of a coding unit of the next-level coded depth 2 is obtained again downward, and based on the preferred reference frame of the coding unit of the coded depth 2, a preferred reference frame of the coding unit of the coded depth 1 is obtained first, and according to the calculated preferred reference frame of the coding unit of the coded depth 1, a preferred reference frame of the current coding unit of the coded depth 64X64 is obtained through calculation.
Since the coding unit has multiple inter-frame partition modes, in order to obtain preferred reference frames of other modes except the 2N × 2N mode of the coding unit, an embodiment of the present application provides a preferred implementation manner, and in the preferred implementation manner, after the step of using the selected preferred reference frame as the preferred reference frame of the coding unit, the preferred reference frame of the coding unit is used as the preferred reference frame of the prediction unit of the other inter-frame partition modes corresponding to the coding unit.
It should be noted that the other inter-frame division modes at least include: n × 2N and 2N × N.
Specifically, the 64 × 64 coding unit, the 32 × 32 coding unit, and the 16 × 16 coding unit have 7 modes, which are respectively: 2 nx 2N, 2 nxn, nx2N, 2 nxnu, 2 nxnd, nlx 2N and nR x 2N; the 8 × 8 coding unit has 3 modes, which are: 2N × 2N, 2N × N, and N × 2N.
As a preferred embodiment, the step may further calculate respective rate-distortion costs corresponding to prediction units of modes other than the coding unit setting mode, using a preferred reference frame of a prediction unit for setting an inter-frame partition mode corresponding to the selected coding unit. Therefore, according to the scheme, the rate distortion cost of each mode except the corresponding set mode can be calculated by performing rate distortion optimization calculation on the optimal reference frame of the prediction unit of the set inter-frame partition mode without the need of going through the whole original reference frame set for each mode prediction unit except the set mode.
For example, in the present embodiment, assuming that the setting mode is specifically a 2N × 2N mode, rate distortion optimization calculation may be performed on the modes other than 2N × 2N of the 8 × 8 coding unit, that is, the 2N × N and N × 2N mode prediction units, using the selected preferred reference frame ref2 of the 8 × 8 coding unit, to obtain 2 rate distortion costs corresponding to the 2N × N and N × 2N mode prediction units; for another example, the rate-distortion optimization calculation may be performed on the mode prediction units other than the 64 × 64 coding unit 2N × 2N, i.e., the 2N × N, N × 2N, 2N × nU, 2N × nD, nL × 2N, and nR × 2N mode prediction units, using the selected preferred reference frame ref1 of the 64 × 64 coding unit, to obtain 6 rate-distortion costs corresponding to the 6 mode prediction units.
Based on the above preferred embodiment, the smallest one of the rate-distortion costs of the prediction unit in the inter-partition mode and the rate-distortion costs of the prediction units in the modes other than the set mode is determined, and the specific mode of the prediction unit of the coding unit corresponding to the smallest rate-distortion cost is selected as the prediction unit mode corresponding to the coding unit. Through the optimal step, the specific mode of the prediction unit with the minimum rate distortion cost corresponding to the coding unit can be obtained, so that preparation is provided for subsequent coding calculation.
For example, in this embodiment, assuming that the setting mode is specifically a 2N × 2N mode, since the rate distortion cost calculated by the 2N × N mode prediction unit of the 8 × 8 coding unit is smaller than the rate distortion cost calculated by the 2N × 2N mode prediction unit, the mode of the prediction unit corresponding to the 8 × 8 coding unit is 2N × N; for another example, since the rate-distortion cost corresponding to nL × 2N is the smallest among the rate-distortion costs calculated by the 2 nx N, N × 2N, 2 nxnu, 2 nxnd, nL × 2N and nR × 2N mode prediction units of the 64 × 64 coding unit, the mode of the prediction unit corresponding to the 64 × 64 coding unit is nL × 2N.
Step S107, using the selected preferred reference frame as the preferred reference frame of the coding unit.
For example, in the present embodiment, assuming that the setting mode is specifically 2N × 2N, and the preferred reference frame of the 2N × 2N mode prediction unit corresponding to the selected 8 × 8 coding unit is ref2, ref2 is selected as the preferred reference frame of the selected 8 × 8 coding unit; for another example, if the preferred reference frame of the 2N × 2N mode prediction unit corresponding to the selected 64 × 64 coding unit is ref1, ref1 is selected as the preferred reference frame of the selected 64 × 64 coding unit.
In summary, the preferred reference frame of the prediction unit of the inter-frame partition mode is selected and set as the preferred reference frame of the coding unit, and when the preferred reference frames of the coding units of 64 × 64, 32 × 32 and 16 × 16 are selected and calculated, only the preferred reference frames of at most 4 next coding depth coding units are calculated without traversing the original reference frame set; in the rate distortion calculation of the prediction unit in the mode other than the set mode, only the preferred reference frame of the prediction unit in the inter-frame division mode is calculated without going through the original reference frame set. Therefore, the method of the scheme can greatly reduce the encoding complexity and the calculation amount.
In the above embodiments, a method for selecting a reference frame of a coding unit is provided, and the present application also provides a device for selecting a reference frame of a coding unit, corresponding to the method for selecting a reference frame of a coding unit. Since the embodiments of the apparatus are substantially similar to the embodiments of the method, they have been described in a relatively simple manner, and reference may be made to the description of the embodiments of the method for relevant points. The device embodiments described below are merely illustrative. The embodiment of the device for selecting the reference frame of the coding unit is as follows:
please refer to fig. 5, which illustrates a schematic diagram of an apparatus for selecting a reference frame of a coding unit according to an embodiment of the present application.
The device for selecting the reference frame of the coding unit comprises: an encoding unit acquisition unit 501, a reference frame set acquisition unit 503, a preferred reference frame selection unit 505, and a preferred reference frame determination unit 507;
the encoding unit obtaining unit 501 is configured to obtain an encoding unit;
the reference frame set obtaining unit 503 is configured to obtain a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
the preferred reference frame selecting unit 505 is configured to perform coding optimization calculation on each reference frame in the reference frame set, and select a preferred reference frame;
the preferred reference frame determining unit 507 is configured to use the selected preferred reference frame as the preferred reference frame of the encoding unit.
Optionally, the reference frame set obtaining unit 503 includes: the device comprises a coding depth judgment subunit, an original reference frame set acquisition subunit and a prediction unit reference frame set acquisition subunit;
the coding depth judgment subunit is configured to judge a level of a coding depth of the coding unit;
the original reference frame set obtaining subunit is configured to receive a determination result of the coding depth determination subunit, and if the coding depth of the coding unit is the lowest level, obtain an original reference frame set;
and the prediction unit reference frame set acquisition subunit is configured to receive a determination result of the coding depth determination subunit, and if the coding depth of the coding unit is not the lowest level, acquire a reference frame set of the prediction unit corresponding to the coding unit.
Optionally, the prediction unit reference frame set obtaining sub-unit is specifically configured to receive a determination result of the coding depth determination sub-unit, and if the coding depth of the coding unit is not the lowest level, obtain a preferred reference frame of each sub-coding unit, which is formed by splitting the preferred reference frame, by using the number of levels, which is lower than the coding depth of the coding unit, corresponding to the coding unit as a splitting strength.
Optionally, the obtaining, by the prediction unit, a reference frame set obtaining subunit, where the obtained inter-frame division mode of the reference frame set of the prediction unit includes: 2N × 2N.
Optionally, the preferred reference frame selecting unit 505 includes: an optimization calculation subunit and an optimal reference frame selection subunit;
the optimization calculation subunit is configured to perform rate distortion optimization calculation on one or more reference frames in the reference frame set to obtain a rate distortion cost corresponding to each reference frame;
and the preferred reference frame selecting subunit is configured to select a rate distortion cost lower than a preset threshold from each rate distortion cost obtained through calculation, and use the reference frame corresponding to the selected rate distortion cost as a preferred reference frame of the reference frame set.
Optionally, the optimization calculation subunit is further configured to perform the rate distortion optimization calculation only once on the same reference frame if the reference frame set has the same reference frame.
Optionally, the apparatus for selecting a reference frame of the coding unit further includes: a preferred reference frame synchronization unit;
and the preferred reference frame synchronization unit is configured to, after the selected preferred reference frame is used as the preferred reference frame of the coding unit, use the preferred reference frame of the coding unit as the preferred reference frame of the prediction unit of the other inter partition mode corresponding to the coding unit.
Optionally, the other inter-frame division modes at least include: n × 2N and 2N × N.
Optionally, the coding unit obtaining unit 501 is specifically configured to obtain the coding unit from an original coding sequence.
In the above embodiments, a method for selecting a reference frame of a coding unit and a device for selecting a reference frame of a coding unit are provided, and an electronic device is further provided; the electronic equipment comprises the following embodiments:
referring to fig. 6, a schematic diagram of an electronic device provided according to an embodiment of the present application is shown.
The electronic device includes: a processor 601; a memory 603;
the memory 603 is used for storing a reference frame selection program, and when the program is read and executed by the processor, the program executes the following operations of acquiring a coding unit; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit; and taking the selected preferred reference frame as the preferred reference frame of the coding unit.
For example, the electronic device is a computer, and the computer acquires the encoding unit; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit; and finally, taking the selected preferred reference frame as the preferred reference frame of the coding unit. Since the electronic device uses the method for selecting the reference frame of the coding unit, please refer to the above description of the embodiment of the method for selecting the reference frame of the coding unit for relevant parts, which is not described herein again.
In the above embodiments, a method for selecting a reference frame of a coding unit, a device for selecting a reference frame of a coding unit, and an electronic device are provided, and in addition, another method for selecting a reference frame of a coding unit is also provided; the embodiment of the method for selecting the reference frame of the coding unit comprises the following steps:
referring to fig. 7, a flowchart of another method for selecting a cu reference frame according to an embodiment of the present application is shown.
The method for selecting the reference frame of the coding unit comprises the following steps:
in step S701, an encoding unit is acquired.
In this embodiment, the obtaining and encoding unit may be implemented as follows: from the original coding sequence as an input source, a coding unit is obtained.
It should be noted that the image of each frame is composed of a plurality of coding units.
Step S703 is to obtain a reference frame set corresponding to the coding unit according to the coding depth of the coding unit.
In this embodiment, the obtaining of the reference frame set corresponding to the coding unit according to the coded depth of the coding unit may be implemented as follows: determining the level of the coding depth of the coding unit, and acquiring a reference frame set corresponding to the level of the coding depth of the coding unit according to the level of the coding depth of the coding unit.
It can be understood that when the coded depths of the coding units are totally divided into 4 levels, the 8 × 8 coding unit is the coded depth of the deepest level (i.e., coded depth ═ 3); the 16 × 16 coding unit is a coded depth of a next deep level (i.e., depth ═ 2); the 32 × 32 coding unit is a coding depth of a level higher than the next deep level (i.e., depth ═ 1); the 64 × 64 coding unit is the coded depth of the highest level (i.e., depth ═ 0). If the coding depth of a coding unit is 3, the coding depth is the lowest level, i.e. this particular coding unit is an 8 × 8 coding unit.
In a specific implementation, when the reference frame set corresponding to the coding unit is obtained according to the coded depth of the coding unit, the following steps need to be performed:
if the coding depth of the coding unit is the lowest level, acquiring an original reference frame set;
and if the coding depth of the coding unit is not the lowest level, acquiring a reference frame set of a prediction unit corresponding to the coding unit.
It should be noted that, it is necessary to determine the coding depth of the coding unit according to the coding size of the coding unit, for example: if the coding size of the coding unit is 32X32, the coding depth of the coding unit is assumed to be 1.
And when the coding depth of the coding unit is judged, acquiring a corresponding reference frame set according to the coding depth of the coding unit.
The following describes a process of acquiring a corresponding reference frame set when the coding unit is at each coding depth.
When the coding depth of the coding unit is 3, it indicates that the coding unit does not include any coding unit of the next coding depth, and the original reference frame set is used as the reference frame set of the coding unit.
For example, in the present embodiment, assuming that the current coding unit is an 8 × 8 coding unit and that the inter partition mode is specifically set to be a 2N × 2N mode, since the 8 × 8 coding unit is already the smallest coding unit, i.e., the coding depth is the lowest level, the original reference frame set, i.e., the reference frame set { ref1, ref2, ref3, ref4, ref5} may be selected as the reference frame set of the 2N × 2N mode prediction unit corresponding to the 8 × 8 coding unit.
For example, in the present embodiment, assuming no other limitation or constraint, the number of original reference frames may be set to 5, that is, the original reference frame set is { ref1, ref2, ref3, ref4, ref5}, and ref1 to ref5 represent a specific original reference frame of a frame respectively.
When the coded depth of the coding unit is 0, 1 or 2, that is, the coded depth of the coding unit is not the lowest level, the four preferred reference frames of the four sub-coding units of the next coded depth of the current coding unit are used to form a reference frame set corresponding to the prediction unit of the current coding unit.
The reference frame set of the prediction unit is a preferred reference frame of each sub-coding unit, which is split by using the number of layers lower than the coding depth of the coding unit as a splitting strength.
When the coding depth of the coding unit is higher than the lowest level, each coding unit consists of four sub-coding units of the next coding depth. The coding unit comprises four sub-coding units of the next coding depth, namely 4 sub-coding units divided by four equal parts; the coding unit of the previous coded depth is also called a parent coding unit, and after being combined, 4 sub-coding units of the next coded depth can form 1 complete parent coding unit. Specifically, in the present embodiment, the 64 × 64 coding unit includes 4 32 × 32 coding units of the next coded depth, the 32 × 32 coding unit includes 4 16 × 16 coding units of the next coded depth, and the 16 × 16 coding unit includes 4 8 × 8 coding units of the next coded depth.
When the coding depth of the coding unit is 0, 1 or 2, that is, for a selected coding unit of 64 × 64, 32 × 32 or 16 × 16, a reference frame set composed of preferred reference frames of four coding units of the next coding depth included in the coding unit is selected as a reference frame set of a prediction unit corresponding to the coding unit and setting an inter-frame partition mode.
For example, in this embodiment, if the selected coding unit is a 64 × 64 coding unit, a reference frame set composed of preferred reference frames of four 32 × 32 coding units of the next coded depth included in the selected 64 × 64 coding unit may be used as a reference frame set of a prediction unit corresponding to the selected 64 × 64 coding unit; in this reference frame set, i.e., there are 4 reference frames, which are respectively the preferred reference frames of 4 32 × 32 coding units, which are assumed to be reference frames ref1, ref1, ref2, and ref 4; assuming that the set mode is specifically 2N × 2N mode, the reference frame set of the 2N × 2N mode PU corresponding to the 64 × 64 CU is { ref1, ref1, ref2, ref4 }.
Step S705, selecting a preferred reference frame from the reference frame set.
In this embodiment, the selecting a preferred reference frame from the reference frame set may be implemented as follows: and selecting one or more reference frames from the acquired reference frame set as the preferred reference frames corresponding to the coding unit.
It should be noted that, when one or more reference frames are selected from the acquired reference frame set, a preferred reference frame may be selected according to a certain criterion. For example: calculating the reference frames in the reference frame set by a rate distortion optimization method.
Step S707, using the selected preferred reference frame as a preferred reference frame of the coding unit.
For example, in the present embodiment, assuming that the setting mode is specifically 2N × 2N, and the preferred reference frame of the 2N × 2N mode prediction unit corresponding to the selected 8 × 8 coding unit is ref2, ref2 is selected as the preferred reference frame of the selected 8 × 8 coding unit; for another example, if the preferred reference frame of the 2N × 2N mode prediction unit corresponding to the selected 64 × 64 coding unit is ref1, ref1 is selected as the preferred reference frame of the selected 64 × 64 coding unit.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transmyedia), such as modulated data signals and carrier waves.
2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.
Claims (12)
1. A method for selecting a reference frame of a coding unit, comprising:
acquiring a coding unit;
acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit, including: performing rate distortion optimization calculation on one or more reference frames in the reference frame set, and selecting the reference frame with the minimum rate distortion cost value as the optimal reference frame of the reference frame set;
taking the selected preferred reference frame as a preferred reference frame of the coding unit;
wherein the reference frame set of the prediction unit comprises: and taking the number of levels lower than the coding depth of the coding unit as splitting strength to split the preferred reference frame of each sub-coding unit.
2. The method of claim 1, wherein the obtaining a reference frame set of a prediction unit corresponding to the coding unit according to the coded depth of the coding unit comprises:
if the coding depth of the coding unit is the lowest level, acquiring an original reference frame set;
and if the coding depth of the coding unit is not the lowest level, acquiring a reference frame set of a prediction unit corresponding to the coding unit.
3. The method of claim 1, wherein the inter-frame partition mode of the reference frame set of the prediction unit comprises: 2N × 2N.
4. The method of claim 1, wherein the performing a rate-distortion optimization calculation on one or more reference frames in the reference frame set; the method comprises the following steps:
and if the reference frames in the reference frame set have the same reference frame, performing the rate distortion optimization calculation on the same reference frame only once.
5. The method according to any of claims 1-4, wherein the step of selecting the selected preferred reference frame as the preferred reference frame of the coding unit is followed by:
and taking the preferred reference frame of the coding unit as the preferred reference frame of the prediction unit of the other inter partition mode corresponding to the coding unit.
6. The method of claim 5, wherein the other inter-frame partition modes at least comprise: n × 2N and 2N × N.
7. The method of claim 5, wherein the obtaining the coding unit comprises:
the coding units are obtained from the original coding sequence.
8. An apparatus for selecting a reference frame of a coding unit, comprising:
an encoding unit acquisition unit for acquiring an encoding unit;
a reference frame set obtaining unit, configured to obtain a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
the optimal reference frame selecting unit is used for performing coding optimization calculation on each reference frame in the reference frame set and selecting an optimal reference frame of the prediction unit corresponding to the coding unit; wherein, the preferred reference frame selecting unit is specifically configured to: performing rate distortion optimization calculation on one or more reference frames in the reference frame set, and selecting the reference frame with the minimum rate distortion cost value as the optimal reference frame of the reference frame set;
a preferred reference frame determining unit, configured to use the selected preferred reference frame as a preferred reference frame of the encoding unit;
wherein the reference frame set of the prediction unit comprises: and taking the number of levels lower than the coding depth of the coding unit as splitting strength to split the preferred reference frame of each sub-coding unit.
9. The apparatus for selecting the reference frame of the coding unit according to claim 8, wherein the reference frame set obtaining unit comprises:
a coded depth judgment subunit, configured to judge a level of a coded depth of the coding unit;
an original reference frame set obtaining subunit, configured to receive a determination result of the coding depth determination subunit, and if the coding depth of the coding unit is a lowest level, obtain an original reference frame set;
and the prediction unit reference frame set acquisition subunit is used for receiving the judgment result of the coding depth judgment subunit, and acquiring the reference frame set of the prediction unit corresponding to the coding unit if the coding depth of the coding unit is not the lowest level.
10. The apparatus according to claim 9, wherein the prediction unit reference frame set obtaining sub-unit is specifically configured to receive a determination result of the coding depth determination sub-unit, and if the coding depth of the coding unit is not the lowest level, obtain a preferred reference frame of each sub-coding unit corresponding to the coding unit and split by using a number of levels lower than the coding depth of the coding unit as a splitting strength.
11. A computing device, wherein the computing device comprises:
a processor;
a memory for storing a reference frame selection program which, when read and executed by the processor, performs the operations of acquiring an encoding unit; acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit; performing coding optimization calculation on one or more reference frames in the reference frame set, and selecting a preferred reference frame of a prediction unit corresponding to the coding unit, including: performing rate distortion optimization calculation on one or more reference frames in the reference frame set, and selecting the reference frame with the minimum rate distortion cost value as the optimal reference frame of the reference frame set; taking the selected preferred reference frame as a preferred reference frame of the coding unit; wherein the reference frame set of the prediction unit comprises: and taking the number of levels lower than the coding depth of the coding unit as splitting strength to split the preferred reference frame of each sub-coding unit.
12. A method for selecting a reference frame of a coding unit, comprising:
acquiring a coding unit;
acquiring a reference frame set corresponding to the coding unit according to the coding depth of the coding unit;
selecting a preferred reference frame of a prediction unit corresponding to the coding unit from the reference frame set, including: performing rate distortion optimization calculation on one or more reference frames in the reference frame set, and selecting the reference frame with the minimum rate distortion cost value as the optimal reference frame of the reference frame set;
taking the selected preferred reference frame as a preferred reference frame of the coding unit;
wherein the reference frame set of the prediction unit comprises: and taking the number of levels lower than the coding depth of the coding unit as splitting strength to split the preferred reference frame of each sub-coding unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710407439.XA CN108989799B (en) | 2017-06-02 | 2017-06-02 | Method and device for selecting reference frame of coding unit and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710407439.XA CN108989799B (en) | 2017-06-02 | 2017-06-02 | Method and device for selecting reference frame of coding unit and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108989799A CN108989799A (en) | 2018-12-11 |
| CN108989799B true CN108989799B (en) | 2022-03-08 |
Family
ID=64501717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710407439.XA Active CN108989799B (en) | 2017-06-02 | 2017-06-02 | Method and device for selecting reference frame of coding unit and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108989799B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111263151B (en) * | 2020-04-26 | 2020-08-25 | 腾讯科技（深圳）有限公司 | Video encoding method, video encoding device, electronic device, and computer-readable storage medium |
| CN112633122B (en) * | 2020-12-17 | 2024-01-23 | 厦门大学 | Front-end mileage calculation method and system of monocular VIO system |
| CN112770179B (en) * | 2021-01-05 | 2023-06-16 | 武汉球之道科技有限公司 | Efficient transmission system for on-line event rebroadcasting |
| CN114286089A (en) * | 2021-09-26 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Reference frame selection method, device, equipment and medium |
| CN114466199A (en) * | 2022-04-12 | 2022-05-10 | 宁波康达凯能医疗科技有限公司 | Reference frame generation method and system applicable to VVC (variable valve timing) coding standard |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581685A (en) * | 2013-10-09 | 2014-02-12 | 合一网络技术(北京)有限公司 | Method and device for selecting H.264 reference frames |
| CN103813166A (en) * | 2014-01-28 | 2014-05-21 | 浙江大学 | Low-complexity method for selecting HEVC coding multiple reference frames |
| CN104618715A (en) * | 2014-07-22 | 2015-05-13 | 腾讯科技（北京）有限公司 | Method and device for obtaining minimal rate-distortion cost |
| CN106034236A (en) * | 2015-03-19 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method, Apparatus and coder for selecting optimal reference frame in HEVC |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI720750B (en) * | 2011-12-28 | 2021-03-01 | 日商Ｊｖｃ建伍股份有限公司 | Dynamic image encoding device and dynamic image encoding method |
| JP5664681B2 (en) * | 2013-03-08 | 2015-02-04 | 株式会社Ｊｖｃケンウッド | Image decoding apparatus, image decoding method, image decoding program, receiving apparatus, receiving method, and receiving program |
| JP6430542B2 (en) * | 2014-06-16 | 2018-11-28 | クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ | Simplified Shifting Merge Candidate and Merge List Derivation in 3D-HEVC |
-
2017
- 2017-06-02 CN CN201710407439.XA patent/CN108989799B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581685A (en) * | 2013-10-09 | 2014-02-12 | 合一网络技术(北京)有限公司 | Method and device for selecting H.264 reference frames |
| CN103813166A (en) * | 2014-01-28 | 2014-05-21 | 浙江大学 | Low-complexity method for selecting HEVC coding multiple reference frames |
| CN104618715A (en) * | 2014-07-22 | 2015-05-13 | 腾讯科技（北京）有限公司 | Method and device for obtaining minimal rate-distortion cost |
| CN106034236A (en) * | 2015-03-19 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method, Apparatus and coder for selecting optimal reference frame in HEVC |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108989799A (en) | 2018-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108989799B (en) | Method and device for selecting reference frame of coding unit and electronic equipment | |
| JP7351463B2 (en) | Image encoding method and device, image decoding method and device, and program | |
| RU2678510C2 (en) | Method and device for coding and decoding motion vector based on reduced predictors-candidates of motion vector | |
| CN110249628B (en) | Video encoder and decoder for predictive partitioning | |
| CN111164974B (en) | Image encoding/decoding method and apparatus, and non-transitory storage medium | |
| US10057586B2 (en) | Method and device for providing depth based block partitioning in high efficiency video coding | |
| JP2020039167A (en) | Calculation apparatus and method | |
| EP3560197A1 (en) | Low-complexity sign prediction for video coding | |
| KR102435595B1 (en) | Method and apparatus to provide comprssion and transmission of learning parameter in distributed processing environment | |
| KR102070431B1 (en) | Method and apparatus for encoding video with restricting bi-directional prediction and block merging, method and apparatus for decoding video | |
| KR20090058954A (en) | Video coding method and apparatus using side matching, and video decoding method and appartus thereof | |
| WO2014190468A1 (en) | Video encoder for images | |
| TW202021354A (en) | Motion vector predictor list generation | |
| KR20230007313A (en) | Parallelized Rate-Distortion Optimized Quantization Using Deep Learning | |
| CN112055203B (en) | Inter-frame prediction method, video coding method and related devices | |
| US20200099928A1 (en) | Encoding method and device therefor, and decoding method and device therefor | |
| CN111263144B (en) | Motion information determination method and equipment | |
| KR20190090867A (en) | Intra prediction mode based image processing method and apparatus therefor | |
| CN114900691B (en) | Encoding method, encoder, and computer-readable storage medium | |
| US20080112631A1 (en) | Method of obtaining a motion vector in block-based motion estimation | |
| CN108924551B (en) | Method for predicting video image coding mode and related equipment | |
| AU2016228184B2 (en) | Method for inducing a merge candidate block and device using same | |
| CN103959788B (en) | Estimation of motion at the level of the decoder by matching of models | |
| CN113747172A (en) | Inter-frame prediction method, encoder, decoder, and computer storage medium | |
| KR101337345B1 (en) | Encoder and method for skipping sub-pel motion estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |