WO2019192309A1 - 确定编码单元的编码代价的方法、装置及计算机可读存储介质 - Google Patents

确定编码单元的编码代价的方法、装置及计算机可读存储介质 Download PDF

Info

Publication number
WO2019192309A1
WO2019192309A1 PCT/CN2019/078300 CN2019078300W WO2019192309A1 WO 2019192309 A1 WO2019192309 A1 WO 2019192309A1 CN 2019078300 W CN2019078300 W CN 2019078300W WO 2019192309 A1 WO2019192309 A1 WO 2019192309A1
Authority
WO
WIPO (PCT)
Prior art keywords
coding
coding unit
unit
determining
cost
Prior art date
Application number
PCT/CN2019/078300
Other languages
English (en)
French (fr)
Inventor
毛煦楠
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to JP2020547348A priority Critical patent/JP7080542B2/ja
Priority to EP19782198.6A priority patent/EP3780611A4/en
Publication of WO2019192309A1 publication Critical patent/WO2019192309A1/zh
Priority to US16/996,358 priority patent/US11070803B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/1883—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit relating to sub-band structure, e.g. hierarchical level, directional tree, e.g. low-high [LH], high-low [HL], high-high [HH]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Definitions

  • the present application relates to the field of computer technology, and in particular, to a method, an apparatus, and a computer readable storage medium for determining an encoding cost of a coding unit.
  • High Efficiency Video Coding is jointly developed by the International Telecommunications Union-Telecommunication Standardization Sector (ITU-T) and the International Standard Organized (ISO) under the management of the International Telecommunication Union.
  • ITU-T International Telecommunications Union-Telecommunication Standardization Sector
  • ISO International Standard Organized
  • a new generation of efficient video coding standards can also be called H.265.
  • HEVC uses more, newer and more flexible coding techniques. Under the same video quality conditions, HEVC can further reduce the code rate of coding.
  • the HEVC standard continues the block-based video compression coding framework in the H.264/AVC standard.
  • the HEVC standard uses a more flexible block structure than the macroblock (Macblock) used in the H.264/AVC standard.
  • Macblock macroblock
  • the size of the MB is fixed, and its fixed size is 16*16 pixels.
  • the coding unit (CU) can have four sizes, which are 64*64 respectively. Pixels, 32*32 pixels, 16*16 pixels, and 8*8 pixels.
  • the coding unit algorithm of the tree structure is used to determine the specific size of the coding unit, thereby completing the division of one frame of image, which is specifically: first, dividing one frame of image into multiple coding tree units (Coding Tree) Unit, CTU), and then traversing the CU in a traversal recursive manner for each CTU, determining the coding cost of each CU, and determining the coding strategy that minimizes the coding cost corresponding to the CTU according to the coding cost of each CU.
  • the division CTU is determined according to the division strategy, thereby realizing division of the CU.
  • each CU in the tree structure needs to perform prediction coding, coding cost calculation and the like to determine the coding cost of the CU and determine whether the CU needs to be divided, the algorithm needs to perform a large number of calculations, and requires more time and coding rate. Lower.
  • the main purpose of the present application is to provide a method and a related device for determining the coding cost of a coding unit.
  • a first aspect of the embodiments of the present application provides a method for determining an encoding cost of a coding unit, which is executed by a server, and may include:
  • the pixel gradient information includes a pixel gradient variance, where the pixel gradient variance is a variance of an image gradient of at least a part of pixels in the coding unit;
  • determining a first coding cost as a second coding cost of the coding unit the second coding cost being used to determine a corresponding coding unit a coding strategy of a coding tree unit, the first coding cost being an encoding cost obtained by predictive coding of the coding unit by using a current size of the coding unit as a size of a prediction unit.
  • the second aspect of the embodiments of the present application provides an apparatus for determining an encoding cost of a coding unit, which may include:
  • a coding unit determining module configured to determine a coding unit that performs predictive coding in an intra prediction mode
  • a gradient information determining module configured to determine pixel gradient information corresponding to the coding unit, where the pixel gradient information includes a pixel gradient variance, where the pixel gradient variance is a variance of an image gradient of at least a part of pixels in the coding unit;
  • a pre-decision module configured to perform a pre-decision on the coding unit according to the pixel gradient information
  • a cost determining module configured to determine, according to a result of the pre-decision of the pre-decision module to the coding unit, a first coding cost as a second coding cost of the coding unit, the second coding
  • the cost is used to determine a coding strategy of a coding tree unit corresponding to the coding unit, where the first coding cost is an encoding cost obtained by predictive coding the coding unit by using a current size of the coding unit as a size of a prediction unit.
  • a third aspect of the embodiments of the present application provides a computer readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the determining coding unit provided by the first aspect of the present application is implemented. The individual steps in the method of encoding the cost.
  • a fourth aspect of the embodiments of the present application provides a terminal device, including: a memory, a processor, and a computer program stored on the memory and running on the processor, where the processor executes the In the case of a computer program, various steps in a method of determining the encoding cost of a coding unit as provided by the first aspect of the embodiments of the present application are implemented.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it can be determined that the coding unit does not need to be divided.
  • the division of the coding unit is completed in advance, and the coding process is divided into sub-coding units, and then the sub-coding unit is used as a new coding unit for predictive coding and the calculation process of the coding cost calculation is calculated, which reduces the calculation amount and improves the calculation. Coding rate.
  • FIG. 1 is a schematic diagram showing an example of dividing a frame image into a plurality of CTUs according to an embodiment of the present application
  • 2A-2B are schematic diagrams showing an example of a representation of a CTU partitioning strategy provided by an embodiment of the present application.
  • 3A-3E are flow diagrams of determining a CTU partitioning strategy by a complete traversal recursive algorithm
  • FIG. 4 is a schematic diagram showing an example of determining a CTU division manner according to a final coding cost of a CTU according to an embodiment of the present application
  • FIG. 5 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application
  • FIG. 6 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application
  • FIGS. 7A-7E are schematic diagrams showing an example of a process for determining a CTU division policy according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application
  • FIG. 9 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present disclosure
  • FIG. 11 is a schematic diagram of an example of a neighbor coding unit according to an embodiment of the present application.
  • FIG. 12 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application
  • FIG. 13 is a schematic structural diagram of an apparatus for determining an encoding cost of a coding unit according to an embodiment of the present disclosure
  • FIG. 14 is a schematic structural diagram of a pre-decision module according to an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of an apparatus for determining an encoding cost of a coding unit according to an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • the CTU is a basic unit for dividing one frame of image. Before encoding a frame of image, it is necessary to divide one frame of image. In the process of dividing the image, the image is first divided into several CTUs. In general, the CTU size is 64*64 pixels. Alternatively, the CTU size can be 32*32 pixels, and further can be 16*16 pixels or 8*8 pixels. For example, the pixels of a frame of I frame image are 640*480 pixels. As shown in FIG. 1 , after dividing the frame image into 70 CTUs of 64*64 pixels, the remaining images are not enough to be further divided into 64*64 pixels. The CTU divides the remaining image into 20 32*32 pixel CTUs.
  • the CU is a coding unit in the HEVC standard, similar to the concept of MB in the H.264/AVC standard.
  • each CU in this frame image is encoded one by one to form a continuous video code stream.
  • the CU is divided by the CTU, and the size of the CU is less than or equal to the size of the CTU.
  • the division of each CTU in the image is determined, the division of the entire frame image can be determined.
  • the CTU is divided according to the partitioning strategy corresponding to the CTU to determine the division of the CU in the CTU.
  • the partitioning strategy corresponding to the CTU can be represented by a tree structure, and the division of the CU of the CTU can be determined according to the tree structure.
  • the CTU partitioning strategy is represented by a tree structure as shown in FIG. 2A, wherein the root node in the tree structure represents a CTU, and the leaf nodes in the tree structure represent CUs in a CTU. If there are sub-nodes under the node in the tree structure, it indicates that the CU corresponding to the node needs to continue to be divided.
  • the CTU partitioning strategy can be determined by a complete traversal recursive coding algorithm.
  • the schematic diagram of the algorithm can be as shown in FIG. 3A-3E, taking the CTU size as 64*64 pixels as an example. .
  • the traversal process can be as shown in Figures 3A and 3B.
  • the CTU is divided into four 32*32 pixel CUs as sub-CUs of the CTU.
  • Each 32*32 pixel CU is further divided into four 16*16 pixel CUs as sub-CUs of 32*32 pixel CUs.
  • Each 16*16 CU is further divided into four 8*8 CUs as sub-CUs of 16*16-pixel CUs, and the four-layer CU forms a full quadtree (except that the last layer has no sub-nodes, There are four child nodes under all nodes on each layer).
  • the CTU is predictively encoded with 64*64 pixels as the size of the prediction unit (PU) to determine the optimal prediction mode, and the CTU is determined in the optimal prediction mode.
  • Coding cost (C1) which is the minimum coding cost of the CTU in the case where the PU partitioning mode is 2N*2N.
  • Predictive coding of each CU of the second layer by using 32*32 pixels as the size of the prediction unit respectively determines the optimal prediction mode of each CU of the second layer, and determines the coding cost of each CU of the second layer in the optimal prediction mode.
  • the coding cost is the minimum coding cost for each CU of the second layer in the case where the PU division mode is 2N*2N.
  • Predictive coding is performed on each CU of the third layer with 16*16 pixels as the size of the prediction unit, respectively, to determine an optimal prediction mode of each CU of the third layer, and to determine the coding cost of each CU of the third layer in the optimal prediction mode.
  • the coding cost is the minimum coding of each CU of the third layer in the case where the PU division mode is 2N*2N cost.
  • Predictive coding is performed on each CU of the fourth layer with 8*8 pixels as the size of the prediction unit, respectively, and an optimal prediction mode of each CU of the fourth layer is determined, and the coding cost of each CU of the fourth layer in the optimal prediction mode is determined.
  • the coding cost is the case where each CU of the fourth layer is in the case where the PU division mode is 2N*2N Minimum coding cost.
  • the optimal prediction mode is a prediction mode in which a coding cost obtained by predictive coding of a CU is minimized among a plurality of prediction modes corresponding to a prediction unit of a certain size, for example, a prediction unit having a size of 64*64 corresponds to four prediction modes.
  • the mode predictively encodes the CUs by using the four prediction modes, and the obtained coding costs are c1, c2, c3, and c4, respectively, and c1 ⁇ c2 ⁇ c3 ⁇ c4, and the prediction mode corresponding to c1 is the optimal prediction. mode.
  • the coding cost can also be called the rate distortion cost, which is used to indicate the difference between the predicted value and the reference value predicted by the CU when predictive coding is performed in a certain prediction mode, wherein the smaller the coding cost, the predicted value and the reference The smaller the difference between the values, the less the number of bits required for encoding and the lower the code rate.
  • the Sum of Absolute Difference (SAD) algorithm, the Sum of Absolute Transformed Difference (SATD) algorithm, and the Sum of Squared Difference (SSD) algorithm can be used.
  • the coding cost of predictive coding of a CU in a certain prediction mode is determined.
  • the traversal process is followed by a recursive process as shown in Figures 3C, 3D and 3E.
  • the minimum coding costs (such as c1, c2, c3, and c4) of the four sub-CUs under the respective CUs of the third layer are respectively summed to obtain another coding cost of each CU of the third layer.
  • the coding cost is the minimum coding cost for each CU of the third layer in the case where the PU division mode is N*N.
  • the minimum coding costs (such as c31, c32, c33, c34) of the four sub-CUs under the respective CUs of the second layer are respectively summed to obtain another coding cost of each CU of the second layer.
  • the coding cost is the minimum coding cost for each CU of the second layer in the case where the PU division mode is N*N.
  • the smaller one of the two minimum coding costs is selected as the minimum coding cost of each CU of the second layer, and the minimum coding cost is the final coding cost of each CU of the third layer (c21, C22, c23, c24).
  • the minimum coding cost (c21, c22, c23, c24) of the four CUs under the CTU of the first layer is summed to obtain another coding cost of the CTU, and the coding cost is CTU in the PU.
  • the minimum coding cost in the case where the division is N*N.
  • it is determined that the CTU needs to be divided and it is determined that the CTU is divided into four CUs, as shown in the figure. B in 4 is shown.
  • determining whether each CU of the second layer needs to be divided according to the minimum coding cost of each CU of the second layer and if the minimum coding cost is the minimum coding cost in the case where the PU division mode is 2N*2N, no division is needed.
  • the minimum coding cost is the minimum coding cost in the case where the PU division mode is N*N
  • division is required.
  • it is determined whether each CU of the third layer needs to be divided according to the minimum coding cost of each CU of the third layer.
  • the division of each CU can be represented as a tree structure as shown in FIG. 2A.
  • the process involved in the complete traversal recursive algorithm is a process of finding a partitioning strategy that minimizes the coding cost of the CTU.
  • the final coding cost of each coding unit determined by the recursive process is The CTU is divided according to the dividing policy, and the CTU can be encoded according to the CU divided in the CTU.
  • the PU is a basic unit used in predictive coding of a CU, and has different PU partitioning modes for different prediction modes.
  • the PU partitioning mode is 2N*2N and N*N.
  • the 2N*2N mode refers to dividing the CU into one PU for predictive coding, that is, one CU includes one PU;
  • N* The N mode refers to dividing the CU into four PUs of the same size for predictive coding, that is, one CU includes four PUs.
  • the coding strategy determined according to the coding algorithm of the tree structure can make the coding cost of the CTU as small as possible, that is, the CTU can be coded with as few bits as possible to reduce the code rate.
  • the calculation process involved is complicated.
  • the calculation process specifically includes calculating the coding cost under different PU partition modes and different prediction modes for each CU in the full quadtree, and calculating the calculated cost.
  • the coding cost is compared to obtain the minimum coding cost of each CU, the calculation amount is large, the complexity is high, the time required is large, and the coding speed is slow.
  • the embodiment of the present application provides a method for determining an encoding cost of a coding unit and a device related thereto, which solves the problem of large computational complexity in determining a CTU partitioning strategy.
  • a flow chart of a method for determining an encoding cost of a coding unit is provided in an embodiment of the present application.
  • the method is typically performed by a video encoding server.
  • the method in this embodiment of the present application may include the following steps S101-S104.
  • S101 Determine a coding unit that performs predictive coding in an intra prediction mode.
  • the coding unit is divided by the CTU, and the size of the coding unit may be any one of 8*8 pixels, 16*16 pixels, 32*32 pixels, and 64*64 pixels.
  • performing predictive coding in the intra prediction mode refers to performing predictive coding by using a coding unit of a current frame as a reference coding unit, where the reference coding unit refers to a coding unit that has been encoded.
  • the coding unit may be directly determined to be a coding unit that performs predictive coding in the intra prediction mode.
  • the coding unit is a coding unit in a non-I frame (such as a P frame or a B frame)
  • the coding unit that performs prediction coding in the intra prediction mode may be determined by referring to an intra mode decision algorithm in the H.264/AVC standard, which may specifically After determining the optimal inter mode by performing interframe predictive coding on the coding unit, determining an average code rate and an average boundary error of the coding unit.
  • the coding unit is determined to be a coding unit that performs prediction in an intra prediction mode.
  • the coding unit is a coding unit in a non-I frame
  • the coding unit that performs prediction coding in the intra prediction mode may be determined according to other manners, which is not limited in the embodiment of the present application.
  • step S102 to S103 are skipped, and step S104 is performed.
  • the first coding cost is determined as the second coding cost of the coding unit.
  • the pixel gradient information corresponding to the coding unit is information obtained according to the image gradient of the pixel in the coding unit.
  • the pixel gradient information may be information obtained from image gradients of partial pixels in the coding unit, and the pixel gradient information may also be based on information of image gradients of all pixels in the coding unit.
  • the pixel gradient information may further include parameters such as a pixel gradient average value, a pixel gradient mean square error, and the like for measuring the smoothness of the image and the complexity of the texture.
  • the pixel gradient average refers to the mean of the image gradients of some or all of the pixels in the coding unit.
  • the pixel gradient mean square error refers to the mean square error of the image gradient of some or all of the pixels in the coding unit.
  • the pixel gradient information of each pixel in the coding unit may be determined by an image gradient algorithm, thereby determining an image gradient of a partial pixel or all pixels in the coding unit, and further may be based on an image gradient of a partial pixel in the coding unit or The image gradient of all pixels determines the pixel gradient information corresponding to the coding unit.
  • the image gradient algorithm may be an image gradient algorithm based on a first derivative, such as a Roberts operator, a Sobel operator or a Prewitt operator, and the like.
  • the image gradient algorithm may also be an image gradient algorithm based on a second derivative, such as a Laplace operator.
  • the partial pixel may be a partial pixel randomly selected among all pixels of the coding unit.
  • the partial pixels may also be partial pixels selected in all pixels of the coding unit according to a certain selection rule.
  • some pixels in the coding unit may be partial pixels selected according to a preset sampling rate. For example, if the preset sampling rate is 90% and the size of the coding unit is 64*64 pixels, that is, 4096 pixels, then some pixels in the coding unit are 3687 pixels out of 4096 pixels, which can be left to right and up to The next sequence samples all the pixels in the coding unit, resulting in 3687 pixels.
  • the coding unit can be divided into pre-decisions according to the pixel gradient variance. If the pixel gradient information further includes a pixel gradient average value, the coding unit may be subjected to a partition pre-decision according to the pixel gradient variance and the pixel gradient average value. If the pixel gradient information further includes a pixel gradient mean square error, the coding unit may be subjected to a partition pre-decision according to the pixel gradient variance and the pixel gradient mean square error.
  • the specific implementation manner of the pre-decision of the coding unit according to the pixel gradient information is specifically introduced in the embodiment shown in FIG. 8 to FIG. 12, and the description in the embodiment shown in FIG. 8 to FIG. 12 can be referred to.
  • the first coding cost is determined as a second coding cost of the coding unit, and the second coding cost is used to determine the coding unit.
  • a partitioning strategy of the corresponding coding tree unit where the first coding cost is an encoding cost obtained by predictive coding the coding unit by using a current size of the coding unit as a size of a prediction unit.
  • the first coding cost is an encoding cost in a case where the coding unit is 2N*2N in the PU division mode. If the size of the prediction unit is only one of the prediction modes corresponding to the current size of the coding unit, the first coding cost is an encoding cost obtained by predictive coding the coding unit in the prediction mode, that is, the prediction mode is Corresponding coding cost. If the size of the prediction unit is different for the prediction mode corresponding to the current size of the coding unit, the first coding cost is the minimum obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit. Coding cost.
  • the coding unit has a size of 32*32 pixels, and a prediction unit with a size of 32*32 pixels corresponds to a prediction mode of only one prediction mode, and the first coding cost is a prediction mode 1 pair coding unit.
  • the coding cost obtained by predictive coding.
  • the prediction modes corresponding to the prediction unit having the size of 32*32 pixels respectively have four types: prediction mode 1, prediction mode 2, prediction mode 3, and prediction mode 4, respectively, and predictive coding of the coding unit by using the four prediction modes respectively.
  • the coding costs are c11, c12, c13, c14, respectively, and c11 ⁇ c12 ⁇ c13 ⁇ c14, then the first coding cost is c11.
  • the second coding cost refers to the final coding cost of the coding unit. If the coding unit is a CTU, the second coding cost of the CTU may be equivalent to the minimum coding cost (c0) of the CTU determined after the recursive process in the process shown in FIG. 3E, since the second coding cost is the first The coding cost, according to the division situation shown in FIG.
  • the second coding cost of the coding unit may be equal to the minimum coding cost in the case where the PU is divided into 2N*2N for the CU and the N*N in the PU division mode in the process shown in FIG. 3C or FIG. 3D.
  • the minimum coding cost in the case of comparison compares the determined minimum coding cost of the CU (c21, c31, etc.), which is used to calculate the upper layer CU of the coding unit in the PU division mode of N*N
  • the minimum coding cost in the case which in turn can determine the second coding cost of the CTU by recursively, and then determine the CTU partitioning strategy.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it may be determined that the coding unit does not need to be divided.
  • the division of the coding unit is terminated in advance, and the coding unit is divided into sub-coding units, and then the sub-coding unit is used as a new coding unit to perform prediction coding, coding cost calculation and the like, thereby reducing the calculation amount and improving the calculation amount. Coding rate.
  • FIG. 6 a flowchart of a method for determining an encoding cost of a coding unit is provided in an embodiment of the present application.
  • the method is typically performed by a video encoding server.
  • the method in this embodiment of the present application may include the following steps S201-S208.
  • S201 Determine a coding unit that performs predictive coding in an intra prediction mode.
  • the coding unit is divided by the CTU, and the size of the coding unit may be any one of 8*8 pixels, 16*16 pixels, 32*32 pixels, and 64*64 pixels.
  • performing predictive coding in the intra prediction mode refers to performing predictive coding by using a coding unit of a current frame as a reference coding unit, where the reference coding unit refers to a coding unit that has been encoded.
  • the coding unit may be directly determined to be a coding unit that performs predictive coding in the intra prediction mode.
  • the coding unit is a coding unit in a non-I frame (such as a P frame or a B frame)
  • the coding unit that performs prediction coding in the intra prediction mode may be determined by referring to an intra mode decision algorithm in the H.264/AVC standard, which may specifically After determining the optimal inter mode by performing interframe predictive coding on the coding unit, determining an average code rate and an average boundary error of the coding unit.
  • the coding unit is determined to be a coding unit that performs prediction in an intra prediction mode.
  • the coding unit is a coding unit in a non-I frame
  • the coding unit that performs prediction coding in the intra prediction mode may be determined according to other manners, which is not limited in the embodiment of the present application.
  • step S207 is performed to determine the first coding cost as the coding unit.
  • the second encoding cost is performed to determine the first coding cost.
  • S202 Determine pixel gradient information corresponding to the coding unit, where the pixel gradient information includes a pixel gradient variance, where the pixel gradient variance is a variance of an image gradient of at least a part of pixels in the coding unit.
  • the pixel gradient information corresponding to the coding unit is information obtained according to the image gradient of the pixel in the coding unit.
  • the pixel gradient information may be information obtained from image gradients of partial pixels in the coding unit, and the pixel gradient information may also be based on information of image gradients of all pixels in the coding unit.
  • the pixel gradient information may further include parameters such as a pixel gradient average value, a pixel gradient mean square error, and the like for measuring the smoothness of the image and the complexity of the texture.
  • the pixel gradient average refers to the mean of the image gradients of some or all of the pixels in the coding unit.
  • the pixel gradient mean square error refers to the mean square error of the image gradient of some or all of the pixels in the coding unit.
  • the pixel gradient information of each pixel in the coding unit may be determined by an image gradient algorithm, thereby determining an image gradient of a partial pixel or all pixels in the coding unit, and further may be based on an image gradient of a partial pixel in the coding unit or The image gradient of all pixels determines the pixel gradient information corresponding to the coding unit.
  • the image gradient algorithm may be an image gradient algorithm based on a first derivative, such as a Roberts operator, a Sobel operator or a Prewitt operator, and the like.
  • the image gradient algorithm may also be an image gradient algorithm based on a second derivative, such as a Laplace operator.
  • the partial pixel may be a partial pixel randomly selected among all pixels of the encoding unit.
  • the partial pixels may also be partial pixels selected in all pixels of the coding unit according to a certain selection rule.
  • some pixels in the coding unit may be partial pixels selected according to a preset sampling rate. For example, if the preset sampling rate is 90% and the size of the coding unit is 64*64 pixels, that is, 4096 pixels, then some pixels in the coding unit are 3687 pixels out of 4096 pixels, which can be left to right and up to The next sequence samples all the pixels in the coding unit, resulting in 3687 pixels.
  • the coding unit can be divided into pre-decisions according to the pixel gradient variance. If the pixel gradient information further includes a pixel gradient average value, the coding unit may be subjected to a partition pre-decision according to the pixel gradient variance and the pixel gradient average value. If the pixel gradient information further includes a pixel gradient mean square error, the coding unit may be subjected to a partition pre-decision according to the pixel gradient variance and the pixel gradient mean square error.
  • the specific implementation manner of the pre-decision of the coding unit according to the pixel gradient information is specifically introduced in the embodiment shown in FIG. 8 to FIG. 12, and the description in the embodiment shown in FIG. 8 to FIG. 12 can be referred to.
  • step S207 is performed, and if the result of the pre-decision of the division of the coding unit is division, step S204 is performed.
  • the coding unit may be divided into four sub coding units by referring to the method of the full quadtree in the prior art.
  • the coding unit can be divided into other numbers of sub-coding units, which are not limited in this embodiment.
  • At least two sub coding units may be determined as new coding units that perform predictive coding in an intra prediction mode, and steps S201 to S208 are performed to determine coding costs of the sub coding units, respectively.
  • the first coding cost is an encoding cost in a case where the coding unit is 2N*2N in the PU division mode. If the size of the prediction unit is only one of the prediction modes corresponding to the current size of the coding unit, the first coding cost is an encoding cost obtained by predictive coding the coding unit in the prediction mode, that is, the prediction mode is Corresponding coding cost. If the size of the prediction unit is different for the prediction mode corresponding to the current size of the coding unit, the first coding cost is the minimum obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit. Coding cost.
  • the coding unit has a size of 32*32 pixels, and a prediction unit with a size of 32*32 pixels corresponds to a prediction mode of only one prediction mode, and the first coding cost is a prediction mode 1 pair coding unit.
  • the coding cost obtained by predictive coding.
  • the prediction modes corresponding to the prediction unit having the size of 32*32 pixels respectively have four types: prediction mode 1, prediction mode 2, prediction mode 3, and prediction mode 4, respectively, and predictive coding of the coding unit by using the four prediction modes respectively.
  • the coding costs are c11, c12, c13, c14, respectively, and c11 ⁇ c12 ⁇ c13 ⁇ c14, then the first coding cost is c11.
  • step S207 is performed; if the sum of the coding costs of the at least two sub coding units is less than the first coding cost, step S208 is performed.
  • the second coding cost refers to the final coding cost of the coding unit. If the coding unit is a CTU, the second coding cost of the CTU may be equivalent to the minimum coding cost (c0) of the CTU determined after the recursive process in the process shown in FIG. 3E, since the second coding cost is the first The coding cost, according to the division situation shown in FIG.
  • the second coding cost of the coding unit may be equal to the minimum coding cost in the case where the PU partitioning mode is 2N*2N for the CU and the N*N in the PU division mode in the process shown in FIG. 3C or FIG. 3D.
  • the minimum coding cost in the case of comparison compares the determined minimum coding cost of the CU (c21, c31, etc.), which is used to calculate the upper layer CU of the coding unit in the PU division mode of N*N
  • the minimum coding cost in the case which in turn can determine the second coding cost of the CTU by recursively, and then determine the CTU partitioning strategy.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it may be determined that the coding unit does not need to be divided.
  • the division of the coding unit is terminated in advance, and the coding unit is divided only when the result of the division pre-decision is divided, and then the coding cost of the sub-coding unit corresponding to the coding unit and the size of the first coding cost are compared. Since the same method of determining the coding cost is also adopted in determining the coding cost of the sub coding unit, the further division of the sub coding unit can be ended in advance, and the division of the coding unit is terminated as early as possible, thereby eliminating a large amount of computational coding.
  • the process of cost, reducing the calculation Quantity, determine the CTU partition strategy as soon as possible, and improve the coding speed.
  • the process of determining the CTU partitioning strategy by using the technical solution of the embodiment of the present application is described below by taking the coding unit into four sub-coding units as an example.
  • the CTU size is 64*64 pixels as an example. See FIGS. 7A-7E.
  • the CTU is determined to be a CU, and the CTU is predictively encoded with 64*64 pixels as the size of the prediction unit to determine an optimal prediction mode of the CTU, and the coding cost of the CTU in the optimal prediction mode is determined, that is, C1, the coding cost is the first coding cost of the CTU.
  • the CTU is divided and determined according to the pixel gradient information of the CTU.
  • the second coding cost of the CTU is determined, as can be seen from FIG. 4, Determine the CTU partitioning strategy, that is, the CTU does not need to be divided, and the division is terminated. Since the CTU partitioning strategy has been determined, it is not necessary to sequentially calculate the encoding cost of the second layer CU, the third layer CU, and the fourth layer CU as in the process shown in FIG. 3B, and it is not necessary to perform the processes of FIG. 3C to FIG. 3E. Going a lot of calculations and improving the coding speed.
  • the CTU is divided into four 32*32 CUs, that is, the CU of the second layer.
  • predictive coding is performed on each CU of the second layer with 32*32 pixels as the size of the prediction unit, respectively, to determine an optimal prediction mode of each CU, and to determine the optimal prediction of each CU of the second layer.
  • the coding cost in the mode which is the first coding cost of each CU of the second layer, namely C21, C22, C23, C24.
  • the pre-decision is performed on each CU of the second layer according to the pixel gradient information of each CU of the second layer.
  • the first coding cost is determined as the second coding cost of the target CU of the second layer, at this time, due to the second target CU of the second layer If the coding cost is determined, the division of the target CU of the second layer is terminated.
  • the coding cost of the third layer CU and the fourth layer CU under the target CU is not calculated in sequence as in the process shown in FIG. 3B, and a large amount of calculation is also omitted, and the coding speed is improved.
  • the target CU of the second layer is divided into four sub-CUs of 16*16 pixels, that is, CUs of the third layer.
  • predictive coding is performed on each CU of the third layer with 16*16 pixels as the size of the prediction unit, respectively, and an optimal prediction mode of each CU of the third layer is determined, and each CU of the third layer is determined to be the most The coding cost in the optimal prediction mode, which is the first coding cost of each CU of the third layer.
  • the pre-decision is performed on each CU of the second layer according to the pixel gradient information of each CU of the third layer.
  • the first coding cost of the target CU of the third layer is determined as the second coding cost of the target CU of the third layer, at this time, due to the third The second coding cost of the target CU of the layer is determined, and the division of the target CU of the third layer is terminated. Since it has been determined that the target CU is not divided, it is not necessary to sequentially calculate the fourth layer CU under the target CU as in the process shown in FIG. 3B, and a large amount of calculation is also omitted, and the encoding speed is improved.
  • the target CU of the third layer is divided into four CUs of 8*8 pixels, that is, CUs of the third layer.
  • predictive coding is performed on each CU of the fourth layer with 8*8 pixels as the size of the prediction unit, respectively, and an optimal prediction mode of each CU of the fourth layer is determined, and each CU of the fourth layer is determined to be the most The coding cost in the optimal prediction mode, which is the first coding cost of each CU of the fourth layer.
  • the second coding cost of the CU in the third layer, the second layer, and the CU in the first layer that has not determined the second coding cost is sequentially determined according to the bottom-up manner, and the CU is The smaller of the two coding costs (the sum of the second coding costs of the four sub-CUs under the CU, and the first coding cost of the CU) is determined as the second coding cost of the CU.
  • the tree structure determined according to the process of FIGS. 7A-7C is as shown in FIG. 7D, wherein it is determined by the process of FIG. 7A that the CU numbered 1 (ie, CTU) needs to be divided. It is determined by the process of FIG. 7B that the CUs numbered 3 and 4 need to be divided, and the CUs coded 2 and 5 do not need to be partitioned, then the second coding cost of the CUs coded 2 and 5 is determined, for their first coding cost, The second coding cost of the CUs numbered 3 and 4 has not been determined.
  • the second coding cost of the CU for which the second coding unit price has not been determined is determined in order from bottom to top, that is, the second coding cost of the CU numbered 6, 3, 4, 1 is determined, as shown in FIG. 7E.
  • the second encoding cost of the CU numbered 6 in the CU of the third layer is determined, and then the second encoding cost of the CU encoded as 3 and 4 in the CU of the second layer is determined, and finally, the number is determined to be 1.
  • the CU is the second coding cost of the CTU.
  • the CTU division manner is determined according to the minimum coding cost of the CU of each layer. If the second coding cost of the CTU is the first coding cost of the CTU, it is determined that the CTU is not divided. It is determined that the tree structure of the CTU has only one layer, that is, the root node, and its corresponding division is shown as A in FIG. 4 . If the second coding cost of the CTU is the sum of the second coding costs of the CU of the second layer, it is determined that the CTU needs to be divided, and it is determined that the CTU is divided into four CUs, as shown by B in FIG. In the case that it is determined that the CTU is divided into four CUs, the division manner of the second layer may be further determined according to the second coding cost of the CU of the second layer, and so on, and the final division of the CTU may be determined.
  • FIG. 8 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present disclosure.
  • the coding unit according to the pixel gradient information is used in the embodiment of the present application.
  • Performing the division pre-decision may include the following steps S301-S303.
  • the variance threshold corresponding to the coding unit there are two cases of the variance threshold corresponding to the coding unit: one is that the variance threshold is a fixed threshold; the other is that the variance threshold is positively correlated with the quantization parameter corresponding to the coding unit, that is, the coding unit corresponds to The larger the quantization parameter, the larger the variance threshold.
  • the quantization parameter reflects the spatial detail compression. If the quantization parameter is small, most of the details will be preserved. If the quantization parameter is increased, the detail is lost and the code rate is reduced, that is, the quantization parameter and the code rate are negatively correlated, that is, the variance threshold and the code rate are negative. Related.
  • the value of the quantization parameter can be determined by the encoder. The value of the quantization parameter ranges from 0 to 52. When the quantization parameter is greater than 0, the variance threshold corresponding to the coding unit may be equal to the quantization parameter corresponding to the coding unit multiplied by X, and X is a value greater than or equal to 1.
  • the pixel gradient variance corresponding to the coding unit is used to reflect the texture and object boundary conditions of the image in the coding unit.
  • the pixel gradient of the coding unit is small, indicating that the image of the coding unit may be a messy texture. Further division of the coding unit does not help to improve the image quality. Since it is a messy texture, the human eye is not sensitive to the coding unit, so no further division is needed. .
  • the variance of the pixel gradient is large, indicating that the coding unit may contain object boundaries. Further division may help to improve the image quality. After further division, the similarity of the sub-coding units corresponding to the coding unit is higher, which helps to improve the compression efficiency.
  • the pre-decision of whether the coding unit is divided according to the pixel gradient variance of the coding unit is completed, and the pixel gradient of the coding unit reflects the texture and boundary of the image in the coding unit, according to the pixel gradient of the coding unit.
  • the variance can end the division of the coding unit in advance, saving computational complexity, while at the same time balancing compression efficiency and image quality.
  • the pixel gradient information described in the embodiment of the present application may further include a pixel gradient average value.
  • FIG. 9 is a schematic flowchart of a method for determining an encoding cost of a coding unit according to an embodiment of the present application, as shown in FIG. 9.
  • the pre-decision of the coding unit according to the pixel gradient information may include the following steps S401-S406.
  • S401 Determine a first threshold corresponding to the coding unit, a second threshold corresponding to the coding unit, and a variance threshold corresponding to the coding unit, where the second threshold is greater than the first threshold.
  • the variance threshold corresponding to the coding unit there are two cases of the variance threshold corresponding to the coding unit: one is that the variance threshold is a fixed threshold; the other is that the variance threshold is positively correlated with the quantization parameter corresponding to the coding unit, that is, the coding unit corresponds to The larger the quantization parameter, the larger the variance threshold.
  • the quantization parameter reflects the spatial detail compression. If the quantization parameter is small, most of the details will be preserved. If the quantization parameter is increased, the detail is lost and the code rate is reduced, that is, the quantization parameter and the code rate are negatively correlated, that is, the variance threshold and the code rate are negative. Related.
  • the value of the quantization parameter can be determined by the encoder. The value of the quantization parameter ranges from 0 to 52. When the quantization parameter is greater than 0, the variance threshold corresponding to the coding unit may be equal to the quantization parameter corresponding to the coding unit multiplied by X, and X is a value greater than or equal to 1.
  • the first threshold and the second threshold are both average thresholds. If the first threshold corresponding to the coding unit is not zero, the second threshold corresponding to the coding unit may be a fixed multiple of the first threshold corresponding to the coding unit, that is, the second threshold corresponding to the coding unit is the first threshold and the corresponding threshold of the coding unit. The product of two coefficients, the second coefficient being greater than one. If the coding unit corresponds to the first threshold being zero, the second threshold corresponding to the coding unit may be a sum of a threshold corresponding to the coding unit and a target mean threshold, and the target mean threshold is a fixed threshold.
  • the first threshold corresponding to the coding unit may be a fixed threshold. If the first threshold corresponding to the coding unit is a fixed threshold, the first threshold is inversely related to the size of the coding unit.
  • the first threshold corresponding to the coding unit may also be related to the average value of the pixel gradient of the coding unit around the coding unit.
  • the determining the first threshold corresponding to the coding unit includes the following steps S501-S503.
  • S501 Determine a plurality of neighbor coding units of the coding unit, where an edge of the neighbor coding unit overlaps with an edge of the coding unit, or a vertex of the neighbor coding unit overlaps with a vertex of the coding unit, where The image in the neighbor coding unit does not coincide with the image in the coding unit.
  • the neighbor coding unit is a coding unit that is around the coding unit and is connected to the coding unit.
  • the neighbor coding unit may be a coding unit that has performed predictive coding.
  • the size of the neighbor coding unit may be the same as the size of the coding unit, and the size of the neighbor coding unit may also be different from the size of the coding unit.
  • the coding unit is CU1 as shown in FIG. 11, CU7, CU17, CU19, CU20, and CU23 have edges with CU1 edges overlapping, CU6, CU10, and CU22.
  • the vertices of CU24 overlap with the vertices of CU1, then CU6, CU7, CU10, CU17, CU19, CU20, CU22, CU23, and CU24 are all neighbor coding units of CU1, and the remaining CUs have no edges or vertices coincident with CU1, therefore,
  • the remaining CUs are not neighbor coding units of the coding unit.
  • the neighbor coding unit may also be a coding unit around the coding unit and in a specific orientation.
  • the neighbor coding unit is a neighbor coding unit around the coding unit and in the upper left direction, the upper right direction, the upper right direction, and the right left direction, that is, in FIG. 11, CU6, CU7, CU10, CU17, CU19 are neighbor coding of CU1. unit.
  • the neighbor coding unit of CU1 may be a coding unit formed by combining CU3, CU4, CU5, and CU6, CU7, A coding unit formed by combining CU8, CU9, CU10, and CU11, a coding unit formed by combining CU16, CU17, CU18, and CU19, and a coding unit formed by combining CU20, CU22, CU23, CU3, CU24, CU25, and CU26.
  • the first neighbor coding unit performs predictive coding in an intra prediction mode.
  • the target number can be set to 4, 5, and 6 values.
  • the neighbor coding units determined in step S501 are all first neighbor coding units; if the coding unit is a coding unit in a non-I frame, The manner described in step S101 determines a first neighbor coding unit among a plurality of neighbor coding units.
  • the determining, according to the size of the first neighbor coding unit and the pixel gradient average of the first neighbor coding unit, a first threshold corresponding to the coding unit including The following steps S601-S603.
  • CU7, CU10, CU17, CU20, CU22, and CU23 are all coding units that perform predictive coding in the intra prediction mode, and sizes of CU7, CU20, CU22, and CU23. Both are equal to the size of CU1, and CU7, CU20, CU22, and CU23 are all second neighbor coding units.
  • the second neighbor coding unit determines a product of the first coefficient and the target average value as a first threshold corresponding to the coding unit, where the target average value is a pixel corresponding to each second neighbor coding unit. The quotient of the sum of the gradient averages and the number of the second neighbor coding units.
  • the first coefficient can be a coefficient greater than one.
  • the pixel gradient average value corresponding to each neighbor coding unit may be determined according to the manner of determining the pixel gradient information corresponding to the coding unit introduced in step S102.
  • the second neighbor coding unit is the coding unit 2, the coding unit 3, the coding unit 4, and the coding unit 5.
  • the corresponding pixel gradient average values are pixel gradient average 1, pixel gradient average 2, and pixel gradient average 3, respectively.
  • the pixel average value is 4, and the target average value is (pixel gradient average value 1 + pixel gradient average value 2+ pixel gradient average value 3 + pixel gradient average value 4) / 4.
  • the first threshold is zero is the minimum value that can be obtained by performing the gradient calculation.
  • step S502 is that the size of the neighbor coding unit can be the same as the size of the coding unit, and the size of the neighbor coding unit can also be different from the size of the coding unit.
  • a way of thresholding if the neighbor coding unit specifically refers to a coding unit whose size is the same as the coding unit, step S502 may be further replaced by: if the first neighbor coding unit of the multiple neighbor coding units And determining, by the first neighbor coding unit, a first threshold corresponding to the coding unit, where the quantity is greater than or equal to the target quantity, where the first neighbor coding unit is determined according to the division condition of the first neighbor coding unit and the pixel gradient average value of the first neighbor coding unit A coding unit that performs predictive coding in an intra prediction mode.
  • the determining, according to the partitioning condition of the first neighbor coding unit, and the pixel gradient average value of the first neighbor coding unit, the first threshold corresponding to the coding unit including: coding in the first neighbor Searching, in the unit, a third neighbor coding unit, where the third neighbor coding unit is an undivided first neighbor coding unit; if the third neighbor coding unit is present, determining a product of the first coefficient and the target average as a first threshold corresponding to the coding unit, where the target average is a quotient of a sum of a pixel gradient average corresponding to each second neighbor coding unit and a quantity of the second neighbor coding unit.
  • the mean threshold corresponding to the size of the coding unit is a fixed threshold, wherein the larger the size of the coding unit, the smaller the mean threshold corresponding to the coding size.
  • three mean thresholds may be designed to respectively correspond to a coding unit having a size of 16*16 pixels, a coding unit having a size of 32*32 pixels, and a coding unit having a size of 64*64 pixels.
  • step S405 is performed; if the pixel gradient average is greater than or equal to the first threshold, step S403 is performed.
  • the pixel gradient average corresponding to the coding unit is used to reflect the smoothness of the image content in the coding unit.
  • the average value of the pixel gradient of the coding unit is small, indicating that the image content in the coding unit is relatively smooth, and there is no need to continue dividing.
  • step S404 is performed; if the pixel gradient average is greater than or equal to the second threshold, step S406 is performed.
  • the average value of the pixel gradient is large, indicating that the content of the image in the coding unit is not smooth, and there is a high possibility that the division needs to be continued.
  • step S405 is performed, and if the pixel gradient variance is greater than or equal to the variance threshold, step S406 is performed.
  • the pixel gradient variance corresponding to the coding unit is used to reflect the texture and object boundary conditions of the image in the coding unit.
  • the pixel gradient of the coding unit is small, indicating that the image of the coding unit may be a messy texture. Further division of the coding unit does not help to improve the image quality. Since it is a messy texture, the human eye is not sensitive to the coding unit, so no further division is needed. .
  • the variance of the pixel gradient is large, indicating that the coding unit may contain object boundaries. Further division may help to improve the image quality. After further division, the similarity of the sub-coding units corresponding to the coding unit is higher, which helps to improve the compression efficiency.
  • the pre-decision of whether the coding unit is divided is performed by combining the pixel gradient variance and the pixel gradient average of the coding unit, and the pixel variance of the coding unit reflects the texture and boundary of the image in the coding unit, and the coding unit
  • the average of the pixel gradient reflects the content smoothness of the image in the coding unit, and comprehensively analyzes the image content in the coding unit, which can accurately determine whether the coding unit is divided, saves computation, and takes into account compression efficiency and image quality.
  • the apparatus for determining the coding cost of the coding unit provided by the embodiment of the present application will be described in detail below with reference to FIG. 13 to FIG.
  • the means for determining the coding cost of the coding unit may be a terminal having a calculation processing function. It should be noted that the apparatus shown in FIG. 13 to FIG. 16 is used to perform the method of the embodiment shown in FIG. 5 to FIG. 12 of the present application. For the convenience of description, only the parts related to the embodiment of the present application are shown. For technical details not disclosed, please refer to the embodiment shown in Figures 5-12 of the present application.
  • the apparatus 70 for determining the coding cost of the coding unit in the embodiment of the present application may include: a coding unit determining module 71, a gradient information determining module 72, a pre-decision module 73, and a cost determining module 74.
  • a coding unit determining module 71 configured to determine a coding unit that performs predictive coding in an intra prediction mode
  • the coding unit is divided by the CTU, and the size of the coding unit may be any one of 8*8 pixels, 16*16 pixels, 32*32 pixels, and 64*64 pixels.
  • performing predictive coding in the intra prediction mode refers to performing predictive coding by using a coding unit of a current frame as a reference coding unit, where the reference coding unit refers to a coding unit that has been encoded.
  • the coding unit determining module 71 may directly determine that the coding unit performs prediction coding in an intra prediction mode. Coding unit.
  • the coding unit that performs prediction coding in the intra prediction mode may be determined by referring to an intra mode decision algorithm in the H.264/AVC standard, which may specifically
  • the coding unit determining module 71 determines the average code rate and the average boundary error of the coding unit after performing the inter prediction coding on the coding unit to select the optimal inter mode, if the average code rate of the coding unit is greater than or equal to the coding unit.
  • the average boundary error is determined, and the coding unit is determined to be a coding unit that performs prediction in the intra prediction mode.
  • the coding unit determination module 71 may also determine the coding unit that performs prediction coding in the intra prediction mode according to other manners, which is not limited in the embodiment of the present application.
  • the cost determination module 74 directly determines the first coding cost as the second coding cost of the coding unit.
  • the gradient information determining module 72 is configured to determine pixel gradient information corresponding to the coding unit, where the pixel gradient information includes a pixel gradient variance, where the pixel gradient variance is a variance of an image gradient of at least a part of pixels in the coding unit;
  • the pixel gradient information corresponding to the coding unit is information obtained by the gradient information determining module 72 according to the image gradient of the pixel in the coding unit.
  • the pixel gradient information may be information obtained by the gradient information determining module 72 according to the image gradient of the partial pixels in the encoding unit, and the pixel gradient information may also be the information of the gradient of the image of all the pixels in the encoding unit by the gradient information determining module 72.
  • the pixel gradient information may further include parameters such as a pixel gradient average value, a pixel gradient mean square error, and the like for measuring the smoothness of the image and the complexity of the texture.
  • the pixel gradient average refers to the mean of the image gradients of some or all of the pixels in the coding unit.
  • the pixel gradient mean square error refers to the mean square error of the image gradient of some or all of the pixels in the coding unit.
  • the gradient information determining module 72 may determine pixel gradient information of each pixel in the coding unit by using an image gradient algorithm, thereby determining an image gradient of a partial pixel in the coding unit, or determining an image of all pixels in the coding unit.
  • the gradient may further determine pixel gradient information corresponding to the coding unit according to an image gradient of a partial pixel in the coding unit or an image gradient of all pixels.
  • the image gradient algorithm may be an image gradient algorithm based on a first derivative, such as a Roberts operator, a Sobel operator or a Prewitt operator, and the like.
  • the image gradient algorithm may also be an image gradient algorithm based on a second derivative, such as a Laplace operator.
  • the partial pixel may be a partial pixel randomly selected by the gradient information determining module 72 in all pixels of the encoding unit.
  • the partial pixels in the coding unit may also be partial pixels selected by the gradient information determining module 72 in all pixels of the coding unit according to a certain selection rule, for example, some pixels in the coding unit may be partial pixels selected according to a preset sampling rate. .
  • the preset sampling rate is 90%
  • the size of the coding unit is 64*64 pixels, that is, 4096 pixels
  • some pixels in the coding unit are 3687 pixels out of 4096 pixels
  • the gradient information determining module 72 can follow from left to right.
  • the right, top-to-bottom sequence samples all pixels in the coding unit, resulting in 3687 pixels.
  • a pre-decision module 73 configured to perform a pre-decision on the coding unit according to the pixel gradient information
  • the pre-decision module 73 can perform a partition pre-decision on the coding unit according to the pixel gradient variance. If the pixel gradient information further includes a pixel gradient average, the pre-decision module 73 may pre-compute the coding unit based on the pixel gradient variance and the pixel gradient average. If the pixel gradient information further includes a pixel gradient mean square error, the pre-decision module 73 may perform a partition pre-decision on the coding unit according to the pixel gradient variance and the pixel gradient mean square error.
  • the pre-decision module 73 may include a threshold determining unit 731, and a result determining unit 732.
  • the threshold determining unit 731 is configured to determine a variance threshold corresponding to the coding unit, and a result determining unit 732 is configured to determine, if the pixel gradient variance is less than the variance threshold, The result of the division pre-decision of the unit is not divided; the result determining unit 732 is further configured to determine, if the pixel gradient variance is greater than or equal to the variance threshold, a result of dividing the pre-decision of the coding unit into a division.
  • the pixel gradient information further includes a pixel gradient average value
  • the threshold determining unit 731 is configured to determine a first threshold corresponding to the coding unit, a second threshold corresponding to the coding unit, and a variance threshold corresponding to the coding unit, where the second threshold is greater than the first threshold
  • a result determining unit 732 configured to determine a division of the coding unit if the pixel gradient average is less than the first threshold The result of the pre-judgment is no division; or the result determination unit 732 is further configured to: if the pixel gradient average is greater than or equal to the first threshold, and the pixel gradient average is less than the second threshold, and If the pixel gradient variance is less than the variance threshold, determining that the result of the partition pre-decision of the coding unit is not divided; or the result determining unit 732 is further configured to: if the pixel gradient average is greater than or equal to a first threshold, and the pixel gradient average is less than the second threshold, and the pixel gradient variance is greater
  • the variance threshold corresponding to the coding unit there are two cases of the variance threshold corresponding to the coding unit: one is that the variance threshold is a fixed threshold; the other is that the variance threshold is positively correlated with the quantization parameter corresponding to the coding unit, that is, the coding unit corresponds to The larger the quantization parameter, the larger the variance threshold.
  • the quantization parameter reflects the spatial detail compression. If the quantization parameter is small, most of the details will be preserved. If the quantization parameter is increased, the detail is lost and the code rate is reduced, that is, the quantization parameter and the code rate are negatively correlated, that is, the variance threshold and the code rate are negative. Related.
  • the value of the quantization parameter can be determined by the encoder. The value of the quantization parameter ranges from 0 to 52. If the quantization parameter is greater than 0, the threshold determining unit 731 may multiply the quantization parameter corresponding to the coding unit by X to determine a variance threshold corresponding to the coding unit, where X is greater than or equal to 1. The value of the threshold determination unit 731 may also determine the fixed variance threshold as the variance threshold corresponding to the coding unit.
  • the first threshold and the second threshold are both average thresholds. If the first threshold corresponding to the coding unit is not zero, the second threshold corresponding to the coding unit may be a fixed multiple of the first threshold corresponding to the coding unit, that is, the threshold determination unit 731 may determine that the second threshold corresponding to the coding unit is the coding unit. The product of the first threshold and the second coefficient, the second coefficient being greater than one. If the coding unit corresponds to the first threshold being zero, the threshold determination unit 731 may determine that the second threshold corresponding to the coding unit may be a sum of a threshold corresponding to the coding unit and a target mean threshold, and the target mean threshold is a fixed threshold.
  • the first threshold corresponding to the coding unit may be a fixed threshold, and the threshold determination unit 731 may determine the average threshold corresponding to the size of the coding unit as the first threshold corresponding to the coding unit. If the first threshold corresponding to the coding unit is a fixed threshold, the first threshold is inversely related to the size of the coding unit.
  • the first threshold corresponding to the coding unit may also be related to the average value of the pixel gradient of the coding unit around the coding unit.
  • the threshold determining unit 731 may be specifically configured to:
  • the neighbor coding unit Determining a plurality of neighbor coding units of the coding unit, an edge of the neighbor coding unit is overlapped with an edge of the coding unit, or a vertex of the neighbor coding unit is overlapped with a vertex of the coding unit, the neighbor coding
  • the image in the unit does not coincide with the image in the coding unit; if the number of the first neighbor coding units in the plurality of neighbor coding units is greater than or equal to the target number, according to the size and location of the first neighbor coding unit Determining, by the first neighbor coding unit, a first threshold value corresponding to the coding unit, the first neighbor coding unit performing prediction coding in an intra prediction mode; if the multiple neighbor coding units are The number of the first neighbor coding units is smaller than the target number, and the average threshold corresponding to the size of the coding unit is determined as the first threshold corresponding to the coding unit.
  • the neighbor coding unit may be a coding unit that is around the coding unit and is connected to the coding unit.
  • the neighbor coding unit may be a coding unit that has performed predictive coding.
  • the size of the neighbor coding unit may be the same as the size of the coding unit, and the size of the neighbor coding unit may also be different from the size of the coding unit.
  • the threshold determining unit 731 determines that CU6, CU7, CU10, CU17, CU19, CU20, CU22, CU23, and CU24 are all neighbor coding units of CU1, and the remaining CUs have no edges or vertices.
  • CU1 coincides, and therefore, the threshold determination unit 731 determines that the remaining CUs are not neighbor cells of the coding unit.
  • the neighbor coding unit may also be a coding unit around the coding unit and in a specific orientation.
  • the neighbor coding unit is a neighbor coding unit around the coding unit and in the upper left direction, the upper right direction, the upper right direction, and the right left direction.
  • the threshold determination unit 731 determines that CU6, CU7, CU10, CU17, and CU19 are CUs. Neighbor coding unit.
  • the threshold determination unit 731 determines that the following coding unit is a neighbor coding unit: a combination of CU3, CU4, CU5, and CU6.
  • the target number can be set to 4, 5, and 6 values.
  • the mean threshold corresponding to the size of the coding unit is a fixed threshold, wherein the larger the size of the coding unit, the smaller the mean threshold corresponding to the coding size.
  • three mean thresholds may be designed to respectively correspond to a coding unit having a size of 16*16 pixels, a coding unit having a size of 32*32 pixels, and a coding unit having a size of 64*64 pixels.
  • the threshold determination unit 731 determines that all the neighbor coding units are first neighbor coding units; if the coding unit is a coding unit in a non-I frame, the threshold The determining unit 731 determines the first neighbor coding unit among the plurality of neighbor coding units by the coding unit determination module 71.
  • the threshold determination unit 731 may be specifically used to And searching, in the first neighbor coding unit, a second neighbor coding unit, where the size of the second neighbor coding unit is greater than or equal to a size of the coding unit. If the second neighbor coding unit is present, the product of the first coefficient and the target average is determined as a first threshold corresponding to the coding unit, and the target average is a pixel gradient average corresponding to each second neighbor coding unit. The quotient of the sum of the values and the number of the second neighbor coding units. If the second neighbor unit does not exist, determining that the first threshold corresponding to the coding unit is zero.
  • CU7, CU10, CU17, CU20, CU22, and CU23 are all coding units that perform predictive coding in the intra prediction mode, and sizes of CU7, CU20, CU22, and CU23. If both are equal to the size of CU1, the threshold determination unit 731 determines that CU7, CU20, CU22, and CU23 are both second neighbor coding units.
  • the threshold determining unit 731 may determine, by the gradient information determining module 72, a pixel gradient average corresponding to each neighbor coding unit, thereby determining a target average value.
  • the second neighbor coding unit is the coding unit 2, the coding unit 3, the coding unit 4, and the coding unit 5.
  • the corresponding pixel gradient average values are pixel gradient average 1, pixel gradient average 2, and pixel gradient average 3, respectively.
  • the pixel gradient average value 4, the threshold value determining unit 731 determines that the target average value is (pixel gradient average value 1 + pixel gradient average value 2+ pixel gradient average value 3 + pixel gradient average value 4) / 4.
  • the first threshold is zero is the minimum value that can be obtained by performing the gradient calculation.
  • the threshold determining unit 731 may be specifically configured to: determine a plurality of neighbor coding units of the coding unit, an edge of the neighbor coding unit, and the coding There is a coincidence of edges of the unit or a vertex of the neighbor coding unit overlaps with a vertex of the coding unit, an image in the neighbor coding unit does not coincide with an image in the coding unit; if the multiple neighbor coding units Determining, according to the partitioning condition of the first neighbor coding unit and the pixel gradient average value of the first neighbor coding unit, a first threshold corresponding to the coding unit, where the number of the first neighbor coding units is greater than or equal to the target number, The first neighbor coding unit performs coding coding for predictive coding in an intra prediction mode.
  • the threshold determining unit 731 may be specifically configured to: search, in the first neighbor coding unit, a third neighbor coding unit, where the third neighbor coding unit is an undivided first neighbor coding unit; Determining, by the third neighbor coding unit, a product of the first coefficient and the target average value as a first threshold corresponding to the coding unit, where the target average value is a sum of average values of pixel gradients corresponding to the respective second neighbor coding units A quotient with the number of the second neighbor coding units.
  • the cost determining module 74 is configured to determine, according to a result of the pre-decision of the coding unit by the pre-decision module 73, that the first coding cost is the second coding cost of the coding unit, where a second coding cost is used to determine a coding strategy of a coding tree unit corresponding to the coding unit, where the first coding cost is obtained by predictive coding the coding unit by using a current size of the coding unit as a size of a prediction unit. Coding cost.
  • the first coding cost is an encoding cost in a case where the coding unit is 2N*2N in the PU division mode. If the size of the prediction unit is only one of the prediction modes corresponding to the current size of the coding unit, the first coding cost is an encoding cost obtained by predictive coding the coding unit in the prediction mode, that is, the prediction mode is Corresponding coding cost. If the size of the prediction unit is different for the prediction mode corresponding to the current size of the coding unit, the first coding cost is the minimum obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit. Coding cost.
  • the coding unit has a size of 32*32 pixels, and a prediction unit with a size of 32*32 pixels corresponds to a prediction mode of only one prediction mode, and the first coding cost is a prediction mode 1 pair coding unit.
  • the coding cost obtained by predictive coding.
  • the prediction modes corresponding to the prediction unit of size 32*32 pixels have four types of prediction mode 1, prediction mode 2, prediction mode 3, and prediction mode 4, respectively, and the cost determination module 74 predicts the coding unit by using the four prediction modes respectively.
  • the encoding costs resulting from the encoding are c11, c12, c13, c14, respectively, and c11 ⁇ c12 ⁇ c13 ⁇ c14, then the cost determination module 74 determines that the first encoding cost is 11.
  • the second coding cost refers to the final coding cost of the coding unit. If the coding unit is a CTU, the second coding cost of the CTU may be equivalent to the minimum coding cost (c0) of the CTU determined after the recursive process in the process shown in FIG.
  • the second coding cost is the first
  • the encoding cost is determined according to the foregoing description of the partitioning strategy according to the minimum coding cost of the CTU, and then the dividing strategy of the CTU may be determined as not dividing, that is, directly coding in the current size of the CTU; if the coding unit is not CTU and not For the CU of the fourth layer, the second coding cost of the coding unit may be equivalent to the minimum coding cost in the case where the PU is divided into 2N*2N for the CU in the process shown in FIG. 3C or FIG. 3D.
  • the minimum coding cost of the CU determined by the PU coding mode is N*N is compared (c21, c31, etc.), and the second coding cost is used to calculate the upper layer node of the coding unit in the PU.
  • the minimum coding cost in the case of N*N is divided, and the second coding cost of the CTU can be determined recursively, and then the CTU division strategy is determined.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it may be determined that the coding unit does not need to be divided.
  • the division of the coding unit is terminated in advance, and the coding unit is divided into sub-coding units, and then the sub-coding unit is used as a new coding unit to perform prediction coding, coding cost calculation and the like, thereby reducing the calculation amount and improving the calculation amount. Coding rate.
  • FIG. 15 is a schematic structural diagram of an apparatus for determining an encoding cost of a coding unit according to an embodiment of the present application.
  • the apparatus may include the coding unit determining module 71 and the gradient information determining module 72 in the foregoing embodiment of FIG.
  • the apparatus may further include a partitioning module 75, wherein:
  • a dividing module 75 configured to divide the coding unit into at least two sub-coding units if a result of the pre-decision of the pre-decision module to the coding unit is a division;
  • the dividing module 75 divides the coding unit into four sub coding units.
  • the division module 75 can also divide the coding unit into other numbers of sub-coding units, which is not limited in the embodiment of the present application.
  • the coding unit determining module 71 is further configured to use the at least two sub coding units as a coding unit that performs predictive coding in an intra prediction mode, where the gradient information determination module 72 is further configured to determine the at least two sub-
  • the pre-decision module 73 is further configured to perform a pre-decision on the at least two coding sub-units according to the pixel gradient information, where the cost determination module 74 is further configured to determine respectively Decoding the encoding cost of at least two sub-coding units;
  • the gradient information determining module 72 may respectively determine pixel gradient information corresponding to the at least two sub-coding units according to the manner of determining the pixel gradient information corresponding to the coding unit; the pre-decision module 73 may follow the pixel gradient information corresponding to the coding unit. Performing a pre-decision on the coding unit to perform a pre-decision on the at least two coding sub-units respectively; the cost determination module 74 may separately determine the coding cost of the at least two sub-coding units in a manner of determining an encoding cost of the coding unit. For details, refer to the corresponding description in the foregoing embodiment corresponding to FIG. 13 .
  • the cost determining module 74 is further configured to determine the first encoding cost as the second encoding of the coding unit if a sum of coding costs of the at least two sub coding units is greater than or equal to the first coding cost cost;
  • the cost determining module 74 is further configured to determine, if the sum of coding costs of the at least two sub coding units is greater than the first coding cost, a sum of coding costs of the at least two sub coding units as the coding unit The second encoding cost.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it may be determined that the coding unit does not need to be divided.
  • the division of the coding unit is terminated in advance, and the coding unit is divided only when the result of the division pre-decision is divided, and then the coding cost of the sub-coding unit corresponding to the coding unit and the size of the first coding cost are compared. Since the same method of determining the coding cost is also adopted in determining the coding cost of the sub coding unit, the further division of the sub coding unit can be ended in advance, and the division of the coding unit is terminated as early as possible, thereby eliminating a large amount of computational coding.
  • the process of cost, reducing the calculation Quantity, determine the CTU partition strategy as soon as possible, and improve the coding speed.
  • FIG. 16 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • the terminal 80 may include at least one processor 801, such as a CPU, an input/output interface 803, a memory 804, and at least one communication bus 802.
  • the communication bus 802 is used to implement connection communication between these components.
  • the input/output interface 803 can include a standard wired interface (such as a USB interface) and a wireless interface (such as a WI-FI interface).
  • the input and output interface 803 can optionally include a user interface (such as a display screen, a keyboard, a mouse, etc.).
  • the memory 804 may be a high speed RAM memory or a non-volatile memory such as at least one disk memory.
  • the memory 804 can also optionally be at least one storage device located remotely from the aforementioned processor 801.
  • a memory 804 as a computer storage medium may include an operating system, a network communication module, a user interface module, a program for determining an encoding cost, and an image.
  • the input/output interface 803 is for acquiring an image and transmitting it to the terminal.
  • the input and output interface can also be used to output an image processed by the processor.
  • the processor 801 can divide the image to obtain a coding unit. Further, the processor 801 can be used to invoke a program for determining an encoding cost stored in the memory 804, and specifically perform the following operations:
  • the pixel gradient information includes a pixel gradient variance, where the pixel gradient variance is a variance of an image gradient of at least a part of pixels in the coding unit;
  • determining a first coding cost as a second coding cost of the coding unit the second coding cost being used to determine a corresponding coding unit a coding strategy of a coding tree unit, the first coding cost being an encoding cost obtained by predictive coding of the coding unit by using a current size of the coding unit as a size of a prediction unit.
  • the processor 801 can also call a program that determines the encoding cost stored in the memory 804 to perform the following operations:
  • the coding unit is divided into at least two sub-coding units;
  • the processor 801 when the processor 801 performs a pre-decision on the coding unit according to the pixel gradient information, the following operations are specifically performed:
  • the result of the partition pre-decision of the coding unit is a partition.
  • the pixel gradient information further includes a pixel gradient average value, which is an average value of image gradients of at least part of pixels within the coding unit; the processor 801 is performing according to the pixel
  • the gradient information is used to divide the pre-decision of the coding unit, the following operations are specifically performed:
  • the result of the division pre-decision of the coding unit is division; or
  • the result of the pre-decision of the division of the coding unit is a division.
  • the processor 801 when performing the determining the first threshold corresponding to the coding unit, the processor 801 specifically performs the following operations:
  • an edge of the neighbor coding unit is overlapped with an edge of the coding unit, or a vertex of the neighbor coding unit is overlapped with a vertex of the coding unit, the neighbor coding
  • the image within the unit does not coincide with the image within the coding unit;
  • the number of the first neighbor coding units in the plurality of neighbor coding units is greater than or equal to the target number, determining, according to the size of the first neighbor coding unit and the pixel gradient average corresponding to the first neighbor coding unit, a first threshold corresponding to the coding unit, where the first neighbor coding unit performs predictive coding in an intra prediction mode;
  • the processor 801 when the determining, by the processor 801, the first threshold corresponding to the coding unit according to the size of the first neighbor coding unit and the pixel gradient average corresponding to the first neighbor coding unit, Do the following:
  • the product of the first coefficient and the target average is determined as a first threshold corresponding to the coding unit, and the target average is a pixel gradient average corresponding to each second neighbor coding unit. a quotient of the sum of the values and the number of the second neighbor coding units;
  • the second threshold corresponding to the coding unit is a product of a first threshold and a second coefficient corresponding to the coding unit.
  • the variance threshold corresponding to the coding unit is positively correlated with the quantization parameter corresponding to the coding unit.
  • the coding cost obtained by predictive coding the coding unit by using the current size of the coding unit as the size of the prediction unit is determined as the final coding cost of the coding unit, and further, it may be determined that the coding unit does not need to be divided.
  • the division of the coding unit is terminated in advance, and the coding unit is divided only when the result of the division pre-decision is divided, and then the coding cost of the sub-coding unit corresponding to the coding unit and the size of the first coding cost are compared. Since the same method of determining the coding cost is also adopted in determining the coding cost of the sub coding unit, the further division of the sub coding unit can be ended in advance, and the division of the coding unit is terminated as early as possible, thereby eliminating a large amount of computational coding.
  • the process of cost, reducing the calculation , CTU as soon as possible to determine the partitioning strategy, improve the encoding speed.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Abstract

本申请公开了确定编码单元的编码代价的方法及装置，所述方法包括：确定以帧内预测模式进行预测编码的编码单元；确定编码单元对应的像素梯度信息，像素梯度信息包括像素梯度方差，像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；根据像素梯度信息对编码单元进行划分预判决；如果对编码单元的划分预判决的结果为不划分，则将第一编码代价确定为编码单元的第二编码代价，第二编码代价被用于确定编码单元对应的编码树单元的划分策略，第一编码代价为以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价。本申请技术方案，通过提前结束对编码单元的划分，可以减少计算量，降低编码复杂度，提高编码速率。

Description

确定编码单元的编码代价的方法、装置及计算机可读存储介质
本申请要求于2018年4月2日提交中国专利局、申请号为201810287548.7、发明名称为“确定编码单元的编码代价的方法及相关装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及确定编码单元的编码代价的方法、装置及计算机可读存储介质。
技术背景
高效视频编码(High Efficiency Video Coding，HEVC)是由国际电信联盟管理下的远程通信标准化组织(International Telecommunications Union-Telecommunication Standardization Sector，ITU-T)和国际标准化组织(International Standard Organized，ISO)共同制定的新一代高效视频编码标准，HEVC又可以称为H.265。相比H.264/高级视频编码(Advanced Video Coding，AVC)标准，HEVC采用了更多、更新、更灵活的编码技术，在相同视频质量条件下，HEVC可进一步降低编码的码率。
HEVC标准延续了H.264/AVC标准中的基于块的视频压缩编码框架，相对于H.264/AVC标准中使用的宏块(Macroblock，MB)，HEVC标准采用了更灵活的块结构。在H.264/AVC标准中，MB的尺寸固定，其固定的尺寸为16*16像素，而在HEVC标准中，编码单元(Coding Unit，CU)可以有四种尺寸，其分别为64*64像素、32*32像素、16*16像素以及8*8像素。在HEVC标准中，通过树形结构的编码单元算法来确定编码单元的具体尺寸，从而完成对一帧图像的划分，其具体为：首先，将一帧图像划分为多个编码树单元(Coding Tree Unit，CTU)，然后对每一个CTU采用遍历递归的方式逐层Z形遍历CU，分别确定各个CU的编码代价，根据各个CU的编码代价确定使得编码该CTU对应的编码代价最小的划分策略，根据该划分策略确定划分CTU，从而实现CU的划分。由于对树形结构中的每个CU都需要进行预测编码、编码代价计算等计算以确定CU的编码代价以及确定CU是否需要划分，该算法需要进行大量的计算，需要的时间较多，编码速率较低。
技术内容
本申请的主要目的在于提供一种确定编码单元的编码代价的方法及相关装置，通过提前结束对CU的划分，可以减少计算量，降低编码复杂度，提高编码速率。
本申请实施例第一方面提供了一种确定编码单元的编码代价的方法，由服务器执行，可包括：
确定以帧内预测模式进行预测编码的编码单元；
确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
根据所述像素梯度信息对所述编码单元进行划分预判决；
如果对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
本申请实施例第二方面提供了一种确定编码单元的编码代价的装置，可包括：
编码单元确定模块，用于确定以帧内预测模式进行预测编码的编码单元；
梯度信息确定模块，用于确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
预判决模块，用于根据所述像素梯度信息对所述编码单元进行划分预判决；
代价确定模块，用于如果所述预判决模块对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
为实现上述目的，本申请实施例第三方面提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时，实现如本申请第一方面提供的确定编码单元的编码代价的方法中的各个步骤。
为实现上述目的，本申请实施例第四方面提供一种终端设备，包括：存储器、处理器及存储在所述存储器上且在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如本申请实施例第一方面提供的确定编码单元的编码代价的方法中的各个步骤。
本申请实施例中，在确定编码单元以帧内预测模式进行预测编码后，通过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，省去了将编码单元划分为子编码单元，然后以子编码单元作为新的编码单元进行预测编码并计算编码代价计算的计算过程，减少了计算量，提高了编码速率。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的将一帧图像划分为若干个CTU的情况的举例示意图；
图2A-图2B是本申请实施例提供的CTU的划分策略的表现形式的举例示意图；
图3A-图3E是一种通过完整的遍历递归算法确定CTU的划分策略的流程示意图；
图4是本申请实施例提供的根据CTU的最终的编码代价确定CTU的划分方式的举例示意图；
图5是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图6是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图7A-7E是本申请实施例提供的确定CTU的划分策略的过程举例示意图；
图8是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图9是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图10是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图11是本申请实施例提供的邻居编码单元的举例示意图；
图12是本申请实施例提供的一种确定编码单元的编码代价的方法的流程示意图；
图13是本申请实施例提供的一种确定编码单元的编码代价的装置的结构示意图；
图14是本申请实施例提供的预判决模块的结构示意图；
图15是本申请实施例提供的一种确定编码单元的编码代价的装置的结构示意图；
图16是本申请实施例提供的一种终端的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
在介绍本申请实施例的方案之前，为便于理解本申请实施例的方案，首先对HEVC标准中与本申请实施例有关的一些定义进行介绍。
1、CTU。CTU是对一帧图像进行划分时的基本单位，在对一帧图像进行编码之前，需要对一帧图像进行划分，其中，在对图像进行划分的过程中，首先将图像划分为若干个CTU。在一般情况下，CTU的尺寸为64*64像素，可选的，CTU的尺寸可以为32*32像素，进一步可以为16*16像素或8*8像素。例如，一帧I帧图像的像素为640*480像素，如图1所示，将这帧图像划分为70个64*64像素的CTU后，剩下的图像不足以再划分为64*64像素的CTU，则将剩下的图像划分为20个32*32像素的CTU。
2、CU。CU是HEVC标准中的编码单位，类似于H.264/AVC标准中MB的概念。在对某一帧图像进行视频编码时，对这一帧图像中的每个CU进行逐个编码，以形成连续的视频码流。CU由CTU划分而来，CU的尺寸小于或等于CTU的尺寸。当确定图像中的每个CTU的划分情况后，即可确定整帧图像的划分情况。
根据CTU对应的划分策略对CTU进行划分可确定CTU中CU的划分情况。CTU对应的划分策略可以用树形结构来表示，根据树形结构可确定CTU的CU的划分情况。例如，CTU的划分策略用树形结构来表示如图2A所示，其中，树形结构中的根节点代表CTU，树形结构中的叶子节点代表CTU中的CU。如果树形结构中的节点下还有子节点，则说明该节点对应的CU需要继续划分，如果树形结构中的节点下没有子节点，则说明该节点不需要继续划分，根据图2A所示的树形结构确定的CTU中CU的划分情况如图2B所示。
如果CTU以帧内预测模式进行预测编码，那么CTU的划分策略可以通过完整的遍历递归编码算法确定，该算法的示意图可以如图3A-3E所示，以CTU的尺寸为64*64像素为例。
遍历过程可以如图3A和图3B所示。
首先如图3A所示，对于一个尺寸为64*64像素的CTU，将CTU划分为4个32*32像素的CU，以作为CTU的子CU。再将每个32*32像素的CU划分为4个16*16像素的CU，以作为32*32像素的CU的子CU。再将每个16*16的CU划分为4个8*8的CU，以作为16*16像素的CU的子CU，四层CU形成满四叉树(除最后一层无任何子节点外，每一层上的所有节点下都有四个子节点)。
在形成满四叉树之后，如图3B所示，以64*64像素作为预测单元(Prediction Unit，PU)的尺寸对CTU进行预测编码确定最优预测模式，确定CTU在最优预测模式下的编码代价(C1)，该编码代价为CTU在PU划分方式为2N*2N的情况下的最小编码代价。分别以32*32像素作为预测单元的尺寸对第二层的各个CU进行预测编码确定第二层的各个CU的最优预测模式，确定第二层的各个CU在最优预测模式下的编码代价(C21，C22，C22，C24)，该编码代价为第二层的各个CU在PU划分方式为2N*2N的情况下的最小编码代价。分别以16*16像素作为预测单元的尺寸对第三层的各个 CU进行预测编码确定第三层的各个CU的最优预测模式，确定第三层的各个CU在最优预测模式下的编码代价(C31，…，C34；C35，…，C38；C39，…，C312；C313，…，C316)，该编码代价为第三层的各个CU在PU划分方式为2N*2N的情况下的最小编码代价。分别以8*8像素作为预测单元的尺寸对第四层的各个CU进行预测编码确定第四层的各个CU的最优预测模式，确定第四层的各个CU在最优预测模式下的编码代价(c1，…，c4；c5，…，c6；…；c57，…，c60；c61，…，c64)，该编码代价为第四层的各个CU在PU划分方式为2N*2N的情况下的最小编码代价。
其中，最优预测模式为某一尺寸的预测单元对应的多种预测模式中能够使得对CU进行预测编码得到的编码代价最小的预测模式，例如，尺寸为64*64的预测单元对应4种预测模式，分别以这4种预测模式对CU进行预测编码，其得到的编码代价分别为c1，c2，c3，c4，并且，c1<c2<c3<c4，则c1对应的预测模式为最优预测模式。
编码代价也可以称之为率失真代价，用于指示在某种预测模式下对CU进行预测编码时预测得到的预测值与参考值之间的差距，其中，编码代价越小，预测值与参考值之间的差距越小，则编码所需的比特数越少，码率越低。具体实现中，可以通过绝对误差和(Sum of Absolute Difference，SAD)算法、绝对变化误差和(Sum of Absolute Transformed Difference，SATD)算法、差值平方和(Sum of Squared Difference，SSD)算法等算法来确定在某种预测模式下对CU进行预测编码时的编码代价。
遍历过程之后是递归过程，递归过程如图3C、3D和3E所示。
首先如图3C所示，分别对第三层的各个CU下的四个子CU的最小编码代价(如c1，c2，c3，c4)进行求和，得到第三层的各个CU的另一个编码代价，该编码代价为第三层的各个CU在PU划分方式为N*N的情况下的最小编码代价。将第三层的各个CU在PU划分方式为2N*2N的情况下的最小编码代价和在PU划分方式为N*N的情况下的最小编码代价进行比较(如将C31和c1+c2+c3+c4比较)，选择两个最小编码代价中较小的一个最小编码代价作为第三层的各个CU的最小编码代价，此最小编码代价为第三层的各个CU的最终的编码代价(c31，c32，…，c316)。
然后如图3D所示，分别对第二层的各个CU下的四个子CU的最小编码代价(如c31，c32，c33，c34)进行求和，得到第二层的各个CU的另一个编码代价，该编码代价为第二层的各个CU在PU划分方式为N*N的情况下的最小编码代价。将第二层的各个CU在PU划分方式为2N*2N的情况下的最小编码代价和在PU划分方式为N*N的情况下的最小编码代价进行比较(如将C21和c31+c32+c33+c34比较)，选择两个最小编码代价中较小的一个最小编码代价作为第二层的各个CU的最小编码代价，此最小编码代价为第三层的各个CU的最终的编码代价(c21，c22，c23，c24)。
接着如图3E所示，对第一层的CTU下的四个CU的最小编码代价(c21，c22，c23，c24)进行求和，得到CTU的另一个编码代价，该编码代价为CTU在PU划分方式为 N*N的情况下的最小编码代价。将CTU在PU划分方式为2N*2N的情况下的最小编码代价(C1)和在PU划分方式为N*N的情况下的最小编码代价(c21+c22+c23+c24)进行比较，选择两个最小编码代价中较小的一个最小编码代价作为CTU的最小编码代价(c0)，此编码代价为CTU的最终的编码代价。
在递归过程之后，根据各层CU的最小编码代价可确定CTU的划分方式。如果CTU的最小编码代价为在PU划分方式为2N*2N的情况下的最小编码代价(c0＝C1)，则确定CTU不划分，确定CTU的树形结构只有一层，即根节点，其对应的划分情况如图4中的A所示。如果CTU的最小编码代价为在PU划分为N*N的情况下的最小编码代价(c0＝c21+c22+c23+c24)，则确定CTU需要划分，确定将CTU划分为四个CU，如图4中的B所示。再分别依次根据第二层的各个CU的最小编码代价确定第二层的各个CU是否需要划分，若该最小编码代价在PU划分方式为2N*2N的情况下的最小编码代价，则不需要划分，若该最小编码代价在PU划分方式为N*N的情况下的最小编码代价，则需要划分。同理，在分别依次根据第三层的各个CU的最小编码代价确定第三层的各个CU是否需要划分。各个CU的划分情况可表示成如图2A所示的树形结构。
由图3A-3E所示的过程可知，完整的遍历递归的算法所涉及的过程为找到使得CTU的编码代价最小的划分策略的过程，通过递归过程所确定的各个编码单元最终的编码代价均被用于确定该划分策略，根据该划分策略可完成对CTU的划分，进而可以根据CTU中划分的CU对该CTU进行编码。
3、PU。PU是在对CU进行预测编码时使用的基本单元，针对于不同的预测模式，有不同的PU划分方式。例如，对于帧内预测模式，PU划分方式有2N*2N和N*N两种，其中，2N*2N的方式是指将CU划分为一个PU进行预测编码，即一个CU包含一个PU；N*N的方式是指将CU划分为四个相同大小的PU进行预测编码，即一个CU包含四个PU。
由图3A-3E所示的过程可知，根据树形结构的编码算法确定的划分策略可使CTU的编码代价尽可能地小，即可以用尽量少的比特对CTU进行编码，以降低码率。虽然降低了码率，但是涉及的计算过程复杂，计算过程具体包括对满四叉树中的每一个CU进行不同的PU划分方式下和不同的预测模式下的编码代价计算、对计算得到的多个编码代价进行比较得到各个CU的最小编码代价，计算量大，复杂度高，需要的时间较多，编码速度慢。
因此，本申请实施例提供一种确定编码单元的编码代价的方法和与其相关的装置，解决在确定CTU的划分策略的过程中计算复杂度大的问题。
下面将结合图5-图12，对本申请实施例提供的确定编码单元的编码代价的方法进行详细介绍。
请参见图5，为本申请实施例提供了一种确定编码单元的编码代价的方法的流程示意图。在本申请的实施例中，该方法通常可由视频编码服务器执行。如图5所示， 本申请实施例的所述方法可以包括以下步骤S101-S104。
S101，确定以帧内预测模式进行预测编码的编码单元。
可以理解的是，编码单元由CTU划分而来，编码单元的尺寸可以为8*8像素、16*16像素、32*32像素、64*64像素中的任意一种尺寸。
本申请实施例中，以帧内预测模式进行预测编码是指以当前帧的编码单元作为参考编码单元进行预测编码，其中，参考编码单元是指已经经过了编码的编码单元。
具体实现中，如果编码单元为I帧(I frame)中的编码单元，由于I帧只采用帧内预测编码，则可以直接确定该编码单元为以帧内预测模式进行预测编码的编码单元。如果编码单元为非I帧(如P帧或B帧)中的编码单元，可参考H.264/AVC标准中帧内模式判决算法确定以帧内预测模式进行预测编码的编码单元，其具体可以为：在对编码单元进行帧间预测编码选择完最佳帧间模式后，确定编码单元的平均码率和平均边界误差，如果编码单元的平均码率大于或等于编码单元的平均边界误差，则确定编码单元为以帧内预测模式进行预测的编码单元。当然，如果编码单元为非I帧中的编码单元，也可以根据其他的方式确定以帧内预测模式进行预测编码的编码单元，本申请实施例不做限制。
可以理解的是，如果所述编码单元的尺寸等于最小编码单元(Small Coding Unit，SCU)的尺寸，即8*8像素，编码单元不能继续划分，则跳过步骤S102～S103，执行步骤S104，将第一编码代价确定为编码单元的第二编码代价。
S102，确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差。
本申请实施例中，编码单元对应的像素梯度信息为根据该编码单元中的像素的图像梯度得到的信息。像素梯度信息可以为根据编码单元中的部分像素的图像梯度得到的信息，像素梯度信息也可以根据编码单元中的所有像素的图像梯度的信息。
除了包括像素梯度方差外，像素梯度信息还可以包括像素梯度平均值、像素梯度均方差等用于衡量图像的平滑程度、纹理复杂程度的参数。其中，像素梯度平均值是指编码单元中的部分像素或所有像素的图像梯度的均值。像素梯度均方差是指编码单元中的部分像素或所有像素的图像梯度的均方差。
具体实现中，可以通过图像梯度算法对确定编码单元中的各个像素的像素梯度信息，从而确定编码单元中的部分像素或者所有像素的图像梯度，进一步可根据编码单元中的部分像素的图像梯度或所有像素的图像梯度确定编码单元对应的像素梯度信息。
图像梯度算法可以为基于一阶导数的图像梯度算法，例如为罗伯特(Roberts)算子、索贝尔(Sobel)算子或Prewitt算子等等。图像梯度算法也可以为基于二阶导数的图像梯度算法，例如为拉普拉斯(Laplace)算子等。
本申请实施例中，如果像素梯度信息为根据编码单元中的部分像素的图像梯度 得到的信息，该部分像素可以是在编码单元的所有像素中随机选取的部分像素。该部分像素也可以是按照一定的选取规则在编码单元的所有像素中选取的部分像素，如编码单元中的部分像素可以是按照预设的采样率选取的部分像素。例如，预设的采样率为90％，编码单元的尺寸为64*64像素，即4096像素，则编码单元中的部分像素为4096像素中3687个像素，可按照从左至右、从上到下的顺序对编码单元中的所有像素进行采样，得到3687个像素。
S103，根据所述像素梯度信息对所述编码单元进行划分预判决。
可以理解的是，如果像素梯度信息为像素梯度方差，则可根据像素梯度方差对编码单元进行划分预判决。如果像素梯度信息还包括像素梯度平均值，则可根据像素梯度方差和像素梯度平均值对编码单元进行划分预判决。如果像素梯度信息还包括像素梯度均方差，则可根据像素梯度方差和像素梯度均方差对编码单元进行划分预判决。其中，根据像素梯度信息对编码单元进行划分预判决的具体实现方式在图8-图12所示的实施例中进行具体介绍，可参见图8-图12所示的实施例中的描述。
S104，如果对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
本申请实施例中，第一编码代价为编码单元在PU划分模式为2N*2N的情况下的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式只有一种，则第一编码代价为在该预测模式下对所述编码单元进行预测编码得到的编码代价，也即该预测模式所对应的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式有多种，则第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的最小编码代价。
举例来进行说明，例如，编码单元的尺寸为32*32像素，尺寸为32*32像素的预测单元对应的预测模式只有预测模式1一种，则第一编码代价为以预测模式1对编码单元进行预测编码所得到的编码代价。尺寸为32*32像素的预测单元对应的预测模式分别有预测模式1、预测模式2、预测模式3以及预测模式4共四种，分别以这4种预测模式对编码单元进行预测编码所得到的编码代价分别为c11，c12，c13，c14，并且，c11<c12<c13<c14，则第一编码代价为c11。
本申请实施例中，第二编码代价指编码单元的最终的编码代价。如果该编码单元为CTU，则CTU的第二编码代价可等同于上述图3E所示的过程中经过递归过程后所确定的CTU的最小编码代价(c0)，由于该第二编码代价为第一编码代价，根据图4所示的划分情况，则可确定该CTU的划分策略为不划分，即直接以CTU当前的尺寸进行编码；如果该编码单元不为CTU并且不为第四层的CU，则编码单元的第二编码代价可等同于上述图3C或图3D所示的过程中经过对CU的在PU划分方式为2N*2N 的情况下的最小编码代价和在PU划分方式为N*N的情况下的最小编码代价进行比较所确定的CU的最小编码代价(c21，c31等)，该第二编码代价被用于计算该编码单元的上一层CU在PU划分方式为N*N的情况下的最小编码代价，进而可以通过递归的方式确定CTU的第二编码代价，然后确定CTU的划分策略。
本申请实施例中，在确定编码单元需要以帧内预测模式进行预测编码以后，通过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，省去了将编码单元划分为子编码单元，然后以子编码单元作为新的编码单元进行预测编码、编码代价计算等计算过程，减少了计算量，提高了编码速率。
请参见图6，为本申请实施例提供了一种确定编码单元的编码代价的方法的流程示意图。在本申请的实施例中，该方法通常可由视频编码服务器执行。如图6所示，本申请实施例的所述方法可以包括以下步骤S201-S208。
S201，确定以帧内预测模式进行预测编码的编码单元。
可以理解的是，编码单元由CTU划分而来，编码单元的尺寸可以为8*8像素、16*16像素、32*32像素、64*64像素中的任意一种尺寸。
本申请实施例中，以帧内预测模式进行预测编码是指以当前帧的编码单元作为参考编码单元进行预测编码，其中，参考编码单元是指已经经过了编码的编码单元。
具体实现中，如果编码单元为I帧(I frame)中的编码单元，由于I帧只采用帧内预测编码，则可以直接确定该编码单元为以帧内预测模式进行预测编码的编码单元。如果编码单元为非I帧(如P帧或B帧)中的编码单元，可参考H.264/AVC标准中帧内模式判决算法确定以帧内预测模式进行预测编码的编码单元，其具体可以为：在对编码单元进行帧间预测编码选择完最佳帧间模式后，确定编码单元的平均码率和平均边界误差，如果编码单元的平均码率大于或等于编码单元的平均边界误差，则确定编码单元为以帧内预测模式进行预测的编码单元。当然，如果编码单元为非I帧中的编码单元，也可以根据其他的方式确定以帧内预测模式进行预测编码的编码单元，本申请实施例不做限制。
可以理解的是，如果所述编码单元的尺寸等于SCU的尺寸，即8*8像素，编码单元不能继续划分，则跳过步骤S202～S206，执行步骤S207，将第一编码代价确定为编码单元的第二编码代价。
S202，确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差。
本申请实施例中，编码单元对应的像素梯度信息为根据该编码单元中的像素的图像梯度得到的信息。像素梯度信息可以为根据编码单元中的部分像素的图像梯度 得到的信息，像素梯度信息也可以根据编码单元中的所有像素的图像梯度的信息。
除了包括像素梯度方差外，像素梯度信息还可以包括像素梯度平均值、像素梯度均方差等用于衡量图像的平滑程度、纹理复杂程度的参数。其中，像素梯度平均值是指编码单元中的部分像素或所有像素的图像梯度的均值。像素梯度均方差是指编码单元中的部分像素或所有像素的图像梯度的均方差。
具体实现中，可以通过图像梯度算法对确定编码单元中的各个像素的像素梯度信息，从而确定编码单元中的部分像素或者所有像素的图像梯度，进一步可根据编码单元中的部分像素的图像梯度或所有像素的图像梯度确定编码单元对应的像素梯度信息。
图像梯度算法可以为基于一阶导数的图像梯度算法，例如为罗伯特(Roberts)算子、索贝尔(Sobel)算子或Prewitt算子等等。图像梯度算法也可以为基于二阶导数的图像梯度算法，例如为拉普拉斯(Laplace)算子等。
本申请实施例中，如果像素梯度信息为根据编码单元中的部分像素的图像梯度得到的信息，该部分像素可以是在编码单元的所有像素中随机选取的部分像素。该部分像素也可以是按照一定的选取规则在编码单元的所有像素中选取的部分像素，如编码单元中的部分像素可以是按照预设的采样率选取的部分像素。例如，预设的采样率为90％，编码单元的尺寸为64*64像素，即4096像素，则编码单元中的部分像素为4096像素中3687个像素，可按照从左至右、从上到下的顺序对编码单元中的所有像素进行采样，得到3687个像素。
S203，根据所述像素梯度信息对所述编码单元进行划分预判决。
可以理解的是，如果像素梯度信息为像素梯度方差，则可根据像素梯度方差对编码单元进行划分预判决。如果像素梯度信息还包括像素梯度平均值，则可根据像素梯度方差和像素梯度平均值对编码单元进行划分预判决。如果像素梯度信息还包括像素梯度均方差，则可根据像素梯度方差和像素梯度均方差对编码单元进行划分预判决。其中，根据像素梯度信息对编码单元进行划分预判决的具体实现方式在图8-图12所示的实施例中进行具体介绍，可参见图8-图12所示的实施例中的描述。
如果对所述编码单元的划分预判决的结果为不划分，执行步骤S207，如果对所述编码单元的划分预判决的结果为划分，执行步骤S204。
S204，将所述编码单元划分为至少两个子编码单元。
具体实现中，可参考现有技术中满四叉树的方式，将编码单元划分为四个子编码单元。当然，也可以将编码单元划分为其他数量的子编码单元，本申请实施例不做限制。
S205，分别确定所述至少两个子编码单元的编码代价。
本申请实施例中，可将至少两个子编码单元确定为新的以帧内预测模式进行预测编码的编码单元，执行步骤S201～S208，以分别确定子编码单元的编码代价。
S206，将所述至少两个子编码单元的编码代价之和与第一编码代价进行比较，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
本申请实施例中，第一编码代价为编码单元在PU划分模式为2N*2N的情况下的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式只有一种，则第一编码代价为在该预测模式下对所述编码单元进行预测编码得到的编码代价，也即该预测模式所对应的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式有多种，则第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的最小编码代价。
举例来进行说明，例如，编码单元的尺寸为32*32像素，尺寸为32*32像素的预测单元对应的预测模式只有预测模式1一种，则第一编码代价为以预测模式1对编码单元进行预测编码所得到的编码代价。尺寸为32*32像素的预测单元对应的预测模式分别有预测模式1、预测模式2、预测模式3以及预测模式4共四种，分别以这4种预测模式对编码单元进行预测编码所得到的编码代价分别为c11，c12，c13，c14，并且，c11<c12<c13<c14，则第一编码代价为c11。
如果所述至少两个子编码单元的编码代价之和大于或等于第一编码代价，执行步骤S207；如果所述至少两个子编码单元的编码代价之和小于所述第一编码代价，执行步骤S208。
S207，将所述第一编码代价确定为所述编码单元的第二编码代价。
S208，将所述至少两个子编码单元的编码代价之和确定为所述编码单元的第二编码代价。
本申请实施例中，第二编码代价指编码单元的最终的编码代价。如果该编码单元为CTU，则CTU的第二编码代价可等同于上述图3E所示的过程中经过递归过程后所确定的CTU的最小编码代价(c0)，由于该第二编码代价为第一编码代价，根据图4所示的划分情况，则可确定该CTU的划分策略为不划分，即直接以CTU当前的尺寸进行编码；如果该编码单元不为CTU并且不为第四层的CU，则编码单元的第二编码代价可等同于上述图3C或图3D所示的过程中经过对CU的在PU划分方式为2N*2N的情况下的最小编码代价和在PU划分方式为N*N的情况下的最小编码代价进行比较所确定的CU的最小编码代价(c21，c31等)，该第二编码代价被用于计算该编码单元的上一层CU在PU划分方式为N*N的情况下的最小编码代价，进而可以通过递归的方式确定CTU的第二编码代价，然后确定CTU的划分策略。
本申请实施例中，在确定编码单元需要以帧内预测模式进行预测编码以后，通过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最 终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，只有在划分预判决的结果为划分的情况下才对编码单元进行划分，然后比较编码单元对应的子编码单元的编码代价和和第一编码代价的大小，由于在确定子编码单元的编码代价时也采用了同样的确定编码代价的方法，因此可以提前结束对子编码单元的进一步划分，通过尽可能的提前结束编码单元的划分，省去了大量计算编码代价的过程，减少了计算量，尽快确定CTU的划分策略，提高编码速度。
下面以将编码单元划分为四个子编码单元为例介绍采用本申请实施例的技术方案确定CTU的划分策略的过程，仍以CTU的尺寸为64*64像素为例，参见图7A-7E。
首先，如图7A所示，将CTU确定为CU，以64*64像素作为预测单元的尺寸对CTU进行预测编码确定CTU的最优预测模式，确定CTU在最优预测模式下的编码代价，即C1，该编码代价为CTU的第一编码代价。根据CTU的像素梯度信息对CTU进行划分判决。
如果对CTU的划分判决的结果为不划分，则将第一编码代价C1确定为CTU的第二编码代价，即c0＝C1，此时，由于CTU的第二编码代价确定，由图4可知，确定CTU的划分策略，即CTU不需要划分，划分终止。由于已经确定了CTU的划分策略，无需像图3B所示的过程一样依次计算第二层CU、第三层CU、第四层CU的编码代价，也无需执行图3C至图3E的过程，省去了大量的计算，提高了编码速度。
如果对CTU的划分判决的结果为划分，则将CTU划分为四个32*32的CU，即第二层的CU。
然后，如图7B所示，分别以32*32像素作为预测单元的尺寸对第二层的各个CU进行预测编码确定各个CU的最优预测模式，并确定第二层的各个CU在最优预测模式下的编码代价，该编码代价为第二层的各个CU的第一编码代价，即C21，C22，C23，C24。分别根据第二层的各个CU的像素梯度信息对第二层的各个CU进行划分预判决。
如果对第二层的目标CU的划分判决的结果为不划分，则将第一编码代价确定为第二层的目标CU的第二编码代价，此时，由于第二层的目标CU的第二编码代价确定，则对第二层的目标CU的划分终止。无需如图3B所示的过程一样依次计算该目标CU下的第三层CU、第四层CU的编码代价，同样省去了大量的计算，提高了编码速度。
如果对第二层的目标CU的划分判决的结果为划分，则将第二层的目标CU划分为四个16*16像素的子CU，即第三层的CU。
如图7C所示，分别以16*16像素作为预测单元的尺寸对第三层的各个CU进行预测编码确定第三层的各个CU的最优预测模式，并确定第三层的各个CU在最优预测模式下的编码代价，该编码代价为第三层的各个CU的第一编码代价。分别根据第三层的各个CU的像素梯度信息对第二层的各个CU进行划分预判决。
如果对第三层的目标CU的划分判决的结果为不划分，则将第三层的目标CU的第一编码代价确定为第三层的目标CU的第二编码代价，此时，由于第三层的目标CU的第二编码代价确定，则对第三层的目标CU的划分终止。由于已经确定了目标CU不划分，无需像图3B所示的过程一样依次计算该目标CU下的第四层CU，也省去了大量的计算，提高了编码速度。
如果对第三层的目标CU的划分判决的结果为划分，则将第三层的目标CU划分为四个8*8像素的CU，即第三层的CU。
如图7C所示，分别以8*8像素作为预测单元的尺寸对第四层的各个CU进行预测编码确定第四层的各个CU的最优预测模式，并确定第四层的各个CU在最优预测模式下的编码代价，该编码代价为第四层的各个CU的第一编码代价。
如果形成了树形结构，则根据从下往上的方式依次确定第三层、第二层以及第一层中的CU中还未确定第二编码代价的CU的第二编码代价，将CU的两个编码代价(CU下的四个子CU的第二编码代价之和，以及CU的第一编码代价)中较小的一个确定为CU的第二编码代价。
例如，按照通过图7A-7C的过程所确定的树形结构如图7D所示，其中，通过图7A的过程确定编号为1的CU(即CTU)需要划分。通过图7B的过程确定编号为3和4的CU需要划分，编码为2和5的CU不需要划分，那么编码为2和5的CU的第二编码代价确定，为他们的第一编码代价，编号为3和4的CU的第二编码代价还未确定。通过7C的过程，确定编号为7～13的CU不需要划分，编码为6的CU需要划分，那么编号为7～13的CU的第二编码代价确定，为他们的第一编码代价，编号为6的CU的第二编码代价还未确定。
然后，按照从下往上的方式依次确定还未确定第二编码单价的CU的第二编码代价，即确定编号为6、3、4、1的CU的第二编码代价，如图7E所示，首先，确定第三层的CU中编号为6的CU的第二编码代价，然后，再确定第二层的CU中编码为3和4的CU的第二编码代价，最后，确定编号为1的CU即CTU的第二编码代价。
在确定了各层的CU的第二编码代价之后，根据各层的CU的最小编码代价确定CTU的划分方式，如果CTU的第二编码代价为CTU的第一编码代价，则确定CTU不划分，确定CTU的树形结构只有一层，即根节点，其对应的划分情况如图4中的A所示。如果CTU的第二编码代价为第二层的CU的第二编码代价之和，确定CTU需要划分，确定将CTU划分为四个CU，如图4中的B所示。在确定CTU划分为四个CU的情况下，进一步可根据第二层的CU的第二编码代价确定第二层的划分方式，依次类推，可确定CTU最终的划分情况。
由图7A-7E可知，只有在最极端的情况下，即对上三层(第一层、第二层以及第三层)的每个CU的划分预判决结果均为划分的情况下，才需要按照图3所示的方式进行遍历和递归，以确定CTU的划分策略，只要上三层中有一个CU的划分预判决 结果为不划分，那么就可以不计算该CU下的CU的最小编码代价，由于确定一个CU的最小编码代价都需要经过比较复杂的计算，因此，本申请实施例的方案可省去大量计算，节省计算的时间，提高编码速度。
请参见图8，为本申请实施例提供了一种确定编码单元的编码代价的方法的流程示意图，如图8所示，本申请实施例的所述根据所述像素梯度信息对所述编码单元进行划分预判决可以包括以下步骤S301-S303。
S301，确定所述编码单元对应的方差阈值。
本申请实施例中，编码单元对应的方差阈值有两种情况：一种是该方差阈值为一个固定阈值；另一种是该方差阈值与编码单元对应的量化参数正相关，即编码单元对应的量化参数越大，则方差阈值越大。
量化参数反映空间细节压缩情况，如果量化参数小，大部分细节都会保留，如果量化参数增大，则细节丢失，码率降低，即量化参数和码率负相关，也即方差阈值与码率负相关。量化参数的取值可以由编码器决定。量化参数的取值范围为0～52，在量化参数大于0的情况下，编码单元对应的方差阈值可以等于编码单元对应的量化参数乘以X，X为大于或等于1的数值。
S302，如果所述编码单元对应的像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分。
编码单元对应的像素梯度方差用于反映编码单元中的图像的纹理和物体边界情况。编码单元的像素梯度方差较小，说明编码单元的图像可能是杂乱纹理，进一步划分编码单元对提高图像质量帮助不大，由于是杂乱纹理，人眼对该编码单元不敏感，所以不需要进一步划分。
S303，如果所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分。
像素梯度方差较大，说明编码单元可能包含物体边界，进一步划分可能有助于提高图像质量，进一步划分后该编码单元对应的子编码单元的相似性更高，有助于提高压缩效率。
本申请实施例中，根据编码单元的像素梯度方差完成对编码单元是否划分的预判决，由于编码单元的像素梯度方差反映了编码单元中的图像的纹理和边界等情况，根据编码单元的像素梯度方差可提前结束对编码单元的划分，节省计算量，同时能兼顾压缩效率和图像质量。
在另一种可能的实现方式中，本申请实施例所述的像素梯度信息还可以包括像素梯度平均值。请参见图9，为本申请实施例提供了一种确定编码单元的编码代价的方法的流程示意图，如图9所示。本申请实施例的所述根据所述像素梯度信息对所述编码单元进行划分预判决可以包括以下步骤S401-S406。
S401，确定所述编码单元对应的第一阈值、所述编码单元对应的第二阈值以及 所述编码单元对应的方差阈值，所述第二阈值大于所述第一阈值。
本申请实施例中，编码单元对应的方差阈值有两种情况：一种是该方差阈值为一个固定阈值；另一种是该方差阈值与编码单元对应的量化参数正相关，即编码单元对应的量化参数越大，则方差阈值越大。
量化参数反映空间细节压缩情况，如果量化参数小，大部分细节都会保留，如果量化参数增大，则细节丢失，码率降低，即量化参数和码率负相关，也即方差阈值与码率负相关。量化参数的取值可以由编码器决定。量化参数的取值范围为0～52，在量化参数大于0的情况下，编码单元对应的方差阈值可以等于编码单元对应的量化参数乘以X，X为大于或等于1的数值。
本申请实施例中，第一阈值和第二阈值均为均值阈值。如果编码单元对应的第一阈值不为零，编码单元对应的第二阈值可以为编码单元对应的第一阈值的固定倍数，即编码单元对应的第二阈值为编码单元对应的第一阈值与第二系数之积，第二系数大于1。如果编码单元对应第一阈值为零，则编码单元对应的第二阈值可以为编码单元对应的阈值与目标均值阈值之和，目标均值阈值为一个固定阈值。
在一种可能的实现方式中，该编码单元对应的第一阈值可以为一个固定阈值。如果编码单元对应的第一阈值为固定阈值，则第一阈值与编码单元的尺寸负相关。
在另一种可能的实现方式中，编码单元对应的第一阈值也可以与编码单元周围的编码单元的像素梯度的平均值相关。具体实现中，如图10所示，所述确定所述编码单元对应的第一阈值，包括以下步骤S501-S503。
S501，确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合。
可以理解的，邻居编码单元为在编码单元周围且与编码单元相接的编码单元。该邻居编码单元可以为已经进行了预测编码的编码单元。邻居编码单元的尺寸可以与编码单元的尺寸相同，邻居编码单元的尺寸也可以与编码单元的尺寸不同。
下面举例来对邻居编码单元进行说明，如图11所示，如果编码单元为图11中所示的CU1，CU7、CU17、CU19、CU20、CU23有边缘有CU1的边缘重合，CU6、CU10、CU22以及CU24的顶点与CU1的顶点存在重合，则CU6、CU7、CU10、CU17、CU19、CU20、CU22、CU23以及CU24均为CU1的邻居编码单元，其余的CU没有边缘或顶点与CU1重合，因此，其余的CU不为编码单元的邻居编码单元。
进一步地，邻居编码单元也可以是在编码单元周围的并且在特定方位上的编码单元。例如，邻居编码单元为在编码单元周围的且处于左上方位、正上方位、右上方位和正左方位的邻居编码单元，即在图11中，CU6、CU7、CU10、CU17、CU19是CU1的邻居编码单元。
如果将邻居编码单元限定为与编码单元的尺寸相同的编码单元，仍以CU1为编 码单元为例，那么CU1的邻居编码单元可以为：CU3、CU4、CU5、CU6组合形成的编码单元，CU7，CU8、CU9、CU10、CU11组合形成的编码单元，CU16、CU17、CU18、CU19组合形成的编码单元，CU20，CU22，CU23，CU3、CU24、CU25、CU26组合形成的编码单元。
需说明的是，上述对邻居编码单元的举例仅作为示例，不对本申请实施例进行限制。
S502，如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码。
具体实现中，目标数量可以设置为4，5，6等值。
本申请实施例中，如果编码单元为I帧中的编码单元，则在步骤S501中确定的邻居编码单元均为第一邻居编码单元；如果编码单元为非I帧中的编码单元，则可以按照步骤S101所描述的方式在多个邻居编码单元中确定第一邻居编码单元。
在一种具体实现中，如图12所示，所述根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值，包括以下步骤S601-S603。
S601，在所述第一邻居编码单元中查找第二邻居编码单元，所述第二邻居编码单元的尺寸大于或等于所述编码单元的尺寸。
例如，在图11所示的邻居编码单元中，CU7、CU10、CU17、CU20、CU22、以及CU23均为以帧内预测模式进行预测编码的编码单元，并且，CU7、CU20、CU22以及CU23的尺寸均等于CU1的尺寸，则CU7、CU20、CU22以及CU23均为第二邻居编码单元。
S602，如果存在所述第二邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商。
第一系数可以为大于1的系数。
具体实现中，可根据上述步骤S102介绍的确定编码单元对应的像素梯度信息的方式确定各个邻居编码单元对应的像素梯度平均值。
例如，第二邻居编码单元为编码单元2、编码单元3、编码单元4、编码单元5，其对应的像素梯度平均值分别为像素梯度平均值1、像素梯度平均值2、像素梯度平均值3、像素梯度平均值4，则目标平均值为(像素梯度平均值1+像素梯度平均值2+像素梯度平均值3+像素梯度平均值4)/4。
S603，如果不存在所述第二邻居单元，则确定所述编码单元对应的第一阈值为零。
可以理解的是，第一阈值为零是进行梯度计算所能得到的最小值。
需要说明的是，步骤S502的方式是针对邻居编码单元的尺寸可以与编码单元的尺寸相同，邻居编码单元的尺寸也可以与编码单元的尺寸不同的情况所设计的确定所述编码单元对应的第一阈值的方式。在另一种可能的实现方式中，如果邻居编码单元特指尺寸与编码单元的尺寸相同的编码单元，那么步骤S502还可替换为：如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的划分情况以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码的编码单元。
具体实现中，所述根据所述第一邻居编码单元的划分情况以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值，包括：在所述第一邻居编码单元中查找第三邻居编码单元，所述第三邻居编码单元为未被划分的第一邻居编码单元；如果存在所述第三邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商。
S503如果所述多个邻居编码单元中所述第一邻居编码单元的数量小于所述目标数量，则将所述编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。
编码单元的尺寸对应的均值阈值为一个固定阈值，其中，编码单元的尺寸越大，则编码尺寸对应的均值阈值越小。具体实现中，可设计三个均值阈值，以分别对应尺寸为16*16像素的编码单元、尺寸为32*32像素的编码单元以及尺寸为64*64像素的编码单元。
S402，将所述编码单元对应的像素梯度平均值与所述第一阈值进行比较。
如果像素梯度平均值小于第一阈值，则执行步骤S405；如果像素梯度平均值大于或等于第一阈值，则执行步骤S403。
编码单元对应的像素梯度平均值用于反映编码单元中的图像内容的平滑情况。编码单元的像素梯度平均值较小，说明编码单元中的图像内容比较平滑，不需要继续划分。
S403，将所述像素梯度平均值与所述第二阈值进行比较。
如果像素梯度平均值小于第二阈值，则执行步骤S404；如果像素梯度平均值大于或等于第二阈值，则执行步骤S406。
像素梯度平均值较大，说明编码单元中的图像的内容不平滑，需要继续划分的可能性大。
S404，将所述编码单元对应的像素梯度方差与所述方差阈值进行比较。
如果像素梯度方差小于方差阈值，则执行步骤S405，如果像素梯度方差大于或等于所述方差阈值，则执行步骤S406。
编码单元对应的像素梯度方差用于反映编码单元中的图像的纹理和物体边界情况。编码单元的像素梯度方差较小，说明编码单元的图像可能是杂乱纹理，进一步划分编码单元对提高图像质量帮助不大，由于是杂乱纹理，人眼对该编码单元不敏感，所以不需要进一步划分。像素梯度方差较大，说明编码单元可能包含物体边界，进一步划分可能有助于提高图像质量，进一步划分后该编码单元对应的子编码单元的相似性更高，有助于提高压缩效率。
S405，确定对所述编码单元的划分预判决的结果为不划分。
S406，确定对所述编码单元的划分预判决的结果为划分。
本申请实施例中，通过结合编码单元的像素梯度方差和像素梯度平均值完成对编码单元是否划分的预判决，编码单元的像素方差反映了编码单元中的图像的纹理和边界等情况，编码单元的像素梯度平均值反映了编码单元中的图像的内容平滑情况，综合分析编码单元中的图像内容，可以实现对编码单元是否划分的准确判决，节省计算量，同时兼顾了压缩效率和图像质量。
下面将结合图13-图16，对本申请实施例提供的确定编码单元的编码代价的装置进行详细介绍。所述确定编码单元的编码代价的装置可以是具备计算处理功能的终端。需要说明的是，图13-图16所示的装置，用于执行本申请图5-图12所示实施例的方法，为了便于说明，仅示出了与本申请实施例相关的部分，具体技术细节未揭示的，请参照本申请图5-图12所示的实施例。
请参见图13，为本申请实施例提供了一种确定编码单元的编码代价的装置的结构示意图。如图13所示，本申请实施例的所述确定编码单元的编码代价的装置70可以包括：编码单元确定模块71、梯度信息确定模块72、预判决模块73和代价确定模块74。
编码单元确定模块71，用于确定以帧内预测模式进行预测编码的编码单元；
可以理解的是，编码单元由CTU划分而来，编码单元的尺寸可以为8*8像素、16*16像素、32*32像素、64*64像素中的任意一种尺寸。
本申请实施例中，以帧内预测模式进行预测编码是指以当前帧的编码单元作为参考编码单元进行预测编码，其中，参考编码单元是指已经经过了编码的编码单元。
具体实现中，如果编码单元为I帧(I frame)中的编码单元，由于I帧只采用帧内预测编码，则编码单元确定模块71可以直接确定该编码单元为以帧内预测模式进行预测编码的编码单元。如果编码单元为非I帧(如P帧或B帧)中的编码单元，可参考H.264/AVC标准中帧内模式判决算法确定以帧内预测模式进行预测编码的编码单元，其具体可以为：编码单元确定模块71在对编码单元进行帧间预测编码选择完最佳帧间模式后，确定编码单元的平均码率和平均边界误差，如果编码单元的平均码率大于或等于编码单元的平均边界误差，则确定编码单元为以帧内预测模式进行预测的编码单元。当然，如果编码单元为非I帧中的编码单元，编码单元确定模块71也 可以根据其他的方式确定以帧内预测模式进行预测编码的编码单元，本申请实施例不做限制。
可以理解的是，如果所述编码单元的尺寸等于SCU的尺寸，即8*8像素，编码单元不能继续划分，则代价确定模块74直接将第一编码代价确定为编码单元的第二编码代价。
梯度信息确定模块72，用于确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
本申请实施例中，编码单元对应的像素梯度信息为梯度信息确定模块72根据该编码单元中的像素的图像梯度得到的信息。像素梯度信息可以为梯度信息确定模块72根据编码单元中的部分像素的图像梯度得到的信息，像素梯度信息也可以为梯度信息确定模块72根据编码单元中的所有像素的图像梯度的信息。
除了包括像素梯度方差外，像素梯度信息还可以包括像素梯度平均值、像素梯度均方差等用于衡量图像的平滑程度、纹理复杂程度的参数。其中，像素梯度平均值是指编码单元中的部分像素或所有像素的图像梯度的均值。像素梯度均方差是指编码单元中的部分像素或所有像素的图像梯度的均方差。
具体实现中，梯度信息确定模块72可以通过图像梯度算法对确定编码单元中的各个像素的像素梯度信息，从而确定编码单元中的部分像素的图像梯度，或者，确定编码单元中的所有像素的图像梯度，进一步可根据编码单元中的部分像素的图像梯度或所有像素的图像梯度确定编码单元对应的像素梯度信息。
图像梯度算法可以为基于一阶导数的图像梯度算法，例如为罗伯特(Roberts)算子、索贝尔(Sobel)算子或Prewitt算子等等。图像梯度算法也可以为基于二阶导数的图像梯度算法，例如为拉普拉斯(Laplace)算子等。
本申请实施例中，如果像素梯度信息为根据编码单元中的部分像素的图像梯度得到的信息，该部分像素可以是梯度信息确定模块72在编码单元的所有像素中随机选取的部分像素。编码单元中的部分像素也可以是梯度信息确定模块72按照一定的选取规则在编码单元的所有像素中选取的部分像素，如编码单元中的部分像素可以是按照预设的采样率选取的部分像素。例如，预设的采样率为90％，编码单元的尺寸为64*64像素，即4096像素，则编码单元中的部分像素为4096像素中3687个像素，梯度信息确定模块72可按照从左至右、从上到下的顺序对编码单元中的所有像素进行采样，得到3687个像素。
预判决模块73，用于根据所述像素梯度信息对所述编码单元进行划分预判决；
可以理解的是，如果像素梯度信息为像素梯度方差，则预判决模块73可根据像素梯度方差对编码单元进行划分预判决。如果像素梯度信息还包括像素梯度平均值，则预判决模块73可根据像素梯度方差和像素梯度平均值对编码单元进行划分预判 决。如果像素梯度信息还包括像素梯度均方差，则预判决模块73可根据像素梯度方差和像素梯度均方差对编码单元进行划分预判决。
可选的，如图14所示，所述预判决模块73可以包括阈值确定单元731，结果确定单元732。
在一种具体实现方式中，阈值确定单元731，用于确定所述编码单元对应的方差阈值；结果确定单元732，用于如果所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；所述结果确定单元732还用于如果所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分。
在另一种具体实现方式中，所述像素梯度信息还包括像素梯度平均值，阈值确定单元731，用于确定所述编码单元对应的第一阈值、所述编码单元对应的第二阈值以及所述编码单元对应的方差阈值，所述第二阈值大于所述第一阈值；结果确定单元732，用于如果所述像素梯度平均值小于所述第一阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者所述结果确定单元732还用于如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者所述结果确定单元732还用于如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分；或者所述结果确定单元732还用于如果所述述像素梯度平均值大于或等于所述第二阈值，则确定对所述编码单元的划分预判决的结果为划分。
本申请实施例中，编码单元对应的方差阈值有两种情况：一种是该方差阈值为一个固定阈值；另一种是该方差阈值与编码单元对应的量化参数正相关，即编码单元对应的量化参数越大，则方差阈值越大。
量化参数反映空间细节压缩情况，如果量化参数小，大部分细节都会保留，如果量化参数增大，则细节丢失，码率降低，即量化参数和码率负相关，也即方差阈值与码率负相关。量化参数的取值可以由编码器决定。量化参数的取值范围为0～52，在量化参数大于0的情况下，阈值确定单元731可以将编码单元对应的量化参数乘以X确定为编码单元对应的方差阈值，X为大于或等于1的数值；阈值确定单元731也可以将固定方差阈值确定为编码单元对应的方差阈值。
本申请实施例中，第一阈值和第二阈值均为均值阈值。如果编码单元对应的第一阈值不为零，编码单元对应的第二阈值可以为编码单元对应的第一阈值的固定倍数，即阈值确定单元731可以确定编码单元对应的第二阈值为编码单元对应的第一阈值与第二系数之积，第二系数大于1。如果编码单元对应第一阈值为零，则阈值确定单元731可以确定编码单元对应的第二阈值可以为编码单元对应的阈值与目标均值 阈值之和，目标均值阈值为一个固定阈值。
在一种可能的实现方式中，该编码单元对应的第一阈值可以为一个固定阈值，阈值确定单元731可以将该编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。如果编码单元对应的第一阈值为固定阈值，则第一阈值与编码单元的尺寸负相关。
在另一种可能的实现方式中，编码单元对应的第一阈值也可以与编码单元周围的编码单元的像素梯度的平均值相关。在确定所述编码单元对应的第一阈值时，所述阈值确定单元731可以具体用于：
确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合；如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码；如果所述多个邻居编码单元中所述第一邻居编码单元的数量小于所述目标数量，则将所述编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。
本申请实施例中，邻居编码单元可以为在编码单元周围且与编码单元相接的编码单元。该邻居编码单元可以为已经进行了预测编码的编码单元。邻居编码单元的尺寸可以与编码单元的尺寸相同，邻居编码单元的尺寸也可以与编码单元的尺寸不同。
下面举例来对邻居编码单元进行说明，如图11所示，如果编码单元为图11中所示的CU1、CU7、CU17、CU19、CU20、CU23有边缘有CU1的边缘重合，CU6、CU10、CU22以及CU24的顶点与CU1的顶点存在重合，则阈值确定单元731确定CU6、CU7、CU10、CU17、CU19、CU20、CU22、CU23以及CU24均为CU1的邻居编码单元，其余的CU没有边缘或顶点与CU1重合，因此，阈值确定单元731确定其余的CU不为编码单元的邻居单元。
进一步地，邻居编码单元也可以是在编码单元周围的并且在特定方位上的编码单元。例如，邻居编码单元为在编码单元周围的且处于左上方位、正上方位、右上方位和正左方位的邻居编码单元，此时，阈值确定单元731确定CU6、CU7、CU10、CU17、CU19是CU的邻居编码单元。
如果将邻居编码单元限定为与编码单元的尺寸相同的编码单元，仍以CU1为编码单元为例，阈值确定单元731确定以下编码单元为邻居编码单元：CU3、CU4、CU5、CU6组合形成的编码单元，CU7、CU8、CU9、CU10、CU11组合形成的编码单元，CU16、CU17、CU18、CU19组合形成的编码单元，CU20、CU22、CU23、CU3、CU24、CU25、CU26组合形成的编码单元。
具体实现中，目标数量可以设置为4、5、6等值。
编码单元的尺寸对应的均值阈值为一个固定阈值，其中，编码单元的尺寸越大，则编码尺寸对应的均值阈值越小。具体实现中，可设计三个均值阈值，以分别对应尺寸为16*16像素的编码单元、尺寸为32*32像素的编码单元以及尺寸为64*64像素的编码单元。
本申请实施例中，如果编码单元为I帧中的编码单元，则阈值确定单元731确定所有的邻居编码单元均为第一邻居编码单元；如果编码单元为非I帧中的编码单元，则阈值确定单元731通过编码单元确定模块71在多个邻居编码单元中确定第一邻居编码单元。
具体实现中，在根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值时，所述阈值确定单元731可以具体用于：在所述第一邻居编码单元中查找第二邻居编码单元，所述第二邻居编码单元的尺寸大于或等于所述编码单元的尺寸。如果存在所述第二邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商。如果不存在所述第二邻居单元，则确定所述编码单元对应的第一阈值为零。
例如，在图11所示的邻居编码单元中，CU7、CU10、CU17、CU20、CU22、以及CU23均为以帧内预测模式进行预测编码的编码单元，并且，CU7、CU20、CU22以及CU23的尺寸均等于CU1的尺寸，则阈值确定单元731确定CU7、CU20、CU22以及CU23均为第二邻居编码单元。
具体实现中，阈值确定单元731可通过梯度信息确定模块72确定各个邻居编码单元对应的像素梯度平均值，进而确定目标平均值。
例如，第二邻居编码单元为编码单元2、编码单元3、编码单元4、编码单元5，其对应的像素梯度平均值分别为像素梯度平均值1、像素梯度平均值2、像素梯度平均值3、像素梯度平均值4，则阈值确定单元731确定目标平均值为(像素梯度平均值1+像素梯度平均值2+像素梯度平均值3+像素梯度平均值4)/4。
可以理解的是，第一阈值为零是进行梯度计算所能得到的最小值。
可选的，在确定所述编码单元对应的第一阈值时，阈值确定单元731也可以具体用于：确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合；如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的划分情况以及所述第一邻居编码单元的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码的编码单 元。
具体实现中，阈值确定单元731可以具体用于：在所述第一邻居编码单元中查找第三邻居编码单元，所述第三邻居编码单元为未被划分的第一邻居编码单元；如果存在所述第三邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商。
代价确定模块74，用于如果所述预判决模块73对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
本申请实施例中，第一编码代价为编码单元在PU划分模式为2N*2N的情况下的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式只有一种，则第一编码代价为在该预测模式下对所述编码单元进行预测编码得到的编码代价，也即该预测模式所对应的编码代价。如果预测单元的尺寸为编码单元当前的尺寸所对应的预测模式有多种，则第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的最小编码代价。
举例来进行说明，例如，编码单元的尺寸为32*32像素，尺寸为32*32像素的预测单元对应的预测模式只有预测模式1一种，则第一编码代价为以预测模式1对编码单元进行预测编码所得到的编码代价。尺寸为32*32像素的预测单元对应的预测模式分别有预测模式1、预测模式2、预测模式3以及预测模式4共四种，代价确定模块74分别以这4种预测模式对编码单元进行预测编码所得到的编码代价分别为c11，c12，c13，c14，并且，c11<c12<c13<c14，则代价确定模块74确定第一编码代价为11。
本申请实施例中，第二编码代价指编码单元的最终的编码代价。如果该编码单元为CTU，则CTU的第二编码代价可等同于上述图3E所示的过程中经过递归过程后所确定的CTU的最小编码代价(c0)，由于该第二编码代价为第一编码代价，根据前述介绍的根据CTU的最小编码代价确定划分策略的介绍，则可确定该CTU的划分策略为不划分，即直接以CTU当前的尺寸进行编码；如果该编码单元不为CTU并且不为第四层的CU，则编码单元的第二编码代价可等同于上述图3C或图3D所示的过程中经过对CU的在PU划分方式为2N*2N的情况下的最小编码代价和在PU划分方式为N*N的情况下的最小编码代价进行比较所确定的CU的最小编码代价(c21，c31等)，该第二编码代价被用于计算该编码单元的上一层节点在PU划分方式为N*N的情况下的最小编码代价，进而可以通过递归的方式确定CTU的第二编码代价，然后确定CTU的划分策略。
本申请实施例中，在确定编码单元需要以帧内预测模式进行预测编码以后，通 过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，省去了将编码单元划分为子编码单元，然后以子编码单元作为新的编码单元进行预测编码、编码代价计算等计算过程，减少了计算量，提高了编码速率。
请参见图15，为本申请实施例提供了一种确定编码单元的编码代价的装置的结构示意图，所述装置可以包括上述图13对应实施例中的编码单元确定模块71、梯度信息确定模块72、预判决模块73和代价确定模块74，进一步地，所述装置还可以包括划分模块75，其中：
划分模块75，用于如果所述预判决模块对所述编码单元的划分预判决的结果为划分，则将所述编码单元划分为至少两个子编码单元；
具体实现中，可参考现有技术中满四叉树的方式，划分模块75将编码单元划分为四个子编码单元。当然，划分模块75也可以将编码单元划分为其他数量的子编码单元，本申请实施例不做限制。
所述编码单元确定模块71还用于将所述至少两个子编码单元作为新的以帧内预测模式进行预测编码的编码单元，所述梯度信息确定模块72还用于分别确定所述至少两个子编码单元对应的像素梯度信息，所述预判决模块73还用于根据所述像素梯度信息分别对所述至少两个编码子单元进行划分预判决，所述代价确定模块74还用于分别确定所述至少两个子编码单元的编码代价；
具体实现中，梯度信息确定模块72可按照确定编码单元对应的像素梯度信息的方式分别确定所述至少两个子编码单元对应的像素梯度信息；预判决模块73可按照根据编码单元对应的像素梯度信息对编码单元进行划分预判决的方式分别对所述至少两个编码子单元进行划分预判决；代价确定模块74可按照确定编码单元的编码代价的方式分别确定所述至少两个子编码单元的编码代价，具体可参考前述图13对应的实施例中的相应描述。
所述代价确定模块74还用于如果所述至少两个子编码单元的编码代价之和大于或等于所述第一编码代价，则将所述第一编码代价确定为所述编码单元的第二编码代价；
所述代价确定模块74还用于如果所述至少两个子编码单元的编码代价之和大于所述第一编码代价，则将所述至少两个子编码单元的编码代价之和确定为所述编码单元的第二编码代价。
本申请实施例中，在确定编码单元需要以帧内预测模式进行预测编码以后，通过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸 作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，只有在划分预判决的结果为划分的情况下才对编码单元进行划分，然后比较编码单元对应的子编码单元的编码代价和和第一编码代价的大小，由于在确定子编码单元的编码代价时也采用了同样的确定编码代价的方法，因此可以提前结束对子编码单元的进一步划分，通过尽可能的提前结束编码单元的划分，省去了大量计算编码代价的过程，减少了计算量，尽快确定CTU的划分策略，提高编码速度。
请参见图16，为本申请实施例提供了一种终端的结构示意图。如图16所示，所述终端80可以包括：至少一个处理器801，例如CPU，输入输出接口803，存储器804，至少一个通信总线802。其中，通信总线802用于实现这些组件之间的连接通信。其中，输入输出接口803可以包括标准的有线接口(如USB接口)、无线接口(如WI-FI接口)。输入输出接口803可选的可以包括用户接口(如显示屏、键盘、鼠标等)。存储器804可以是高速RAM存储器，也可以是非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器804可选的还可以是至少一个位于远离前述处理器801的存储装置。如图15所示，作为一种计算机存储介质的存储器804中可以包括操作系统、网络通信模块、用户接口模块、确定编码代价的程序以及图像。
在图15所示的终端80中，输入输出接口803用于获取图像并发送给终端。可选的，输入输出接口还可以用于输出处理器处理后的图像。处理器801可以对图像进行划分，得到编码单元。进一步地，处理器801可以用于调用存储器804中存储的确定编码代价的程序，并具体执行以下操作：
确定以帧内预测模式进行预测编码的编码单元；
在所述编码单元确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
根据所述像素梯度信息对所述编码单元进行划分预判决；
如果对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
在一个实施例中，所述处理器801还可以调用存储器804中存储的确定编码代价的程序执行以下操作：
如果对所述编码单元的划分预判决的结果为划分，则将所述编码单元划分为至少两个子编码单元；
将所述至少两个子编码单元作为新的以帧内预测模式进行预测编码的编码单 元，执行所述确定编码单元的编码代价的方法，分别确定所述至少两个子编码单元的编码代价；
如果所述至少两个子编码单元的编码代价之和大于或等于所述第一编码代价，则将所述第一编码代价确定为所述编码单元的第二编码代价；
如果所述至少两个子编码单元的编码代价之和小于所述第一编码代价，则将所述至少两个子编码单元的编码代价之和确定为所述编码单元的第二编码代价。
在一个实施例中，所述处理器801在执行根据所述像素梯度信息对所述编码单元进行划分预判决时，具体执行以下操作：
确定所述编码单元对应的方差阈值；
如果所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；
如果所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分。
在一个实施例中，所述像素梯度信息还包括像素梯度平均值，所述像素梯度平均值为编码单元内的至少部分像素的图像梯度的平均值；所述处理器801在执行根据所述像素梯度信息对所述编码单元进行划分预判决时，具体执行以下操作：
确定所述编码单元对应的第一阈值、所述编码单元对应的第二阈值以及所述编码单元对应的方差阈值，所述第二阈值大于所述第一阈值；
如果所述像素梯度平均值小于所述第一阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者
如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者
如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分；或者
如果所述述像素梯度平均值大于或等于所述第二阈值，则确定对所述编码单元的划分预判决的结果为划分。
在一个实施例中，所述处理器801在执行确定所述编码单元对应的第一阈值时，具体执行以下操作：
确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合；
如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平 均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码；
如果所述多个邻居编码单元中所述第一邻居编码单元的数量小于所述目标数量，则将所述编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。
在一个实施例中，所述处理器801在执行根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平均值确定所述编码单元对应的第一阈值时，具体执行以下操作：
在所述第一邻居编码单元中查找第二邻居编码单元，所述第二邻居编码单元的尺寸大于或等于所述编码单元的尺寸；
如果存在所述第二邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商；
如果不存在所述第二邻居单元，则确定所述编码单元对应的第一阈值为零。
在一个实施例中，所述编码单元对应的第二阈值为所述编码单元对应的第一阈值与第二系数之积。
在一个实施例中，所述编码单元对应的方差阈值与所述编码单元对应的量化参数正相关。
本申请实施例中，在确定编码单元需要以帧内预测模式进行预测编码以后，通过确定该编码单元的像素梯度信息并根据该编码单元的像素梯度信息对编码单元进行划分预判决，在划分预判决的结果为不划分的情况下，将以编码单元当前的尺寸作为预测单元的尺寸对编码单元进行预测编码得到的编码代价确定为编码单元的最终的编码代价，进而可以确定编码单元不需要划分，提前结束对该编码单元的划分，只有在划分预判决的结果为划分的情况下才对编码单元进行划分，然后比较编码单元对应的子编码单元的编码代价和和第一编码代价的大小，由于在确定子编码单元的编码代价时也采用了同样的确定编码代价的方法，因此可以提前结束对子编码单元的进一步划分，通过尽可能的提前结束编码单元的划分，省去了大量计算编码代价的过程，减少了计算量，尽快确定CTU的划分策略，提高编码速度。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本发明较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (15)

  1. 一种确定编码单元的编码代价的方法，由服务器执行，其特征在于，包括：
    确定以帧内预测模式进行预测编码的编码单元；
    确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
    根据所述像素梯度信息对所述编码单元进行划分预判决；
    如果对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    如果对所述编码单元的划分预判决的结果为划分，则将所述编码单元划分为至少两个子编码单元；
    将所述至少两个子编码单元作为新的以帧内预测模式进行预测编码的编码单元，执行所述确定编码单元的编码代价的方法，分别确定所述至少两个子编码单元的编码代价；
    如果所述至少两个子编码单元的编码代价之和大于或等于所述第一编码代价，则将所述第一编码代价确定为所述编码单元的第二编码代价；
    如果所述至少两个子编码单元的编码代价之和小于所述第一编码代价，则将所述至少两个子编码单元的编码代价之和确定为所述编码单元的第二编码代价。
  3. 根据权利要求1所述的方法，其特征在于，所述根据所述像素梯度信息对所述编码单元进行划分预判决，包括：
    确定所述编码单元对应的方差阈值；
    如果所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；
    如果所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分。
  4. 根据权利要求1所述的方法，其特征在于，所述像素梯度信息还包括像素梯度平均值，所述像素梯度平均值为编码单元内的至少部分像素的图像梯度的平均值；
    所述根据所述像素梯度信息对所述编码单元进行划分预判决，包括：
    确定所述编码单元对应的第一阈值、所述编码单元对应的第二阈值以及所述编码单元对应的方差阈值，所述第二阈值大于所述第一阈值；
    如果所述像素梯度平均值小于所述第一阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差小于所述方 差阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分；或者如果所述述像素梯度平均值大于或等于所述第二阈值，则确定对所述编码单元的划分预判决的结果为划分。
  5. 根据权利要求4所述的方法，其特征在于，所述确定所述编码单元对应的第一阈值，包括：
    确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合；
    如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码；
    如果所述多个邻居编码单元中所述第一邻居编码单元的数量小于所述目标数量，则将所述编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平均值确定所述编码单元对应的第一阈值，包括：
    在所述第一邻居编码单元中查找第二邻居编码单元，所述第二邻居编码单元的尺寸大于或等于所述编码单元的尺寸；
    如果存在所述第二邻居编码单元，则将第一系数与目标平均值之积确定为所述编码单元对应的第一阈值，所述目标平均值为各个第二邻居编码单元对应的像素梯度平均值的总和与所述第二邻居编码单元的数量之商；
    如果不存在所述第二邻居单元，则确定所述编码单元对应的第一阈值为零。
  7. 根据权利要求4-6任一项所述的方法，其特征在于，所述编码单元对应的第二阈值为所述编码单元对应的第一阈值与第二系数之积。
  8. 根据权利要求3或4所述的方法，其特征在于，所述编码单元对应的方差阈值与所述编码单元对应的量化参数正相关。
  9. 一种确定编码单元的编码代价的装置，其特征在于，包括：
    编码单元确定模块，用于确定以帧内预测模式进行预测编码的编码单元；
    梯度信息确定模块，用于确定所述编码单元对应的像素梯度信息，所述像素梯度信息包括像素梯度方差，所述像素梯度方差为编码单元内的至少部分像素的图像梯度的方差；
    预判决模块，用于根据所述像素梯度信息对所述编码单元进行划分预判决；
    代价确定模块，用于如果所述预判决模块对所述编码单元的划分预判决的结果为不划分，则将第一编码代价确定为所述编码单元的第二编码代价，所述第二编码代价被用于确定所述编码单元对应的编码树单元的划分策略，所述第一编码代价为以所述编码单元当前的尺寸作为预测单元的尺寸对所述编码单元进行预测编码得到的编码代价。
  10. 根据权利要求9所述的装置，其特征在于，所述装置还包括：
    划分模块，用于如果所述预判决模块对所述编码单元的划分预判决的结果为划分，则将所述编码单元划分为至少两个子编码单元；
    所述编码单元确定模块还用于将所述至少两个子编码单元作为新的以帧内预测模式进行预测编码的编码单元，所述梯度信息确定模块还用于分别确定所述至少两个子编码单元对应的像素梯度信息，所述预判决模块还用于根据所述像素梯度信息分别对所述至少两个编码子单元进行划分预判决，所述代价确定模块还用于分别确定所述至少两个子编码单元的编码代价；
    所述代价确定模块还用于如果所述至少两个子编码单元的编码代价之和大于或等于所述第一编码代价，则将所述第一编码代价确定为所述编码单元的第二编码代价；
    所述代价确定模块还用于如果所述至少两个子编码单元的编码代价之和大于所述第一编码代价，则将所述至少两个子编码单元的编码代价之和确定为所述编码单元的第二编码代价。
  11. 根据权利要求9所述的装置，其特征在于，所述判决模块，包括：
    阈值确定单元，用于确定所述编码单元对应的方差阈值；
    结果确定单元，用于如果所述像素梯度方差小于所述方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；
    所述结果确定单元还用于如果所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分。
  12. 根据权利要求9所述的装置，其特征在于，所述像素梯度信息还包括像素梯度平均值，所述像素梯度平均值为编码单元内的至少部分像素的图像梯度的平均值；
    所述预判决模块，包括：
    阈值确定单元，用于确定所述编码单元对应的第一阈值、所述编码单元对应的第二阈值以及所述编码单元对应的方差阈值，所述第二阈值大于所述第一阈值；
    结果确定单元，用于如果所述像素梯度平均值小于所述第一阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者
    所述结果确定单元还用于如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差小于所述 方差阈值，则确定对所述编码单元的划分预判决的结果为不划分；或者
    所述结果确定单元还用于如果所述像素梯度平均值大于或等于所述第一阈值，并且，所述像素梯度平均值小于所述第二阈值，并且，所述像素梯度方差大于或等于所述方差阈值，则确定对所述编码单元的划分预判决的结果为划分；或者
    所述结果确定单元还用于如果所述述像素梯度平均值大于或等于所述第二阈值，则确定对所述编码单元的划分预判决的结果为划分。
  13. 根据权利要求12所述的装置，其特征在于，所述阈值确定单元具体用于：
    确定所述编码单元的多个邻居编码单元，所述邻居编码单元的边缘与所述编码单元的边缘存在重合或所述邻居编码单元的顶点与所述编码单元的顶点存在重合，所述邻居编码单元内的图像与所述编码单元内的图像不重合；
    如果所述多个邻居编码单元中第一邻居编码单元的数量大于或等于目标数量，则根据所述第一邻居编码单元的尺寸以及所述第一邻居编码单元对应的像素梯度平均值确定所述编码单元对应的第一阈值，所述第一邻居编码单元以帧内预测模式进行预测编码；
    如果所述多个邻居编码单元中所述第一邻居编码单元的数量小于所述目标数量，则将所述编码单元的尺寸对应的均值阈值确定为编码单元对应的第一阈值。
  14. 一种计算机存储介质，其特征在于，所述计算机存储介质存储有多条指令，所述指令适于由处理器加载并执行如权利要求1-8任意一项的方法步骤。
  15. 一种终端，其特征在于，包括：处理器和存储器；其中，所述存储器存储有计算机程序，所述计算机程序适于由所述处理器加载并执行如权利要求1-8任意一项的方法步骤。
PCT/CN2019/078300 2018-04-02 2019-03-15 确定编码单元的编码代价的方法、装置及计算机可读存储介质 WO2019192309A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2020547348A JP7080542B2 (ja) 2018-04-02 2019-03-15 符号化単位の符号化コストの決定方法並びにその装置、コンピュータ可読記憶媒体、端末、及びコンピュータプログラム
EP19782198.6A EP3780611A4 (en) 2018-04-02 2019-03-15 METHOD AND DEVICE FOR DETERMINING THE ENCODING COSTS OF AN ENCODING UNIT AND COMPUTER-READABLE STORAGE MEDIUM
US16/996,358 US11070803B2 (en) 2018-04-02 2020-08-18 Method and apparatus for determining coding cost of coding unit and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810287548.7 2018-04-02
CN201810287548.7A CN110351556B (zh) 2018-04-02 2018-04-02 确定编码单元的编码代价的方法及相关装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/996,358 Continuation US11070803B2 (en) 2018-04-02 2020-08-18 Method and apparatus for determining coding cost of coding unit and computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2019192309A1 true WO2019192309A1 (zh) 2019-10-10

Family

ID=68100114

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/078300 WO2019192309A1 (zh) 2018-04-02 2019-03-15 确定编码单元的编码代价的方法、装置及计算机可读存储介质

Country Status (5)

Country Link
US (1) US11070803B2 (zh)
EP (1) EP3780611A4 (zh)
JP (1) JP7080542B2 (zh)
CN (1) CN110351556B (zh)
WO (1) WO2019192309A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112437310A (zh) * 2020-12-18 2021-03-02 重庆邮电大学 一种基于随机森林的vvc帧内编码快速cu划分决策方法
CN112929657A (zh) * 2021-01-22 2021-06-08 郑州轻工业大学 基于梯度与方差的h.266/vvc快速cu划分决策方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112702603A (zh) * 2019-10-22 2021-04-23 腾讯科技（深圳）有限公司 视频编码方法、装置、计算机设备和存储介质
CN110708551A (zh) * 2019-10-22 2020-01-17 腾讯科技（深圳）有限公司 视频编码方法、装置、计算机可读存储介质和计算机设备
CN111372079B (zh) * 2020-03-11 2021-01-22 南华大学 一种vvc帧间cu深度快速划分方法
CN111294591B (zh) * 2020-05-09 2020-09-25 腾讯科技（深圳）有限公司 视频信息处理方法、多媒体信息处理方法、装置
CN111770340B (zh) * 2020-07-23 2022-03-15 腾讯科技（深圳）有限公司 视频编码方法、装置、设备以及存储介质
US11490090B2 (en) * 2020-12-30 2022-11-01 Ati Technologies Ulc Text and line detection in video encode by using co-sited gradient and variance values

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104883566A (zh) * 2015-05-27 2015-09-02 复旦大学 一种适用于hevc标准的帧内预测块大小划分的快速算法
US20170041616A1 (en) * 2015-08-03 2017-02-09 Arris Enterprises Llc Intra prediction mode selection in video coding
CN107071418A (zh) * 2017-05-05 2017-08-18 上海应用技术大学 一种基于决策树的hevc帧内编码单元快速划分方法
CN107105297A (zh) * 2017-05-19 2017-08-29 华侨大学 一种针对3d‑hevc深度图帧内预测编码的快速优化方法
CN104539949B (zh) * 2014-12-12 2018-02-06 北方工业大学 Hevc屏幕编码中基于边缘方向的快速分块的方法及装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110112240A (ko) * 2010-04-06 2011-10-12 삼성전자주식회사 비디오 부호화 방법 및 장치, 비디오 복호화 방법 및 장치
US10154257B2 (en) * 2012-04-13 2018-12-11 Sun Patent Trust Image coding method in which offset is applied when an image is coded, and image decoding method in which offset is applied when an image is decoded
US11178407B2 (en) * 2012-11-19 2021-11-16 Texas Instruments Incorporated Adaptive coding unit (CU) partitioning based on image statistics
KR20140089486A (ko) * 2013-01-04 2014-07-15 삼성전자주식회사 스케일러블 비디오의 부호화 및 복호화를 위한 움직임 보상 방법 및 장치
JP2015027022A (ja) * 2013-07-29 2015-02-05 Kddi株式会社 動画像符号化装置、動画像符号化方法、およびプログラム
KR102169610B1 (ko) * 2013-08-21 2020-10-23 삼성전자주식회사 인트라 예측 모드 결정 방법 및 장치
CN105391999B (zh) * 2015-10-30 2018-08-17 北京奇艺世纪科技有限公司 一种编码模式判断方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104539949B (zh) * 2014-12-12 2018-02-06 北方工业大学 Hevc屏幕编码中基于边缘方向的快速分块的方法及装置
CN104883566A (zh) * 2015-05-27 2015-09-02 复旦大学 一种适用于hevc标准的帧内预测块大小划分的快速算法
US20170041616A1 (en) * 2015-08-03 2017-02-09 Arris Enterprises Llc Intra prediction mode selection in video coding
CN107071418A (zh) * 2017-05-05 2017-08-18 上海应用技术大学 一种基于决策树的hevc帧内编码单元快速划分方法
CN107105297A (zh) * 2017-05-19 2017-08-29 华侨大学 一种针对3d‑hevc深度图帧内预测编码的快速优化方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CAIXIABAI: "Fast coding tree unit decision for HEVC intracoding", ICCE- CHINA WORKSHOP (ICCE- CHINA, 30 April 2013 (2013-04-30), XP032583169 *
See also references of EP3780611A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112437310A (zh) * 2020-12-18 2021-03-02 重庆邮电大学 一种基于随机森林的vvc帧内编码快速cu划分决策方法
CN112437310B (zh) * 2020-12-18 2022-07-08 重庆邮电大学 一种基于随机森林的vvc帧内编码快速cu划分决策方法
CN112929657A (zh) * 2021-01-22 2021-06-08 郑州轻工业大学 基于梯度与方差的h.266/vvc快速cu划分决策方法

Also Published As

Publication number Publication date
JP2021516504A (ja) 2021-07-01
EP3780611A1 (en) 2021-02-17
CN110351556A (zh) 2019-10-18
US20200382778A1 (en) 2020-12-03
EP3780611A4 (en) 2021-06-02
JP7080542B2 (ja) 2022-06-06
CN110351556B (zh) 2021-03-02
US11070803B2 (en) 2021-07-20

Similar Documents

Publication Publication Date Title
WO2019192309A1 (zh) 确定编码单元的编码代价的方法、装置及计算机可读存储介质
TWI666926B (zh) 使用樹結構的塊分割
KR101155767B1 (ko) 비디오 데이터의 인코딩을 위한 인코딩 유형 및 예측 모드선택
US20150016521A1 (en) Video encoder for images
TWI677239B (zh) 結合多個去雜訊化技術及並行分組圖像補塊的非局部適應性環路濾波器
JP2017507554A (ja) 非対称パーティション並びにエンコーダ側の検索パターン、検索範囲、及び分割のためのアプローチを使用するイントラブロックコピー予測
US20220159292A1 (en) Filtering method for removing blocking artifact and apparatus
MX2012011165A (es) Metodo de prediccion espacial, metodo de decodificacion de imagenes y metodo de codificacion de imagenes.
KR20190117708A (ko) 부호화유닛 심도 확정 방법 및 장치
WO2022104498A1 (zh) 帧内预测方法、编码器、解码器以及计算机存储介质
WO2016180129A1 (zh) 预测模式选择方法、装置及设备
WO2019114225A1 (zh) 编码单元划分确定方法及装置、计算设备及可读存储介质
BR112021014671A2 (pt) Transformada discreta de cosseno de formato adaptativo para particionamento geométrico com um número adaptativo de regiões
US20240080439A1 (en) Intra-frame predictive coding method and system for 360-degree video and medium
WO2020248715A1 (zh) 基于高效率视频编码的编码管理方法及装置
JP7481332B2 (ja) デコードのための予測方法及びその装置、並びにコンピュータ記憶媒体
Lin et al. Fast binary tree partition decision in H. 266/FVC intra Coding
BR112021006138A2 (pt) método de predição de componente de imagem aplicado a um decodificador, dispositivo de predição de componente de vídeo, aplicado a um decodificador, e mídia de armazenamento legível por computador
WO2020056767A1 (zh) 视频图像分量的预测方法、装置及计算机存储介质
WO2020140214A1 (zh) 解码预测方法、装置及计算机存储介质
US9471992B2 (en) Moving image processing apparatus, moving image processing method, and computer product
CN110933410B (zh) Hevc视频帧间预测方法和装置
WO2023193550A1 (en) Method and apparatus for dimd region-wise adaptive blending, and encoder/decoder including the same
EP4258662A1 (en) Method and apparatus for dimd edge detection adjustment, and encoder/decoder including the same
RU2781240C1 (ru) Способ и устройство предсказания для декодирования и компьютерный носитель данных

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19782198

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020547348

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019782198

Country of ref document: EP

Effective date: 20201102