WO2020037501A1 - 码率分配方法、码率控制方法、编码器和记录介质 - Google Patents

码率分配方法、码率控制方法、编码器和记录介质 Download PDF

Info

Publication number
WO2020037501A1
WO2020037501A1 PCT/CN2018/101569 CN2018101569W WO2020037501A1 WO 2020037501 A1 WO2020037501 A1 WO 2020037501A1 CN 2018101569 W CN2018101569 W CN 2018101569W WO 2020037501 A1 WO2020037501 A1 WO 2020037501A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
frame
code rate
unit
current
Prior art date
Application number
PCT/CN2018/101569
Other languages
English (en)
French (fr)
Inventor
陈秋伯
郑萧桢
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2018/101569 priority Critical patent/WO2020037501A1/zh
Priority to CN201880042003.8A priority patent/CN110800298A/zh
Publication of WO2020037501A1 publication Critical patent/WO2020037501A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/41—Bandwidth or redundancy reduction
    • H04N1/411—Bandwidth or redundancy reduction for the transmission or storage or reproduction of two-tone pictures, e.g. black and white pictures
    • H04N1/413—Systems or arrangements allowing the picture to be reproduced without loss or modification of picture-information
    • H04N1/417—Systems or arrangements allowing the picture to be reproduced without loss or modification of picture-information using predictive or differential encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Definitions

  • the present disclosure relates to a code rate allocation method, a code rate control method, an encoder, and a recording medium, and particularly to a video encoding code rate allocation method, a control method, an encoder, and a recording medium.
  • signal coding technology especially for example, video coding technology for image signals, is constantly developing.
  • Signal coding technology especially video coding technology, generally includes multiple processes such as prediction, transformation, quantization, and entropy coding.
  • the coding code rate (that is, the number of coding bits) is usually controlled, that is, the video coding code rate is adjusted by controlling the quantization parameter.
  • the key in encoding bit rate control is how to make reasonable encoding bit rate allocation for each encoding level of the video, so as to maximize the quality of image reconstruction when the target bit rate is met.
  • video coding usually requires high real-time processing performance, so it is difficult to achieve pre-analysis, and the code rate can only be allocated by prediction.
  • a non-average allocation strategy is generally used. For example, a method of representing image complexity by calculating a gradient or variance of an image block, and performing a code rate allocation based on the calculated image complexity may be adopted.
  • these calculations require additional computing resources, and the image complexity calculated based on gradients or variances does not predict the code rate allocation well, which will cause the code rate allocation to be inaccurate.
  • the present disclosure is made to solve the problem that the code rate allocation in the prior art cannot be accurately performed.
  • An aspect of the present disclosure provides a code rate allocation method, including: acquiring a first reference unit of a current first image unit as a coding target and compression information of the first reference unit; and according to the first reference unit, The compression information allocates a coding rate to the current first image unit.
  • Another aspect of the present disclosure provides a code rate control method, including: performing the code rate allocation method of one aspect of the present disclosure, allocating an encoding code rate to the current first image unit as a target code rate; The encoding parameters are adjusted during the encoding process of the current first image unit, so that the actual encoding bit rate approaches the target bit rate.
  • Another aspect of the present disclosure provides an encoder, including at least: a quantization module that uses a quantization parameter to quantize a transform coefficient of a coding transform; and a bit rate control module that executes the bit rate control according to another aspect of the present disclosure.
  • a method to adjust the quantization parameter of the quantization module is a method to adjust the quantization parameter of the quantization module.
  • Another aspect of the present disclosure provides a computer-readable recording medium storing executable instructions that, when executed by a processor, cause the processor to perform the code rate allocation method according to an aspect of the present disclosure or another aspect of the present disclosure.
  • the code rate control method of one aspect is a computer-readable recording medium storing executable instructions that, when executed by a processor, cause the processor to perform the code rate allocation method according to an aspect of the present disclosure or another aspect of the present disclosure.
  • the code rate allocation and device, the code rate control method, the encoder, and the computer-readable recording medium of the present disclosure it is possible to provide users with a more accurate, simple, and less resource consuming code rate allocation and code rate control scheme so The image reconstruction quality of encoding process is maximized.
  • FIG. 1 is a schematic diagram of a video coding architecture of an application scenario of a code rate allocation method according to an embodiment of the present disclosure.
  • FIG. 2 schematically illustrates an example block diagram of an information compression process such as reference frame compression to which the video encoding framework shown in FIG. 1 is to be combined.
  • FIG. 3 schematically illustrates a data structure example of the reference frame compression shown in FIG. 2.
  • FIG. 4 schematically illustrates a schematic flowchart of a code rate allocation method according to an embodiment of the present disclosure.
  • FIG. 5 schematically illustrates a brief flowchart of encoding code rate allocation steps in a code rate allocation method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at the image frame level by using reference frame compression information according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at the image block level by using reference frame compression information according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at a compression unit level using reference frame compression information according to an embodiment of the present disclosure.
  • FIG. 1 schematically illustrates a video coding framework of an application scenario of a code rate allocation method according to an embodiment of the present disclosure.
  • video coding may mainly include prediction, transformation, quantization, and entropy coding.
  • FIG. 1 illustrates a currently popular video encoding framework M, which is represented by a dotted line, and can also be regarded as an encoder as a whole, for example.
  • the video encoding framework M may receive the output of a video sequence from the video sequence module S, and the video encoding framework M may include: at least an inter prediction module Rd and / or an intra prediction module Ri A prediction module; a transform module V that performs a coding transformation to obtain a transform coefficient; a quantization module Q that obtains a quantized coefficient after quantizing a transform coefficient obtained by the transform module V by a quantization parameter; Encoding module E.
  • intra-prediction and inter-prediction are two types of prediction, the purpose of which is to remove the redundant information of the current image block to be encoded by using the prediction block information.
  • intra prediction uses the information of the frame image to obtain prediction block data
  • the process may include dividing the image block to be encoded into several sub-image blocks; then, for each sub-image block, using neighboring pixels around the image block to generate The predicted pixel block of the current image block
  • the inter prediction uses the information of the reference frame to obtain the predicted block data, the process of which includes dividing the image block to be encoded into several sub-image blocks; then, for each sub-image block, search and The image block that most closely matches the current sub-image block is used as the prediction block.
  • An image frame that uses only the intra prediction mode in encoding is called an I frame
  • an image frame that uses both intra prediction and inter prediction is called a P frame or a B frame.
  • the transformation module V After predicting a pixel block, the corresponding pixel value of the sub-image block and the prediction block can be subtracted to obtain a residual. Then, in the transformation module V, a transformation matrix may be used for the residuals to remove the correlation of the image block residuals to improve the coding efficiency.
  • the transformation may generally adopt a two-dimensional transformation, that is, the encoding unit multiplies the residual information of the data block with an NxN transformation matrix and its transposed matrix, and the transformation coefficient is obtained after the multiplication.
  • a quantization parameter may be used to quantize the transform coefficient to obtain a quantization coefficient.
  • the quantization coefficient is subjected to entropy coding to obtain coding mode information and / or a bit stream, such as an intra prediction mode and motion vector information.
  • the encoding mode information and / or the bit stream are stored and / or sent to a decoding end.
  • an entropy coded bit stream can be obtained first and then entropy decoded to obtain the corresponding residuals.
  • the predicted image block corresponding to the image block is calculated, and the value of each pixel in the current sub-image block is obtained according to the residual of the predicted image block and the image block.
  • the video encoding framework M may further include an inverse quantization module IQ and an inverse transform module IV for feeding back image information to the prediction module.
  • the prediction module may further include a loop filter module F and a reference frame buffer module Bf.
  • the loop filtering module F is configured to filter the feedback image information (for example, restored image frames obtained after inverse quantization and inverse transformation, etc.) to remove or reduce distortion caused by quantization loss, thereby obtaining
  • a reference frame used for the inter prediction the reference frame buffer module Bf is configured to buffer the reference frame for use during inter prediction or for intra prediction (shown by a dashed line between Bf and Ri) (Indicates that although the reference information in the cache module is mainly used by inter prediction, the reference information can also be read from the cache module to perform code rate allocation during intra prediction).
  • the reference frame buffer module Bf shown here is included in the video encoding frame M, it goes without saying that the reference frame buffer module Bf may also be an external memory existing outside the video encoding frame M.
  • the video encoding framework M generally also needs to have a bit rate control module B.
  • the bit rate control module B usually plays a very important role in the video encoding process.
  • the purpose of bit rate control is to dynamically adjust the coding parameters to ensure that the compressed video code stream can be transmitted in real time through a channel with a fixed bandwidth, or to meet the compressed code in a specific time, while ensuring a certain image quality.
  • the stream file must be smaller than a certain size constraint. Because different video sequences have different contents and characteristics, if the same encoding parameters are used for encoding, a very different output bit rate and different degrees of video distortion will be generated. In order to overcome this problem, a bit rate control scheme is required. Coding parameters are adaptively adjusted according to the characteristics of different video images.
  • the code rate control algorithm can include two steps, code rate allocation and coding parameter adjustment.
  • the first step is code rate allocation.
  • the code rate allocation refers to assigning a predetermined coding code rate, that is, a target code rate (target number of bits) before encoding at each coding level.
  • the encoding level may include, for example, from the highest to the lowest: an image set GOP (Group Of Pictures) level, an image frame level and an image block level, or even a smaller image unit level.
  • the efficiency of intra prediction is lower than inter prediction. Therefore, under the same encoding quality, the number of bits in the encoded I frame is relatively much higher than the number of bits in the encoded P or B frame, so the I frame is accurately coded. Rate allocation is more conducive to improving encoding efficiency and image quality.
  • each coding level After the code rate allocation, each coding level obtains a pre-assigned coding code rate (number of coding bits), and then adjusts the coding parameters in the second step so that the actual coding rate (the actual number of coding bits) and the code allocation
  • standard-based video encoders have great flexibility in the selection of encoding parameters such as the direction mode of intra prediction, the motion vector of inter prediction, and the quantization parameters in the quantization process, which will affect the encoded video.
  • the bit rate of the code stream has a great impact.
  • the code rate control algorithm usually enables the encoder to select appropriate parameters between a series of encoding parameter sets, so that the actual encoding code rate (actual encoding bit number) approaches the pre-allocated target code rate (target bit number).
  • the quantization parameter is a key parameter that has a more direct impact on the encoding code rate. For example, when other parameters are fixed, a larger quantization parameter usually causes the encoder to output a smaller encoding bit rate when encoding the same video, and a smaller quantization parameter causes the encoder to output a larger encoding bit rate when encoding the same video Encoding bit rate.
  • the quantization parameter QP (Quantization Parameter) is usually calculated according to the target bit rate (target bit number). For example, formula (1) can be a calculation method.
  • the code rate control module B is represented by a gray box in FIG. 1 to emphasize that the coding allocation method of the present disclosure is an improved solution for the code allocation in the code rate control module B.
  • TargetBits is the target bit rate (target bit number) at the image frame level, which can be calculated from the target bit rate and the frame rate of the video sequence;
  • numOfPixel is the number of all pixels in the current image frame;
  • ⁇ and ⁇ are parameters Is usually an empirical value, which can be obtained by statistical analysis of a large number of videos.
  • the logarithmic form and ⁇ and ⁇ values in formula (1) can be given by an empirical code rate control model. It can be known from formula (1) that the target bit rate is very important in the calculation of the quantization parameters.
  • the accurate target bit rate that is, the accurate bit rate allocation, is the key to improving the quality of video encoded images.
  • inter prediction referring to Figs. 2 and 3, it is explained that information compression processing such as reference frame compression is usually required in the video encoding and decoding process.
  • intra prediction does not require information compression processing such as reference frame compression, but only because the reference frame used for inter prediction can be directly used as a reference frame during code rate allocation.
  • FIG. 2 schematically illustrates an example block diagram of an information compression process such as reference frame compression to be combined with the video encoding framework shown in FIG. 1.
  • the video encoding framework M shown in FIG. 1 can be regarded as a video encoder M.
  • the video encoder M requires a large amount of, for example, reference frame data for inter prediction (in addition, the intra prediction also requires, for example, reference pixel data to form a reference frame for intra prediction), and the video encoder M
  • these reference frame data are usually stored in the external memory Mo, and reading and writing these reference frame data will consume a large amount of memory bandwidth.
  • reference frame compression is often used to compress the reference frame data in the video encoding and decoding process, thereby effectively reducing the memory access bandwidth and external memory power consumption.
  • the transformation loop filtering module F finally performs filtering processing to generate the reference frame data for inter prediction.
  • the reference frame data is compressed by the reference frame compression module C, and the compressed reference frame data is stored in the external memory Mo.
  • the compressed reference frame data is read from the external memory Mo, decompressed by the reference frame decompression module DC, and the decompressed reference frame data is transmitted to the video encoding.
  • the encoder M then performs prediction by the inter prediction module Rd of the video encoder M.
  • the code rate allocation method of the present disclosure mainly uses, for example, reference frame data that is compressed by the reference frame compression module C and stored in the external memory Mo, for example, including reference frame-related information.
  • a code rate allocation scheme implemented by compressing at least one type of compression information such as the number of bits, compression rate, and compression quality.
  • FIG. 3 schematically illustrates a data structure example of the reference frame compression shown in FIG. 2.
  • reference frame compression in order to meet the characteristics of random prediction and large access to reference frames between video frames, reference frame compression usually has the following characteristics:
  • the compression unit CU of the reference frame may be 4x4 or 8x8 pixels, for example.
  • the image strip tile is generally composed of multiple compression units (CU), and the start address is determined or located by other information;
  • the compressed reference frame information stored in the external memory after the reference frame compression may mainly include: compressed reference frame data; and header information, the header information is not limited to but mainly includes, for example, the compression unit CU, for example, At least one type of compression information, such as the number of compressed bits, the compression rate, and the compression quality.
  • the key lies in how to encode the video at various levels.
  • Reasonable code rate allocation is made in advance, so that the image reconstruction quality is maximized under the condition that the actual encoding code rate is as close as possible to the pre-assigned target code rate. This requires high accuracy of the target bit rate.
  • the most efficient code rate allocation method is: firstly pre-analyze the compressed video content to obtain the code rate allocation for each image level, and then compress the video according to the code rate allocation.
  • video encoders usually require high real-time processing performance, it is difficult to achieve pre-analysis, and only code rate allocation can be performed by prediction.
  • the average allocation method will cause the reconstruction quality of image areas (such as image blocks, etc.) that are difficult to compress to decrease, which is mainly reflected in:
  • the same code rate allocation causes the image quality of the I frame to decrease significantly.
  • I-frames are used as reference frames for subsequent P-frames. I-frames with lower reconstruction quality will result in lower inter-frame prediction efficiency of subsequent P-frames.
  • the coding code rate (number of coding bits) of the fixed allocation I frame is several times that of the P frame, but at a certain code rate, a large number of bits have been used to encode the I frame, and the remaining The number of bits is not enough to maintain the relatively high quality of the subsequent P frames, so it will cause the quality of the subsequent P frames to decrease.
  • the complexity of the I frame itself is not high, a higher code rate allocation is not required.
  • the present disclosure provides a code rate allocation method for video coding in order to solve the existing problem of the above-mentioned code rate allocation scheme.
  • the method uses the compression information (for example, the number of compressed bits, the compression rate, etc.) of a reference unit such as a reference frame. , Compression quality, etc.) as a reference basis for bit rate allocation, based on which the bit rate can be dynamically allocated to the image area to improve the quality of the reconstructed image at a certain bit rate.
  • the compression information (e.g., compression rate, etc.) of the reference unit may be provided by, for example, a reference unit compression module (e.g., reference frame compression module C), without requiring additional calculation and storage resources, and compared to It has higher prediction accuracy.
  • a reference unit compression module e.g., reference frame compression module C
  • FIG. 1 and FIG. 2 and FIG. 3 and their corresponding descriptions are merely examples, and the technical solutions of the present disclosure are not limited to these examples.
  • Figures 2 and 3 show situations such as reference frames and inter-prediction, that is, the image unit to be processed is set to a frame (image frame level). Of pictures (Pictures) level, image frame level and image block level, even smaller image units. Therefore, it goes without saying that the technical solution of the present disclosure is of course applicable to various levels other than the image frame level.
  • the code rate allocation method is a main part of the code rate control module B (ie, the code rate control method) in the encoding process shown in FIG. 1.
  • FIG. 4 schematically illustrates a schematic flowchart of a code rate allocation method according to an embodiment of the present disclosure.
  • the code rate allocation method includes: obtaining a reference unit and compression information (step S1), and assigning a code rate (step S2).
  • the first reference unit of the current first image unit as the object of encoding and the compression information of the first reference unit are acquired.
  • the current first image unit may be, for example, any one of an image set GOP, an image frame, an image block, and the like that are not encoded and compressed as an encoding target.
  • the first reference unit may be, for example, any one of a reference picture set GOP, a reference frame, a reference picture block, and the like of the current first picture unit.
  • the compression information may include, for example, at least one of a number of compression bits, a compression rate, a compression quality, and the like.
  • the current first image unit is allocated with an encoding bit number, that is, an encoding code rate.
  • the first reference unit (for example, a reference frame) and the current first image unit (for the current image frame) are image units in the same scene image, or the first reference unit (for example, a reference frame) is preferred.
  • the current first image unit (current image frame) is an image unit in a scene image in which a scene mutation does not occur. In this way, better reference accuracy can be guaranteed.
  • the so-called scene mutation refers to that the scene video content changes significantly. Such a mutation can be judged by comparing the histogram statistical results of two frames of images, or comparing other features.
  • the first reference unit may be a first image unit before the current first image unit, or may be a first fusion of a plurality of first image units before the current first image unit.
  • Picture unit may be a well-known image fusion process in the art.
  • a corresponding weight may be assigned to each image unit according to the degree of matching of pixels and / or feature points in multiple image units, so as to use the weight to divide multiple images. Units are merged into one image unit.
  • the first reference unit is a first image unit immediately before the current first image unit.
  • the first reference unit is preferably the immediately previous image frame of the current image frame. This is because the possibility of a sudden change in the scene in the immediately preceding image frame is low, which can guarantee better reference accuracy.
  • the current first image unit may include a plurality of second image units.
  • the current first image unit when the current first image unit is an image set GOP, the current first image unit, that is, the image set GOP, may include multiple second image units, that is, image frames; in the current first image unit, In the case of an image frame, the current first image unit, that is, the image frame may include a plurality of second image units, that is, image blocks.
  • the first reference unit may include a plurality of second reference units.
  • the first reference unit when the current first picture unit is a picture set GOP, the first reference unit is a reference picture set GOP, and the first reference unit, that is, the reference picture set GOP, may include multiple as the first The second reference unit of the reference image of the two image units (image frames) is the reference image frame; when the current first image unit is an image frame, the first reference unit is a reference image frame, and the first A reference unit, that is, a reference image frame, may include a plurality of second reference units, that is, reference image blocks that are reference images of the second image unit (image block).
  • FIG. 5 schematically illustrates a brief flowchart of encoding code rate allocation steps in a code rate allocation method according to an embodiment of the present disclosure.
  • the allocation of the encoding code rate may specifically include: calculation of the allocation rate of the code rate (step S2-1) and determination of the allocation of the encoding code rate (step S2-2).
  • the bit rate allocation ratio of each of the second image units in the current first image unit may be calculated according to the compression information of the first reference unit.
  • the coding code of each of the second image units may be determined according to the code rate allocation ratio and the coding code rate of the current first picture unit allocated in advance. rate.
  • the compression information may include the number of compressed bits of the first reference unit and the number of compressed bits of the plurality of second reference units.
  • the calculation of the bit rate allocation ratio may further specifically include: calculating each location according to the ratio of the number of compressed bits of each second reference unit to the number of compressed bits of the first reference unit.
  • the bit rate allocation ratio of the second image unit is described.
  • the code rate allocation method of the present disclosure it is first proposed to use compression information to characterize The technical solution of image complexity, that is, in the allocation of the encoding bit rate (step S2) shown in FIG. 4, according to the compression information (for example, the number of compressed bits) of the reference unit (for example, the reference frame),
  • the current image unit (for example, the current image frame) allocates the number of coding bits, that is, the code rate.
  • FIG. 6 is a schematic diagram for describing the use of reference frame compression information to characterize the complexity of an arbitrary region of a current image frame according to an embodiment of the present disclosure.
  • an arbitrary-shaped image area (for example, an image block, etc.)
  • S_cur of the current image frame uses, for example, the immediately preceding image frame of the current image frame as the previous reference frame (here, the previous reference frame is Because the immediately previous image frame can generally be regarded as the image frame in the same scene image, or the image frame in the scene image where no mutation has occurred)
  • the number of compression units CU in the reference frame covered by S_cur be 1, 2, ... n respectively, then the texture complexity of S_cur can be expressed by the following formula (2):
  • Bits (CUi) represents the number of compressed bits corresponding to the i-th reference frame compression unit CU.
  • Reference frame compression has the same processes as residual prediction and entropy coding, so it approximates the compression potential of image blocks.
  • texture complexity is used as the basis for code rate allocation at each level of encoding in video coding. Because the compression complexity information of the previous frame is used to estimate the texture complexity, this disclosure is generally not suitable for the current image frame as the first scene switching Frame, but can be applied to frames before and after the same scene (or unmuted scene). In addition, when texture rate is used to allocate code rates to different coding levels, there are different degrees of adaptability and different degrees of constraint descriptions are needed. This will be analyzed and explained in different embodiments described below.
  • FIG. 6 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at the image frame level by using reference frame compression information according to an embodiment of the present disclosure.
  • the encoding level in the current picture unit is an image frame level or higher (that is, including an image frame level, an image set GOP level, or even an image set GOP group level or higher than the image set GOP level, etc.) ),
  • the allocation of the encoding bit rate (step S2) shown in FIG. 4 according to the reference frame corresponding to the position of the current image frame, that is, the number of compressed bits of the corresponding reference frame and the corresponding reference.
  • the original code rate (original number of bits) of the frame when it is not compressed is used to calculate the compression rate of the corresponding reference frame, and then, using the calculated compression rate, code rate allocation is performed on the current image frame.
  • I-frames are only because the image quality of subsequent P-frames or B-frames will also improve accordingly when the image quality of I-frames is improved. Therefore, you can also perform the code rate only for I-frames when performing code rate allocation. Allocation, of course, goes without saying that P frames or B frames also apply. Because it is for I frames, S_cur here is the image size of the entire frame.
  • the code rate allocation of the I frames can be dynamically adjusted according to the texture complexity, that is, for example, more bits are allocated to the I frames that are difficult to compress, and they are allocated. Give the easily compressed P or B frames a small number of bits, so that the image quality of subsequent P or B frames can be improved while ensuring the quality of the I frame.
  • Bits (x) represents the encoding code rate (number of encoding bits) allocated to x, that is, Bits (I_frame) represents the number of encoding bits allocated to the current I frame, that is, encoding code rate
  • Bits (GOP) represents allocation to
  • the current image frame is composed of multiple image frames and the encoding level is higher than the upper image unit (here, the image set GOP) of the current image frame, which is the encoding bit rate; here, the first frame in the image set GOP is preferred I-frames, followed by P-frames or B-frames (usually, this situation is more common, of course, it does not rule out the existence of multiple I-frames and I-frames in different locations);
  • formula (3) is only an example , There are only I-frame and P-frame in this image set, I_MAX_ratio and P_ratio are the pre-allocated code rate allocation ratio between I-frame and P-frame; parameter ⁇ is shown in formula (4), which is the reason for the
  • the compression ratio of the reference frame is preferably used as the parameter ⁇ (because it can characterize the ease of compression of the reference frame), the parameter ⁇ does not have to be the compression ratio of the reference frame, as long as the I-frame and Allocation of the code rate between P frames or B frames is sufficient.
  • the parameter ⁇ adjusts the proportion of the encoding code rate (the number of encoding bits) occupied by the I frame to achieve the final code rate allocation for the current image frame.
  • the dotted line connection between Bf and Ri in FIG. 1 indicates that the intra prediction may not need to read the reference frame from the buffer for prediction, but the reference frame or the reference frame compression information may be read from the buffer for code rate allocation. No other constraints are needed in this case where the current picture unit is above the picture frame level.
  • the current image unit is not a frame but an image set GOP level larger than the image frame level, or even an image set GOP level or more In the case of GOP group level, etc.
  • the x in the above-mentioned Bits (x) should be adjusted accordingly. For example, if Bits (I_frame) in Equation (3) is changed to Bits (GOP), then Bits (GOP) in Equation (3) is changed to Bits (GOP Group) accordingly.
  • the Bits (GOP) in Equation (3) may of course be Bits (GOP group), as long as it is a picture unit (for example, GOP, GOP group, etc.) that is larger than a frame and can be composed of frames.
  • the P-frame or B-frame other than the I-frame is also applicable to the technical solution of the embodiment of the present disclosure.
  • the code rate allocation method in the embodiment of the present disclosure may further include: acquiring a current image unit (here, for example, a current image frame) and encoding level higher than the current image unit
  • step S3 the acquired upper-level unit is used
  • a coding rate (number of coding bits) of a picture unit here, for example, a picture set GOP
  • the current picture unit here, for example, a current picture frame
  • bit rate allocation is performed according to compression information of a reference unit (here, for example, a reference frame) Bit rate allocation.
  • the compression information for example, the number of compressed bits
  • code rate allocation that is, to use the ease of compression of the reference frame to represent the complexity of the image texture of the current image frame, thereby achieving more accuracy and simplicity.
  • code rate allocation that consumes less resources is used, so that the image reconstruction during encoding processing can maximize the reconstruction quality.
  • FIG. 7 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at the image block level by using reference frame compression information according to an embodiment of the present disclosure.
  • the reference unit corresponding to the position of the current image unit (for example, n image blocks S_cur_1, S_cur_2 ... S_cur_n in the current image frame) may correspond.
  • the number of compressed bits of a reference unit (for example, n reference image blocks S_ref_1, S_ref_2 ...
  • an upper image unit for example, the current The ratio of the number of compressed bits in the entire reference unit (for example, a reference frame) in the image frame), and then, using the calculated ratio, allocate a bit rate to the current image unit (for example, the image block S_cur_1, etc.).
  • the following description is made by taking an image block-level code rate allocation for an I frame as an example.
  • the purpose of I-frames is because the image quality of subsequent P-frames or B-frames will also improve with the premise that the image quality of I-frames often improves.
  • P-frames or B-frames Be applicable.
  • the image block level is a large range of image blocks, that is, the value of n shown in FIG. 6 is as small as possible (for example, n can be less than 4), and it can also be a large block slice or tile division in video coding, for example.
  • FIG. 6 shows an example.
  • Bits (S_cur_x) represents the encoding code rate (number of encoding bits) allocated to the current image block S_cur_x in the current image frame
  • Bits (I_frame) represents allocation to a composition of a plurality of the current image blocks and the encoding level is higher than
  • the number of encoding bits of the current image block (here, for example, the current image frame I_frame) is the encoding bit rate
  • S_cur_x is the current image block
  • S_ref_x is the reference image block corresponding to the position of the current image block S_cur_x
  • n is greater than 1 Integer.
  • the function TC is the number of compressed bits.
  • the encoding bit rate of the current image frame can also be allocated by using a method similar to the formula (5), that is, according to the proportion of the reference image frame corresponding to the current image frame in the reference GOP or reference GOP set to the current The encoding code rate of the GOP or GOP set to which the picture frame belongs is allocated.
  • Bits (I_frame) is preferably obtained from the code rate allocation in the above-mentioned Embodiment 2 in the present disclosure, so that a more accurate code rate allocation can be further achieved, thereby further maximizing the image reconstruction quality during encoding processing.
  • Bits (I_frame) can also be obtained by the allocation method as in formula (5), or obtained by existing methods such as code rate average allocation, or by existing methods such as using the gradient or variance of image blocks. Obtained by the non-code rate average allocation method for code rate allocation.
  • the constraint conditions are: for still video sequences, there are no other constraints; for motion video sequences, the larger the image block, the more accurate the bit rate allocation, which It is because the texture complexity estimation of large image blocks is not sensitive to the movement of image content, while small blocks have a greater impact.
  • the shape of the image area may be any shape.
  • Bits (I_frame) in Equation (5) can of course be Bits (GOP) or Bits (GOP Group), as long as it is an image unit (such as a frame, GOP, GOP) that is larger than an image block and can be composed of an image block. Group, etc.).
  • the code rate allocation method may further include: acquiring a plurality of current image units (here, for example, current image blocks) and encoding levels higher than the current image unit
  • the encoding bit rate (number of encoding bits) of the upper-level image unit here, for example, an image frame is composed of image blocks.
  • step S2 the obtained The encoding bit rate (number of encoding bits) of the superordinate image unit (here, for example, a frame), and according to the compression information of the reference unit (here, for example, a reference image block), the current image unit (here, for example, the current image block) is performed Bit rate allocation.
  • the compression information for example, the number of compressed bits
  • the complexity of the image texture is characterized by the compression difficulty of the image block of the reference frame, thereby achieving more accurate.
  • the code rate allocation is simpler and consumes less resources, so that the image reconstruction during encoding processing can maximize the reconstruction quality.
  • the compression unit level is a basic coding unit for video coding, such as MB in H.264 and CTU / CU in H.265.
  • FIG. 8 is a schematic diagram for describing the complexity of an arbitrary image region of a current image frame at a compression unit level using reference frame compression information according to an embodiment of the present disclosure.
  • the coding level of the current picture unit is a smaller level, such as a compression unit level, in the allocation of the coding code rate shown in FIG. 4 (step S2)
  • the reference unit corresponding to the position of the current image unit for example, n compression units 1, 2, ..., n in the current image block S_cur
  • the corresponding reference unit for example, n reference compression in the reference image block S_ref
  • the number of compressed bits of units 1, 2, ..., n) to calculate a reference unit (e.g., a reference unit) of an upper picture unit (e.g., the current picture block S_cur) of the corresponding reference unit at an encoding level higher than the current picture unit
  • the ratio of the number of compressed bits in the entire image block S_ref) and then, using the calculated ratio, the current image unit (for example, the compression unit 1 of the image block S_cur, etc.) is allocated a bit rate.
  • FIG. 7 shows an example, assuming that code rate allocation is needed for the compression units 1, 2, ..., n in the image block S_cur of the current image frame, using a reference compression unit such as the reference image block S_ref of the previous reference frame.
  • the number of compressed bits of the reference frame at the corresponding positions of 1, 2, ..., n to estimate the texture complexity of the compression units 1, 2, ..., n in the image block S_cur, and the calculation of the code rate allocation can be as follows (6):
  • Bits (S_cu_m) represents the encoding bit rate (encoding bits) of the current compression unit S_cu_m in the current image block S_cur allocated to the current image frame
  • Bits (S_cur) represents the allocation to a composition of a plurality of the current compression units.
  • the encoding level is higher than the encoding bit rate of the upper image unit of the current compression unit (here, for example, the current image block S_cur); that is, the encoding bit rate
  • S_cu_m is the current compression unit
  • S_ref_m is the reference compression corresponding to the position of the current compression unit S_cu_m Unit
  • n is an integer greater than 1.
  • the function TC is a texture complexity function of the above formula (2).
  • Bits (S_cur) is preferably obtained from the code rate allocation in Embodiment 3 in the present disclosure, which can further achieve a more accurate code rate allocation, thereby further maximizing the image reconstruction quality during encoding processing.
  • Bits (S_cur) can also be obtained from existing methods such as code rate average allocation, or from existing non-code rate average uses such as image block gradients or variances. Method to obtain.
  • the shape of the image area may be any shape.
  • Bits (S_cur) in equation (7) can of course be Bits (I_frame), Bits (GOP), or Bits (GOP group), as long as it is an image unit (for example, a compression unit) that is larger than the compression unit (for example, Image block, frame, GOP, GOP group, etc.).
  • the method for assigning bit rate in the embodiment of the present disclosure may further include: acquiring a current image unit (here, for example, a current compression unit) and encoding level higher than the current image unit
  • the encoding bit rate (number of encoding bits) of the upper-level image unit here, for example, an image block is constituted by a compression unit.
  • step S2 the obtained The encoding bit rate (number of encoding bits) of the upper image unit (here, for example, an image block), and according to the compression information of the reference unit (here, for example, a reference compression unit), the current image unit (here, for example, the current compression unit) Perform code rate allocation.
  • the reference unit here, for example, a reference compression unit
  • the current image unit here, for example, the current compression unit
  • the compression information for example, the number of compression bits
  • the code rate that is, using the compression ease of the compression unit in the image block of the reference frame to represent the complexity of the image texture Degree, thereby achieving more accurate, simpler, and less resource-consuming code rate allocation, so that the image reconstruction during encoding processing can maximize the reconstruction quality.
  • the above embodiments only exemplify the case of two levels of coding levels, and of course, there may be cases of two or more levels of coding.
  • the coding bit rate allocation method in the embodiment of the present disclosure may be used for calculation, or the existing technology may be used. It is self-evident that it is calculated by, for example, a code rate average allocation method, or a non-code rate average allocation method such as a gradient or a variance.
  • the code rate allocation method in the embodiment of the present disclosure is a step in the code rate control method shown in FIG. 1.
  • a code rate control method can be provided, which can The method includes: performing the code rate allocation method according to the embodiment of the present disclosure (for example, as shown in the flowcharts of FIGS. 4 and 5), allocating a coding code rate to the current picture unit as a target code rate; and in the current first picture, During the coding process of the unit, the coding parameters are adjusted so that the actual coding code rate approaches the target code rate.
  • the image reconstruction during the encoding process can also be used to maximize the reconstruction quality.
  • code rate allocation method and the code rate control method on the other hand of the embodiments of the present disclosure may also be stored in a computer-readable storage medium as a computer program, and the computer program may include code / Computer-executable instructions cause a computer to execute the code rate allocation method shown in the flowcharts of FIGS. 4 and 5 of the present disclosure, and variations thereof.
  • the computer-readable storage medium may be, for example, any medium capable of containing, storing, transmitting, propagating, or transmitting instructions.
  • a readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium.
  • Specific examples of readable storage media include: magnetic storage devices such as magnetic tapes or hard disk drives (HDD); optical storage devices such as optical disks (CD-ROM); memories such as random access memory (RAM) or flash memory; and / or wired / Wireless communication link.
  • the computer program may be configured to have computer program code including, for example, a computer program module.
  • computer program module the division manner and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions.
  • program module combinations are executed by a computer (or processor), the computer For example, the flow of the code rate allocation method described above in conjunction with FIGS. 4-5 and its variants can be performed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本公开提供一种码率分配方法、码率控制方法、编码器和记录介质。其中，所述码率分配方法包括：获取作为编码对象的当前第一图像单元的第一参考单元和所述第一参考单元的压缩信息；根据所述第一参考单元的所述压缩信息，对所述当前第一图像单元进行编码比特数的分配。

Description

码率分配方法、码率控制方法、编码器和记录介质 技术领域
本公开涉及一种码率分配方法、码率控制方法、编码器和记录介质，特别是涉及一种视频编码码率分配方法、控制方法、编码器和记录介质。
背景技术
随着计算机技术及信号处理技术的快速发展，信号编码技术尤其是例如针对图像信号的视频编码技术正不断蓬勃发展。
在信号编码技术尤其是视频编码技术中，一般包括预测、变换、量化和熵编码等多种处理。其中，在量化处理过程中，通常要进行编码码率(即编码比特数)控制，即通过控制量化参数来调节视频编码码率。在编码码率控制中关键在于如何对视频各编码层级进行合理的编码码率分配，以使得满足目标码率的情况下图像重构质量最大化。
然而，视频编码通常要求较高的实时处理性能，因而难以实现预分析，只能通过预测的方式进行码率分配。
现有的编码码率分配方法中一般采用非平均分配策略。例如，可以采用通过计算图像块的梯度或方差来表征图像复杂度，并根据计算出的图像复杂度来进行编码码率分配的方法。但这些计算需要消耗额外的计算资源，并且基于梯度或方差计算出的图像复杂度也并不能很好地预测编码码率分配，会导致编码码率分配不够准确。
发明内容
本公开就是为了解决现有技术中不能准确的进行编码码率分配而做出的。
本公开的一个方面提供了一种码率分配方法，包括：获取作为编码对象的当前第一图像单元的第一参考单元和所述第一参考单元的压缩信息；根据所述第一参考单元的所述压缩信息，对所述当前第一图像单元进行编码码率的分配。
本公开的另一个方面提供了一种码率控制方法，包括：进行本公开的一个方面的所述码率分配方法，对所述当前第一图像单元分配编码码率，作为目标码率；在所述当前第一图像单元的编码过程中调整编码参数，使得实际编码码率逼近所述目标码率。
本公开的另一方面提供一种编码器，至少包括：量化模块，利用量化参数，对编码变换的变换系数进行量化；码率控制模块，通过执行本公开的另一方面的所述码率控制方法来调整所述量化模块的所述量化参数。
本公开的另一方面提供一种计算机可读记录介质，存储有可执行指令，该指令被处理器执行时使该处理器执行本公开的一个方面的所述码率分配方法或者本公开的另一个方面的所述码率控制方法。
根据本公开的码率分配及装置、码率控制方法、编码器、和计算机可读记录介质，能够为用户提供更准确、更简便且消耗资源少的码率分配及码率控制方案，以使编码处理的图像重构质量最大化。
附图说明
为了更完整地理解本公开及其优势，现在将参考结合附图的以下描述，其中：
图1示意性示出了本公开实施例的码率分配方法的应用场景的视频编码构架简图。
图2示意性示出了图1所示的所述视频编码构架所要结合到的例如参考帧压缩这样的信息压缩处理的示例框图。
图3示意性示出了图2所示的所述参考帧压缩的数据结构示例图。
图4示意性示出了本公开实施例的码率分配方法的简要流程图。
图5示意性示出了本公开实施例的码率分配方法中的编码码率分配 步骤的简要流程图。
图6是用于说明本公开实施例的利用参考帧压缩信息在图像帧级来表征当前图像帧的任意图像区域复杂度的示意图。
图7是用于说明本公开实施例的利用参考帧压缩信息在图像块级来表征当前图像帧的任意图像区域复杂度的示意图。
图8用于说明本公开实施例的利用参考帧压缩信息在压缩单元级来表征当前图像帧的任意图像区域复杂度的示意图。
具体实施方式
以下，将参照附图来描述本公开的实施例。
图1示意性示出了本公开实施例的码率分配方法的应用场景的视频编码框架简图。
通常，视频编码可以主要包括预测、变换、量化和熵编码等。作为本公开实施例的码率分配方法的一种应用场景的示例，这里，图1示出一种当前流行的视频编码框架M，其以虚线进行表示，也可以整体视为例如是编码器。
如图1所示，该视频编码框架M可以接收来自视频序列模块S的视频序列的输出，在所述视频编码框架M中可以具备：至少包括帧间预测模块Rd和/或帧内预测模块Ri的预测模块；进行编码变换以得到变换系数的变换模块V；将经所述变换模块V得到变换系数经量化参数量化后得到量化系数的量化模块Q；以及对所述量化系数进行熵编码的熵编码模块E。
此外，在预测模块中，帧内预测和帧间预测是两种类型的预测，其目的在于利用预测块信息去除当前待编码图像块的冗余信息。
具体而言，帧内预测利用本帧图像的信息获得预测块数据，其过程可以包括将待编码图像块划分成若干个子图像块；然后，针对每个子图像块，利用图像块周边的邻近像素生成当前图像块的预测像素块；帧间预测利用参考帧的信息获得预测块数据，其过程包括将待编码图像块划分成若干个子图像块；然后，针对每个子图像块，在参考图像 中搜索与当前子图像块最匹配的图像块作为预测块。在编码中仅使用帧内预测模式的图像帧被称为I帧，同时使用帧内预测及帧间预测的图像帧被称为P帧或B帧。
此外，在经所述帧内预测模块Ri或所述帧间预测模块Rd(这里，以虚线表示帧内预测模块Ri与帧间预测模块Rd这两者的输出可以是“或”的关系)获得预测像素块后，可以将该子图像块与预测块的相应像素值相减得到残差。然后，在所述变换模块V中，针对所述残差可以使用变换矩阵去除图像块残差的相关性，以提高编码效率。所述变换例如通常可以采用二维变换，即在编码端将数据块的残差信息分别与一个NxN的变换矩阵及其转置矩阵相乘，相乘之后得到的是变换系数。然后，在所述量化模块Q中，可以利用量化参数来对所述变换系数进行量化而得到量化系数。然后，在所述熵编码模块E中，对所述量化系数进行熵编码，得到例如帧内预测模式、运动矢量信息等编码模式信息和/或比特流。最后，将所述编码模式信息和/或所述比特流进行存储和/或发送到解码端。
这里，对于所述解码端虽未图示，但在所述解码端，通常可以首先获得熵编码比特流后进行熵解码，得到相应的残差，根据解码得到的运动矢量或帧内预测等信息计算出图像块对应的预测图像块，根据预测图像块与图像块的残差得到当前子图像块中各像素点的值。
如图1所示，所述视频编码框架M还可以具备：用于向所述预测模块反馈图像信息的反量化模块IQ和反变换模块IV。此外，所述预测模块中还可以包括：环路滤波模块F和参考帧缓存模块Bf。所述环路滤波模块F用于对所述反馈的图像信息(例如，经反量化及反变换后得到的恢复图像帧等)进行滤波，以去除或减小由量化损失产生的失真，从而得到例如用于所述帧间预测的参考帧；所述参考帧缓存模块Bf用于缓存所述参考帧，以备帧间预测时使用或帧内预测时使用(Bf和Ri之间以虚线示出表示尽管缓存模块中的参考信息主要由帧间预测来使用，但在帧内预测时也可以从缓存模块中读取参考信息来进行码率分配)。此外，这里所示的所述参考帧缓存模块Bf虽然被包括于视频编码框架 M内，但不言而喻，该参考帧缓存模块Bf也可以是存在于视频编码框架M外的外部存储器。
此外，如图1所示，所述视频编码框架M通常还需要具备码率控制模块B。而且，所述码率控制模块B在视频编码过程中通常起到非常重要的作用。具体而言，码率控制的目的是通过动态地调整编码参数，在保证一定图像质量的前提下使压缩后的视频码流能通过固定带宽的信道实时传输，或是满足特定时间内压缩后码流文件必须要小于某个规定尺寸的约束。由于不同的视频序列其内容和特征都各不相同，若使用同样的编码参数进行编码就会产生差别很大的输出码率及不同程度的视频失真，为了克服这个问题就需要码率控制方案能够根据不同视频图像的特征自适应地调整编码参数。
一般而言，码率控制算法可以包括两个步骤，码率分配和编码参数调整。第一步是码率分配，所述码率分配是指在各个编码层级进行编码前预先分配指定的编码码率即目标码率(目标比特数)。其中，编码层级例如可以从高到低依次包括：图像集GOP(Group Of Pictures)级、图像帧级及图像块级、甚至更小的图像单元级等。一般而言，帧内预测的效率低于帧间预测，因此在相同编码质量下，编码I帧的比特数比编码P或B帧的比特数会相对高很多，所以对I帧进行准确的码率分配更有利于提高编码效率和提升图像质量。
进行码率分配后，各个编码层级得到预先分配的编码码率(编码比特数)，第二步再通过所述编码参数调整，使得实际编码码率(实际编码比特数)与在所述编码分配中预先分配的编码码率(编码比特数)相吻合。通常，基于标准的视频编码器在例如帧内预测的方向模式、帧间预测的运动矢量、量化过程中的量化参数等这些编码参数的选择方面具有很大的灵活性，这会对编码后视频码流的码率有很大的影响。因此，码率控制算法通常使得编码器能够在一系列编码参数集合之间选择合适的参数，以实现实际编码码率(实际编码比特数)逼近预先分配的目标码率(目标比特数)。在这些编码参数之中，量化参数是对编码码率有更直接影响的关键参数。例如，在其他参数固定的情况下，较大的量 化参数通常会导致编码器编码相同视频时输出较小的编码码率，反之较小的量化参数会导致编码器编码相同视频时输出较大的编码码率。量化参数QP(Quantization Parameter)通常根据目标码率(目标比特数)来计算，例如，式(1)可以为一种计算方法。
这里，在图1中以灰色框表示了所述码率控制模块B，是为了强调本公开的编码分配方法就是针对该码率控制模块B中的所述编码分配进行的一种改良方案。
Figure PCTCN2018101569-appb-000001
式中，TargetBits为图像帧级的目标码率(目标比特数)，可以由目标码率及视频序列的帧率计算而得；numOfPixel为当前图像帧的所有像素点个数；α和β为参数，通常是经验值，可以由大量视频的统计分析而得到，例如，式(1)中的对数形式及α、β值可以由经验码率控制模型给出。由公式(1)可知，目标码率在量化参数计算中非常重要，给出准确的目标码率，也即准确的码率分配，是提升视频编码图像质量的关键。
以帧间预测为例，参考图2、3来说明视频编解码过程中通常还需要例如参考帧压缩这样的信息压缩处理。这里，以帧间预测为例不是说帧内预测就不需要例如参考帧压缩这样的信息压缩处理，而仅是因为帧间预测所用的参考帧可以直接用来作为码率分配时的参考帧。
图2示意性示出了图1所示的所述视频编码构架所要结合的例如参考帧压缩这样的信息压缩处理的示例框图。
如图2所示，图1所示的所述视频编码构架M即可以视为一种视频编码器M。通常，所述视频编码器M需要大量的例如参考帧数据进行帧间预测(另外，帧内预测也同样需要例如参考像素数据以形成参考帧来进行帧内预测)，而在所述视频编码器M的电路设计中这些参考帧数据通常保存于外部存储器Mo中，而读写这些参考帧数据会消耗大量的存储带宽。随着视频分辨率不断提升，急剧增长的带宽需求以及访存功耗已成为制约嵌入式系统或者移动设备的关键因素。为了解决上述问题，参考帧压缩常常被用于压缩视频编解码过程中的参 考帧数据，从而有效减小访存带宽以及外部存储器功耗。具体而言，例如，视频序列信号经视频编码器M的变换、量化等模块的处理后，最终由变换环路滤波模块F进行滤波处理而生成用于帧间预测的所述参考帧数据。然后，所述参考帧数据经参考帧压缩模块C进行压缩，将被压缩的参考帧数据存储到所述外部存储器Mo中。相应地，当进行帧间预测时，从所述外部存储器Mo中读出被压缩的参考帧数据，经参考帧解压模块DC进行解压缩，将被解压缩的参考帧数据传输给所述视频编码器M，然后，由所述视频编码器M的所述帧间预测模块Rd进行预测。
此外，本公开的码率分配方法主要就是利用例如经所述参考帧压缩模块C压缩并存储于所述外部存储器Mo中的例如包含与所述参考帧相关的信息在内的参考帧数据中的例如压缩比特数、压缩率、压缩质量等至少一种压缩信息来实现的码率分配方案。
为此，下面，参考图3来说明所述参考帧压缩的参考帧数据结构。
图3示意性示出了图2所示的所述参考帧压缩的数据结构示例图。
如图3所示，一般而言，为了满足视频帧间预测随机并大量访问参考帧的特性，参考帧压缩通常具有以下特点：
a.属于帧内压缩，仅利用帧内信息进行残差预测，参考帧的压缩单元CU例如可以为4x4或者8x8个像素；
b.以图像条带tile大小进行读写外部存储器，图像条带tile一般由多个压缩单元CU(Compression Unit)组成，并且开始地址是确定的或者通过其它信息定位到；
c.为了支持高并行解码吞吐率，通常存储到外部存储器时需要存储额外的信息指示每个压缩单元CU的例如压缩比特数、压缩率、压缩质量等至少一种压缩信息。因此，经过参考帧压缩后存储于外部存储器中的被压缩的参考帧信息可以主要包括：压缩后的参考帧数据；和头信息，所述头信息不限于但主要包含所述压缩单元CU的例如压缩比特数、压缩率、压缩质量等至少一种压缩信息。
如上所述可知，在一定的传输带宽下，需要通过控制例如量化参 数(其调节实际编码码率的大小)这样的编码参数来调节视频编码码流的大小，其关键在于如何对视频各编码层级预先进行合理的码率分配，使得实际编码码率尽量接近预先分配的目标码率情况下图像重建质量最大化。这就要求目标码率有很高的准确性。通常，最高效的码率分配方式是：首先对所压缩的视频内容进行预分析，得出各图像层级的码率分配，然后再根据码率分配来对视频进行压缩。然而，在实际产品中，视频编码器通常要求较高的实时处理性能，难以实现预分析，只能通过预测的方式进行码率分配。
此外，现有的码率分配主要有：
1)平均分配法
即，对各编码层次采用平均分配的策略。
但由于图像内容和特征是动态变化的，该平均分配法会导致图像比较难压缩的图像区域(例如，图像块等)重建质量下降，其主要体现在：
a.例如，在帧级码率分配上，由于I帧需要的编码码率(编码比特数)远大于P帧，相同的码率分配使得I帧图像质量大幅下降。同时I帧作为后续P帧的参考帧，较低重建质量的I帧会导致后续P帧的帧间预测效率降低。
b.例如，在帧内图像块级码率分配上，对于那些细节纹理较多的图像区域，平均分配法会导致这些区域图像质量下降，作为对比，那些比较平坦的图像区域由于容易压缩并不需要很多的码率分配。
2)非平均分配法
即，对各编码层次采用非平均分配的其他策略。
但由于这些现有的非平均分配法过于粗糙，而同样会导致重建图像质量下降问题，主要体现在：
a.例如，在帧级码率分配上，固定分配I帧的编码码率(编码比特数)为P帧的数倍，然而在一定码率下，大量比特数已用于编码I帧，剩余比特数不够后续P帧再保持相对较高的质量， 因此会引发后续P帧的质量下降；此外，如果I帧本身复杂度不高，也不需要较高的码率分配。
b.例如，在I帧内图像块级码率分配上，采用一些表征图像复杂度的计算用于码率分配，如图像块梯度或者方差计算，然而这些计算需要耗费额外的计算资源，并且基于梯度和方差的复杂度并不能很好地预测码率分配，导致码率分配不够准确。
总之，现有方法难以使得视频码率，尤其是I帧码率，能够根据图像内容和特征进行动态合理地调整，或者难以对I帧进行较准确的码率分配。
本公开就是为了解决现有的上述码率分配方案的问题而提供了一种视频编码的码率分配方法，该方法利用例如参考帧这样的参考单元的压缩信息(例如，压缩比特数、压缩率、压缩质量等)作为码率分配的参考依据，可据此动态地对图像区域进行码率分配，提升一定码率条件下重建图像的质量。此外，该参考单元(例如，参考帧)的压缩信息(例如，压缩率等)可由例如参考单元压缩模块(例如，参考帧压缩模块C)提供，不需要额外耗费计算和存储资源，并且相比之下具有更高的预测准确度。
此外，这里要指出的是：上述的图1及图2、图3及其相应的记载内容仅仅是示例，本公开的技术方案等并不局限于这些示例。例如，图2、3示出的是参考帧和帧间预测等情形，即作为处理对象的图像单元设为帧(图像帧级)，但如上所述，编码层级可以包括：图像集GOP(Group Of Pictures)级、图像帧级及图像块级，甚至更小的图像单元等。故，不言而喻，本公开的技术方案当然也同样可以适用于图像帧级以外的各种层级。
下面，参考图4来说明本公开的码率分配方法。而且，如上所述，该码率分配方法是如图1所示的编码过程中的码率控制模块B(即码率控制方法)的主要部分。
图4示意性示出了本公开实施例的码率分配方法的简要流程图。
如图4所示，所述码率分配方法包括：参考单元及压缩信息的获取(步骤S1)、以及编码码率的分配(步骤S2)。
在参考单元及压缩信息的获取(步骤S1)中，获取作为编码对象的当前第一图像单元的第一参考单元和所述第一参考单元的压缩信息。其中，所述当前第一图像单元可以为例如作为编码对象的未被编码压缩的图像集GOP、图像帧、图像块等之中的任一者。所述第一参考单元可以为例如所述当前第一图像单元的参考图像集GOP、参考帧、参考图像块等中的任一者。另外，所述压缩信息可以包括例如压缩比特数、压缩率、压缩质量等中的至少一个。
在编码码率的分配(步骤S2)中，根据所述第一参考单元的所述压缩信息，对所述当前第一图像单元进行编码比特数即编码码率的分配。
此外，这里优选所述第一参考单元(例如参考帧)与所述当前第一图像单元(当前图像帧)是同一场景图像中的图像单元，或者优选所述第一参考单元(例如参考帧)与所述当前第一图像单元(当前图像帧)是未发生场景突变的场景图像中的图像单元。这样，可以保障更好的参考准确度。其中，所谓场景突变是指场景视频内容会发生显著变化，这种突变可以通过例如比较两帧图像的直方图统计结果、或者其他特征比较来进行判断。
此外，所述第一参考单元可以为所述当前第一图像单元之前的一个第一图像单元，也可以为所述当前第一图像单元之前的多个第一图像单元融合而成的一个第一图像单元。这里，所谓“融合”就是本领域公知的图像融合处理，例如，可以根据多个图像单元中像素和/或特征点的匹配程度分配给各个图像单元相应的权重，从而利用该权重将多个图像单元融合成一个图像单元。
此外，更优选所述第一参考单元为所述当前第一图像单元紧邻的前一个第一图像单元。例如，在当前第一图像单元为当前图像帧的情况下，第一参考单元优选为当前图像帧紧邻的前一个图像帧。这是因 为紧邻的前一个图像帧发生场景突变的可能性很低，从而能够保障更好的参考准确度。
另外，所述当前第一图像单元可以包括多个第二图像单元。例如，在所述当前第一图像单元为图像集GOP的情况下，所述当前第一图像单元即图像集GOP可以包括多个第二图像单元即图像帧；在所述当前第一图像单元为图像帧的情况下，所述当前第一图像单元即图像帧可以包括多个第二图像单元即图像块。相应地，所述第一参考单元可以包括多个第二参考单元。例如，在所述当前第一图像单元为图像集GOP的情况下，所述第一参考单元就为参考图像集GOP，所述第一参考单元即参考图像集GOP可以包括多个作为所述第二图像单元(图像帧)的参考图像的第二参考单元即参考图像帧；在所述当前第一图像单元为图像帧的情况下，所述第一参考单元就为参考图像帧，所述第一参考单元即参考图像帧可以包括多个作为所述第二图像单元(图像块)的参考图像的第二参考单元即参考图像块。
下面，关于图4所示的流程的具体细节，参考图5来进行说明。
图5示意性示出了本公开实施例的码率分配方法中的编码码率分配步骤的简要流程图。
如图5所示，所述编码码率的分配(步骤S2)可以具体包括：码率分配比例的计算(步骤S2-1)和分配编码码率的确定(步骤S2-2)。
在码率分配比例的计算(步骤S2-1)中，可以根据所述第一参考单元的压缩信息计算所述当前第一图像单元中各所述第二图像单元的码率分配比例。
在分配编码码率的确定(步骤S2-2)中，可以根据所述码率分配比例与预先分配的所述当前第一图像单元的编码码率确定每个所述第二图像单元的编码码率。
另外，所述压缩信息可以包括所述第一参考单元的压缩比特数、所述多个第二参考单元的压缩比特数。
这样，所述码率分配比例的计算(步骤S2-1)还可以具体包括：根据每个所述第二参考单元的压缩比特数占所述第一参考单元的压缩比特数的比例计算各所述第二图像单元的码率分配比例。
下面，列举具体实施例来说明本公开的优选实施例，所述优选实施例的主要流程均可以应用到图4、图5所示的流程及其相关流程细节当中。此外，下列实施例例如是以所述压缩信息包括编码过程常被利用到的压缩比特数(编码比特数)为例来进行说明的。当然，不言而喻，当压缩信息包括压缩率、压缩质量等时，本公开的技术方案也同样适用。
[实施例1]
首先，相对于现有技术中采用的以图像块梯度或者方差等来表征图像复杂度以用于码率分配方案，在本公开的所述码率分配方法中，首次提出了以压缩信息来表征图像复杂度的技术方案，即在图4所示的编码码率的分配(步骤S2)中，根据参考单元(例如，参考帧)的压缩信息(例如，压缩比特数)，对作为编码对象的当前图像单元(例如，当前图像帧)进行编码比特数即码率的分配。
图6是用于说明本公开实施例的利用参考帧压缩信息来表征当前图像帧的任意区域复杂度的示意图。
如图6所示，当前图像帧的一个任意形状的图像区域(例如，图像块等)S_cur，使用例如当前图像帧紧邻的前一个图像帧作为前一参考帧(这里，使用前一参考帧是因为一般紧邻的前一个图像帧可以视为同一场景图像中的图像帧，或未发生突变的场景图像中的图像帧)相应位置覆盖的参考帧压缩单元的压缩比特数之和来表征图像区域S_cur的纹理复杂度TC(Texture Complexity)。设S_cur所覆盖的参考帧中的压缩单元CU的编号分别为1，2，...n，则S_cur的纹理复杂度可以由下式(2)表达：
Figure PCTCN2018101569-appb-000002
式中，Bits(CUi)表示第i个参考帧压缩单元CU对应的压缩比特数。这样，利用参考帧压缩后的编码码率(压缩比特数)来表征纹理 复杂度，相比利用图像块梯度或方差等计算方法具有更好的准确度，这是因为：
1.参考帧压缩与视频编码过程一样具有残差预测、熵编码等过程，因而较好地近似了图像块的压缩潜能。
2.由上述可知，在一个结合参考帧压缩的视频编码架构M中，参考帧经压缩后会将包含每个压缩单元比特数的头信息存于外部存储器中，因而不需要额外计算来产生Bits(CUi)。
通常，纹理复杂度被用于作为视频编码中各层级编码的码率分配的依据，由于采用前一帧的压缩信息进行纹理复杂度估计，本公开一般不适合当前图像帧为场景切换的第一帧的情况，而可以适用于同一场景(或未突变场景)的前后帧。此外，利用纹理复杂度对不同编码层级进行码率分配时，存在不同程度的适应性，需要不同程度的约束说明，对此将在下述不同实施例中进行分析和说明。
[实施例2]
图6是用于说明本公开实施例的利用参考帧压缩信息在图像帧级来表征当前图像帧的任意图像区域复杂度的示意图。
如图6所示那样，在所述当前图像单元的所述编码层级为图像帧级或以上(即，包括图像帧级、图像集GOP级、甚至图像集GOP级以上的图像集GOP群级等)的情况下，在图4所示的所述编码码率的分配(步骤S2)中，可以根据与当前图像帧的位置相对应的参考帧即对应参考帧的压缩比特数和所述对应参考帧的未被压缩时的原始码率(原始比特数)来计算出所述对应参考帧的压缩率，然后，利用计算出的所述压缩率，对所述当前图像帧进行码率分配。
具体而言，下面以针对I帧的图像帧级的码率分配为例进行说明。这里针对I帧仅是因为往往I帧的图像质量提高的前提下后续的P帧或B帧的图像质量也会随着相应提高，所以在进行码率分配时也可以仅针对I帧进行码率分配，当然，不言而喻，P帧或B帧也同样适用。由于针对I帧，故这里的S_cur即为整帧图像大小，可以根据纹理复杂度来动态调节所述I帧的码率分配，即例如分配给难压缩的I帧较 多的比特数，而分配给易压缩的P帧或B帧较少的比特数，从而在保证I帧质量的前提下使得后续P帧或B帧的图像质量得到提升。
作为一种算法可以如下式(3)、(4)：
Figure PCTCN2018101569-appb-000003
Figure PCTCN2018101569-appb-000004
式中，Bits(x)表示分配给x的编码码率(编码比特数)，即Bits(I_frame)表示分配给当前的I帧的编码比特数即编码码率，Bits(GOP)表示分配给包括当前图像帧的由多个图像帧构成且编码层级高于所述当前图像帧的上级图像单元(这里，为图像集GOP)编码比特数即编码码率；这里，优选图像集GOP中第一帧为I帧，后续都为P帧或者B帧(通常，这种情形较为常见，当然也不排除存在多个I帧且I帧存在于不同位置的情形)；公式(3)仅为一种示例，该图像集中仅存在I帧和P帧的情况，I_MAX_ratio和P_ratio是预先分配的I帧和P帧之间的码率分配比例；参数α如式(4)所示，其为由与当前图像帧的位置相对应的参考单元帧即对应参考帧的压缩比特数和所述对应参考帧的未被压缩时的原始码率(原始比特数)计算得出的所述对应参考帧的压缩率，其可以反映参考帧的纹理复杂度，压缩率越大，说明参考帧越难压缩，纹理越复杂，反之，说明参考帧越容易压缩，纹理越不复杂。此外，虽然这里优选使用参考帧的压缩率来作为参数α(因为可以表征参考帧的压缩难易程度)，但参数α并非必须是参考帧的压缩率，只要能根据纹理丰富程度调整I帧和P帧或B帧之间的码率分配即可。此外，由式(3)可知，由参数α来调节I帧所占编码码率(编码比特数)的比例，以实现最终针对当前图像帧的码率分配。图1中Bf和Ri之间用虚线连接表示，帧内预测可以不需要从缓存中读取参考帧进行预测，但是可以从缓存中读取参考帧或参考帧压缩信息进行码率分配。在当前图像单元为图像帧级以上的此种情况下不需 要其它约束条件。
此外，这里仅是针对I帧的图像帧级的码率分配的示例，不言而喻，在当前图像单元不是帧而是大于图像帧级的图像集GOP级、甚至图像集GOP级以上的图像集GOP群级等的情况下，要相应地对上述Bits(x)中的x作调整。例如，式(3)中的Bits(I_frame)若变更为Bits(GOP)，则相应地式(3)中的Bits(GOP)要变更为Bits(GOP群)。此外，式(3)中的Bits(GOP)当然也可以是Bits(GOP群)，只要是能由帧构成的比帧大的图像单元(例如GOP、GOP群等)即可。而且，不言而喻，除了I帧以外的P帧或B帧也同样适用本公开实施例的技术方案。
此外，由式(3)可见，本公开实施例的所述码率分配方法中还可以包括：获取由多个当前图像单元(这里，例如当前图像帧)构成且编码层级高于该当前图像单元的上级图像单元(这里，例如由帧构成图像集GOP)的编码码率(编码比特数)，这样，在图4所示的所述码率的分配(步骤S3)中，利用所获取的上级图像单元(这里，例如图像集GOP)的编码码率(编码比特数)，并根据参考单元(这里，例如参考帧)的压缩信息，对所述当前图像单元(这里，例如当前图像帧)进行码率分配。
由此，通过利用参考帧的压缩信息(例如，压缩比特数)来进行码率分配，即利用参考帧的压缩难易程度来表征当前图像帧的图像纹理复杂程度，从而实现更准确、更简便且消耗资源少的码率分配，以使编码处理时的图像重构能够实现重构质量的最大化。
[实施例3]
下面，参照图7进一步说明另外一种编码比特数分配的情形。
图7是用于说明本公开实施例的利用参考帧压缩信息在图像块级来表征当前图像帧的任意图像区域复杂度的示意图。
如图7所示那样，在所述当前图像单元的所述编码层级小于图像帧级(即，包括任意图像区域(即图像块)、甚至更小的例如压缩单元等)的情况下，在图4所示的所述编码码率的分配(步骤S2)中，可以根据与当前图像单元(例如，当前图像帧中的n个图像块S_cur_1， S_cur_2…S_cur_n)的位置相对应的参考单元即对应参考单元(例如，参考帧中的n个参考图像块S_ref_1，S_ref_2…S_ref_n)的压缩比特数，计算出所述对应参考单元在编码层级高于所述当前图像单元的上级图像单元(例如，当前图像帧)的参考单元(例如，参考帧)整体中所占压缩比特数的比例，然后，利用计算出的所述比例，对所述当前图像单元(例如图像块S_cur_1等)进行码率分配。
具体而言，下面以针对I帧的图像块级的码率分配为例进行说明。同样，这里针对I帧仅是因为往往I帧的图像质量提高的前提下后续的P帧或B帧的图像质量也会随着相应提高，当然，不言而喻，P帧或B帧也同样适用。此外，优选图像块级为大范围的图像块，即图6所示的n值尽量小(例如，n可以小于4)，也可以例如视频编码中的大块slice或者tile划分。图6示出了一种示例，假设需要对当前图像帧的n个图像块进行码率分配，使用例如前一参考帧S_ref_1、S_ref_2、…、S_ref_n对应位置的参考帧压缩比特数来估计S_cur_1、S_cur_2、…、S_cur_n的纹理复杂度，码率分配计算可以如下式(5)：
Figure PCTCN2018101569-appb-000005
式中，Bits(S_cur_x)表示分配给当前图像帧中的当前图像块S_cur_x的编码码率(编码比特数)，Bits(I_frame)表示分配给由多个所述当前图像块构成且编码层级高于所述当前图像块的上级图像单元(这里，例如为当前图像帧I_frame)编码比特数即编码码率；S_cur_x为当前图像块；S_ref_x为当前图像块S_cur_x位置对应的参考图像块；n为大于1的整数。函数TC为压缩比特数。
在另外的实施例中，当前图像帧的编码码率也可以用类似式(5)的方法进行分配，即根据当前图像帧对应的参考图像帧在参考GOP或参考GOP集中所占的比例对当前图像帧所属的GOP或GOP集的编码码率进行分配。此外，Bits(I_frame)优选由本公开中的上述实施例2的码率分配而得，这样能够进一步实现更准确的码率分配，从而实现编码处理时的图像重构质量的进一步最大化。当然，不言而喻， Bits(I_frame)也可以通过如公式(5)的分配方式得到，或者由现有的例如码率平均分配法获得，或者由现有的例如利用图像块的梯度或方差等来进行码率分配的非码率平均分配法获得。
此外，在当前图像单元为小于图像帧级的此种图像块级的情况下约束条件为：对于静止视频序列，无其它约束；对于运动视频序列，图像块越大，码率分配越准确，这是因为大图像块纹理复杂度估计对图像内容的运动不敏感，而小块影响较大。
此外，这里所述图像区域即图像块的形状可以为任意形状。
此外，这里仅是针对I帧的图像块级的码率分配的示例，不言而喻，除了I帧以外的P帧或B帧也同样适用本公开实施例的技术方案。此外，式(5)中的Bits(I_frame)当然也可以是Bits(GOP)、或Bits(GOP群)，只要是能由图像块构成的比图像块大的图像单元(例如帧、GOP、GOP群等)即可。
此外，由式(3)可见，本公开实施例的所述码率分配方法中还可以包括：获取由多个当前图像单元(这里，例如当前图像块)构成且编码层级高于该当前图像单元的上级图像单元(这里，例如由图像块构成图像帧)的编码码率(编码比特数)，这样，在图4所示的所述编码码率的分配(步骤S2)中，利用所获取的上级图像单元(这里，例如帧)的编码码率(编码比特数)，并根据参考单元(这里，例如参考图像块)的压缩信息，对所述当前图像单元(这里，例如当前图像块)进行码率分配。
由此，通过利用参考帧的图像块的压缩信息(例如，压缩比特数)来进行码率分配，即利用参考帧的图像块的压缩难易程度来表征图像纹理复杂程度，从而实现更准确、更简便且消耗资源少的码率分配，以使编码处理时的图像重构能够实现重构质量的最大化。
[实施例4]
下面，参照图8进一步说明编码层级小于图像帧级的情形之中的压缩单元级的情形。其中，所述压缩单元级为视频编码的基本编码单元，例如H.264中的MB、H.265中的CTU/CU等。
图8是用于说明本公开实施例的利用参考帧压缩信息在压缩单元级来表征当前图像帧的任意图像区域复杂度的示意图。
如图8所示那样，在所述当前图像单元的所述编码层级为更小层级例如压缩单元级的情况下，在图4所示的所述编码码率的分配(步骤S2)中，可以根据与当前图像单元(例如，当前图像块S_cur中的n个压缩单元1，2，…，n)的位置相对应的参考单元即对应参考单元(例如，参考图像块S_ref中的n个参考压缩单元1，2，…，n)的压缩比特数，计算出所述对应参考单元在编码层级高于所述当前图像单元的上级图像单元(例如，当前图像块S_cur)的参考单元(例如，参考图像块S_ref)整体中所占压缩比特数的比例，然后，利用计算出的所述比例，对所述当前图像单元(例如图像块S_cur的压缩单元1等)进行码率分配。
具体而言，下面以针对I帧的图像块中的压缩单元的码率分配为例进行说明。同样，这里针对I帧仅是因为往往I帧的图像质量提高的前提下后续的P帧或B帧的图像质量也会随着相应提高，当然，不言而喻，P帧或B帧也同样适用。图7示出了一种示例，假设需要对当前图像帧的图像块S_cur中的压缩单元1、2、…、n进行码率分配，使用例如前一参考帧的参考图像块S_ref的参考压缩单元1、2、…、n对应位置的参考帧压缩比特数来估计图像块S_cur中的压缩单元1、2、…、n的纹理复杂度，码率分配计算可以如下式(6)：
Figure PCTCN2018101569-appb-000006
式中，Bits(S_cu_m)表示分配给当前图像帧的当前图像块S_cur中的当前压缩单元S_cu_m的编码码率(编码比特数)，Bits(S_cur)表示分配给由多个所述当前压缩单元构成且编码层级高于所述当前压缩单元的上级图像单元(这里，例如为当前图像块S_cur)的编码比特数即编码码率；S_cu_m为当前压缩单元；S_ref_m为当前压缩单元S_cu_m位置对应的参考压缩单元；n为大于1的整数。函数TC为上述式(2)的纹理复杂度函数。
此外，Bits(S_cur)优选由本公开中的上述实施例3的码率分配而得，这样能够进一步实现更准确的码率分配，从而实现编码处理时的图像重构质量的进一步最大化。当然，不言而喻，Bits(S_cur)也可以由现有的例如码率平均分配法获得，或者由现有的例如利用图像块的梯度或方差等来进行码率分配的非码率平均分配法获得。
此外，在当前图像单元为小于图像帧级的此种压缩单元级的情况下约束条件为：对于静止视频序列，无其它约束；对于运动视频序列，需要保证当前图像块S_cur所对应的图像内容如图7所示为刚体平移运动。
此外，这里所述图像区域即图像块的形状可以为任意形状。
此外，这里仅是针对I帧的图像块级的码率分配的示例，不言而喻，除了I帧以外的P帧或B帧也同样适用本公开实施例的技术方案。此外，式(7)中的Bits(S_cur)当然也可以是Bits(I_frame)、Bits(GOP)、或Bits(GOP群)，只要是能由压缩单元构成的比压缩单元大的图像单元(例如图像块、帧、GOP、GOP群等)即可。
此外，由式(3)可见，本公开实施例的所述码率分配方法中还可以包括：获取由多个当前图像单元(这里，例如当前压缩单元)构成且编码层级高于该当前图像单元的上级图像单元(这里，例如由压缩单元构成图像块)的编码码率(编码比特数)，这样，在图4所示的所述编码码率的分配(步骤S2)中，利用所获取的上级图像单元(这里，例如图像块)的编码码率(编码比特数)，并根据参考单元(这里，例如参考压缩单元)的压缩信息，对所述当前图像单元(这里，例如当前压缩单元)进行码率分配。
由此，通过利用参考帧的图像块中的压缩单元的压缩信息(例如，压缩比特数来进行码率分配，即利用参考帧的图像块中的压缩单元的压缩难易程度来表征图像纹理复杂程度，从而实现更准确、更简便且消耗资源少的码率分配，以使编码处理时的图像重构能够实现重构质量的最大化。
此外，上述各实施例仅示例了两层编码层级的情形，当然，还会有 两层以上编码层级的情形。在有两层以上编码层级的情况下，在进行获取各个上级图像单元的编码码率的计算时，既可以利用本公开实施例的所述编码码率分配方法来计算，也可以利用现有技术的例如码率平均分配法，或者梯度或方差等的非码率平均分配法来计算，这是不言而喻的。
此外，如上所述，本公开实施例的所述码率分配方法是图1所示的码率控制方法中一个步骤，这样，作为本公开另一方面可以提供一种码率控制方法，其可以包括：进行本公开实施例的所述码率分配方法(例如，图4、5流程图所示)，对所述当前图像单元分配编码码率，作为目标码率；在所述当前第一图像单元的编码过程中调整编码参数，使得实际编码码率逼近所述目标码率。
由此，同样可以使编码处理时的图像重构能够实现重构质量的最大化。
此外，不言而喻，本公开实施例的所述码率分配方法及另一方面的所述码率控制方法同样可以作为计算机程序而存储于计算机可读存储介质中，该计算机程序可以包括代码/计算机可执行指令，使计算机执行例如本公开的图4、图5这样的流程图所示的所述码率分配方法及其变形。
此外，计算机可读存储介质，例如可以是能够包含、存储、传送、传播或传输指令的任意介质。例如，可读存储介质可以包括但不限于电、磁、光、电磁、红外或半导体系统、装置、器件或传播介质。可读存储介质的具体示例包括：磁存储装置，如磁带或硬盘(HDD)；光存储装置，如光盘(CD-ROM)；存储器，如随机存取存储器(RAM)或闪存；和/或有线/无线通信链路。
另外，计算机程序可被配置为具有例如包括计算机程序模块的计算机程序代码。应当注意，模块的划分方式和个数并不是固定的，本领域技术人员可以根据实际情况使用合适的程序模块或程序模块组合，当这些程序模块组合被计算机(或处理器)执行时，使得计算机可以执行例如上面结合图4-5所描述的码率分配方法的流程及其变形。
本领域技术人员可以理解，本公开的各个实施例和/或权利要求中记 载的特征可以进行多种组合或/或结合，即使这样的组合或结合没有明确记载于本公开中。特别地，在不脱离本公开精神和教导的情况下，本公开的各个实施例和/或权利要求中记载的特征可以进行多种组合和/或结合。所有这些组合和/或结合均落入本公开的范围。
尽管已经参照本公开的特定示例性实施例示出并描述了本公开，但是本领域技术人员应该理解，在不背离所附权利要求及其等同物限定的本公开的精神和范围的情况下，可以对本公开进行形式和细节上的多种改变。因此，本公开的范围不应该限于所述实施例，而是应该不仅由所附权利要求来进行确定，还由所附权利要求的等同物来进行限定。

Claims (16)

  1. 一种码率分配方法，其特征在于，包括：
    获取作为编码对象的当前第一图像单元的第一参考单元和所述第一参考单元的压缩信息；
    根据所述第一参考单元的所述压缩信息，对所述当前第一图像单元进行编码码率的分配。
  2. 根据权利要求1所述的码率分配方法，其特征在于，
    所述当前第一图像单元包括多个第二图像单元，所述第一参考单元包括多个第二参考单元，所述第二参考单元是所述第二图像单元的参考图像。
  3. 根据权利要求2所述的码率分配方法，其特征在于，
    所述根据所述第一参考单元的所述压缩信息，对所述当前第一图像单元进行编码码率的分配包括：
    根据所述第一参考单元的压缩信息计算所述当前第一图像单元中各所述第二图像单元的码率分配比例；
    根据所述码率分配比例与预先分配的所述当前第一图像单元的编码码率确定每个所述第二图像单元的编码码率。
  4. 根据权利要求3所述的码率分配方法，其特征在于，
    所述第一图像单元为图像集、图像帧或图像块。
  5. 根据权利要求4所述的码率分配方法，其特征在于，
    所述压缩信息包括所述第一参考单元的压缩比特数、所述多个第二参考单元的压缩比特数。
  6. 根据权利要求5所述的码率分配方法，其特征在于，
    所述根据所述第一参考单元的压缩信息计算所述当前第一图像单元中各所述第二图像单元的码率分配比例包括：
    根据每个所述第二参考单元的压缩比特数占所述第一参考单元的压缩比特数的比例计算各所述第二图像单元的码率分配比例。
  7. 根据权利要求3所述的码率分配方法，其特征在于，
    所述第一图像单元是图像集，所述第二图像单元是图像帧，所述第二图像单元包括I帧、P帧、B帧中至少一种。
  8. 根据权利要求7所述的码率分配方法，其特征在于，
    所述压缩信息包括所述第一参考单元的压缩率。
  9. 根据权利要求8所述的码率分配方法，其特征在于，
    所述根据所述第一参考单元的压缩信息计算所述当前第一图像单元中各所述第二图像单元的码率分配比例包括：
    根据所述第一参考单元的压缩率调整预先分配的所述当前第二图像单元中I帧和P帧的比例，或I帧和B帧的比例，或I帧、P帧和B帧的比例得到所述I帧、P帧或B帧的码率分配比例。
  10. 根据权利要求8所述的码率分配方法，其特征在于，
    所述预先分配的所述当前第二图像单元中I帧和P帧，或I帧和B帧，或I帧、P帧和B帧的比例是通过离线统计得到的I帧和P帧，或I帧和B帧，或I帧、P帧和B帧能够分配的最大比例。
  11. 根据权利要求1所述的码率分配方法，其特征在于，
    所述第一参考单元为所述当前第一图像单元之前的一个第一图像单元或多个第一图像单元融合而成的一个第一图像单元。
  12. 根据权利要求11所述的码率分配方法，其特征在于，
    所述第一参考单元为所述当前第一图像单元紧邻的前一个第一图像单元。
  13. 一种码率控制方法，包括：
    进行权利要求1-11中任一项所述的码率分配方法，对所述当前第一图像单元分配编码码率，作为目标码率；
    在所述当前第一图像单元的编码过程中调整编码参数，使得实际编码码率逼近所述目标码率。
  14. 根据权利要求13所述的码率控制方法，其中，
    所述编码参数包括所述编码过程中的量化参数。
  15. 一种编码器，至少包括：
    量化模块，利用量化参数，对编码变换的变换系数进行量化；
    码率控制模块，通过执行权利要求14所述的码率控制方法来调整所述量化模块的所述量化参数。
  16. 一种计算机可读的记录介质，存储有可执行指令，该指令被处理器执行时使该处理器执行权利要求1-12中任一项所述的码率分配方法或者权利要求13或14所述的码率控制方法。
PCT/CN2018/101569 2018-08-21 2018-08-21 码率分配方法、码率控制方法、编码器和记录介质 WO2020037501A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/101569 WO2020037501A1 (zh) 2018-08-21 2018-08-21 码率分配方法、码率控制方法、编码器和记录介质
CN201880042003.8A CN110800298A (zh) 2018-08-21 2018-08-21 码率分配方法、码率控制方法、编码器和记录介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/101569 WO2020037501A1 (zh) 2018-08-21 2018-08-21 码率分配方法、码率控制方法、编码器和记录介质

Publications (1)

Publication Number Publication Date
WO2020037501A1 true WO2020037501A1 (zh) 2020-02-27

Family

ID=69438496

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/101569 WO2020037501A1 (zh) 2018-08-21 2018-08-21 码率分配方法、码率控制方法、编码器和记录介质

Country Status (2)

Country Link
CN (1) CN110800298A (zh)
WO (1) WO2020037501A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111277864B (zh) 2020-02-18 2021-09-10 北京达佳互联信息技术有限公司 直播数据的编码方法、装置、流转系统及电子设备
CN113099161A (zh) * 2021-04-13 2021-07-09 北京中科深智科技有限公司 一种基于深度神经网络的会议视频重建方法和系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1138932A (zh) * 1994-11-23 1996-12-25 摩托罗拉公司 在视频压缩系统中用于确定比特分配的方法和装置
US5933194A (en) * 1995-11-01 1999-08-03 Samsung Electronics Co., Ltd Method and circuit for determining quantization interval in image encoder
CN1909662A (zh) * 2006-08-07 2007-02-07 浙江大学 采用帧目标比特数不平均分配的视频压缩码率控制方法
CN105847796A (zh) * 2016-03-31 2016-08-10 乐视控股（北京）有限公司 一种用于视频编码的比特分配方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7075987B2 (en) * 2002-09-23 2006-07-11 Intel Corporation Adaptive video bit-rate control
CN100481943C (zh) * 2006-12-30 2009-04-22 清华大学 一种视频编码码率控制方法
US20110255594A1 (en) * 2010-04-15 2011-10-20 Soyeb Nagori Rate Control in Video Coding
CN102123282B (zh) * 2011-03-10 2013-02-27 西安电子科技大学 基于Wyner-Ziv视频编码系统的GOP层编码方法
CN102868883B (zh) * 2012-06-26 2013-11-13 中国科学技术大学 一种视频编码的码率控制方法
CN105898303A (zh) * 2015-12-24 2016-08-24 乐视云计算有限公司 码率控制方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1138932A (zh) * 1994-11-23 1996-12-25 摩托罗拉公司 在视频压缩系统中用于确定比特分配的方法和装置
US5933194A (en) * 1995-11-01 1999-08-03 Samsung Electronics Co., Ltd Method and circuit for determining quantization interval in image encoder
CN1909662A (zh) * 2006-08-07 2007-02-07 浙江大学 采用帧目标比特数不平均分配的视频压缩码率控制方法
CN105847796A (zh) * 2016-03-31 2016-08-10 乐视控股（北京）有限公司 一种用于视频编码的比特分配方法及装置

Also Published As

Publication number Publication date
CN110800298A (zh) 2020-02-14

Similar Documents

Publication Publication Date Title
US9258567B2 (en) Method and system for using motion prediction to equalize video quality across intra-coded frames
TWI399097B (zh) 用於編碼視訊之系統及方法，以及電腦可讀取媒體
CN1283100C (zh) 基于时间和空间复杂性自适应编码运动图像的方法及装置
RU2559738C2 (ru) Способ и устройство для кодирования/декодирования вектора движения
CN110740318A (zh) 用于视频处理和视频译码的自动自适应长期参考帧选择
TWI451766B (zh) 用於視頻編碼之電力效率高的運動估計技術
JP2004527960A (ja) メディアプロセッサにおけるｍｐｅｇ２デコード処理の動的複雑性予測及び調整
US8665955B2 (en) Method of storing pictures in a memory using compression coding and cost function including power consumption
CN102271257A (zh) 图像处理设备和方法及程序和计算机可读介质
JP2015530035A (ja) 映像符号化方法、およびこの方法を用いる映像符号化装置
TW202218428A (zh) 圖像編碼方法、圖像解碼方法及相關裝置
Pan et al. Frame-level Bit Allocation Optimization Based on<? brk?> Video Content Characteristics for HEVC
CN110832856A (zh) 用于减小视频编码波动的系统及方法
CN117480778A (zh) 残差编码和视频编码方法、装置、设备和系统
WO2020037501A1 (zh) 码率分配方法、码率控制方法、编码器和记录介质
US20150163486A1 (en) Variable bitrate encoding
CN115118976A (zh) 一种图像编码方法、可读介质及其电子设备
JP2024514348A (ja) ビデオ符号化方法、装置と電子機器
US20110249959A1 (en) Video storing method and device based on variable bit allocation and related video encoding and decoding apparatuses
WO2022021422A1 (zh) 视频编码方法、编码器、系统以及计算机存储介质
JPH0998427A (ja) 動画像符号化装置
US7804897B1 (en) Method for implementing an improved quantizer in a multimedia compression and encoding system
CN113973205A (zh) 基于视频内容特征的码率控制比特分配方法及存储介质
Chen et al. CNN-based fast HEVC quantization parameter mode decision
CN114793282A (zh) 带有比特分配的基于神经网络的视频压缩

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18931020

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18931020

Country of ref document: EP

Kind code of ref document: A1