CN116916027A - Code rate control method and device for image block level of P frame or B frame - Google Patents

Code rate control method and device for image block level of P frame or B frame Download PDF

Info

Publication number
CN116916027A
CN116916027A CN202310781599.6A CN202310781599A CN116916027A CN 116916027 A CN116916027 A CN 116916027A CN 202310781599 A CN202310781599 A CN 202310781599A CN 116916027 A CN116916027 A CN 116916027A
Authority
CN
China
Prior art keywords
frame
coded
image block
inter
coding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310781599.6A
Other languages
Chinese (zh)
Inventor
刘鹏飞
温安君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ASR Microelectronics Co Ltd
Original Assignee
ASR Microelectronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ASR Microelectronics Co Ltd filed Critical ASR Microelectronics Co Ltd
Priority to CN202310781599.6A priority Critical patent/CN116916027A/en
Publication of CN116916027A publication Critical patent/CN116916027A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application discloses a code rate control method for image block level of a P frame or a B frame. And calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded. And according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded, combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame, and calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded. The application takes the image block level quantization parameter as the operation core of the image block level code rate control algorithm of the P frame or the B frame, and is more suitable for hardware realization.

Description

Code rate control method and device for image block level of P frame or B frame
Technical Field
The application relates to a digital video coding technology, in particular to a code rate control method of image block level of P frame or B frame, which is suitable for hardware realization.
Background
Video coding is a technique that characterizes video information by compressing redundant components in video images and using as little data as possible. Common video coding standards are HEVC (High Efficiency Video Coding, high efficiency video coding, also known as h.265), AVC (Advanced Video Coding, advanced video coding, also known as h.264), etc.
Video coding techniques use image blocks as the most basic coding units. For example, in HEVC, a basic Unit of video Coding is a CU (Coding Unit). A CU may be an image block of 64 pixels by 64 pixels, 32 pixels by 32 pixels, 16 pixels by 16 pixels, 8 pixels by 8 pixels size. An image block of 64 pixels by 64 pixels size is also called LCU (Largest Coding Unit, maximum coding unit).
In order to increase the video compression rate as much as possible, a common video coding algorithm often adopts a lossy compression algorithm, so that the coded and reconstructed video is different from the original video (distortion is generated). Both HEVC and AVC belong to lossy compression algorithms. The coding performance of a lossy compression algorithm needs to be commonly measured by the coding bit Rate (Rate) and the Distortion introduced by the coding (loss). In the video coding process, the coding bit rate R and the coding distortion D are mutually restricted and contradicted, and reducing the coding bit rate R tends to increase the coding distortion D, and reducing the coding distortion D tends to increase the coding bit rate R. Rate distortion optimization (Rate Distortion Optimization, RDO) is to reduce the coding distortion D as much as possible at a certain coding bit rate R or to reduce the coding bit rate R as much as possible at a certain coding distortion D. Common video coding algorithms all use a rate distortion optimization method based on lagrangian multipliers (Lagrange multiplier) λ, which is to obtain the minimum coding rate distortion Cost (Rate Distortion Cost, RD Cost) J, j=d+λ×r.
In video coding, each coding mode has coding distortion D and coding bit rate R, and coding rate distortion cost J for each coding mode can be calculated. When video coding is performed, the coding mode with the minimum coding rate distortion cost J is selected for coding, so that the optimal coding performance can be obtained. The process of selecting an optimal coding Mode for each image block to be coded by the encoder through a rate distortion optimization method is called Mode selection (Mode Decision) of the image block to be coded.
In order to remove information redundancy in the spatial and temporal domains, video coding techniques encode input video frames using intra-frame coding techniques and inter-frame coding techniques. Only intra-frame coding techniques are used to encode I-frames. The hybrid encoding is P-frames and B-frames using intra-frame and inter-frame coding techniques. The type of encoded frame is determined by the group of pictures (GOP, group Of Pictures) parameters set by the user. Fig. 1 shows a common group of pictures structure for an I-frame, P-frame or B-frame. When the I frame is encoded, the information of the video frame is only used for encoding, and the information of other encoded video frames is not referred to. All image blocks within an I-frame are encoded as intra-coded blocks. P-frames and B-frames are encoded using information not only from the present video frame, but also from one or more other encoded video frames. The image blocks inside the P-frame and the B-frame may be encoded as intra-frame encoded blocks or inter-frame encoded blocks, depending on which encoding rate the distortion cost J is smaller.
In real life, the channel bandwidth capacity used to transmit video is limited. If the coding bit rate of the video is too high, the video transmission congestion and even packet loss are caused if the channel bandwidth capacity is exceeded. If the coding bit rate of the video is too low, the channel bandwidth is not fully utilized, and higher video quality cannot be obtained. Therefore, it is necessary to control the output code rate of the video encoder to match the channel bandwidth capacity using a code rate control technique.
Rate Control (Rate Control) aims to make the output code Rate of the video encoder equal to a preset target code Rate by adjusting the coding parameters of the video encoder, and simultaneously reduce coding distortion D as much as possible so as to improve video coding quality. The common code rate control algorithm generally realizes the code rate control task through two links of target bit allocation and target bit control.
The target bit allocation links are typically performed at three levels, GOP (group of pictures, i.e., a set of temporally consecutive video frames), video frame level, and block level, respectively. The target comparison allocation at GOP level is to obtain the target number of coded bits of GOP based on the target coded bit rate of video. The target bit allocation of the video frame level is to obtain the target coding bit number of each video frame in the GOP according to the target coding bit number of the GOP. The target bit allocation of the image block level is to obtain the target coding bit number of each image block in the video frame according to the target coding bit number of the video frame and the bit allocation weight of each image block in the video frame.
The target bit control links are typically performed at the video frame level and at the image block level. At the video frame level, a lagrangian multiplier λ and a quantization parameter (quantization parameter) QP for the video frame level are calculated from the target number of encoding bits of the video frame. At the image block level, the Lagrangian multiplier lambda and the quantization parameter QP of the image block level of the image block are calculated according to the target coding bit number of a certain image block, the Lagrangian multiplier lambda and the quantization parameter QP of the video frame level of the video frame to which the image block belongs and the Lagrangian multiplier lambda and the quantization parameter QP of the image block level of the previous coded image block, and are used for the video coding process of the image block.
Both the lagrangian multiplier λ and the quantization parameter QP are divided into video frame level and picture block level. The lagrangian multiplier λ and the quantization parameter QP at the video frame level are used in the target bit allocation and target bit control link at the video frame level (i.e., rate control at the video frame level), and the lagrangian multiplier λ and the quantization parameter QP at the image block level are used in the target bit allocation and target bit control link at the image block level (i.e., rate control at the image block level). The lagrangian multipliers at the video frame level are sometimes referred to simply as video frame level lagrangian multipliers, and lagrangian multipliers for video frames. The lagrangian multipliers at the image block level are sometimes simply referred to as image block level lagrangian multipliers, and lagrangian multipliers for image blocks. The abbreviation for quantization parameter and so on.
Because of the large amount of computation in video coding algorithms, to increase video coding speed and achieve real-time coding and transmission, the industry typically uses Application Specific Integrated Circuits (ASICs) to hardware accelerate the video coding process. An application specific integrated circuit that hardware accelerates the video encoding process is generally referred to simply as a hardware video encoder.
Among common rate control algorithms, the rate control algorithm at the image block level of the P frame or the B frame generally comprises a large number of complex floating point operations, which is not beneficial to hardware implementation. For example, in the HEVC rate control algorithm proposed by JCTVC-K0103, the rate control algorithm at the image block level of the P frame or B frame not only includes a large number of floating point multiplication and floating point division operations, but also includes more complex floating point exponent and floating point logarithm operations, so that the hardware implementation difficulty is very high and the cost is very high. Meanwhile, in a code rate control algorithm represented by the JCTCVC-K0103 proposal, more multiplication and division operations are introduced into the operation taking the Lagrangian multiplier as a core, and the calculation cost, the storage cost and the bandwidth cost are large when hardware is realized due to the wide value range, high precision requirement and large bit width occupation of the Lagrangian multiplier.
Therefore, it is necessary to design a code rate control method suitable for the P-frame or B-frame image block level of a hardware video encoder, more accurately finish the target bit allocation of the image block in the P-frame or B-frame on the basis of not introducing a large amount of floating point operations, and calculate the target coding bit number of the image block and the corresponding coding parameters (such as lagrangian multiplier or quantization parameters), so that the actual coding bit number of the image block is more close to the target coding bit number, and the code rate control process of the P-frame or B-frame is more accurate.
Disclosure of Invention
The technical problems to be solved by the application are as follows: how to make rate control at the picture block level of P-frames or B-frames more suitable for implementation in hardware.
In order to solve the technical problems, the application provides a code rate control method of the image block level of a P frame or a B frame, which comprises the following steps. Step S1: and carrying out inter-frame coding mode prediction on the image block to be coded, namely calculating the prediction cost corresponding to each inter-frame coding mode of the image block to be coded, and only reserving one or more inter-frame coding modes with the lowest prediction cost as inter-frame coding mode candidates and reserving the inter-frame coding mode prediction cost corresponding to each inter-frame coding mode candidate. Step S2: and selecting the minimum inter-frame coding mode prediction cost from inter-frame coding mode prediction costs corresponding to all inter-frame coding mode candidates of the image block to be coded to represent the inter-frame coding complexity of the image block to be coded, and taking the sum of the inter-frame coding complexity of all the image blocks in the video frame as the inter-frame coding complexity of the video frame. Step S3: calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded; the inter-frame coding complexity of each image block inside the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block at the same position inside the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame. Step S4: and according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded, combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame, and calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded.
Preferably, in the step S2, only inter-coding complexity at the picture block level and the video frame level of each of the P-frame and the B-frame is calculated.
Further, in the step S3, if the current P frame or B frame to be encoded is the first P frame or B frame after a certain I frame, the corresponding previous encoded P frame or B frame is the last P frame or B frame before the I frame.
Further, in the step S3, the calculation formula is as follows:wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the inter-coding complexity, ω, of the current image block to be coded pic For the inter-frame coding complexity of the current P frame or B frame to be coded, T Pic Bit is the target coding Bit number of the current P frame or B frame to be coded H The bit number needed for the pre-estimated header information of the current P frame or B frame to be coded.
Further, the step S4 is represented by the formula six:obtaining; wherein QP is new Representing quantization parameters of image blocks to be encoded in the P frame or the B frame to be encoded; f (f) -1 Is an inverse of the first function f; i (n) new The target coding bit number of the image block to be coded in the P frame or B frame to be coded; i (n-1) old Representing the actual number of encoding bits of the co-located picture block inside the previous encoded P-frame or B-frame of the P-frame or B-frame to be encoded; g is a second function, g (1) is a constant; QP (QP) old Quantization parameters representing co-located image blocks within a previously encoded P-frame or B-frame.
Further, the first function f is represented by formula one:acquiring; the first function f characterizes: the same image block adopts different image block level quantization parameters when P frame or B frame coding is carried out, and the ratio of different actual coding bit numbers obtained by the image block is related to the difference value of different image block level quantization parameters adopted by the image block. The second function g is represented by equation two: />Acquiring; the second function g characterizes: and when P frame or B frame coding is carried out, the image blocks at the same position on different video frames adopt the same image block level quantization parameters, and the ratio of the actual coding bit number obtained by the image blocks at the same position on different video frames is related to the ratio of the inter-frame coding complexity of the two image blocks at the same position.
Further, deriving based on the first formula and the second formula to obtain a third formula: further deriving equation four: wherein the function f -1 Is the inverse of function f; let m=n-1, formula four be converted to formula five +.> Let omega again n ＝ω n-1 I.e. the inter-coding complexity of the image block to be coded inside the n-th frame is replaced by the inter-coding complexity of the image block to be coded in the same position inside the n-1 th frame>The sixth equation is obtained.
Optionally, step S5 is further included after step S4; step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
The application also provides a code rate control device of the image block level of the P frame or the B frame, which comprises an inter-frame coding mode prediction module, an inter-frame coding complexity calculation module, an image block bit allocation module and an image block bit control module. The inter-frame coding mode prediction module is used for predicting the inter-frame coding mode of the image block to be coded to obtain inter-frame coding mode candidates and prediction cost of the image block to be coded. The inter-frame coding complexity calculation module is used for selecting the minimum inter-frame coding mode prediction cost to represent the inter-frame coding complexity of the image block to be coded; the sum of the inter-coding complexity of all image blocks in a video frame is taken as the inter-coding complexity of the video frame. The image block bit allocation module is used for calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded; the inter-frame coding complexity of each image block inside the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block at the same position inside the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame. The image block bit control module is used for calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded and combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame.
Further, the image block bit control module is followed by a scaling module; the conversion module is used for obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
The application has the technical effects that: (1) The image block level quantization parameter is used as an operation core of an image block level code rate control algorithm of a P frame or a B frame, and the operation logic is relatively simple and is more suitable for hardware realization. (2) The image preprocessing stage is not required to be added, the hardware cost is low, and the realization cost is low. (3) And the information of the image blocks at the same position in the encoded video frame is utilized to calculate the image block level quantization parameter of the P frame or the B frame to be encoded, so that the calculation accuracy is high, and the accuracy of the image block level code rate control of the P frame or the B frame is effectively improved.
Drawings
Fig. 1 is a schematic diagram of a common group of pictures structure of I, P, and B frames.
Fig. 2 is a flow chart of a method for controlling the bit rate at the picture block level of a P frame or a B frame according to the present application.
Fig. 3 is a schematic diagram of an image block at the same position inside an mth frame, an n-1 frame, and an nth frame.
Fig. 4 is a schematic structural diagram of a coding rate control device at the picture block level of a P frame or a B frame according to the present application.
The reference numerals in the drawings illustrate: 1 is an intra-frame coding mode prediction module, 2 is an intra-frame coding complexity calculation module, 3 is an image block bit allocation module, 4 is an image block bit control module, and 5 is a conversion module.
Detailed Description
Referring to fig. 2, the method for controlling the bit rate at the image block level of the P frame or the B frame according to the present application includes the following steps.
Step S1: and carrying out inter-frame coding mode prediction on the image block to be coded, namely calculating the prediction cost corresponding to each inter-frame coding mode of the image block to be coded, and only reserving one or more inter-frame coding modes with the lowest prediction cost as inter-frame coding mode candidates and reserving the inter-frame coding mode prediction cost corresponding to each inter-frame coding mode candidate. This step is a step in the existing video coding process where inter-coding mode prediction is performed for all tiles of P and B frames.
In a hardware video encoder, in order to reduce cost and power consumption, when a mode selection is performed on an image block to be encoded using a rate distortion optimization method, a process of mode selection is generally divided into an encoding mode prediction stage and an encoding mode rate distortion optimization stage. The coding mode prediction stage is completed by an intra-frame or inter-frame coding mode prediction module, and the coding mode rate distortion optimization stage is completed by an intra-frame or inter-frame coding mode rate distortion optimization module.
Essentially, the coding mode prediction stage and the coding mode rate distortion optimization stage adopt a rate distortion optimization method to screen the coding mode of the image block to be coded. The two stages differ mainly in the accuracy of the calculation method of the coding rate distortion cost J employed. In the coding mode prediction stage, a coding rate distortion cost J calculation method with less calculation amount and lower precision is generally adopted; calculating the coding distortion D typically uses the SAD (Sum of Absolute Difference, sum of absolute errors) algorithm or the SATD (Sum of Absolute Transformed Difference, sum of absolute transform errors) algorithm; calculating the encoding bit rate R typically uses a simplified bit rate estimation algorithm, such as an exponential golomb algorithm; and the coding mode prediction stage typically does not introduce transform and quantization operations. In the code mode rate distortion optimization stage, a code rate distortion cost J calculation method with larger calculation amount and higher precision is generally adopted; calculating the coding distortion D typically uses an SSD (Sum of Squared Differences, sum of square errors) algorithm; calculating the coding bit rate R typically uses CABAC (Context Adaptive Binary Arithmatic Coding, context adaptive binary arithmetic coding) algorithm or CAVLC (Context Adaptive Variable Length Coding ) algorithm; and the coding mode rate distortion optimization stage typically introduces transform and quantization operations. Conventionally, a coding rate distortion cost J of low calculation amount and low precision calculated in the coding mode prediction stage is referred to as a coding mode prediction cost.
When the image block to be encoded is encoded, the encoding mode prediction stage performs preliminary screening on a plurality of encoding modes, and one or a plurality of encoding mode candidates with lower prediction cost are screened out by calculating the prediction cost corresponding to each encoding mode. And in the subsequent coding mode rate distortion optimization stage, accurate calculation is carried out on prediction costs corresponding to the coding mode candidates to obtain accurate coding rate distortion costs J of candidate coding modes, and finally, a coding mode with the minimum coding rate distortion costs J is selected by a mode selection module to serve as an optimal coding mode of the image block to be coded.
Step S2: and selecting the minimum inter-frame coding mode prediction cost from inter-frame coding mode prediction costs corresponding to all inter-frame coding mode candidates of the image block to be coded to represent the inter-frame coding complexity of the image block to be coded, and taking the sum of the inter-frame coding complexity of all the image blocks in the video frame as the inter-frame coding complexity of the video frame. Preferably, this step calculates only the inter-coding complexity at the picture block level and the video frame level for each P-frame and B-frame, thereby serving the P-frame or B-frame that follows. When the inter-frame coding complexity of the image block and the video frame is calculated in this step, the calculation result of the existing inter-frame coding mode prediction in the video encoder is multiplexed, and no extra image preprocessing stage is required to be introduced.
Step S3: and calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded. The inter-frame coding complexity of each image block inside the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block at the same position inside the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame.
If the current P frame or B frame to be encoded is the first P frame or B frame after a certain I frame, the corresponding previous encoded P frame or B frame is the last P frame or B frame before the I frame. Assuming that the P frame or B frame to be encoded is the nth frame, since it has not yet started encoding, the inter-coding mode prediction operation is not started either, and of course, the inter-coding complexity at the image block level and the video frame level cannot be obtained. To solve this problem, the present application exploits the correlation in the time domain of successive video frames, i.e., successive video frames are similar in content, and the encoded n-2 th, n-1 th and n-th frames are considered similar in content, so that each image block level inter-coding complexity of an n-th frame multiplexes the inter-coding complexity of an identical position image block of an n-1 th frame, or an n-2 th frame when an n-1 th frame is an I-frame, and the video frame level inter-coding complexity of an n-th frame multiplexes the video frame level inter-coding complexity of an n-1 th frame, or an n-2 th frame when an n-1 th frame is an I-frame. Since the n-1 th frame, the n-2 th frame (when the n-1 th frame is an I frame) has already been encoded, the inter-coding complexity of its intra-picture blocks and the video frame level inter-coding complexity are known. Through the multiplexing, the P frame or the B frame to be encoded can calculate the target encoding bit number of each image block to be encoded in the P frame or the B frame to be encoded without special pretreatment before encoding, and the calculation accuracy can meet the requirement of a code rate control algorithm, which is an innovation of the application.
As an example, the calculation formula of the target encoding bit number of the image block to be encoded is as follows: wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the inter-coding complexity, ω, of the current image block to be coded pic For the inter-frame coding complexity of the current P frame or B frame to be coded, T Pic Bit is the target coding Bit number of the current P frame or B frame to be coded H The bit number needed for the pre-estimated header information of the current P frame or B frame to be coded.
Step S4: and according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded, combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame to be coded, and calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded.
As an example, the quantization parameter of the image block to be encoded is represented by formula six: and (5) obtaining. Wherein QP is new Representing quantization parameters of the image block to be encoded inside the P frame or B frame to be encoded. f (f) -1 Is an inverse of the first function f. I (n) new Representing that the image block to be encoded inside the P frame or B frame to be encoded adopts QP new The actual coding bit number after coding is performed, and at this time, the image block to be coded in the P frame or B frame to be coded is not yet coded, so that the actual coding ratio cannot be obtainedThe number of bits, here the target number of coded bits of the image block to be coded inside the P-frame or B-frame to be coded is approximately replaced. I (n-1) old Representing the actual number of encoding bits of the co-located picture block inside the previous encoded P-frame or B-frame of the P-frame or B-frame to be encoded. g is a second function and g (1) is a constant. QP (QP) old Quantization parameters representing co-located image blocks within a previously encoded P-frame or B-frame.
Referring to fig. 1, the mth frame and the nth frame are two P frames or B frames. The mth frame is an encoded P frame or B frame, and the nth frame is a P frame or B frame to be encoded. For a certain image block position inside a video frame, use I (m) old The position image block representing the inside of the mth frame adopts QP old The actual number of coded bits after coding is I (m) new The position image block representing the inside of the mth frame adopts QP new The actual number of coded bits after coding is I (n) old The position image block representing the inside of the nth frame adopts QP old The actual number of coded bits after coding is I (n) new The position image block representing the inside of the nth frame adopts QP new The actual number of coded bits after coding, wherein QP old And QP (QP) new Representing two different image block level quantization parameters. With omega m Representing inter-coding complexity, ω, of the image block at that position within the mth frame n Representing the inter-coding complexity of the image block at that position within the nth frame. Fig. 3 is a schematic diagram of an image block at the same position inside an mth frame, an n-1 frame, and an nth frame.
The first function f is represented by equation one:and (5) obtaining. The physical meaning of the formula one is that the same image block is inter-coded by using different image block level quantization parameters, and the ratio of the actual coded bit number obtained after coding is related to the difference value of the image block level quantization parameters, and the relation can be expressed by a first function f. The first function f characterizes: the same image block adopts different image block level quantization parameters when P frame or B frame coding is carried out, and different actual codes are obtained by the image blockThe ratio of the number of bits is related to the difference of the different image block level quantization parameters employed for the image block. The specific expression form of the first function f can be obtained by calculating the actual parameters of the encoded P frame or B frame and performing function fitting on the two sides of the equal sign of the formula one, which is not described here again.
The second function g is represented by equation two:and (5) obtaining. The physical meaning of the formula two is that the same image block level quantization parameter is used for inter-coding the image blocks at the same position on different video frames, and the ratio of the actual coding bit number obtained after coding is related to the ratio of the inter-coding complexity of the two co-located image blocks, and the relation can be represented by a second function g. The second function g characterizes: and when P frame or B frame coding is carried out, the image blocks at the same position on different video frames adopt the same image block level quantization parameters, and the ratio of the actual coding bit number obtained by the image blocks at the same position on different video frames is related to the ratio of the inter-frame coding complexity of the two image blocks at the same position. The specific expression form of the second function g can be obtained by calculating the actual parameters of the encoded P frame or B frame and performing function fitting on the two sides of the equal sign of the formula two, which are not described here again.
In the practice process of the application, a plurality of experiments are performed in advance, and the expression of the first function f and the expression of the second function g are obtained by adopting a function fitting mode, and are built in a video encoder in advance before encoding, and belong to a known function relation. And the first function f and the second function g only contain a small amount of floating point multiplication division operation, do not contain complex floating point function operation (such as floating point exponent operation and floating point logarithm operation), and have the difficulty and cost of hardware implementation far lower than those of a common code rate control algorithm.
Deriving based on the first and second formulas to obtain a third formula: further derivation can yield equation four: /> Wherein the function f -1 Is the inverse of function f, which is known as a result of -1 Also known. In the fourth formula, the mth frame is an encoded P frame or B frame, and its encoding process is completed, then the actual number of encoded bits I (m) corresponding to the intra-co-located image block is calculated old Picture block level quantization parameter QP used in encoding old And inter-coding complexity omega of image blocks m Are known amounts. The nth frame is the P frame or B frame to be encoded, I (n) new The original text represents the actual coding bit number of the image block to be coded in the P frame or the B frame to be coded, and the n frame does not start to be coded yet, so that the actual coding bit number of the image block to be coded in the n frame cannot be obtained. In order to deduce the image block level quantization parameter required by the image block coding process before the image block to be coded is coded, the application approximately adopts the target coding bit number of the image block to be coded to replace the actual coding bit number, and the error introduced by the approximate substitution is small because the target coding bit number of the image block to be coded is the expected actual coding bit number of the image block to be coded after the actual coding process. I (n) new The target number of coded bits of the block of pictures to be coded inside the P-frame or B-frame to be coded is replaced, which is a known quantity. QP (QP) new The actual number of encoding bits representing the image block to be encoded at the same position within the nth frame is the desired target number of encoding bits I (n) new In this case, the image block level quantization parameter to be used is an unknown amount, and is also an amount to be obtained. Omega n The inter-coding complexity of an image block to be coded, representing the same position inside the nth frame, is an unknown quantity since the image block has not yet been coded.
To calculate the picture to be coded inside the P frame or B frame to be coded by using the formula IVQuantization parameter QP of image block new Let m=n-1, i.e. the mth frame is the nth-1 encoded P frame or B frame, i.e. the mth frame is the previous encoded P frame or B frame to the nth P frame or B frame to be encoded, then equation four can be converted into equation five
Due to the temporal correlation of consecutive video frames (consecutive video frames are similar in content), the content of co-located tiles within an nth P-frame or B-frame and an nth-1P-frame or B-frame can be considered to be highly similar, i.e., the inter-frame coding complexity of co-located tiles within an nth-1P-frame or B-frame and an nth-1P-frame or B-frame is highly similar, i.e., ω n ≈ω n-1 . To understand equation five, ω is referred to herein n And omega n-1 Further simplifying the relationship of omega n ＝ω n-1 ThenSince the second function g is known, g (1) is a constant. Due to omega n And omega n-1 Highly similar, let omega n ＝ω n-1 Is small, the quantization parameter QP of the image block to be coded is not new Is a great influence on the calculation of (2). Thus, equation six can be obtained.
Optionally, step S5 is further included after step S4, indicated by the dashed arrow and the dashed box. Step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier. The method can be used for encoding the image block to be encoded and obtaining the actual encoding bit number and inter-frame encoding complexity.
Referring to fig. 4, the device for controlling the bit rate of the image block level of the P frame or the B frame according to the present application includes an inter-frame coding mode prediction module 1, an inter-frame coding complexity calculation module 2, an image block bit allocation module 3, and an image block bit control module 4. The apparatus shown in fig. 4 corresponds to the method shown in fig. 2.
The inter-frame coding mode prediction module 1 is used for predicting an inter-frame coding mode of an image block to be coded to obtain an inter-frame coding mode candidate of the image block to be coded and prediction cost thereof.
The inter-frame coding complexity calculating module 2 is used for selecting the minimum inter-frame coding mode prediction cost to represent the inter-frame coding complexity of the image block to be coded; the sum of the inter-coding complexity of all image blocks in a video frame is taken as the inter-coding complexity of the video frame.
The image block bit allocation module 3 is configured to calculate a target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to a ratio of an inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to an inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded. The inter-frame coding complexity of each image block inside the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block at the same position inside the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame.
The image block bit control module is used for calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded and combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the co-located image block in the previous coded P frame or the B frame.
Optionally, a scaling module 5 is also included after the image block bit control module 4, indicated by the dashed arrow and the dashed box. The conversion module 5 is configured to obtain a lagrangian multiplier of the image block to be encoded from the quantization parameter of the image block to be encoded according to a conversion relationship between the quantization parameter and the lagrangian multiplier. The method can be used for encoding the image block to be encoded and obtaining the actual encoding bit number and intra-frame encoding complexity of the image block to be encoded.
The application provides a code rate control method suitable for image block level inside a P frame or a B frame of a hardware video encoder. The method utilizes the inter-frame coding mode prediction cost generated in the P frame or B frame internal image block coding process to represent the inter-frame coding complexity of the P frame or B frame internal image block, and accordingly, the target bit allocation of the P frame or B frame internal image block is completed, the hardware cost is low, and the realization cost is low; meanwhile, the image block level quantization parameter of the image block to be coded in the P frame or the B frame to be coded is calculated by utilizing the information of the coded image block in the P frame or the B frame to be coded, the logic relationship is simple, a large amount of complex floating point operations are not introduced, and the difficulty and the cost of hardware implementation are effectively reduced.
The application is similar to the technical targets of Chinese application patent application No. 202310177185.2 and patent application No. 2023, 2 and 26, namely, code rate control method and device for image block level of P frame and/or B frame, and is used for code rate control of image block level of P frame or B frame; but the technical solutions adopted by the two are significantly different. In 202310177185.2, the core of all formula derivation and operation is based on the block-level lagrangian multiplier. In the present application, the core of all formula derivation and operation is based on the image block level quantization parameter. If the image block-level Lagrange multiplier is used as an operation core, more multiplication and division operations are introduced to the operation of the Lagrange multiplier, and the storage space cost and the access bandwidth cost generated by storing and accessing the Lagrange multiplier are also higher due to the wide value range, high precision requirement and large bit width occupation of the Lagrange multiplier, and the larger bit width of the multiplier and the divider when the Lagrange multiplier is realized by hardware. The application uses the image block level quantization parameter as the operation core, and the operation of the quantization parameter is mainly addition and subtraction operation, thereby avoiding a large number of multiplication and division operations, and the cost of storage space and access bandwidth generated by storing and accessing the quantization parameter when the hardware is realized is also smaller due to the small value range and small bit width occupation of the quantization parameter. Therefore, compared with 202310177185.2, the method has lower operation cost, storage cost and bandwidth cost, and is more suitable for hardware implementation.
The above are only preferred embodiments of the present application, and are not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (10)

1. A code rate control method of image block level of P frame or B frame is characterized by comprising the following steps;
step S1: inter-frame coding mode prediction is carried out on the image block to be coded, namely, the prediction cost corresponding to each inter-frame coding mode is calculated on the image block to be coded, only one or more inter-frame coding modes with the lowest prediction cost are reserved to serve as inter-frame coding mode candidates, and meanwhile, the inter-frame coding mode prediction cost corresponding to each inter-frame coding mode candidate is reserved;
step S2: selecting the minimum inter-frame coding mode prediction cost from inter-frame coding mode prediction costs corresponding to all inter-frame coding mode candidates of an image block to be coded to represent the inter-frame coding complexity of the image block to be coded, and taking the sum of the inter-frame coding complexity of all the image blocks in a video frame as the inter-frame coding complexity of the video frame;
step S3: calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded; the inter-frame coding complexity of each image block in the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block in the same position in the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame;
step S4: and according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded, combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame, and calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded.
2. The method according to claim 1, wherein in the step S2, only inter-coding complexity of each of the P-frame and B-frame at the picture block level and the video frame level is calculated.
3. The method according to claim 1, wherein in the step S3, if the current P frame or B frame to be encoded is the first P frame or B frame after a certain I frame, the corresponding previous encoded P frame or B frame is the last P frame or B frame before the I frame.
4. The method for controlling the bit rate at the picture block level of a P frame or a B frame according to claim 1, wherein in the step S3, the calculation formula is as follows:wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the inter-coding complexity, ω, of the current image block to be coded pic For the inter-frame coding complexity of the current P frame or B frame to be coded, T Pic Bit is the target coding Bit number of the current P frame or B frame to be coded H The bit number needed for the pre-estimated header information of the current P frame or B frame to be coded.
5. The method for rate control at the picture block level of a P frame or a B frame according to claim 1, wherein said step S4 is represented by formula six:obtaining; wherein QP is new Representing quantization parameters of image blocks to be encoded in the P frame or the B frame to be encoded; f (f) -1 Is an inverse of the first function f; i (n) new The target coding bit number of the image block to be coded in the P frame or B frame to be coded; i (n-1) old Representing the previous one of the P-or B-frames to be encodedThe actual number of encoding bits for the co-located image blocks within the encoded P-frames or B-frames; g is a second function, g (1) is a constant; QP (QP) old Quantization parameters representing co-located image blocks within a previously encoded P-frame or B-frame.
6. The method for rate control at the picture block level of a P-frame or B-frame according to claim 5, wherein said first function f is represented by formula one:acquiring; the first function f characterizes: the same image block adopts different image block level quantization parameters when P frame or B frame coding is carried out, and the ratio of different actual coding bit numbers obtained by the image block is related to the difference value of different image block level quantization parameters adopted by the image block;
the second function g is represented by equation two:acquiring; the second function g characterizes: and when P frame or B frame coding is carried out, the image blocks at the same position on different video frames adopt the same image block level quantization parameters, and the ratio of the actual coding bit number obtained by the image blocks at the same position on different video frames is related to the ratio of the inter-frame coding complexity of the two image blocks at the same position.
7. The method for rate control at the picture block level of a P-frame or B-frame as claimed in claim 6, wherein the derivation is based on equation one and equation two to obtain equation three: further deriving equation four: />Wherein let m=n-1, formula four is converted into formula five +.>Let omega again n ＝ω n-1 I.e. the inter-coding complexity of the image block to be coded inside the n-th frame is replaced by the inter-coding complexity of the image block to be coded in the same position inside the n-1 th frame>The sixth equation is obtained.
8. The picture block level rate control method of a P-frame or B-frame according to claim 1, further comprising step S5 after step S4; step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
9. The code rate control device of the image block level of the P frame or the B frame is characterized by comprising an inter-frame coding mode prediction module, an inter-frame coding complexity calculation module, an image block bit allocation module and an image block bit control module;
the inter-frame coding mode prediction module is used for predicting the inter-frame coding mode of the image block to be coded to obtain an inter-frame coding mode candidate of the image block to be coded and prediction cost thereof;
the inter-frame coding complexity calculation module is used for selecting the minimum inter-frame coding mode prediction cost to represent the inter-frame coding complexity of the image block to be coded; taking the sum of the inter-frame coding complexity of all image blocks in the video frame as the inter-frame coding complexity of the video frame;
the image block bit allocation module is used for calculating the target coding bit number of each image block to be coded in the P frame or the B frame to be coded according to the ratio of the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded to the inter-frame coding complexity of the P frame or the B frame to be coded and combining the target coding bit number of the P frame or the B frame to be coded; the inter-frame coding complexity of each image block in the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the image block in the same position in the previous coded P frame or the B frame, and the inter-frame coding complexity of the P frame or the B frame to be coded multiplexes the inter-frame coding complexity of the previous coded P frame or the B frame;
the image block bit control module is used for calculating the quantization parameter of the image block to be coded in the P frame or the B frame to be coded according to the target coding bit number and the inter-frame coding complexity of the image block to be coded in the P frame or the B frame to be coded and combining the actual coding bit number, the quantization parameter and the inter-frame coding complexity of the image block at the same position in the previous coded P frame or the B frame.
10. The device for rate control at the tile level of a P-frame or B-frame according to claim 9, further comprising a scaling module after said tile bit control module; the conversion module is used for obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
CN202310781599.6A 2023-06-28 2023-06-28 Code rate control method and device for image block level of P frame or B frame Pending CN116916027A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310781599.6A CN116916027A (en) 2023-06-28 2023-06-28 Code rate control method and device for image block level of P frame or B frame

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310781599.6A CN116916027A (en) 2023-06-28 2023-06-28 Code rate control method and device for image block level of P frame or B frame

Publications (1)

Publication Number Publication Date
CN116916027A true CN116916027A (en) 2023-10-20

Family

ID=88365929

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310781599.6A Pending CN116916027A (en) 2023-06-28 2023-06-28 Code rate control method and device for image block level of P frame or B frame

Country Status (1)

Country Link
CN (1) CN116916027A (en)

Similar Documents

Publication Publication Date Title
JP4828925B2 (en) Encoder
CN102100068B (en) Speculative start point selection for motion estimation iterative search
JPWO2009157581A1 (en) Image processing apparatus and image processing method
EP1705925A2 (en) Motion compensation using scene change detection
CN101946516A (en) The decision of macro block increment quantization parameter fast
JPWO2009157580A1 (en) Image processing apparatus and image processing method
CN102100059B (en) Simple next search position selection for motion estimation iterative search
CN112075082A (en) Method and apparatus for video encoding and decoding for CABAC-based neural network implementation
JPWO2017122604A1 (en) Moving picture coding apparatus, moving picture coding method, and storage medium storing moving picture coding program
KR20040093604A (en) Method of determining reference picture and block mode, the apparatus therefor, method of determining block mode, and the apparatus therefor for fast motion estimation
CN101325714A (en) Method and apparatus for processing transformation data, method and apparatus for encoding and decoding
CN104601992B (en) SKIP mode quick selecting methods based on Bayesian Smallest Risk decision
US20070133689A1 (en) Low-cost motion estimation apparatus and method thereof
CN101331773A (en) Two pass rate control techniques for video coding using rate-distortion characteristics
CN114915789B (en) Method, system, device and medium for optimizing Lagrange multiplier between frames
JP3872849B2 (en) Video encoding device
CN108989818B (en) Image coding parameter adjusting method and device
CN115118976A (en) Image coding method, readable medium and electronic device thereof
JP2007235944A (en) Method and system for selecting modes for encoding macroblocks in sequence of frames of video
CN1457196A (en) Video encoding method based on prediction time and space domain conerent movement vectors
WO2006025904A1 (en) Fast motion estimation for multiple reference pictures
CN116916027A (en) Code rate control method and device for image block level of P frame or B frame
JP2002010260A (en) Motion vector detection method and motion picture encoding apparatus
CN116939207A (en) Code rate control method and device for image block level of I frame
CN110800298A (en) Code rate allocation method, code rate control method, encoder, and recording medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination