CN116939207A - Code rate control method and device for image block level of I frame - Google Patents
Code rate control method and device for image block level of I frame Download PDFInfo
- Publication number
- CN116939207A CN116939207A CN202310779930.0A CN202310779930A CN116939207A CN 116939207 A CN116939207 A CN 116939207A CN 202310779930 A CN202310779930 A CN 202310779930A CN 116939207 A CN116939207 A CN 116939207A
- Authority
- CN
- China
- Prior art keywords
- frame
- intra
- coded
- image block
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013139 quantization Methods 0.000 claims abstract description 67
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000014509 gene expression Effects 0.000 claims description 9
- 238000002474 experimental method Methods 0.000 claims description 2
- 238000004422 calculation algorithm Methods 0.000 abstract description 18
- 238000005457 optimization Methods 0.000 description 13
- 238000007781 pre-processing Methods 0.000 description 6
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000009795 derivation Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/19—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses a code rate control method of an image block level of an I frame. Intra-coding mode prediction is performed on the image block to be coded. And selecting the minimum intra-frame coding mode prediction cost to represent the intra-frame coding complexity of the image block to be coded. And calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded. And according to the target coding bit number and the intra-frame coding complexity of the image blocks to be coded in the I frame to be coded, combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the co-located image blocks in the previous coded I frame to calculate the quantization parameter of the image blocks to be coded in the I frame to be coded. The application takes the image block level quantization parameter as the operation core of the code rate control algorithm, and is more suitable for hardware realization.
Description
Technical Field
The application relates to a digital video coding technology, in particular to a code rate control method of image block level of an I frame, which is suitable for hardware realization.
Background
Video coding is a technique that characterizes video information by compressing redundant components in video images and using as little data as possible. Common video coding standards are HEVC (High Efficiency Video Coding, high efficiency video coding, also known as h.265), AVC (Advanced Video Coding, advanced video coding, also known as h.264), etc.
Video coding techniques use image blocks as the most basic coding units. For example, in HEVC, a basic Unit of video Coding is a CU (Coding Unit). A CU may be an image block of 64 pixels by 64 pixels, 32 pixels by 32 pixels, 16 pixels by 16 pixels, 8 pixels by 8 pixels size. An image block of 64 pixels by 64 pixels size is also called LCU (Largest Coding Unit, maximum coding unit).
In order to increase the video compression rate as much as possible, a common video coding algorithm often adopts a lossy compression algorithm, so that the coded and reconstructed video is different from the original video (distortion is generated). Both HEVC and AVC belong to lossy compression algorithms. The coding performance of a lossy compression algorithm needs to be commonly measured by the coding bit Rate (Rate) and the Distortion introduced by the coding (loss). In the video coding process, the coding bit rate R and the coding distortion D are mutually restricted and contradicted, and reducing the coding bit rate R tends to increase the coding distortion D, and reducing the coding distortion D tends to increase the coding bit rate R. Rate distortion optimization (Rate Distortion Optimization, RDO) is to reduce the coding distortion D as much as possible at a certain coding bit rate R or to reduce the coding bit rate R as much as possible at a certain coding distortion D. Common video coding algorithms all use a rate distortion optimization method based on lagrangian multipliers (Lagrange multiplier) λ, which is to obtain the minimum coding rate distortion Cost (Rate Distortion Cost, RD Cost) J, j=d+λ×r.
In video coding, each coding mode has coding distortion D and coding bit rate R, and coding rate distortion cost J for each coding mode can be calculated. When video coding is performed, the coding mode with the minimum coding rate distortion cost J is selected for coding, so that the optimal coding performance can be obtained. The process of selecting an optimal coding Mode for each image block to be coded by the encoder through a rate distortion optimization method is called Mode selection (Mode Decision) of the image block to be coded.
In order to remove information redundancy in the spatial and temporal domains, video coding techniques encode input video frames using intra-frame coding techniques and inter-frame coding techniques. Only intra-frame coding techniques are used to encode I-frames. The hybrid encoding is P-frames and B-frames using intra-frame and inter-frame coding techniques. The type of encoded frame is determined by the group of pictures (GOP, group Of Pictures) parameters set by the user. Fig. 1 shows a common group of pictures structure of I, P, B frames. When the I frame is encoded, the information of the video frame is only used for encoding, and the information of other encoded video frames is not referred to. All image blocks within an I-frame are encoded as intra-coded blocks. P-frames and B-frames are encoded using information not only from the present video frame, but also from one or more other encoded video frames. The image blocks inside the P-frame and the B-frame may be encoded as intra-frame encoded blocks or inter-frame encoded blocks, depending on which encoding rate the distortion cost J is smaller.
In real life, the channel bandwidth capacity used to transmit video is limited. If the coding bit rate of the video is too high, the video transmission congestion and even packet loss are caused if the channel bandwidth capacity is exceeded. If the coding bit rate of the video is too low, the channel bandwidth is not fully utilized, and higher video quality cannot be obtained. Therefore, it is necessary to control the output code rate of the video encoder to match the channel bandwidth capacity using a code rate control technique.
Rate Control (Rate Control) aims to make the output code Rate of the video encoder equal to a preset target code Rate by adjusting the coding parameters of the video encoder, and simultaneously reduce coding distortion D as much as possible so as to improve video coding quality. The common code rate control algorithm generally realizes the code rate control task through two links of target bit allocation and target bit control.
The target bit allocation links are typically performed at three levels, GOP (group of pictures, i.e., a set of temporally consecutive video frames), video frame level, and block level, respectively. The target bit allocation at GOP level is to obtain the target number of coded bits of GOP based on the target coded bit rate of video. The target bit allocation of the video frame level is to obtain the target coding bit number of each video frame in the GOP according to the target coding bit number of the GOP. The target bit allocation of the image block level is to obtain the target coding bit number of each image block in the video frame according to the target coding bit number of the video frame and the bit allocation weight of each image block in the video frame.
The target bit control links are typically performed at the video frame level and at the image block level. At the video frame level, a lagrangian multiplier λ and a quantization parameter (quantization parameter) QP for the video frame level are calculated from the target number of encoding bits of the video frame. At the image block level, the Lagrangian multiplier lambda and the quantization parameter QP of the image block level of the image block are calculated according to the target coding bit number of a certain image block, the Lagrangian multiplier lambda and the quantization parameter QP of the video frame level of the video frame to which the image block belongs and the Lagrangian multiplier lambda and the quantization parameter QP of the image block level of the previous coded image block, and are used for the video coding process of the image block.
Both the lagrangian multiplier λ and the quantization parameter QP are divided into video frame level and picture block level. The lagrangian multiplier λ and the quantization parameter QP at the video frame level are used in the target bit allocation and target bit control link at the video frame level (i.e., rate control at the video frame level), and the lagrangian multiplier λ and the quantization parameter QP at the image block level are used in the target bit allocation and target bit control link at the image block level (i.e., rate control at the image block level). The lagrangian multipliers at the video frame level are sometimes referred to simply as video frame level lagrangian multipliers, and lagrangian multipliers for video frames. The lagrangian multipliers at the image block level are sometimes simply referred to as image block level lagrangian multipliers, and lagrangian multipliers for image blocks. The abbreviation for quantization parameter and so on.
Because of the large amount of computation in video coding algorithms, to increase video coding speed and achieve real-time coding and transmission, the industry typically uses Application Specific Integrated Circuits (ASICs) to hardware accelerate the video coding process. An application specific integrated circuit that hardware accelerates the video encoding process is generally referred to simply as a hardware video encoder.
In hardware video encoders, rate control at the picture block level of an I-frame has been a difficulty. The main reason is that if the code rate of the image block level of the I frame is to be accurately controlled, a special preprocessing stage is added to evaluate the intra-frame coding complexity of each image block in the I frame before the I frame is coded, the bit allocation weight of each image block is calculated, and then the target coding bit number of each image block and the corresponding coding parameters (such as lagrangian multiplier or quantization parameters) are calculated. However, adding specialized preprocessing stages in hardware video encoders is not only costly, but also increases the delay of video encoding. Meanwhile, in a code rate control algorithm represented by the JCTCVC-K0103 proposal, more multiplication and division operations are introduced into the operation taking the Lagrangian multiplier as a core, and the calculation cost, the storage cost and the bandwidth cost are large when hardware is realized due to the wide value range, high precision requirement and large bit width occupation of the Lagrangian multiplier.
Therefore, it is necessary to design a rate control method suitable for the image block level of the I frame of the hardware video encoder, on the basis of not increasing a special preprocessing stage, more accurately evaluate the intra-frame coding complexity of the image block in the I frame, and calculate the target coding bit number and the corresponding coding parameters (such as lagrangian multiplier or quantization parameters) of the image block, so that the actual coding bit number of the image block is closer to the target coding bit number, and the rate control process of the I frame is more accurate.
Disclosure of Invention
The technical problems to be solved by the application are as follows: how to make the rate control at the picture block level of the I-frame more suitable to be implemented in hardware.
In order to solve the technical problems, the application provides a code rate control method of the image block level of an I frame, which comprises the following steps. Step S1: and carrying out intra-frame coding mode prediction on the image block to be coded, namely calculating the prediction cost corresponding to each intra-frame coding mode of the image block to be coded, and only reserving one or more intra-frame coding modes with the lowest prediction cost as intra-frame coding mode candidates and reserving the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate. Step S2: and selecting the minimum intra-frame coding mode prediction cost from the intra-frame coding mode prediction costs corresponding to all the intra-frame coding mode candidates of the image block to be coded to represent the intra-frame coding complexity of the image block to be coded, and taking the sum of the intra-frame coding complexity of all the image blocks in the video frame as the intra-frame coding complexity of the video frame. Step S3: calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded; intra-coding complexity of an I-frame to be encoded multiplexes the intra-coding complexity of a previous encoded video frame; the intra-coding complexity of each image block within an I-frame to be encoded multiplexes the intra-coding complexity of co-located image blocks within a previous encoded video frame. Step S4: according to the target coding bit number and the intra-frame coding complexity of the image block to be coded in the I frame to be coded, combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the image block at the same position in the previous coded I frame to calculate the quantization parameter of the image block to be coded in the I frame to be coded; the intra-coding complexity of each tile within an I-frame to be encoded multiplexes the intra-coding complexity of co-located tiles within a previous encoded video frame.
Preferably, in the step S2, only the intra-coding complexity of the video frame preceding each I-frame is calculated.
Further, the step S3 is represented by the formula:obtaining; wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the intra coding complexity, ω, of the current image block to be coded pic For the intra-coding complexity, T, of the current I frame to be coded Pic Bit is the target coding Bit number of the current I frame to be coded H The number of bits needed for the pre-estimated current I frame header information to be coded.
Further, in the step S4, an image block level quantization parameter QP of the I frame to be encoded new The fifth formula: obtaining; wherein f -1 Is an inverse of the first function f; i (n) new The target coding bit number of the image block to be coded in the I frame to be coded; i (m) old Representing the actual number of encoding bits for the co-located image block within the previous encoded I-frame; g is a second function; omega n-1 Intra-coding complexity representing co-located image blocks within a previously encoded video frame of an I-frame to be encoded; omega m Intra coding complexity representing co-located image blocks within a previous coded I-frame; QP (QP) old Image block level quantization parameters representing co-located image blocks within a previous encoded I-frame.
Further, the first function f is represented by formula one:acquiring; the first function f characterizes: the same image block adopts different image block level quantization parameters when I frame coding is carried out, and the image block obtains the relation between the ratio of different actual coding bit numbers and the difference value of different image block level quantization parameters adopted by the image block. The second function g is represented by equation two: />Acquisition ofThe method comprises the steps of carrying out a first treatment on the surface of the The second function g characterizes: and when I frame coding is carried out, the image blocks at the same position on different video frames adopt the same image block level quantization parameters, and the ratio of the actual coding bit numbers obtained by the image blocks at the same position on different video frames is related to the ratio of the intra-frame coding complexity of the two image blocks at the same position.
Further, the expression of the first function f and the expression of the second function g are obtained through experiments in advance, obtained through a function fitting mode and stored in a video encoder.
Further, deriving based on the first formula and the second formula to obtain a third formula: further deriving equation four Wherein omega n Intra-coding complexity ω of co-located image blocks to be coded inside the previous coded video frame from the I-frame to be coded n-1 And replacing to obtain a formula five.
Optionally, step S5 is further included after step S4. Step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
The application also discloses a code rate control device of the image block level of the I frame, which comprises an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an image block bit allocation module and an image block bit control module. The intra-frame coding mode prediction module is used for carrying out intra-frame coding mode prediction on the image block to be coded to obtain one or more intra-frame coding modes with the lowest prediction cost and corresponding intra-frame coding mode prediction cost. The intra-frame coding complexity calculation module is used for selecting the minimum intra-frame coding mode prediction cost to represent the intra-frame coding complexity of the image block to be coded; the sum of the intra-coding complexity of all image blocks within a video frame is taken as the intra-coding complexity of the video frame. The image block bit allocation module is used for calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded; intra-coding complexity of an I-frame to be encoded multiplexes the intra-coding complexity of a previous encoded video frame; the intra-coding complexity of each image block within an I-frame to be encoded multiplexes the intra-coding complexity of co-located image blocks within a previous encoded video frame. The image block bit control module is used for calculating the quantization parameter of the image block to be coded in the I frame to be coded according to the target coding bit number and the intra-frame coding complexity of the image block to be coded in the I frame to be coded and combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the co-located image block in the previous coded I frame; the intra-coding complexity of each tile within an I-frame to be encoded multiplexes the intra-coding complexity of co-located tiles within a previous encoded video frame.
Optionally, a scaling module is further included after the image block bit control module; the conversion module is used for obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
The application has the technical effects that: (1) The image block level quantization parameter is used as an operation core of a code rate control algorithm, and the operation on the quantization parameter is mainly addition and subtraction operation, so that the method is more suitable for hardware implementation. (2) The image preprocessing stage is not required to be added, the hardware cost is low, and the realization cost is low. (3) And the information of the co-located image blocks in the encoded video frame is utilized to calculate the image block level quantization parameter of the I frame to be encoded, so that the calculation accuracy is high, and the accuracy of the image block level code rate control of the I frame is effectively improved.
Drawings
Fig. 1 is a schematic diagram of a common group of pictures structure of I, P, and B frames.
Fig. 2 is a flow chart of a method for controlling the bit rate at the picture block level of an I frame according to the present application.
Fig. 3 is a schematic diagram of an image block at the same position inside an mth frame, an n-1 frame, and an nth frame.
Fig. 4 is a schematic structural diagram of a rate control device at the picture block level of an I frame according to the present application.
The reference numerals in the drawings illustrate: 1 is an intra-frame coding mode prediction module, 2 is an intra-frame coding complexity calculation module, 3 is an image block bit allocation module, 4 is an image block bit control module, and 5 is a conversion module.
Detailed Description
Referring to fig. 2, the method for controlling the bit rate at the picture block level of the I frame according to the present application includes the following steps.
Step S1: and carrying out intra-frame coding mode prediction on the image block to be coded, namely calculating the prediction cost corresponding to each intra-frame coding mode of the image block to be coded, and only reserving one or more intra-frame coding modes with the lowest prediction cost as intra-frame coding mode candidates and reserving the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate. This step is a step in the existing video coding process, and intra coding mode prediction is performed on all image blocks of I frames, P frames, and B frames.
In a hardware video encoder, in order to reduce cost and power consumption, when a mode selection is performed on an image block to be encoded using a rate distortion optimization method, a process of mode selection is generally divided into an encoding mode prediction stage and an encoding mode rate distortion optimization stage. The coding mode prediction stage is completed by an intra-frame or inter-frame coding mode prediction module, and the coding mode rate distortion optimization stage is completed by an intra-frame or inter-frame coding mode rate distortion optimization module.
Essentially, the coding mode prediction stage and the coding mode rate distortion optimization stage adopt a rate distortion optimization method to screen the coding mode of the image block to be coded. The two stages differ mainly in the accuracy of the calculation method of the coding rate distortion cost J employed. In the coding mode prediction stage, a coding rate distortion cost J calculation method with less calculation amount and lower precision is generally adopted; calculating the coding distortion D typically uses the SAD (Sum of Absolute Difference, sum of absolute errors) algorithm or the SATD (Sum of Absolute Transformed Difference, sum of absolute transform errors) algorithm; calculating the encoding bit rate R typically uses a simplified bit rate estimation algorithm, such as an exponential golomb algorithm; and the coding mode prediction stage typically does not introduce transform and quantization operations. In the code mode rate distortion optimization stage, a code rate distortion cost J calculation method with larger calculation amount and higher precision is generally adopted; calculating the coding distortion D typically uses an SSD (Sum of Squared Differences, sum of square errors) algorithm; calculating the coding bit rate R typically uses CABAC (Context Adaptive Binary Arithmatic Coding, context adaptive binary arithmetic coding) algorithm or CAVLC (Context Adaptive Variable Length Coding ) algorithm; and the coding mode rate distortion optimization stage typically introduces transform and quantization operations. Conventionally, a coding rate distortion cost J of low calculation amount and low precision calculated in the coding mode prediction stage is referred to as a coding mode prediction cost.
When the image block to be encoded is encoded, the encoding mode prediction stage performs preliminary screening on a plurality of encoding modes, and one or a plurality of encoding mode candidates with lower prediction cost are screened out by calculating the prediction cost corresponding to each encoding mode. And in the subsequent coding mode rate distortion optimization stage, accurate calculation is carried out on prediction costs corresponding to the coding mode candidates to obtain accurate coding rate distortion costs J of candidate coding modes, and finally, a coding mode with the minimum coding rate distortion costs J is selected by a mode selection module to serve as an optimal coding mode of the image block to be coded.
Step S2: and selecting the minimum intra-frame coding mode prediction cost from the intra-frame coding mode prediction costs corresponding to all the intra-frame coding mode candidates of the image block to be coded to represent the intra-frame coding complexity of the image block to be coded, and taking the sum of the intra-frame coding complexity of all the image blocks in the video frame as the intra-frame coding complexity of the video frame. Preferably, this step calculates only the intra-coding complexity of the video frame preceding each I-frame, thereby serving the following I-frames. When the intra-frame coding complexity of the image block and the video frame is calculated in this step, the calculation result of the existing intra-frame coding mode prediction in the video encoder is multiplexed, and no extra preprocessing stage is required to be introduced.
Step S3: and calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded. For an I-frame to be encoded, the intra-coding complexity of each image block within the I-frame to be encoded multiplexes the intra-coding complexity of the co-located image block within the previous encoded video frame, and the intra-coding complexity of the I-frame to be encoded multiplexes the intra-coding complexity of the previous encoded video frame. Assuming that the I frame to be encoded is the nth frame, since it has not yet started encoding, the intra-frame encoding mode prediction operation is not started either, and of course, the intra-frame encoding complexity at the image block level and the video frame level cannot be obtained. To solve this problem, the present application exploits the temporal correlation of successive video frames, i.e., successive video frames are similar in content, and the encoded n-1 th frame is considered similar to the content of the n-th frame, so that each image block level intra-coding complexity of the n-th frame multiplexes the intra-coding complexity of the co-located image block of the n-1 th frame, and the video frame level intra-coding complexity of the n-th frame multiplexes the video frame level intra-coding complexity of the n-1 th frame. Since the n-1 frame has already been encoded, the intra-coding complexity of its intra-image blocks and the video frame-level intra-coding complexity are known. Through multiplexing, the I frame to be encoded can calculate the target encoding bit number of each image block to be encoded in the I frame to be encoded without special pretreatment before encoding, and the calculation accuracy can meet the requirement of a code rate control algorithm.
As an example, the step S3 is represented by the formula:and (5) obtaining. Wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the intra coding complexity, ω, of the current image block to be coded pic For the intra-coding complexity, T, of the current I frame to be coded Pic Bit is the target coding Bit number of the current I frame to be coded H The number of bits needed for the pre-estimated current I frame header information to be coded.
Step S4: and according to the target coding bit number and the intra-frame coding complexity of the image blocks to be coded in the I frame to be coded, combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the co-located image blocks in the previous coded I frame to calculate the quantization parameter of the image blocks to be coded in the I frame to be coded. Wherein the intra-coding complexity of each picture block within an I-frame to be encoded is multiplexed by the intra-coding complexity of a co-located picture block within a previous encoded video frame.
Image block level quantization parameter QP of I frame to be encoded new The fifth formula:solving, this is one example. Wherein f -1 Is an inverse of the first function f. The first function f characterizes: the same image block adopts different image block level quantization parameters when I frame coding is carried out, and the image block obtains the relation between the ratio of different actual coding bit numbers and the difference value of different image block level quantization parameters adopted by the image block. I (n) new Representing that the image block to be encoded inside the I frame to be encoded adopts lambda new The actual number of encoded bits after encoding is performed, at this time, the image block to be encoded in the I frame to be encoded is not yet encoded, so the actual number of encoded bits cannot be obtained, and here, the target number of encoded bits of the image block to be encoded in the I frame to be encoded is approximately replaced. I (m) old Representing the actual number of coded bits of the co-located picture block inside the previous coded I-frame. g is a second function, characterizing: proceeding withAnd the image blocks at the same position on different video frames adopt the same image block level quantization parameter in the I frame coding, and the ratio of the actual coding bit number obtained by the image blocks at the same position on different video frames is related to the ratio of the intra-frame coding complexity of the two co-located image blocks. Omega n-1 Representing the intra-coding complexity of co-located image blocks within the n-1 th frame (the previous encoded video frame of the I-frame to be encoded). Omega m Representing the intra-coding complexity of co-located image blocks inside the mth frame (the previous encoded I frame). QP (QP) old Image block level quantization parameters representing co-located image blocks within a previous encoded I-frame.
Referring to fig. 1, the mth frame and the nth frame are two I frames adjacent to each other, and zero or more P frames or B frames may exist between them. Assuming that the nth frame is an I frame to be encoded, all video frames preceding the nth frame are encoded video frames. For a certain image block position inside a video frame, use I (m) old The position image block representing the inside of the mth frame adopts QP old The actual number of coded bits after coding is I (m) new The position image block representing the inside of the mth frame adopts QP new The actual number of coded bits after coding is I (n) old The position image block representing the inside of the nth frame adopts QP old The actual number of coded bits after coding is I (n) new The position image block representing the inside of the nth frame adopts QP new The actual number of coded bits after coding, wherein QP old And QP (QP) new Representing two different image block level quantization parameters. With omega m Representing intra-coding complexity, ω, of the image block at that position within the mth frame n Intra-frame coding complexity, ω, of an image block representing the position inside the nth frame n-1 Representing the intra coding complexity of the image block at that position within the n-1 frame. Fig. 3 is a schematic diagram of an image block at the same position inside an mth frame, an n-1 frame, and an nth frame.
The first function f is represented by equation one:and (5) obtaining. Formula oneThe physical meaning is that the same image block is subjected to intra-frame coding by using different image block level quantization parameters, and the ratio of the actual coding bit number obtained after coding is related to the difference value of the image block level quantization parameters, and the relation can be expressed by a first function f. The specific expression form of the first function f can be obtained by calculating the actual parameters of the encoded I frame and performing function fitting on the two sides of the equal sign of the formula one, which is not described herein.
The second function g is represented by equation two:and (5) obtaining. The physical meaning of the second formula is that the same image block level quantization parameter is used for carrying out intra-frame coding on image blocks at the same position on different video frames, and the ratio of the actual coding bit number obtained after coding is related to the ratio of the intra-frame coding complexity of the two co-located image blocks, and the relation can be represented by a second function g. The specific expression form of the second function g can be obtained by calculating the actual parameters of the encoded I frame and performing function fitting on the two sides of the equal sign of the formula two, which are not described herein again.
In the specific implementation of the application, a great amount of experimental calculation data are used in advance, the expression of the first function f and the expression of the second function g are obtained by adopting a function fitting mode, and the expressions are stored in a video encoder and belong to known functional relations.
Deriving based on the first and second formulas to obtain a third formula: further derivation can yield equation four: /> Wherein the function f -1 Is the inverse of function f, which is known as a result of -1 Also known. In formula four, the mth frame is an encoded I frame, and its encoding process is completed, then the actual number of encoded bits I (m) corresponding to the intra-co-located image block is calculated old Picture block level quantization parameter QP used in encoding old And intra coding complexity omega of image blocks m Are known amounts. The nth frame is the I frame to be encoded, I (n) new The original text represents the actual coding bit number of the image block to be coded in the I frame to be coded, and the n frame does not start to be coded yet, so that the actual coding bit number of the image block to be coded in the I frame to be coded cannot be obtained. In order to derive the lagrangian multiplier required by the coding process of the image block to be coded before the image block to be coded is coded, the application approximately replaces the actual coding bit number with the target coding bit number of the image block to be coded, and the "target coding bit number" of the image block to be coded is the expected "actual coding bit number" of the image block to be coded after the actual coding process, so that the error introduced by the approximate replacement is small. I (n) new The target number of coded bits of the image block to be coded inside the I-frame to be coded is replaced, which is a known quantity. QP (QP) new The actual number of encoding bits representing the image block to be encoded at the same position within the nth frame is the desired target number of encoding bits I (n) new In this case, the image block level quantization parameter to be used is an unknown amount, and is also an amount to be obtained. Omega n The intra-coding complexity of an image block to be coded, representing the same position inside the nth frame, is an unknown quantity since the image block has not yet been coded. To calculate quantization parameter QP of image block to be encoded inside I frame to be encoded using equation four new The application uses the correlation in the time domain of the continuous video frames to multiplex the intra-frame coding complexity of the image blocks at the same position of the coded n-1 frame adjacent to the n frame to replace the intra-frame coding complexity of the image blocks to be coded in the n frame to be coded. Since the sampling time of the n-1 th frame and the n th frame are very close, the contents of the two video frames are very similar, and the frames of the image blocks at the same positionThe error of the replacement operation of the intra-coding complexity is small, and the quantization parameter QP of the image block to be coded is not generated new Is a great influence on the calculation of (2). Thus, since the encoding of the co-located image block on the n-1 th frame is completed, its intra-frame encoding complexity ω n-1 Is a known quantity. In equation four, let ω n-1 Instead of omega n The fifth formula is obtained.
Optionally, step S5 is further included after step S4, indicated by the dashed arrow and the dashed box. Step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier. The method can be used for encoding the image block to be encoded and obtaining the actual encoding bit number and intra-frame encoding complexity of the image block to be encoded.
Referring to fig. 4, the rate control device for image block level of I-frame according to the present application includes an intra-frame coding mode prediction module 1, an intra-frame coding complexity calculation module 2, an image block bit allocation module 3, and an image block bit control module 4. The apparatus shown in fig. 4 corresponds to the method shown in fig. 2.
The intra-frame coding mode prediction module 1 is configured to perform intra-frame coding mode prediction on an image block to be coded, so as to obtain one or more intra-frame coding modes with the lowest prediction cost and corresponding intra-frame coding mode prediction costs.
The intra-frame coding complexity calculating module 2 is used for selecting the minimum intra-frame coding mode prediction cost to represent the intra-frame coding complexity of the image block to be coded; the sum of the intra-coding complexity of all image blocks within a video frame is taken as the intra-coding complexity of the video frame.
The image block bit allocation module 3 is configured to calculate a target coding bit number of each image block to be coded in the I frame to be coded according to a ratio of an intra-frame coding complexity of the image block to be coded in the I frame to be coded to an intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded. Intra-coding complexity of an I-frame to be encoded multiplexes the intra-coding complexity of a previous encoded video frame. The intra-coding complexity of each image block within an I-frame to be encoded multiplexes the intra-coding complexity of co-located image blocks within a previous encoded video frame.
The image block bit control module 4 is configured to calculate a quantization parameter of the image block to be encoded in the I frame to be encoded according to the target encoding bit number and the intra-frame encoding complexity of the image block to be encoded in the I frame to be encoded, and by combining the actual encoding bit number, the quantization parameter and the intra-frame encoding complexity of the co-located image block in the previous I frame to be encoded. The intra-coding complexity of each picture block within an I-frame to be encoded is multiplexed by the intra-coding complexity of the co-located picture blocks within the previous encoded video frame.
Optionally, a scaling module 5 is also included after the image block bit control module 4, indicated by the dashed arrow and the dashed box. The conversion module 5 is configured to obtain a lagrangian multiplier of the image block to be encoded from the quantization parameter of the image block to be encoded according to a conversion relationship between the quantization parameter and the lagrangian multiplier. The method can be used for encoding the image block to be encoded and obtaining the actual encoding bit number and intra-frame encoding complexity of the image block to be encoded.
The application provides a code rate control method suitable for the image block level of an I frame of a hardware video encoder. The method utilizes the intra-frame coding mode prediction cost generated in the process of coding the image blocks in the video frame to represent the intra-frame coding complexity of the image blocks in the video frame, and accordingly, the target bit allocation of the image blocks on the I frame is completed, the image preprocessing stage is not required to be increased, the hardware cost is low, and the realization cost is low; meanwhile, the information of the coded image blocks in the coded video frames is used for calculating the image block level quantization parameters of the image blocks to be coded in the I frames to be coded, so that the calculation accuracy is high, the actual coding bit number of the image blocks is more approximate to the target coding bit number, and the accuracy of the image block level code rate control of the I frames is effectively improved.
The application is similar to the technical targets of the application publication number CN116248882A and the Chinese patent application of the application publication day 2023, 6 and 9, namely a code rate control method and a code rate control device for the image block level of an I frame, and is used for code rate control of the image block level of the I frame; but the technical solutions adopted by the two are significantly different. In CN116248882a, the core of all formula derivation and operation is based on the block-level lagrangian multiplier. In the present application, the core of all formula derivation and operation is based on the image block level quantization parameter. If the image block-level Lagrange multiplier is used as an operation core, more multiplication and division operations are introduced to the operation of the Lagrange multiplier, and the storage space cost and the access bandwidth cost generated by storing and accessing the Lagrange multiplier are also higher due to the wide value range, high precision requirement and large bit width occupation of the Lagrange multiplier, and the larger bit width of the multiplier and the divider when the Lagrange multiplier is realized by hardware. The application uses the image block level quantization parameter as the operation core, and the operation of the quantization parameter is mainly addition and subtraction operation, thereby avoiding a large number of multiplication and division operations, and the cost of storage space and access bandwidth generated by storing and accessing the quantization parameter when the hardware is realized is also smaller due to the small value range and small bit width occupation of the quantization parameter. Therefore, compared with CN116248882a, the present application has lower operation cost, storage cost and bandwidth cost, and is more suitable for hardware implementation.
The above are only preferred embodiments of the present application, and are not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. The code rate control method of the image block level of the I frame is characterized by comprising the following steps of;
step S1: carrying out intra-frame coding mode prediction on the image block to be coded, namely calculating the prediction cost corresponding to each intra-frame coding mode of the image block to be coded, only reserving one or more intra-frame coding modes with the lowest prediction cost as intra-frame coding mode candidates, and reserving the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate;
step S2: selecting the minimum intra-frame coding mode prediction cost from the intra-frame coding mode prediction costs corresponding to all the intra-frame coding mode candidates of the image block to be coded to represent the intra-frame coding complexity of the image block to be coded, and then taking the sum of the intra-frame coding complexity of all the image blocks in the video frame as the intra-frame coding complexity of the video frame;
step S3: calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded; intra-coding complexity of an I-frame to be encoded multiplexes the intra-coding complexity of a previous encoded video frame; the intra-frame coding complexity of each image block inside the I frame to be coded multiplexes the intra-frame coding complexity of the same-position image block inside the previous coded video frame;
step S4: according to the target coding bit number and the intra-frame coding complexity of the image block to be coded in the I frame to be coded, combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the image block at the same position in the previous coded I frame to calculate the quantization parameter of the image block to be coded in the I frame to be coded; the intra-coding complexity of each tile within an I-frame to be encoded multiplexes the intra-coding complexity of co-located tiles within a previous encoded video frame.
2. The method according to claim 1, wherein in the step S2, only the intra-coding complexity of a video frame preceding each I frame is calculated.
3. The method for rate control at the picture block level of an I frame according to claim 1, wherein said step S3 is defined by the formula:obtaining; wherein T is blk_curr For the target number of coded bits, ω, of the current image block to be coded blk_curr For the intra coding complexity, ω, of the current image block to be coded pic For the intra-coding complexity, T, of the current I frame to be coded Pic Bit is the target coding Bit number of the current I frame to be coded H The number of bits needed for the pre-estimated current I frame header information to be coded.
4. The method according to claim 1, wherein in step S4, the image block level quantization parameter QP of the I frame to be encoded new The fifth formula:obtaining; wherein f -1 Is an inverse of the first function f; i (n) new The target coding bit number of the image block to be coded in the I frame to be coded; i (m) old Representing the actual number of encoding bits for the co-located image block within the previous encoded I-frame; g is a second function; omega n-1 Intra-coding complexity representing co-located image blocks within a previously encoded video frame of an I-frame to be encoded; omega m Intra coding complexity representing co-located image blocks within a previous coded I-frame; QP (QP) old Image block level quantization parameters representing co-located image blocks within a previous encoded I-frame.
5. The method for rate control at the picture block level of an I-frame as claimed in claim 4, wherein said first function f is represented by formula one:acquiring; the first function f characterizes: the same image block adopts different image block level quantization parameters when I frame coding is carried out, and the image block obtains the relationship between the ratio of different actual coding bit numbers and the difference value of different image block level quantization parameters adopted by the image block;
the second function g is represented by equation two:acquiring; the second function g characterizes: and when I frame coding is carried out, the image blocks at the same position on different video frames adopt the same image block level quantization parameters, and the ratio of the actual coding bit numbers obtained by the image blocks at the same position on different video frames is related to the ratio of the intra-frame coding complexity of the two image blocks at the same position.
6. The method for controlling the bit rate of the image block level of an I frame according to claim 5, wherein the expression of the first function f and the expression of the second function g are obtained by obtaining data through experiments in advance and are stored in a video encoder in a function fitting mode.
7. The method for rate control at the picture block level of an I-frame according to claim 5, wherein the deriving is based on equation one and equation two to obtain equation three:I(m) old the method comprises the steps of carrying out a first treatment on the surface of the Further deriving equation four: />Wherein omega n Intra-coding complexity ω of co-located image blocks to be coded inside the previous coded video frame from the I-frame to be coded n-1 And replacing to obtain a formula five.
8. The method for rate control at the picture block level of an I-frame according to claim 1, further comprising step S5 after step S4; step S5: and obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
9. The code rate control device of the image block level of the I frame is characterized by comprising an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an image block bit allocation module and an image block bit control module;
the intra-frame coding mode prediction module is used for carrying out intra-frame coding mode prediction on the image block to be coded to obtain one or more intra-frame coding modes with the lowest prediction cost and corresponding intra-frame coding mode prediction cost;
the intra-frame coding complexity calculation module is used for selecting the minimum intra-frame coding mode prediction cost to represent the intra-frame coding complexity of the image block to be coded; taking the sum of the intra-frame coding complexity of all image blocks in the video frame as the intra-frame coding complexity of the video frame;
the image block bit allocation module is used for calculating the target coding bit number of each image block to be coded in the I frame to be coded according to the ratio of the intra-frame coding complexity of the image block to be coded in the I frame to be coded to the intra-frame coding complexity of the I frame to be coded and combining the target coding bit number of the I frame to be coded; intra-coding complexity of an I-frame to be encoded multiplexes the intra-coding complexity of a previous encoded video frame; the intra-frame coding complexity of each image block inside the I frame to be coded multiplexes the intra-frame coding complexity of the same-position image block inside the previous coded video frame;
the image block bit control module is used for calculating the quantization parameter of the image block to be coded in the I frame to be coded according to the target coding bit number and the intra-frame coding complexity of the image block to be coded in the I frame to be coded and combining the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the co-located image block in the previous coded I frame; the intra-coding complexity of each tile within an I-frame to be encoded multiplexes the intra-coding complexity of co-located tiles within a previous encoded video frame.
10. The device for rate control at the tile level of an I-frame according to claim 9, further comprising a scaling module after said tile bit control module; the conversion module is used for obtaining the Lagrange multiplier of the image block to be encoded according to the conversion relation between the quantization parameter and the Lagrange multiplier.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310779930.0A CN116939207A (en) | 2023-06-28 | 2023-06-28 | Code rate control method and device for image block level of I frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310779930.0A CN116939207A (en) | 2023-06-28 | 2023-06-28 | Code rate control method and device for image block level of I frame |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116939207A true CN116939207A (en) | 2023-10-24 |
Family
ID=88378236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310779930.0A Pending CN116939207A (en) | 2023-06-28 | 2023-06-28 | Code rate control method and device for image block level of I frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116939207A (en) |
-
2023
- 2023-06-28 CN CN202310779930.0A patent/CN116939207A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4828925B2 (en) | Encoder | |
| US7532764B2 (en) | Prediction method, apparatus, and medium for video encoder | |
| US8582904B2 (en) | Method of second order prediction and video encoder and decoder using the same | |
| KR101377883B1 (en) | Non-zero rounding and prediction mode selection techniques in video encoding | |
| Merritt et al. | x264: A high performance H. 264/AVC encoder | |
| CN101325714B (en) | Method and apparatus for processing transformation data, method and apparatus for encoding and decoding | |
| CN101946516A (en) | The decision of macro block increment quantization parameter fast | |
| CN106034235B (en) | Method and system for calculating coding distortion degree and controlling coding mode | |
| KR20120042910A (en) | Template matching for video coding | |
| JPWO2009157581A1 (en) | Image processing apparatus and image processing method | |
| KR20050119422A (en) | Method and apparatus for estimating noise of input image based on motion compenstion and, method for eliminating noise of input image and for encoding video using noise estimation method, and recording medium for storing a program to implement the method | |
| JP2006025429A (en) | Coding method and circuit device for executing this method | |
| KR100597397B1 (en) | Method For Encording Moving Picture Using Fast Motion Estimation Algorithm, And Apparatus For The Same | |
| JP2017069866A (en) | Moving image encoder, moving image encoding method and computer program for encoding moving image | |
| CN114915789B (en) | Method, system, device and medium for optimizing Lagrange multiplier between frames | |
| CN108989818B (en) | Image coding parameter adjusting method and device | |
| JP2011091772A (en) | Image encoder | |
| CN116939207A (en) | Code rate control method and device for image block level of I frame | |
| CN116916027A (en) | Code rate control method and device for image block level of P frame or B frame | |
| JP2010212785A (en) | Moving image coding apparatus and moving image coding method | |
| CN116886921A (en) | Code rate control method and device for video frame level of I frame | |
| CN116248882A (en) | Code rate control method and device for I frame image block level | |
| CN116320434A (en) | Code rate control method and device for P frame and/or B frame image block level | |
| KR100336497B1 (en) | Rate Control Apparatus and Method Using Spatial Prediction Error Model for Moving Picture Coding | |
| CN115550656A (en) | I frame code rate control method and device suitable for hardware implementation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |