WO2024066649A1 - 一种视频处理方法及相关设备 - Google Patents
一种视频处理方法及相关设备 Download PDFInfo
- Publication number
- WO2024066649A1 WO2024066649A1 PCT/CN2023/106374 CN2023106374W WO2024066649A1 WO 2024066649 A1 WO2024066649 A1 WO 2024066649A1 CN 2023106374 W CN2023106374 W CN 2023106374W WO 2024066649 A1 WO2024066649 A1 WO 2024066649A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- prediction
- current block
- weight
- value
- values
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 105
- 238000000034 method Methods 0.000 claims description 200
- 230000008569 process Effects 0.000 claims description 64
- 230000003044 adaptive effect Effects 0.000 claims description 58
- 239000002131 composite material Substances 0.000 claims description 56
- 239000013598 vector Substances 0.000 claims description 28
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000013442 quality metrics Methods 0.000 claims description 21
- 238000013139 quantization Methods 0.000 claims description 18
- 238000005259 measurement Methods 0.000 claims description 12
- 230000008521 reorganization Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 5
- 230000001174 ascending effect Effects 0.000 claims description 4
- 150000001875 compounds Chemical class 0.000 abstract description 17
- 230000006870 function Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 13
- 238000006073 displacement reaction Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 238000001914 filtration Methods 0.000 description 7
- 230000009466 transformation Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000009795 derivation Methods 0.000 description 3
- 238000013144 data compression Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 101100025317 Candida albicans (strain SC5314 / ATCC MYA-2876) MVD gene Proteins 0.000 description 1
- 101150079299 MVD1 gene Proteins 0.000 description 1
- 101150020780 MVD2 gene Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
- H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Definitions
- the present application relates to the field of audio and video technology, specifically to the field of video coding technology, and in particular to a video processing method, a video processing device, a computer device, a computer-readable storage medium, and a computer program product.
- a block-based hybrid coding framework is adopted to divide the original video data into a series of coding blocks (Coding Unit, CU), and video coding methods such as prediction, transformation and entropy coding are combined to achieve video data compression.
- video coding standards such as AV1 (the first generation video coding standard developed by the Alliance for Open Media, Alliance for Open Media Video 1) and the AOM (Alliance for Open Media) next-generation standard AV2 (the second generation video coding standard developed by the Alliance for Open Media, Alliance for Open Media Video 2) under development contain a prediction mode called compound prediction.
- the prediction mode of the compound prediction allows the use of multiple reference video signals for weighted prediction; however, it is found in practice that the current compound prediction has low prediction accuracy when performing weighted prediction, which affects the encoding and decoding performance of the video.
- a video processing method and related devices are provided.
- an embodiment of the present application provides a video processing method, which is executed by a computer device, and the method includes:
- a composite prediction is performed on a current block in a video bitstream to obtain N reference prediction values of the current block, where N is an integer greater than 1; the current block is a coding block to be decoded in the video bitstream; the N reference prediction values are derived from the N reference blocks of the current block; the N reference blocks are coding blocks referenced when decoding the current block in the video bitstream; there is a one-to-one correspondence between the reference prediction values and the reference blocks; according to the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block, the target weight group includes one or more weight values; the importance is used to characterize the degree of influence of the corresponding reference prediction value on the decoding performance of the current block; and based on the weight values in the target weight group, the N reference prediction values are weighted predicted to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block.
- an embodiment of the present application provides a video processing method, which is executed by a computer device, and the method includes:
- the current frame in the video is divided and processed to obtain the current block; the current block is compositely predicted to obtain N reference prediction values of the current block, where N is an integer greater than 1; the N reference prediction values are derived from the N reference blocks of the current block; the N reference blocks are coding blocks referenced when encoding the current block in the video; there is a one-to-one correspondence between the reference prediction values and the reference blocks; according to the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block, the target weight group includes one or more weight values; the importance is used to characterize the influence of the corresponding reference prediction value on the coding performance of the current block; based on the weight value in the target weight group, the N reference prediction values are weighted predicted to obtain the prediction value of the current block, and the prediction value of the current block is used to reconstruct the decoded image corresponding to the current block; and based on the prediction value of the current block, the video is encoded to generate a video bitstream.
- an embodiment of the present application provides a video processing device, the device comprising:
- a processing unit is used to perform composite prediction on a current block in a video code stream to obtain N reference prediction values of the current block, where N is an integer greater than 1; the current block is a coding block to be decoded in the video code stream; the N reference prediction values are derived from the N reference blocks of the current block; the N reference blocks are coding blocks referenced when decoding the current block in the video code stream; there is a one-to-one correspondence between the reference prediction values and the reference blocks; a determination unit is used to determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values, the target weight group including one or more weight values; the importance is used to characterize the degree of influence of the corresponding reference prediction value on the decoding performance of the current block; and the processing unit is also used to perform weighted prediction processing on the N reference prediction values based on the weight values in the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding
- an embodiment of the present application provides a video processing device, the device comprising:
- a processing unit is used to divide the current frame in the video to obtain the current block; the processing unit is also used to perform composite prediction on the current block to obtain N reference prediction values of the current block; N is an integer greater than 1; the N reference prediction values are derived from the N reference blocks of the current block; the N reference blocks are coding blocks referenced when encoding the current block in the video; there is a one-to-one correspondence between the reference prediction values and the reference blocks; a determination unit is used to determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values, the target weight group including one or more weight values; the importance is used to characterize the degree of influence of the corresponding reference prediction value on the coding performance of the current block; the processing unit is also used to perform weighted prediction processing on the N reference prediction values based on the weight values in the target weight group to obtain the prediction value of the current block, the prediction value of the current block is used to reconstruct the decoded image corresponding to the current block; and the processing unit is also used to
- an embodiment of the present application provides a computer device, the computer device comprising:
- a processor is adapted to execute computer-readable instructions; a computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the above-mentioned video processing method is implemented.
- an embodiment of the present application provides a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions are loaded by a processor and executed as the above-mentioned video processing method.
- an embodiment of the present application provides a computer program product, which includes computer-readable instructions, and the computer-readable instructions are stored in a computer-readable storage medium.
- a processor of a computer device reads the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computer device performs the above-mentioned video processing method.
- FIG. 1a is a basic workflow diagram of a video encoding provided by an exemplary embodiment of the present application.
- FIG. 1 b is a schematic diagram of an inter-frame prediction provided by an exemplary embodiment of the present application.
- FIG. 2 is a schematic diagram of the structure of a video processing system provided by an exemplary embodiment of the present application.
- FIG. 3 is a flowchart of a video processing method provided by an exemplary embodiment of the present application.
- FIG. 4 is a flowchart of a video processing method provided by another exemplary embodiment of the present application.
- FIG. 5 is a schematic diagram of the structure of a video processing device provided by an exemplary embodiment of the present application.
- FIG. 6 is a schematic structural diagram of a video processing device provided by another exemplary embodiment of the present application.
- FIG. 7 is a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application.
- a video can be composed of one or more video frames, each of which contains part of the video signal of the video.
- the video signal can be obtained by camera or computer generated. Due to the different statistical characteristics corresponding to different acquisition methods, the compression encoding method of the video may also be different.
- HEVC High Efficiency Video Coding, the international video coding standard HEVC/H.265)
- VVC Very Video Coding, the international video coding standard VVC/H.266
- AV1 Alliance for Open Media Video 1, the first generation video coding standard developed by the Open Media Alliance
- AV2 Alliance for Open Taking Media Video 2 (the second-generation video coding standard developed by the Open Media Alliance)
- AVS3 Audio Video Coding Standard 3, the third-generation audio and video source coding standard
- Block partition structure According to the size of the current input frame (i.e., the video frame being encoded or decoded), the current frame can be divided into several non-overlapping processing units, and each processing unit will perform similar compression operations. This processing unit is called CTU (coding tree unit) or LCU (Largest Coding Unit). CTU can be further divided into more refined units to obtain one or more basic coding units, called CU (coding unit or coding block). Each CU is the most basic element in the encoding link. The subsequent embodiments of this application describe various encoding and decoding processing flows that may be used for each CU.
- CTU coding tree unit
- LCU Large Coding Unit
- Prediction coding including intra-frame prediction and inter-frame prediction modes.
- the original video signal contained in the current CU in the current frame i.e., the CU being encoded or decoded in the current frame
- the current CU can also be called the current block
- the video frame where the current block is located is called the current frame
- the reference CU used for predicting the current block can also be called the reference block of the current block
- the video frame where the reference block is located is called the reference frame.
- the encoder needs to decide for the current CU to select the most appropriate prediction coding mode from many possible prediction coding modes and tell the decoder.
- the prediction coding mode may include:
- Intra-frame prediction (Intra (picture) Prediction): The reconstructed video signal used for prediction comes from the area that has been encoded and reconstructed in the same video frame, that is, the current block and the reference block are in the same video frame.
- the basic idea of intra-frame prediction is to use the correlation of adjacent pixels in the same video frame to remove spatial redundancy.
- adjacent pixels refer to the reconstructed pixels of the encoded CU around the current CU in the same video frame.
- Inter-picture prediction The reconstructed video signal used for prediction comes from other video frames that have been encoded and are different from the current frame, that is, the current block and the reference block are located in different video frames.
- Transform & Quantization The residual video signal is transformed into the transform domain through DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform) and other transformation operations, which are called transform coefficients.
- DFT Discrete Fourier Transform
- DCT Discrete Cosine Transform
- the residual video signal in the transform domain is further subjected to lossy quantization operations, which loses certain information, making the quantized signal conducive to compression expression.
- the encoder also needs to select one of the transforms for the current CU and inform the decoder.
- the degree of quantization is usually determined by QP (Quantization Parameters).
- QP Quantization Parameters
- the transform coefficients representing a larger range of values will be quantized to the same output, which usually results in greater distortion and lower bitrates.
- the QP value is small, the transform coefficients representing a smaller range of values will be quantized to the same output, which usually results in less distortion and a higher bitrate.
- Entropy Coding or Statistical Coding The quantized transform domain signal will be statistically compressed and encoded according to the frequency of occurrence of each value, and finally output as a binary (0 or 1) video stream. At the same time, encoding will generate other information, such as the selected prediction coding mode, motion vector, etc. These other information also need to be entropy encoded to reduce the bit rate. Among them, statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).
- VLC variable length coding
- CABAC context-based binary arithmetic coding
- Loop Filtering The encoded CU can be reconstructed into a decoded image corresponding to the CU after undergoing inverse quantization, inverse transformation and prediction compensation (the reverse operation of 2) to 4) above). Compared with the original image, the reconstructed decoded image has some information that is different from the original image due to the influence of quantization, resulting in distortion. Therefore, a filter can be used to filter the reconstructed decoded image to effectively reduce the degree of distortion caused by quantization.
- the filter can be, for example, a deblocking filter, SAO (Sample Adaptive Offset) or ALF (Adaptive Loop Filter), etc.
- Figure 1a is an exemplary basic workflow diagram of a video encoder.
- Figure 1a takes the current block as the k-th CU in the current frame (current image) ( sk [x,y] as shown in Figure 1a) as an example for explanation, k is a positive integer, and k is less than or equal to the total number of CUs contained in the current frame.
- sk [x,y] represents the pixel point (referred to as pixel) with coordinates [x, y] in the k-th CU, x represents the horizontal coordinate of the pixel, and y represents the vertical coordinate of the pixel; sk [x,y] can obtain a prediction signal after motion compensation or intra-frame prediction, and the prediction signal Subtract the original signal s k [x, y] to obtain the residual video signal uk [x, y]; then transform and quantize the residual video signal uk [x, y].
- the data output by the quantization process has two different destinations, A and B:
- the data output by the quantization process can be sent to an entropy encoder for entropy encoding to obtain an encoded bit stream (i.e., a video bit stream), and the bit stream is output to a buffer for storage and waiting to be transmitted.
- an entropy encoder for entropy encoding to obtain an encoded bit stream (i.e., a video bit stream)
- the bit stream is output to a buffer for storage and waiting to be transmitted.
- the data output by the quantization process can be dequantized and inversely transformed to obtain the inversely transformed residual video signal u′ k [x, y]. Then, the inversely transformed residual video signal u′ k [x, y] is compared with the prediction signal Add together to get a new prediction signal And the new prediction signal The new prediction signal can then be sent to the current image buffer for storage. After intra-frame prediction processing, we get The new prediction signal After loop filtering, the reconstructed signal s′ k [x, y] can be obtained, and the reconstructed signal s′ k [x, y] is sent to the decoded image buffer for storage to generate the reconstructed video. The reconstructed signal s′ k [x, y] is obtained after motion compensation prediction processing.
- the reference block may be represented by m x and m y , respectively representing the horizontal and vertical components of the motion vector of the reference block.
- mainstream video coding standards such as HEVC, VVC, AVS3, AV1, and AV2 all use a block-based hybrid coding framework, which can divide the original video data into a series of coding blocks, combined with video coding methods such as prediction, transformation, and entropy coding to achieve video data compression.
- motion compensation is a commonly used prediction method for video coding. Motion compensation is based on the redundant characteristics of video content in the time domain or spatial domain, and derives the prediction value of the current block from the encoded reference block. This type of prediction method includes: inter-frame prediction, intra-frame block copy prediction, intra-frame string copy prediction, etc. In specific coding implementations, these prediction methods may be used alone or in combination.
- the displacement vector is used to indicate the displacement of the current block (or the same-position block of the current block) relative to one or more reference blocks.
- the displacement vector may have different names. This article uniformly describes them in the following way: 1) The displacement vector in inter-frame prediction is called motion vector (MV); 2) The displacement vector in intra-frame block copy prediction is called block vector (BV); 3) The displacement vector in intra-frame string copy prediction is called string vector (SV).
- MV motion vector
- BV block vector
- SV string vector
- Inter-frame prediction uses the correlation in the video time domain and uses the adjacent encoded image pixels to predict the pixels of the current image, so as to effectively remove the video time domain redundancy, and can effectively save the bits of the encoded residual data.
- Figure 1b is a schematic diagram of an inter-frame prediction provided by an embodiment of the present application.
- P is the current frame
- Pr is the reference frame
- B is the current block
- Br is the reference block of B.
- B' and B have the same coordinate position in the image (that is, B' is the same-position block of B), Br coordinates are ( xr , yr ), and B' coordinates are (x, y).
- inter-frame prediction includes two MV prediction technologies: Merge and AMVP (Advanced Motion Vector Prediction).
- the Merge mode will establish an MV candidate list for the current PU (Prediction Unit), in which there are 5 candidate MVs (and their corresponding reference images). Traverse these 5 candidate MVs and select the one with the lowest rate-distortion cost as the optimal MV. If the codec establishes the candidate list in the same way, the encoder only needs to transmit the index of the optimal MV in the candidate list.
- DMVP dynamic motion vector prediction
- the current mainstream video coding standards allow the use of multiple reference frames for inter-frame prediction.
- the AV1 standard and the AOM next-generation standard AV2 currently under development include a prediction mode called compound prediction.
- This compound prediction mode allows the current block to use two reference frames for inter-frame prediction and
- the prediction value of the current block is derived by weighted combination of the inter-prediction value, or the inter-prediction value derived from a reference frame and the intra-prediction value derived from the current frame are weighted combined to derive the prediction value of the current block.
- the current block refers to the coding block being encoded (or decoded).
- the inter-prediction value and the intra-prediction value are subsequently referred to as reference prediction values.
- P(x,y) is the prediction value of the current block
- P 0 (x,y) and P 1 (x,y) are two reference prediction values corresponding to the current block (x,y)
- w(x,y) is the weight applied to the first reference prediction value P 0 (x,y).
- integer calculations are usually used instead of floating-point calculations.
- the weight value w(x,y) and the reference prediction values P 0 (x,y) and P 1 (x,y) are all integers, and right shift operation is used instead of division. ">>6" means right shift by 6 bits. 64 can be divided by right shift by 6 bits. 32 is the bias added for rounding.
- composite prediction uses a special weighting mode, that is, P0 and P1 have equal weight values, and the weights corresponding to the reference prediction values corresponding to different positions are set to fixed values.
- P0 and P1 have equal weight values
- the weights corresponding to the reference prediction values corresponding to different positions are set to fixed values.
- the video bitstream is first entropy decoded to obtain information on various prediction coding modes and quantized transform coefficients, and then each transform coefficient is dequantized and inversely transformed to obtain a residual video signal.
- the prediction signal corresponding to the CU hereinafter referred to as the prediction value
- the residual video signal and the prediction signal are added to obtain a reconstructed video signal, which can be used to reconstruct the decoded image corresponding to the CU.
- the reconstructed video signal needs to undergo a loop filtering operation to generate the final output signal.
- the embodiment of the present application provides a video processing solution, which can be applied to a video encoder or video compression product using composite prediction (or weighted prediction based on multiple reference frames).
- the general principle of the video processing solution is as follows:
- composite prediction can be used for the CU contained in the video frame to obtain N reference prediction values of the CU, where N is an integer greater than 1; and according to the importance of the N reference prediction values, appropriate weights are adaptively selected for the CU for weighted prediction to obtain the prediction value of the CU.
- the so-called weighted prediction refers to the use of adaptively selected weights to perform weighted prediction processing on the N reference prediction values of the CU. Then, the video is encoded according to the prediction value of the CU to generate a video stream, which is sent to the decoding end.
- a composite prediction of the CU in the video stream can be determined based on the information of the prediction coding mode, and appropriate weights can be adaptively selected for the CU according to the importance of the N reference prediction values for weighted prediction; then, the N reference prediction values are weighted predicted based on the adaptively selected weights to obtain the prediction value of the CU, and the decoded image corresponding to the CU is reconstructed using the prediction value of the CU.
- the current video coding standard stipulates that equal weight values are used to perform weighted prediction on reference prediction values derived from different reference blocks in composite prediction.
- the reference prediction values derived from different reference blocks may have unequal importance.
- the use of equal weight values cannot reflect the difference in importance of the predicted reference values.
- the use of existing standards will affect the prediction accuracy.
- the embodiments of the present application improve the current video coding standard, fully consider the importance of reference prediction values derived from different reference blocks in composite prediction, and allow adaptive selection of appropriate weights for CU based on the importance of each reference prediction value in composite prediction for weighted prediction. This expands the weighted prediction method in the video coding standard, can improve the prediction accuracy of the CU, and thus improve the coding performance.
- the video processing system 20 may include an encoding device 201 and a decoding device 202.
- the encoding device 201 is located at the encoding end, and the decoding device is located at the decoding end.
- the encoding device 201 can be a terminal or a server, and the decoding device 202 can be a terminal or a server.
- a communication connection can be established between the encoding device 201 and the decoding device 202.
- the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, Smart speakers, smart watches, car terminals, smart TVs, etc., but not limited to these.
- the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
- the encoding device 201 can obtain a video to be encoded, which can be obtained by shooting with a camera or generated by a computer.
- the camera can be a hardware component provided in the encoding device 201, for example, the camera can be a common camera, a stereo camera, a light field camera, etc. provided in a terminal.
- the camera can also refer to a hardware device connected to the encoding device 201, such as a camera connected to a server.
- a video includes one or more video frames
- the encoding device 201 can divide each video frame into one or more CUs and encode each CU.
- a composite prediction can be performed on the CU being encoded (hereinafter referred to as the current block) to obtain N reference prediction values of the current block, and the importance of each reference prediction value is determined by comprehensively considering factors such as the bit rate consumed during weighted prediction processing and the quality loss of the current block during the encoding process, and then based on the importance of each reference prediction value, a suitable target weight group is adaptively selected for the current block, and the target weight group may include one or more weight values; then, the weight values in the target weight group are used to perform weighted prediction processing on the N reference prediction values to obtain the prediction value of the current block.
- the prediction value of the current block can be understood as the prediction signal corresponding to the current block, and the prediction value of the current block can be used to reconstruct the decoded image corresponding to the current block.
- the N reference prediction values of the current block are derived from the N reference blocks of the current block; one reference prediction value corresponds to one reference block.
- the video frame where the reference block is located is the reference frame, and the video frame where the current block is located is the current frame.
- the positional relationship between the N reference blocks and the current block may include but is not limited to any of the following: 1 The N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video code stream; 2 The N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video code stream; 3 One or more reference blocks among the N reference blocks are located in the current frame, and the remaining reference blocks among the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video code stream; 4 Both the N reference blocks and the current block are located in the current frame.
- the prediction mode of the composite prediction of the embodiment of the present application includes an inter-frame prediction mode, that is, allowing the use of at least two reference frames for inter-frame prediction; it also includes a combined prediction mode, that is, allowing the use of at least one reference frame for inter-frame prediction and allowing the use of the current frame for intra-frame prediction; it also includes an intra-frame prediction mode, that is, allowing the use of the current frame for intra-frame prediction.
- the derivation methods of the N reference prediction values of the current block may include any of the following: 1
- the N reference prediction values of the current block are derived after performing inter-frame prediction using the N reference blocks of the current block respectively, and at this time, the N reference prediction values can all be called inter-frame prediction values.
- 2 Among the N reference prediction values of the current block at least one reference prediction value is derived after performing inter-frame prediction using at least one reference block among the N reference blocks of the current block, and this part of the reference prediction values can be called inter-frame prediction values; the remaining reference prediction values are derived after performing intra-frame prediction using the remaining reference blocks among the N reference blocks, and this part of the reference prediction values can be called intra-frame prediction values.
- the encoding device 201 performs transform coding, quantization, entropy coding and other operations on the video based on the prediction value of the CU contained in the video frame to obtain a video code stream, and sends the video code stream to the decoding device 202 so that the decoding device 202 decodes the video code stream.
- the decoding device 202 may decode the video bitstream after receiving the video bitstream sent by the encoding device 201, and reconstruct the video corresponding to the video bitstream. Specifically, on the one hand, the decoding device 202 may perform entropy decoding on the video bitstream to obtain the prediction mode and quantized transform coefficients of each CU in the video bitstream, and perform composite prediction on the current block (i.e., the CU being decoded) according to the prediction mode of the current block to obtain N reference prediction values of the current block; and determine whether the current block is allowed to use adaptive weighted prediction.
- a target weight list can be determined from one or more weight lists according to the importance of the N reference prediction values, and a target weight group for weighted prediction is determined for the current block from the target weight list, the target weight group containing one or more weight values. Then, the weight values in the target weight group are used to directly The N reference prediction values are weighted predicted to obtain the prediction value of the current block. If it is determined that the current block does not allow the use of adaptive weighted prediction, the N reference prediction values can be weighted predicted according to the existing video coding standard, for example, each reference prediction value is weighted predicted with an equal weight value to obtain the prediction value of the current block.
- the decoding device 202 performs inverse quantization and inverse transformation on the quantized transform coefficients to obtain the residual signal value of the current block, and superimposes the predicted value and the residual signal value of the current block to obtain the reconstruction value of the current block, and then reconstructs the decoded image corresponding to the current block according to the reconstruction value.
- the decoded image can be used as a reference image used when decoding other CUs, and can also be used to reconstruct video.
- composite prediction is used in the video encoding and decoding process, and the importance of reference prediction values derived from different reference blocks is fully considered in the composite prediction, allowing adaptive selection of appropriate weights for CU based on the importance of each reference prediction value to perform weighted prediction in the composite prediction, thereby expanding the weighted prediction method in the video coding standard, and can improve the prediction accuracy of the CU, thereby improving the encoding and decoding performance.
- FIG. 3 is a flow chart of a video processing method provided by the embodiment of the present application.
- the video processing method can be executed by a decoding device in the above-mentioned video processing system.
- the video processing method described in this embodiment may include the following steps S301-S303:
- N reference prediction values are derived from N reference blocks of the current block; the N reference blocks are coding blocks referenced when decoding the current block in the video bitstream; and there is a one-to-one correspondence between the reference prediction values and the reference blocks.
- the video code stream contains one or more video frames, and each video frame may contain one or more coding blocks.
- the decoding device may obtain the coding block from the video code stream, and the decoding device may use the coding block to be decoded as the current block.
- the above-mentioned N reference prediction values may be derived from N reference blocks, and one reference prediction value corresponds to one reference block, that is, the N reference prediction values are obtained based on the N reference blocks, and the reference prediction values correspond to the reference blocks one by one.
- the N reference blocks are the coding blocks referenced when decoding the current block in the video code stream.
- the video frame where the reference block is located may be a reference frame
- the video frame where the current block is located may be a current frame.
- the positional relationship between the N reference blocks and the current block includes any of the following: 1
- the N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame may belong to different video frames in the video code stream.
- the N reference blocks are located in the same reference frame, and the same reference frame and the current frame may belong to different video frames in the video code stream.
- N reference blocks are located in the current frame, and the remaining reference blocks of the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video code stream.
- the four reference blocks are: reference block 1, reference block 2, reference block 3, reference block 4; reference block 1 is located in the current frame, the remaining reference block 2, reference block 3, reference block 4 are all located in reference frame 1, and reference frame 1 and the current frame belong to different video frames in the video code stream; for another example, reference block 1 and reference block 2 are located in the current frame, the remaining reference block 3 can be located in reference frame 1, and reference block 4 is located in a reference frame 2, and reference frame 1, reference frame 2 and the current frame belong to different video frames in the video code stream.
- the prediction mode of the composite prediction in the embodiment of the present application includes both an inter-frame prediction mode, that is, allowing the use of at least two reference frames for inter-frame prediction; and a combined prediction mode, that is, allowing the use of at least one reference frame for inter-frame prediction, while allowing the use of the current frame for intra-frame prediction; and also includes an intra-frame prediction mode, that is, allowing the use of the current frame for intra-frame prediction.
- the derivation methods of the N reference prediction values of the current block may include any of the following: 1
- the N reference prediction values of the current block are derived by using the N reference blocks of the current block to perform inter-frame prediction respectively.
- the N reference prediction values can all be called inter-frame prediction values.
- 2 Among the N reference prediction values of the current block at least one reference prediction value is derived by using at least one reference block in the N reference blocks of the current block to perform inter-frame prediction. This part of the reference prediction values can be called inter-frame prediction values; the remaining reference prediction values are derived by using the remaining reference blocks in the N reference blocks to perform inter-frame prediction.
- the reference prediction values derived after intra-frame prediction are referred to as intra-frame prediction values.
- the N reference blocks may be derived from different video frames, specifically from the current frame where the current block is located or from different reference frames, so that they can be applied to different composite prediction scenarios, ensuring the encoding and decoding performance of composite prediction in different scenarios.
- the decoding device may first determine whether the current block meets the conditions for adaptive weighted prediction, that is, determine the conditions for adaptive weighted prediction. If the current block meets the conditions for adaptive weighted prediction, step S302 is executed; by determining whether the current block meets the conditions for adaptive weighted prediction, the decoding device can adaptively select weights for weighted prediction, further improve prediction accuracy, and thus improve encoding performance.
- the condition that the current block satisfies the adaptive weighted prediction includes at least one of the following:
- the sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that all coding blocks in the frame sequence are allowed to use adaptive weighted prediction; wherein the frame sequence refers to a sequence composed of multiple video frames in a video bitstream in order. It should be understood that when the sequence header of the frame sequence contains the first indication field, the first indication field is used to indicate that all coding blocks contained in the entire frame sequence are allowed to use adaptive weighted prediction.
- the first indication field can be expressed as seq_acp_flag.
- the first indication field can indicate whether the coding blocks in the frame sequence are allowed to use adaptive weighted prediction according to the value. If the above-mentioned first indication field is a first preset value (such as 1), it indicates that all coding blocks in the frame sequence are allowed to use adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the above-mentioned first indication field is a second preset value (such as 0), it indicates that all coding blocks in the frame sequence are not allowed to use adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.
- a first preset value such as 1
- a second preset value such as 0
- the slice header of the current slice to which the current block belongs contains a second indication field, and the second indication field indicates that the coding blocks in the current slice allow the use of adaptive weighted prediction, wherein a video frame can be divided into multiple image slices, each image slice contains one or more coding blocks; the current slice refers to the image slice to which the current block belongs, that is, the image slice being decoded. It should be understood that when the second indication field is included in the slice header of the current slice, the second indication field can be used to indicate that all coding blocks contained in the current slice allow the use of adaptive weighted prediction.
- the second indication field can be expressed as slice_acp_flag, and the second indication field can indicate whether the coding blocks in the current slice are allowed to use adaptive weighted prediction according to the value. If the above-mentioned second indication field is the first preset value (such as 1), it indicates that all coding blocks in the current slice are allowed to use adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the above-mentioned second indication field is the second preset value (such as 0), it indicates that all coding blocks in the current slice are not allowed to use adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.
- the first preset value such as 1
- the second indication field is the second preset value (such as 0)
- the frame header of the current frame where the current block is located includes a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction. It should be understood that when the frame header of the current frame includes the third indication field, the third indication field can be used to indicate that all coding blocks included in the current frame allow the use of adaptive weighted prediction.
- the third indication field can be represented as pic_acp_flag, and the third indication field indicates whether the current frame allows the use of adaptive weighted prediction based on its value. If the above-mentioned third indication field is a first preset value (such as 1), it indicates that the coding block in the current frame allows the use of adaptive weighted prediction, and it can be determined that the current block meets the adaptive weighted prediction conditions; if the third indication field is a second preset value (such as 0), it can indicate that the coding block in the current frame does not allow the use of adaptive weighted prediction, and it can be determined that the current block does not meet the adaptive weighted prediction conditions.
- a first preset value such as 1
- the third indication field is a second preset value (such as 0)
- the current block uses at least two reference frames for inter-frame prediction.
- the current block uses at least one reference frame for inter-frame prediction and uses the current frame for intra-frame prediction. For example, the current block uses reference frame one for inter-frame prediction and uses the current frame for intra-frame prediction, then it can be determined that the current block meets the conditions of adaptive weighted prediction.
- the motion type of the current block is the specified motion type. For example, if the motion type of the current block is simple_translation (simple balance), it is determined that the current block meets the conditions for adaptive weighted prediction.
- the current block uses a preset motion vector prediction mode.
- the current block uses a preset motion vector prediction mode of NEAR_NEARMV. Then, it can be determined that the current block meets the condition of adaptive weighted prediction.
- MVs can be predicted by spatially adjacent blocks in the current frame or temporally adjacent blocks in the reference frame.
- each reference frame has a separate list of predicted MVs; for composite inter-frame prediction, different
- the prediction mv list corresponding to the reference frame constitutes the prediction MV group list, and allows the use of multiple prediction MV modes, such as NEAR_NEARMV, NEAR_NEWMV, NEW_NEAR_MV, NEW_NEWMV, GLOBAL_GLOBALMV, JOINT_NEWMV, etc.
- NEAR_NEARMV NEAR_NEWMV
- NEW_NEAR_MV NEW_NEWMV
- GLOBAL_GLOBALMV JOINT_NEWMV
- NEAR_NEARMV indicates that the MVs corresponding to the two reference frames are the MVs in the predicted MV group
- NEAR_NEWMV The first MV is the first predicted MV in the predicted MV group, and the second MV is derived based on the MVD decoded from the video bitstream and the second predicted MV in the predicted MV group.
- MVD Motion Vector Difference
- MVP Motion Vector Prediction
- NEW_NEARMV The first MV is derived based on the MVD decoded from the video bitstream and the first predicted MV in the predicted MV group, and the second MV is the second predicted MV in the predicted MV group.
- NEW_NEWMV The first MV is derived based on MVD1 decoded from the video bitstream and the first predicted MV in the predicted MV group.
- the second MV is derived based on MVD2 decoded from the video bitstream and the second predicted MV in the predicted MV group.
- GLOBAL_GLOBALMV derive MV based on per-frame level global motion information.
- NEW_NEWMV Similar to NEW_NEWMV, but the video bitstream contains only one MVD, and the other MVD is derived based on the information between reference frames.
- the preset motion vector prediction mode may be one or more of the above: NEAR_NEARMV, NEAR_NEWMV, NEW_NEARMV, NEW_NEWMV, GLOBAL_GLOBALMV, JOINT_MV. It should be noted that the preset motion vector prediction mode is not limited to the above MV prediction mode. For other standards, such as H.265 and H.266, the motion vector prediction mode may also be determined in combination with merge and AMVP.
- the current block uses a preset interpolation filter.
- the video encoding process usually includes many encoding tools, including various types of interpolation filters, such as linear interpolation filters, CIC (integral-comb cascade) interpolation filters, and so on.
- the preset interpolation filter in the embodiment of the present application can be any of the various types of interpolation filters.
- the preset interpolation filter can be a linear interpolation filter.
- the current block does not use a specific coding tool, such as the current block does not use the motion vector optimization method based on optical flow, then it can be determined that the current block meets the conditions of adaptive weighted prediction.
- video coding standards such as AV2 and H.266 allow the use of the motion vector optimization method based on optical flow, which is a method for refining the motion vector based on the derivation of the optical flow equation.
- the specific conditions include one or more of the following (that is, the specific conditions can include one or more of 1 and 2): 1
- the position relationship between the reference frame used in the composite prediction and the current frame in the video bitstream meets the preset relationship; wherein, the position relationship meets the preset relationship and includes any of the following: the reference frames used are all located before the current frame; the reference frames used are all located after the current frame; some of the reference frames used are located before the current frame, and the remaining reference frames are located after the current frame.
- the video bitstream contains multiple video frames, and any video frame corresponds to a frame display time in the video.
- the position relationship can actually be understood as the order of frame display time.
- the reference frames used are all located before the current frame, which can be understood as: the frame display time of the reference frame is earlier than the frame display time of the current frame; the reference frames used are all located after the current frame, which can be understood as: the frame display time of the reference frame is later than the frame display time of the current frame.
- the preset threshold can be set according to the requirements.
- the importance of the reference prediction values corresponding to the reference frames can be measured by an importance metric value.
- the importance difference between the reference prediction values corresponding to the reference frames used in the composite prediction can be determined according to the importance metric values between the reference prediction values corresponding to the reference frames used.
- the above-mentioned current block satisfies the conditions for adaptive weighted prediction and can be used separately or in combination.
- the conditions for the current block to satisfy the adaptive weighted prediction may include: the sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction, and the motion type of the current block is a specified motion type.
- the frame header of the current frame where the current block is located contains a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction and the current block uses a preset motion vector prediction mode; and so on; the present application does not impose any limitation on this.
- the decoding device is supported to flexibly configure each condition item in the conditions for adaptive weighted prediction, which can be applied to different compound prediction scenarios to ensure the encoding and decoding performance of compound prediction in different scenarios.
- the importance of the reference prediction value can be comprehensively determined based on factors such as the bit rate consumed during weighted prediction processing and the quality loss of the current block during the encoding process. For example, when the current block uses a certain reference prediction value for weighted prediction, the bit rate consumption increases significantly, indicating that the reference prediction value is not effective enough in reducing the bit rate consumption and its importance is low. Another example is that when the current block uses a certain reference prediction value for weighted prediction, the quality loss increases significantly, indicating that the reference prediction value is less effective in reducing the quality loss and its importance is low. According to the importance of the reference prediction value, the degree of influence of the reference prediction value on the decoding performance of the current block can be determined.
- the target weight group contains one or more weight values, which will act on each reference prediction value during the weighted prediction process. If the importance of a reference prediction value is low, then the reference prediction value corresponds to a weight value with a smaller value in the target weight group, and if the importance of a reference prediction value is high, then the reference prediction value corresponds to a weight value with a larger value in the target weight group. That is to say, the target weight group is selected based on the influence of each reference prediction value on factors such as bit rate consumption and quality loss, so that the comprehensive cost of the weighted prediction process (i.e., the cost of bit rate consumption, the cost of quality loss, the cost of bit rate consumption and quality loss) is smaller and the encoding and decoding performance is better.
- the comprehensive cost of the weighted prediction process i.e., the cost of bit rate consumption, the cost of quality loss, the cost of bit rate consumption and quality loss
- the video bitstream includes one or more weight lists, each weight list includes one or more weight groups, and a weight group includes one or more weight values; the number of weight values included in each weight group is allowed to be the same or different, the values of the weight values included in each weight group are allowed to be the same or different, and the order of the weight values included in each weight group is allowed to be the same or different.
- a weight list includes one or more weight values
- Weight list 1 is represented as: ⁇ 2,4,6,8,10,12,14 ⁇ /16. This means that weight list 1 contains 7 weight groups, namely: weight group 1: ⁇ 2 ⁇ /16; weight group 2: ⁇ 4 ⁇ /16; weight group 3: ⁇ 6 ⁇ /16, and so on, weight group 7: ⁇ 14 ⁇ /16.
- Weight list 2 is represented as: ⁇ 14,8,4,12,2 ⁇ /16. This means that weight list 2 contains 5 weight groups, namely: weight group 1: ⁇ 14 ⁇ /16; weight group 2: ⁇ 8 ⁇ /16; and so on, weight group 5: ⁇ 2 ⁇ /16.
- Weight list 3 is represented as: ⁇ 4, ⁇ 8,8 ⁇ ,12 ⁇ /16. This indicates that weight list 3 contains 3 weight groups, namely: weight group 1: ⁇ 4 ⁇ /16; weight group 2: ⁇ 8,8 ⁇ /16; weight group 3: ⁇ 12 ⁇ /16.
- each weight list is allowed to be different, such as the number of weight groups in weight list 1 is 7, the number of weight groups in weight list 2 is 5, and the number of weight groups in weight list 1 is 3.
- the number of weight values contained in each weight group is allowed to be the same, for example, the number of weight values contained in each weight group in weight list 1 is 1.
- the data of weight values contained in each weight group is also allowed to be different, for example, the number of weight values contained in weight group 1 in weight list 3 is 1, but the number of weight values contained in weight group 2 in weight list 3 is 2.
- weight values in each weight group are allowed to be the same, for example: weight group 1 in weight list 1 contains weight value 2/16, and weight group 1 in weight list 2 also contains weight value 2/16.
- weight values in each weight group are also allowed to be different, for example, weight group 1 in weight list 1 contains weight value 2/16, but weight group 2 in weight list 1 contains weight value 4/16. It is understandable that the sum of the weight values provided by a weight group should be equal to 1. In the above example, although each weight group contains only one weight value, each weight value is less than 1.
- each weight group implicitly contains another weight value, that is, in a specific application, each weight group actually provides two weight values, and the sum of these two weight values is 1; for example, weight group 1 in weight list 1 only contains weight value 2/16, but in a specific application, weight group 1 actually provides two weight values, weight value 2/16 and weight value 14/16.
- weight group 3 in weight list 3 only contains weight value 12/16, but in a specific application, weight group 3 actually provides two weight values, weight value 12/16 and weight value 4/16. It can be seen that in the embodiment of the present application, when When the sum of the weight values contained in a weight group in the weight list is less than 1, the weight value implied by the weight group can be obtained by calculation.
- Weight list 4 contains 4 weight groups, namely weight group 1: ⁇ 2,14 ⁇ /16; weight group 2: ⁇ 4,12 ⁇ /16; weight group 3: ⁇ 6,10 ⁇ /16; weight group 4: ⁇ 8,8 ⁇ /16.
- Weight list 5 contains 2 weight groups, namely weight group 1: ⁇ 4,12 ⁇ /16; weight group 2: ⁇ 10,6 ⁇ /16.
- step S302 may include steps s31-s32:
- the decoding device determines the target weight list from one or more weight lists according to the importance of the N reference prediction values, it can be implemented by comprehensively considering the number of weight lists and the importance measurement value of each reference prediction value.
- the specific implementation method may include the following methods:
- Method 1 When there is only one weight list in the video bitstream, the decoding device may directly determine the only weight list in the video bitstream as the target weight list.
- Method 2 When there are multiple weight lists in the video bitstream, the decoding device may introduce an importance metric of the reference prediction value, and determine a target weight list from the multiple weight lists according to the importance metric of the N reference prediction values.
- M is a positive integer greater than or equal to 1
- the M+1 groups of weight lists can be recorded as ⁇ w_list1, w_list2...w_listM+1 ⁇ , respectively.
- One weight list corresponds to one threshold interval, that is, the number of threshold intervals is also M+1.
- the decoding device can directly set M+1 threshold intervals according to the requirements.
- the decoding device can obtain M thresholds and divide M+1 threshold intervals according to the M thresholds, such as obtaining M thresholds, which are T1, T2,..., TM; then the decoding device divides the M thresholds into M+1 threshold intervals according to the M thresholds, which are [0, T1], (T1, T2], (T2, T3],..., (TM, + ⁇ ); wherein [0, T1] can correspond to w_list1, (T1, T2] can correspond to w_list2, and so on. T is an integer greater than or equal to 0.
- the decoding device can obtain the importance metric values of N reference prediction values, and calculate the importance difference between the N reference prediction values.
- the importance difference between any two reference prediction values is measured by the difference between the importance metric values of any two reference prediction values.
- the decoding device can determine the threshold interval where the absolute value of the importance difference between the N reference prediction values is located; and determine the weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values is located as the target weight list.
- the importance metric values of any two reference prediction values are D0 and D1 respectively, and the importance difference between the any two reference prediction values is expressed as D0-D1
- the importance metric value is an indicator for measuring the degree of importance, but there can be multiple situations for the measurement benchmark.
- the measurement benchmark can be that the larger the importance metric value, the higher the degree of importance; for another example: the measurement benchmark can also be that the smaller the importance metric value, the higher the degree of importance; this measurement benchmark is not limited in this application.
- the embodiments of the present application can also use other numerical characteristics of the absolute value of the importance difference between the N reference prediction values, such as the maximum value, minimum value, average value, etc. after the square of each absolute value; to determine the target weight list, and the present application does not limit this.
- the decoding device determines the target weight list by the threshold interval of the difference between the importance metric values of the N reference prediction values, so as to determine the weight value suitable for reconstruction of the current block, which is beneficial to improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.
- the N reference prediction values of the current block may include a first reference prediction value and a second reference prediction value
- the video code stream may include a first weight list and a second weight list.
- the decoding device may compare the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value; if it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the first weight list is determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the decoding device determines the second weight list as the target weight list.
- the decoding device compares the sizes of D0 and D1; if D0>D1, the decoding device can determine the first weight list w_list1 as the target weight list; if D0 ⁇ D1, the decoding device can determine the first weight list w_list2 as the target weight list.
- the weight values in the first weight list and the weight values in the second weight list can also be set separately.
- the decoding device determines the corresponding weight list as the target weight list for the current block based on the numerical relationship between the two reference prediction values, which can improve the specificity of the target weight list, help improve the prediction accuracy of the current block, and thus improve the encoding and decoding performance.
- the N reference prediction values of the current block include a first reference prediction value and a second reference prediction value; the video bitstream includes a first weight list, a second weight list, and a third weight list.
- the decoding device can call a mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. If the symbol value is a first preset value (such as -1), the first weight list is determined as the target weight list; if the symbol value is If the symbol value is a second preset value (such as 0), the second weight list is determined as the target weight list; if the symbol value is a third preset value (such as 1), the third weight list is determined as the target weight list.
- the first weight list, the second weight list and the third weight list are different weight lists; or, two weight lists in the first weight list, the second weight list and the third weight list are allowed to be the same weight list.
- the importance metric of the first reference prediction value is D0
- the importance metric of the second reference prediction value is D1.
- sign() represents the data sign function.
- the decoding device determines the corresponding weight list from the three weight lists as the target weight list for the current block according to the numerical relationship between the two reference prediction values, which can improve the pertinence of the target weight list, help improve the prediction accuracy of the current block, and thus improve the encoding and decoding performance.
- the decoding device can first use method (1) to determine the candidate weight list through the threshold interval where the absolute value of the importance difference between the two reference prediction values is located, and then use method (2) to compare the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value.
- the decoding device If it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the decoding device directly determines the candidate weight list as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the decoding device determines the weight list corresponding to the weight value opposite to the weight value in the candidate weight list as the target weight list.
- the decoding device can first adopt method (1) to determine the threshold interval where the absolute value of the importance difference between the first reference prediction value and the second reference prediction value is located, and determine the three weight lists corresponding to the threshold interval as candidate weight lists, that is, the candidate weight list can include the first weight list, the second weight list, and the third weight list. Then the decoding device uses method (2) to call the mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain the symbol value.
- the decoding device determines the first weight list as the target weight list; if the symbol value is the second preset value, the decoding device determines the second weight list as the target weight list; if the symbol value is the third preset value, the decoding device determines the third weight list as the target weight list.
- method (1) is used to determine three weight lists ⁇ w_list1, w_list2, w_list3 ⁇ as candidate weight lists, and then a mathematical symbol function is called to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value.
- the decoding device determines w_list1 as the target weight list; when the symbol value is 0, the decoding device determines w_list2 as the target weight list; when the symbol value is 1, the decoding device determines w_list3 as the target weight list.
- reference prediction value i is derived from reference block i, and the video frame where reference block i is located is reference frame i; i is an integer and i is less than or equal to N; the video frame where the current block is located is the current frame; the importance metric value of reference prediction value i can be determined by any of the following methods:
- Method 1 Calculated based on the frame display order (Picture Order Count, POC) of the current frame in the video bitstream and the frame display order of the reference frame i in the video bitstream.
- Method 2 Calculated according to the frame display order of the current frame in the video code stream, the frame display order of the reference frame i in the video code stream, and the quality metric Q, wherein the quality metric Q can be determined according to various situations, and this application is not limited to this.
- the quality metric Q of the reference frame i may be derived according to the quantization information of the current block, for example, the quality metric Q may be set to the base_qindex (basic quantization index) of the reference frame i, and the base_qindex of any reference frame may be different or the same.
- the quality metric Q of the reference frame i may also be derived according to other coding information, for example, the quality metric Q of the reference frame i may be derived according to the coding information difference between the coded CU in the reference frame i and the coded CU in the current frame.
- the decoding device can calculate the difference between the frame display order of the current frame in the video code stream and the frame display order of the reference frame i in the video code stream; then, using the objective function, determine the importance metric value of the reference prediction value i based on the above difference, the quality metric Q and the importance metric value list.
- the above importance metric list contains the corresponding relationship between f (x) and the reference importance metric, as shown in Table 1:
- the importance metric value of the reference frame i can be calculated based on the orientation relationship between the reference frame i and the current frame and the quality metric Q.
- the decoding device can establish a corresponding relationship between the reference orientation relationship and the reference importance metric value. For example, if the reference orientation relationship is that the reference frame is before the current frame, the reference importance metric value can correspond to the first value; if the reference orientation relationship is that the reference frame is after the current frame, the reference importance metric value can correspond to the second value. Then, the decoding device can calculate the importance metric value of the reference prediction value i based on the reference importance metric value and the quality metric Q corresponding to the reference frame i.
- Method three Calculate the importance metric score of reference frame i according to the calculation results of method one and method two, and sort the importance metric scores of the reference frames corresponding to N reference prediction values in ascending order, and determine the index of reference frame i in the sorting as the importance metric value of reference prediction value i.
- the first importance metric value of the reference prediction value i can be calculated by the above method one, and the second importance metric value of the reference prediction value i can be calculated by the above method two, and then the importance metric score (such as score) of the reference frame i is calculated according to the first importance metric value and the second importance metric value, and then the importance metric scores of the reference frames corresponding to the N reference prediction values are sorted in order from small to large, and the index of the reference frame i in the sorting is determined as the importance metric value of the reference prediction value i.
- the importance metric score such as score
- the importance metric score of reference frame one corresponding to reference prediction value 1 is 20, the importance metric score of reference frame two corresponding to reference prediction value 2 is 30, and the importance metric score of reference frame three corresponding to reference prediction value 3 is 40; then, the importance metric scores of the reference frames corresponding to the three reference prediction values are sorted in ascending order, and the sorting result is: reference frame one, reference frame two, reference frame three; among them, the index of reference frame one in the sorting is 1, then the importance metric value of reference prediction value 1 is 1; the index of reference frame two in the sorting is 2, then the importance metric value of reference prediction value 2 is 2; the index of reference frame three in the sorting is 3, then the importance metric value of reference prediction value 3 is 3.
- the important metric score of the reference frame i can be calculated according to the first importance metric value and the second importance metric value as follows: 1 weighted sum of the first importance metric value and the second importance metric value to obtain the importance metric value of the reference prediction value i. 2 average the first importance metric value and the second importance metric value to obtain the importance metric value of the reference prediction value i.
- Method 4 In order to obtain a more accurate importance metric, the decoding device may adjust the calculation results of method 1, method 2 or method 3 based on the prediction mode of the reference prediction value i to obtain the importance metric of the reference prediction value i; wherein the prediction mode of the reference prediction value i includes any one of the following: inter-frame prediction mode, intra-frame prediction mode.
- the calculation results of method 1, method 2 or method 3 may be adjusted according to an adjustment function to obtain the importance metric of the reference prediction value i.
- the importance metric value; D is the calculation result of the above method 1, method 2 or method 3, and a and b can be adaptively set according to the prediction mode.
- any one of the above methods 1 to 4 can be used to determine the importance metric of the reference prediction value according to the needs, and this application does not limit this.
- the decoding device can determine the importance metric according to various methods, which can ensure the reliability of the importance metric, thereby facilitating the improvement of the prediction accuracy of the current block, thereby improving the encoding and decoding performance.
- the decoding device selects a target weight group from the target weight list, which can be divided into the following two cases:
- the target weight list is represented as ⁇ 2,14 ⁇ , ⁇ 4,12 ⁇ , ⁇ 6,10 ⁇ , ⁇ 8,8 ⁇ /16, and 4 weight groups are included in the target weight list.
- the index of the target weight group for weighted prediction of the current block can be indicated in the video bitstream, and the index of the target weight group is encoded using a truncated unary code for binary encoding, or a multi-symbol entropy encoding method.
- the truncated unary code is when the maximum value Max of the syntax element to be encoded is known.
- the decoding device needs to decode the index of the target weight group used for weighted prediction from the video bitstream, and select the target weight group from the target weight list according to the index of the target weight group.
- the index of the target weight group can indicate the position in the target weight list; for example, the target weight list in the above example contains 4 weight groups; the decoding device decodes the target weight group from the video bitstream and the index is 2, then the position in the target weight list is determined according to the index of the target weight group, that is, it is in the second position in the target weight list (that is, weight group 2), then the determined target weight group is ⁇ 4,12 ⁇ /16.
- the decoding device can determine the target weight group according to the index decoded from the video bitstream, and can improve the processing efficiency of the target weight group selection for determining the target weight group, thereby improving the decoding efficiency.
- the decoding device determines a target weight list based on the importance of N reference prediction values, and selects a target weight group for weighted prediction from the target weight list. By screening the weight list, a suitable weight group can be selected for decoding and reconstruction of the current block, which can improve the prediction accuracy of the current block and thus improve the encoding and decoding performance.
- the number of weight values actually provided by the target weight group should correspond to the number of reference prediction values.
- the number of reference prediction values is N, and correspondingly, the number of weight values actually provided by the target weight group is also N.
- N 3, and the three reference prediction values are reference prediction value 1, reference prediction value 2, and reference prediction value 3.
- the target weight group may contain three weight values, namely, weight value 1, weight value 2, and weight value 3; wherein, reference prediction value 1 corresponds to weight value 1, reference prediction value 2 corresponds to weight value 2, and reference prediction value 3 corresponds to weight value 3.
- weighted prediction processing is performed on N reference prediction values based on the target weights to obtain the prediction value of the current block in the following two ways (1) and (2):
- the decoding device may use the weight values in the target weight group to perform weighted summation processing on the N reference prediction values to obtain the prediction value of the current block.
- P(x,y) is the prediction value of the current block
- P0 (x,y), P1 (x,y)... PN-1 (x,y) represent N reference prediction values respectively
- w1 represents the weight value corresponding to the first reference prediction value corresponding to the current block (x,y)
- w2 represents the weight value corresponding to the second reference prediction value corresponding to the current block (x,y)
- wn represents the Nth reference prediction value corresponding to the current block (x,y).
- the current block uses two reference prediction values for weighted prediction.
- the number of weight values included in the target weight group is one, that is, it includes an implicit weight value.
- P(x,y) is the prediction value of the current block
- P0 (x,y) and P1 (x,y) are the two reference prediction values corresponding to the current block (x,y)
- w(x,y) is the weight value in the target weight group applied to the first reference prediction value P0 (x,y)
- 1-w(x,y) is the implicit weight value and is applied to the second reference prediction value P1 (x,y).
- the decoding device can use the weight values in the target weight group to perform weighted processing on the two reference prediction values in the form of integer calculation to obtain the prediction value of the current block.
- “>>4” means right shift by 4 bits, that is, by right shifting by 4 bits, the data in the weighted processing process can be divided by 16, and 8 is the bias added for rounding.
- P 0 (x, y), P 1 (x, y), w(x, y), and P(x, y) are all integers;
- P(x, y) is the prediction value of the current block,
- P 0 (x, y) and P 1 (x, y) are the two reference prediction values corresponding to the current block (x, y), and w(x, y) is the weight value applied to the first prediction value P 0 (x, y) (that is, the weight value in the target weight group).
- “>>6” means a 6-bit right shift, that is, a 6-bit right shift can make the data in the weighted processing process divisible by 64, and 32 is the bias added for rounding.
- the decoding device performs weighted summation processing on the N reference prediction values respectively through the weight values in the target weight group, or performs weighted summation processing in the form of integer calculation to obtain the prediction value of the current block, which can ensure the accuracy of the prediction value and thus help improve the decoding performance for the current block.
- a superposition process may be performed based on the residual video signal of the current block and the prediction value to obtain a reconstruction value of the current block, and a decoded image corresponding to the current block may be reconstructed based on the reconstruction value of the current block.
- the decoded image corresponding to the current block may be used as a reference image for weighted prediction of other coding blocks, and on the other hand, the decoded image corresponding to the current block may also be used to reconstruct the current frame of the current block, and finally the video may be reconstructed based on the reconstructed multiple video frames.
- a decoding device performs composite prediction on a current block in a video bitstream to obtain N reference prediction values of the current block, where the current block refers to a coding block being decoded in the video bitstream, and N is an integer greater than 1; the decoding device determines a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values; weighted prediction processing is performed on the N reference prediction values based on the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block.
- Composite prediction is used in the video decoding process, and the importance of the reference prediction value is fully considered in the composite prediction, so that in the composite prediction, the importance of each reference prediction value is adaptively selected for the current block for weighted prediction, which can improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.
- FIG 4 is a flow chart of another video processing method provided in an embodiment of the present application.
- the video processing method can be executed by an encoding device in a video processing system.
- the video processing method described in this embodiment may include the following steps S401-S405:
- the current block refers to a coding block in the video that is being encoded by the coding device.
- the video may include one or more video frames, and the current frame refers to the video frame being encoded.
- the coding device may divide the current frame in the video to obtain one or more coding blocks, and the current block refers to any one of the one or more coding blocks in the current frame that is being encoded.
- N is an integer greater than 1
- the N reference prediction values are derived from N reference blocks of the current block
- the N reference blocks are coding blocks referenced when encoding the current block in the video; there is a one-to-one correspondence between the reference prediction values and the reference blocks.
- the above-mentioned N reference prediction values may be derived from N reference blocks, and one reference prediction value corresponds to one reference block.
- the video frame where the reference block is located may be a reference frame
- the video frame where the current block is located is a current frame.
- the positional relationship between the N reference blocks and the current block includes any of the following: 1
- the N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame may belong to different video frames in the video.
- the N reference blocks are located in the same reference frame, and the same reference frame and the current frame may belong to different video frames in the video.
- One or more of the N reference blocks are located in the current frame, and the remaining reference blocks of the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video. 4 Both the N reference blocks and the current block are located in the current frame.
- the prediction mode of the composite prediction in the embodiment of the present application includes both an inter-frame prediction mode, that is, allowing the use of at least two reference frames for inter-frame prediction; and a combined prediction mode, that is, allowing the use of at least one reference frame for inter-frame prediction, while allowing the use of the current frame for intra-frame prediction; and also includes an intra-frame prediction mode, that is, allowing the use of the current frame for intra-frame prediction.
- the encoding device performs compound prediction on the current block
- the N reference prediction values of the current block obtained may include any of the following: 1
- the current block uses N reference blocks for inter-frame prediction to obtain N reference prediction values of the current block.
- the N reference prediction values of the current block are derived after using the N reference blocks of the current block to perform inter-frame prediction respectively.
- the current block uses at least one reference prediction value among the N reference prediction values for inter-frame prediction and uses the remaining reference blocks among the N reference blocks for intra-frame prediction, thereby obtaining N reference prediction values.
- some of the N reference prediction values are derived after using at least one reference block among the N reference blocks of the current block for inter-frame prediction; the remaining reference prediction values are derived after using the remaining reference blocks among the N reference blocks for intra-frame prediction.
- the importance of the reference prediction value can be comprehensively determined based on factors such as the bit rate consumed during weighted prediction processing and the quality loss of the current block during the encoding process. For example, when the current block uses a certain reference prediction value for weighted prediction, the bit rate consumption increases significantly, indicating that the reference prediction value is not effective enough in reducing bit rate consumption and its importance is low. Another example is that when the current block uses a certain reference prediction value for weighted prediction, the quality loss increases significantly, indicating that the reference prediction value is less effective in reducing quality loss and its importance is low.
- the target weight group contains one or more weight values, which will act on each reference prediction value during the weighted prediction process.
- the target weight group is selected based on the influence of each reference prediction value on factors such as bit rate consumption and quality loss, so that the comprehensive cost of the weighted prediction process (i.e., the cost of bit rate consumption, the cost of quality loss, the cost of bit rate consumption and quality loss) is smaller and the encoding and decoding performance is better.
- the bit rate consumed when the encoding device uses the weight value in the target weight group to perform weighted prediction processing on N reference prediction values is less than the preset bit rate threshold; or, based on the weight value in the target weight group, the N reference prediction values are weighted predicted, so that the quality loss of the current block in the encoding process is less than the preset loss threshold; or, the bit rate consumed when the weight value in the target weight group is used to perform weighted prediction processing on N reference prediction values is less than the preset bit rate threshold, and based on the weight value in the target weight group, the N reference prediction values are weighted predicted, so that the quality loss of the current block in the encoding process is less than the preset loss threshold.
- the preset bit rate threshold and the preset loss threshold can be pre-set according to actual needs, such as the corresponding bit rate threshold and loss threshold can be statistically analyzed according to historical encoding and decoding records, so that the comprehensive cost of the weighted prediction process is relatively small.
- the encoding device can constrain the selection of the target weight group according to at least one of the preset bit rate threshold and the preset loss threshold, and can select a suitable target weight group according to actual needs, which is conducive to reducing the encoding cost and improving the encoding and decoding performance.
- each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values contained in each weight group is allowed to be the same or different, and the weight values contained in each weight group are allowed to be the same or different.
- the target weight group of weighted prediction may include steps s41-s42:
- the specific implementation method of determining the target weight list from one or more weight lists may include the following methods:
- Method 1 When there is a weight list during the encoding process, the weight list can be directly used as the target weight list.
- Method 2 When there are multiple weight lists during the encoding process, an importance metric of the reference prediction value may be introduced, and a target weight list may be determined from the multiple weight lists according to the importance metric values of the N reference prediction values.
- the encoding device can obtain the importance metric values of N reference prediction values, and calculate the importance difference between the N reference prediction values.
- the importance difference between any two reference prediction values is measured by the difference between the importance metric values of any two reference prediction values; specifically, the difference between the importance metrics of any two reference prediction values can be calculated, and the difference between the importance metric values of any two reference prediction values is used as the importance difference between any two reference prediction values; then, the threshold interval where the absolute value of the importance difference between the N reference prediction values is located is determined; and the weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values is located is determined as the target weight list.
- the N reference prediction values of the current block may include a first reference prediction value and a second reference prediction value
- the weight list existing in the encoding process includes a first weight list and a second weight list.
- the encoding device may compare the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value; if it is determined that the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, the first weight list is determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the second weight list is determined as the target weight list.
- the weight values in the first weight list are opposite to the weight values in the second weight list.
- the N reference prediction values of the current block include a first reference prediction value and a second reference prediction value; the weight lists existing in the encoding process include a first weight list, a second weight list and a third weight list.
- the encoding device can call a mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value. If the above symbol value is a first preset value (such as -1), the first weight list is determined as the target weight list; if the above symbol value is a second preset value (such as 0), the second weight list is determined as the target weight list; if the above symbol value is a third preset value (such as 1), the third weight list is determined as the target weight list.
- Method (1) can be used alone, or they can be combined to determine the target weight list.
- Method (1) can be used to determine the weight list corresponding to the threshold interval where the absolute value of the importance difference is located as the candidate weight list.
- the N reference prediction values include the first reference prediction value and the second prediction value.
- Method (2) is used to compare the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value.
- the candidate weight list is directly determined as the target weight list; if it is determined that the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, the weight list corresponding to the weight value opposite to the weight value in the candidate weight list is determined as the target weight list.
- any one of the N reference prediction values is represented as reference prediction value i
- the reference prediction value i is derived from the reference block i
- the video frame where the reference block i is located is the reference frame i
- i is an integer and i is less than or equal to N
- the video frame where the current block is located is the current frame
- the importance metric value of the reference prediction value i can be determined by any of the following methods:
- Method 1 Calculated based on the frame display order (POC) of the current frame in the video and the frame display order of the reference frame i in the video.
- the encoding device can calculate the difference between the frame display order of the current frame in the video and the frame display order of the reference frame i in the video, and use the absolute value of the difference as the importance measure of the reference prediction value i.
- the frame display order of the current frame in the video be cur_poc
- the frame display order of the reference frame i in the video be ref_poc
- the reference prediction value i is the value of the importance measure of the reference prediction value i.
- Method 2 Calculated according to the frame display order of the current frame in the video, the frame display order of the reference frame i in the video and the quality metric Q.
- the quality metric Q can be determined according to a variety of situations, and the present application is not limited to this.
- the quality metric Q of the reference frame i can be derived according to the quantization information of the current block.
- the quality metric Q can be set to the base_qindex (basic quantization index) of the reference frame i.
- the base_qindex of any reference frame can be different or the same.
- the quality metric Q of the reference frame i can also be derived according to other coding information.
- the quality metric Q of the reference frame i can be derived based on the difference in coding information between the encoded CU in the reference frame i and the encoded CU in the current frame.
- the encoding device may calculate the difference between the frame display order of the current frame in the video and the frame display order of the reference frame i in the video; then, using the objective function, determine the importance metric value of the reference prediction value i based on the above difference, the quality metric Q, and the importance metric value list.
- the importance metric value of the reference frame i can be calculated based on the orientation relationship between the reference frame i and the current frame and the quality metric Q.
- a correspondence between the reference orientation relationship and the reference importance metric value can be established. For example, if the reference orientation relationship is that the reference frame is before the current frame, the reference importance metric value can correspond to the first value; if the reference orientation relationship is that the reference frame is after the current frame, the reference importance metric value can correspond to the second value. Then, the encoding device can calculate the importance metric value of the reference prediction value i based on the reference importance metric value and the quality metric Q corresponding to the reference frame i.
- Method three Calculate the importance metric score of reference frame i according to the calculation results of method one and method two, and sort the importance metric scores of the reference frames corresponding to N reference prediction values in ascending order, and determine the index of reference frame i in the sorting as the importance metric value of reference prediction value i.
- the first importance metric value of the reference prediction value i can be calculated by the above method one, and the second importance metric value of the reference prediction value i can be calculated by the above method two, and then the importance metric score (such as score) of the reference frame i is calculated according to the first importance metric value and the second importance metric value, and then the importance metric scores of the reference frames corresponding to the N reference prediction values are sorted in order from small to large, and the index of the reference frame i in the sorting is determined as the importance metric value of the reference prediction value i.
- the importance metric score such as score
- the important metric score of the reference frame i can be calculated according to the first importance metric value and the second importance metric value as follows: 1 weighted sum of the first importance metric value and the second importance metric value to obtain the importance metric value of the reference prediction value i. 2 average the first importance metric value and the second importance metric value to obtain the importance metric value of the reference prediction value i.
- Method 4 In order to obtain a more accurate importance measurement value, the calculation results of method 1, method 2 or method 3 can be adjusted based on the prediction mode of reference prediction value i to obtain the importance measurement value of reference prediction value i; wherein the prediction mode of reference prediction value i includes any one of the following: inter-frame prediction mode, intra-frame prediction mode.
- any one of the above methods 1 to 4 may be used to determine the importance metric of the reference prediction value according to needs, and this application does not limit this.
- the target weight list includes one or more weight groups.
- the target weight list includes multiple weight groups, and a target weight group for weighted prediction is selected from the target weight list.
- the encoding device may first obtain the encoding performance when weighted prediction processing is performed on N reference prediction values using each weight group in the target weight list, and then determine the weight group with the best encoding performance in the target weight list as the target weight group.
- the bit rate consumed when weighted prediction processing is performed on N reference prediction values using the weight values in the target weight group is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block during the encoding process when weighted prediction processing is performed on N reference prediction values based on the weight values in the target weight group is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list; or,
- the bit rate consumed when the weight values in the target weight group perform weighted prediction processing on N reference prediction values is the minimum value of the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the current block corresponding to the weighted prediction processing on N reference prediction values based on the weight values in the target weight group during the encoding process is the minimum value of the quality loss corresponding to the ownership reorganization in the target weight list.
- the encoding device can determine the weight group with the best encoding performance as the target weight group, and
- the encoding device determines the target weight list based on the importance of the N reference prediction values, and selects the target weight group for weighted prediction from the target weight list. By screening the weight list, a suitable weight group can be selected for reconstruction of the current block, which can improve the prediction accuracy of the current block, thereby improving the encoding performance.
- step S404 may refer to the specific implementation of step S303 above.
- the encoding device may encode the video based on the prediction value of the current block and the index of the target weight group to generate a video bitstream, wherein encoding the video based on the prediction value of the current block and the index of the target weight group to generate a video bitstream can refer to the above corresponding encoding description.
- the encoding device divides the current frame in the video to obtain the current block, which refers to the encoding block being encoded in the video; performs composite prediction on the current block to obtain N reference prediction values of the current block; determines a target weight group for weighted prediction for the current block based on the importance of the N reference prediction values; performs weighted prediction processing on the N reference prediction values based on the target weight group to obtain the prediction value of the current block; and encodes the video based on the prediction value of the current block to generate a video bitstream.
- Composite prediction is used in the video encoding process, and the importance of the reference prediction value derived in the composite prediction is realized in the composite prediction to realize the adaptive selection of appropriate weights for the current block based on the importance of each reference prediction value in the composite prediction for weighted prediction, which can improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.
- FIG. 5 is a schematic diagram of the structure of a video processing device provided in an embodiment of the present application.
- the video processing device can be set in the computer device provided in the embodiment of the present application, and the computer device can be the decoding device mentioned in the above method embodiment.
- the video processing device shown in Figure 5 can be a computer-readable instruction (including program code) running in the computer device, and the video processing device can be used to execute some or all of the steps in the method embodiment shown in Figure 3.
- the video processing device may include the following units:
- the processing unit 501 is used to perform composite prediction on a current block in a video bitstream to obtain N reference prediction values of the current block, where N is an integer greater than 1; the current block is a coding block to be decoded in the video bitstream; the N reference prediction values are derived from N reference blocks of the current block; the N reference blocks are coding blocks referenced when decoding the current block in the video bitstream; the reference prediction values correspond to the reference blocks in a one-to-one correspondence;
- a determination unit 502 is used to determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values, where the target weight group includes one or more weight values; the importance is used to characterize the influence of the corresponding reference prediction value on the decoding performance of the current block; and
- the processing unit 501 is further used to perform weighted prediction processing on the N reference prediction values based on the weight values in the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block.
- the video frame where the reference block is located is the reference frame
- the video frame where the current block is located is the current frame
- the positional relationship between the N reference blocks and the current block includes any of the following: the N reference blocks are respectively located in N reference frames, and the N reference frames and the current frame belong to different video frames in the video code stream; the N reference blocks are located in the same reference frame, and the same reference frame and the current frame belong to different video frames in the video code stream; one or more reference blocks among the N reference blocks are located in the current frame, and the remaining reference blocks among the N reference blocks are located in one or more reference frames, and one or more reference frames and the current frame belong to different video frames in the video code stream; and the N reference blocks and the current block are both located in the current frame.
- the processing unit 501 is further used to: determine the conditions for adaptive weighted prediction; and if the current block satisfies The condition of adaptive weighted prediction is to determine the target weight group for weighted prediction for the current block according to the importance of the N reference prediction values.
- the conditions that the current block satisfies adaptive weighted prediction include at least one of the following: the sequence header of the frame sequence to which the current block belongs contains a first indication field, and the first indication field indicates that the coding block in the frame sequence allows the use of adaptive weighted prediction; the frame sequence is a sequence composed of video frames in a video code stream; the slice header of the current slice to which the current block belongs contains a second indication field, and the second indication field indicates that the coding block in the current slice allows the use of adaptive weighted prediction; the current slice is an image slice to which the current block belongs, and the image slice is obtained by dividing the current frame where the current block is located; the frame header of the current frame where the current block is located contains a third indication field, and the third indication field indicates that the coding block in the current frame allows the use of adaptive weighted prediction; in the composite prediction process, the current block uses at least two reference frames for inter-frame prediction; in the composite prediction process, when The previous block uses at least one reference frame for inter-frame
- the video bitstream includes one or more weight lists, each weight list includes one or more weight groups, each weight group includes one or more weight values, the number of weight values in each weight group is allowed to be the same or different, and the value of the weight value included in each weight group is allowed to be the same or different;
- the determination unit 502 can be specifically used to:
- the number of weight lists in the video bitstream is M+1, one weight list corresponds to one threshold interval, and M is a positive integer greater than or equal to 1;
- the determination unit 502 can be specifically used to: obtain importance measurement values of N reference prediction values, and calculate the importance difference between the N reference prediction values, the importance difference between any two reference prediction values is measured by the difference between the importance measurement values of any two reference prediction values; determine the threshold interval where the absolute value of the importance difference between the N reference prediction values lies; and determine the weight list corresponding to the threshold interval where the absolute value of the importance difference between the N reference prediction values lies as the target weight list.
- the N reference prediction values of the current block include a first reference prediction value and a second reference prediction value;
- the video code stream includes a first weight list and a second weight list;
- the determination unit 502 can be specifically used to: compare the importance metric value of the first reference prediction value with the importance metric value of the second reference prediction value; if the importance metric value of the first reference prediction value is greater than the importance metric value of the second reference prediction value, determine the first weight list as the target weight list; and if the importance metric value of the first reference prediction value is less than or equal to the importance metric value of the second reference prediction value, determine the second weight list as the target weight list; wherein the sum of the weight values in the first weight list and in the second weight list in the same order is 1.
- the N reference prediction values of the current block include a first reference prediction value and a second reference prediction value;
- the video code stream includes a first weight list, a second weight list and a third weight list;
- the determination unit 502 can be specifically used to: call a mathematical symbol function to process the difference between the importance metric value of the first reference prediction value and the importance metric value of the second reference prediction value to obtain a symbol value; if the symbol value is a first preset value, the first weight list is determined as the target weight list; if the symbol value is a second preset value, the second weight list is determined as the target weight list; and if the symbol value is a third preset value, the third weight list is determined as the target weight list; wherein the first weight list, the second weight list and the third weight list are different weight lists respectively; or, two weight lists in the first weight list, the second weight list and the third weight list are allowed to be the same weight list.
- any one of the N reference prediction values is represented as reference prediction value i, and the reference prediction value i is derived from the reference block i, and the video frame where the reference block i is located is the reference frame i; i is an integer and i is less than or equal to N; the video frame where the current block is located is the current frame; the importance metric value of the reference prediction value i is determined by any of the following methods: Method 1: According to The frame display order of the current frame in the video code stream and the frame display order of the reference frame i in the video code stream are calculated; Method 2: The frame display order of the reference frame i in the video code stream and the quality metric Q are calculated, and the frame display order of the current frame in the video code stream is calculated; Method 3: The important metric score of the reference frame i is calculated according to the calculation results of Method 1 and the calculation results of Method 2, and the important metric scores of the reference frames corresponding to the N reference prediction values are sorted in order from small to large, and the index of the reference frame
- the number of weight groups included in the target weight list is greater than 1; the determination unit 502 can be specifically used to: decode the index of the target weight group used for weighted prediction from the video bit stream; and select the target weight group from the target weight list according to the index of the target weight group; wherein the index of the target weight group is encoded using a truncated unary code for binary encoding, or using a multi-symbol entropy encoding method.
- the processing unit 501 can be specifically used to: use the weight values in the target weight group to perform weighted summation processing on N reference prediction values respectively to obtain the prediction value of the current block; or, use the weight values in the target weight group to perform weighted summation processing on N reference prediction values respectively in the form of integer calculation to obtain the prediction value of the current block.
- a composite prediction is performed on the current block in the video bitstream to obtain N reference prediction values of the current block.
- the current block refers to the coding block being decoded in the video bitstream.
- a target weight group for weighted prediction is adaptively selected for the current block; based on the target weight group, the N reference prediction values are weighted predicted to obtain the prediction value of the current block, and the prediction value of the current block is used to reconstruct the decoded image corresponding to the current block.
- Composite prediction is used in the video decoding process, and the importance of the reference prediction value is fully considered in the composite prediction, so that the appropriate weight is determined for the current block based on the importance of each reference prediction value in the composite prediction to perform weighted prediction, which can improve the prediction accuracy of the current block, thereby improving the encoding and decoding performance.
- FIG. 6 is a schematic diagram of the structure of a video processing device provided in an embodiment of the present application.
- the video processing device can be set in the computer device provided in the embodiment of the present application, and the computer device can be the encoding device mentioned in the above method embodiment.
- the video processing device shown in Figure 6 can be a computer-readable instruction (including program code) running in the computer device, and the video processing device can be used to execute some or all of the steps in the method embodiment shown in Figure 4.
- the video processing device may include the following units:
- the processing unit 601 is used to divide the current frame in the video to obtain a current block
- the processing unit 601 is further configured to perform composite prediction on the current block to obtain N reference prediction values of the current block, where N is an integer greater than 1; the N reference prediction values are derived from the N reference blocks of the current block; the N reference blocks are coding blocks referenced when encoding the current block in the video; and the reference prediction values correspond to the reference blocks in a one-to-one manner;
- a determination unit 602 is used to determine a target weight group for weighted prediction for the current block according to the importance of the N reference prediction values, where the target weight group includes one or more weight values; the importance is used to characterize the influence of the corresponding reference prediction value on the encoding performance of the current block;
- the processing unit 601 is further used to perform weighted prediction processing on the N reference prediction values based on the weight values in the target weight group to obtain a prediction value of the current block, and the prediction value of the current block is used to reconstruct a decoded image corresponding to the current block;
- the processing unit 601 is further configured to encode the video based on the prediction value of the current block to generate a video bit stream.
- the determination unit 602 can be specifically used to: determine a target weight list from one or more weight lists based on the importance of N reference prediction values; each weight list contains one or more weight groups, each weight group contains one or more weight values, the number of weight values contained in each weight group is allowed to be the same or different, and the values of the weight values contained in each weight group are allowed to be the same or different; and select a target weight group for weighted prediction from the target weight list.
- the bit rate consumed when weighted prediction processing is performed on N reference prediction values using the weight values in the target weight group is less than a preset bit rate threshold; or, weighted prediction processing is performed on the N reference prediction values based on the weight values in the target weight group, so that the quality loss of the current block during the encoding process is less than a preset loss threshold; or, the bit rate consumed when weighted prediction processing is performed on the N reference prediction values using the weight values in the target weight group is less than a preset bit rate threshold, and weighted prediction processing is performed on the N reference prediction values based on the weight values in the target weight group, so that the quality loss of the current block during the encoding process is less than a preset loss threshold.
- the quality loss is less than the preset loss threshold.
- the target weight group is a weight group with optimal coding performance in the target weight list; wherein, the optimal coding performance includes: the bit rate consumed when weighted prediction processing is performed on N reference prediction values using the weight values in the target weight group is the minimum value among the consumed bit rates corresponding to the ownership reorganization in the target weight list; or, the quality loss of the corresponding current block in the encoding process when weighted prediction processing is performed on N reference prediction values based on the weight values in the target weight group is the minimum value among the quality losses corresponding to the ownership reorganization in the target weight list; or, the bit rate consumed when weighted prediction processing is performed on N reference prediction values using the weight values in the target weight group is the minimum value among the consumed bit rates corresponding to the ownership reorganization in the target weight list, and the quality loss of the corresponding current block in the encoding process when weighted prediction processing is performed on N reference prediction values based on the weight values in the target weight group is the minimum value among the quality losses corresponding to the ownership reorganization in
- the current frame in the video is divided and processed to obtain the current block, which refers to the coding block being encoded in the video; the current block is compound predicted to obtain N reference prediction values of the current block; according to the importance of the N reference prediction values, a target weight group for weighted prediction is determined for the current block; based on the target weight group, the N reference prediction values are weighted predicted to obtain the prediction value of the current block; based on the prediction value of the current block, the video is encoded to generate a video bitstream.
- the embodiment of the present application also provides a structural schematic diagram of a computer device, and the structural schematic diagram of the computer device can be seen in Figure 7;
- the computer device can be the above-mentioned encoding device or decoding device;
- the computer device may include: a processor 701, an input device 702, an output device 703 and a memory 704.
- the above-mentioned processor 701, input device 702, output device 703 and memory 704 are connected via a bus.
- the memory 704 is used to store computer-readable instructions, and the computer-readable instructions include program instructions.
- the processor 701 is used to execute the program instructions stored in the memory 704.
- the processor 701 executes the steps of the video processing method related to decoding as performed by the above-mentioned decoding device by running the executable program code in the memory 704.
- the processor 701 executes each step of the video processing method related to encoding performed by the above-mentioned encoding device by running the executable program code in the memory 704.
- the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions include program instructions.
- the processor executes the above program instructions, it can execute the method in the embodiment corresponding to Figures 3 and 4 above. Therefore, it will not be repeated here.
- the program instructions can be deployed on a computer device, or executed on multiple computer devices located at one location, or, executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.
- a computer program product which includes computer-readable instructions, and the computer-readable instructions are stored in a computer-readable storage medium.
- the processor of the computer device reads the computer-readable instructions from the computer-readable storage medium, and the processor executes the computer-readable instructions, so that the computer device can execute the method in the embodiment corresponding to Figures 3 and 4 above, so it will not be repeated here.
- a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium.
- the program When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.
- the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
一种视频处理方法,由计算机设备执行,包括:对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值;当前块是视频码流中待进行解码处理的编码块;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频码流中针对当前块进行解码时所参考的编码块;参考预测值与参考块之间一一对应(S301);根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的解码性能的影响程度(S302);及基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像(S303)。
Description
本申请要求于2022年09月30日提交中国专利局、申请号为2022112305931、发明名称为“一种视频处理方法及相关设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及音视频技术领域,具体涉及视频编码技术领域,尤其涉及一种视频处理方法、一种视频处理装置、一种计算机设备、一种计算机可读存储介质及一种计算机程序产品。
在现有的视频编码技术中,采用基于块的混合编码框架,可将原始的视频数据分成一系列的编码块(Coding Unit,CU),结合预测、变换和熵编码等视频编码方法,实现视频数据的压缩。为了取得更好的预测效果,目前主流的视频编码标准如AV1(由开放媒体联盟制定的第一代视频编码标准,Alliance for Open Media Video 1)标准和正在研发的AOM(开放媒体联盟,Alliance for Open Media)下一代标准AV2(由开放媒体联盟制定的第二代视频编码标准,Alliance for Open Media Video 2)中包含一种称为复合预测(compound prediction)的预测模式,该复合预测的预测模式允许使用多种参考视频信号来进行加权预测;但实践发现,目前的复合预测在进行加权预测时,预测准确性不高,影响了视频的编解码性能。
发明内容
根据本申请提供的各种实施例,提供了一种视频处理方法及相关设备。
一方面,本申请实施例提供了一种视频处理方法,由计算机设备执行,该方法包括:
对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;当前块是视频码流中待进行解码处理的编码块;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频码流中针对当前块进行解码时所参考的编码块;参考预测值与参考块之间一一对应;根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的解码性能的影响程度;及基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像。
一方面,本申请实施例提供了一种视频处理方法,由计算机设备执行,该方法包括:
对视频中的当前帧进行划分处理,得到当前块;对当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频中针对当前块进行编码时所参考的编码块;参考预测值与参考块之间一一对应;根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的编码性能的影响程度;基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建该当前块对应的解码图像;及基于当前块的预测值对视频进行编码生成视频码流。
一方面,本申请实施例提供了一种视频处理装置,该装置包括:
处理单元,用于对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;当前块是视频码流中待进行解码处理的编码块;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频码流中针对当前块进行解码时所参考的编码块;参考预测值与参考块之间一一对应;确定单元,用于根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的解码性能的影响程度;及处理单元,还用于基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像。
一方面,本申请实施例提供了一种视频处理装置,该装置包括:
处理单元,用于对视频中的当前帧进行划分处理,得到当前块;处理单元,还用于对当前块进行复合预测,得到当前块的N个参考预测值;N为大于1的整数;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频中针对当前块进行编码时所参考的编码块;参考预测值与参考块之间一一对应;确定单元,用于根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的编码性能的影响程度;处理单元,还用于基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建所述当前块对应的解码图像;及处理单元,还用于基于当前块的预测值对视频进行编码生成视频码流。
一方面,本申请实施例提供一种计算机设备,该计算机设备包括:
处理器,适于执行计算机可读指令;计算机可读存储介质,计算机可读存储介质中存储有计算机可读指令,计算机可读指令被处理器执行时,实现如上述的视频处理方法。
一方面,本申请实施例提供一种计算机可读存储介质,该计算机可读存储介质存储有计算机可读指令,该计算机可读指令被处理器加载并执行如上述视频处理方法。
一方面,本申请实施例提供了一种计算机程序产品,该计算机程序产品包括计算机可读指令,该计算机可读指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机可读指令,处理器执行该计算机可读指令,使得该计算机设备执行上述视频处理方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1a是本申请一个示例性实施例提供的一种视频编码的基本工作流程图。
图1b是本申请一个示例性实施例提供的一种帧间预测的示意图。
图2是本申请一个示例性实施例提供的一种视频处理系统的结构示意图。
图3是本申请一个示例性实施例提供的一种视频处理方法的流程示意图。
图4是本申请另一个示例性实施例提供的一种视频处理方法的流程示意图。
图5是本申请一个示例性实施例提供的一种视频处理装置的结构示意图。
图6是本申请另一个示例性实施例提供的一种视频处理装置的结构示意图。
图7是本申请一个示例性实施例提供的一种计算机设备的结构示意图。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
下面对本申请中涉及的技术术语进行介绍:
一、视频编码
视频可由一个或多个视频帧构成,每个视频帧包含该视频的部分视频信号。视频信号的获取方式可以分为摄像机拍摄到或者计算机生成两种方式。由于不同获取方式对应的统计特性不同,视频的压缩编码方式也可能有所区别。
在主流视频编码技术中,以HEVC(High Efficiency Video Coding,国际视频编码标准HEVC/H.265)、VVC(Versatile Video Coding,国际视频编码标准VVC/H.266),AV1(Alliance for Open Media Video 1,由开放媒体联盟制定的第一代视频编码标准),AV2(Alliance for Open
Media Video 2,由开放媒体联盟制定的第二代视频编码标准)以及AVS3(Audio Video coding Standard 3,第三代音视频信源编码标准)为例,采用了混合编码框架,在该混合编码框架下允许对视频进行如下一系列的操作和处理:
1)块划分结构(block partition structure):根据输入的当前帧(即正在被编码或解码的视频帧)的大小,可将当前帧划分成若干个不重叠的处理单元,每个处理单元将进行类似的压缩操作。这个处理单元被称作CTU(coding tree unit,编码树单元),或者LCU(Largest Coding Unit,最大编码单元)。CTU可以再继续往下进行更加精细的划分,得到一个或多个基本编码的单元,称之为CU(coding unit,编码单元或编码块)。每个CU是一个编码环节中最基本的元素,本申请后续各实施例中描述的是针对每一个CU可能采用的各种编解码处理流程。
2)预测编码:包括帧内预测和帧间预测等模式,当前帧中的当前CU(即当前帧中正在被编码或解码的CU)包含的原始视频信号经过选定的参考CU中已重建视频信号的预测之后,得到残差视频信号。此处,当前CU又可称为当前块,当前块所在的视频帧称为当前帧,当前块预测时所使用的参考CU又可称为当前块的参考块,参考块所在的视频帧称为参考帧。其中,编码端需要为当前CU决定在众多可能的预测编码模式中选择最合适的一种预测编码模式并告诉解码端。其中,预测编码模式可包括:
a、帧内预测(Intra(picture)Prediction):预测所使用的重建视频信号来自于同一视频帧内已经编码重建过的区域,即当前块与参考块位于同一视频帧中。其中,帧内预测的基本思想就是利用同一视频帧内相邻像素的相关性去除空间冗余。在视频编码中,相邻像素指的就是同一视频帧内当前CU周围的已编码CU的重建(reconstructed)像素。
b、帧间预测(Inter(picture)Prediction):预测所使用的重建视频信号来自已经编码过的,不同于当前帧的其他视频帧,即当前块与参考块分别位于不同的视频帧中。
3)变换编码及量化(Transform&Quantization):残差视频信号经过DFT(Discrete Fourier Transform,离散傅里叶变换),DCT(Discrete Cosine Transform,离散余弦变换)等变换操作,可将残差视频信号转换到变换域中,称之为变换系数。在变换域中的残差视频信号,进一步的进行有损的量化操作,丢失掉一定的信息,使得量化后的信号有利于压缩表达。
在一些视频编码标准中,可能有多于一种变换方式可以选择,因此,编码端也需要为当前CU选择其中的一种变换,并告知解码端。量化的精细程度通常由QP(Quantization Parameters,量化参数)来决定,当QP取值较大时,表示更大取值范围的变换系数将被量化为同一个输出,因此通常会带来更大的失真及较低的码率;相反,当QP取值较小时,表示较小取值范围的变换系数将被量化为同一个输出,因此通常会带来较小的失真,同时对应较高的码率。
4)熵编码(Entropy Coding)或统计编码:量化后的变换域信号,将根据各个值出现的频率,进行统计压缩编码,最后输出二值化(0或者1)的视频码流。同时,编码会产生其他信息,例如选择的预测编码模式,运动矢量等,这些其他信息也需要进行熵编码以降低码率。其中,统计编码是一种无损编码方式,可以有效的降低表达同样的信号所需要的码率。常见的统计编码方式有变长编码(Variable Length Coding,VLC)或者基于上下文的二值化算术编码(Content Adaptive Binary Arithmetic Coding,CABAC)。
5)环路滤波(Loop Filtering):已经编码过的CU,经过反量化,反变换及预测补偿的操作(上述2)~4)的反向操作),可重建该CU对应的解码图像。重建的解码图像与原始图像相比,由于存在量化的影响,部分信息与原始图像有所不同,产生失真(Distortion),因此可利用滤波器对重建的解码图像进行滤波操作,有效的降低量化所产生的失真程度。滤波器例如可以是去块效应滤波(deblocking),SAO(样本自适应补偿,Sample Adaptive Offset)或者ALF(Adaptive Loop Filter,自适应环路滤波)等等,由于这些经过滤波后的重建的解码图像,将作为后续需要编码的其他CU的参考CU,应用于其他CU的预测过程中,所以上述的滤波操作也被称为环路滤波,及在编码环路内的滤波操作。
基于上述步骤1)-5)的相关描述,本申请实施例提供了一种视频编码器的基本工作流程
图。请参见图1a,图1a为示例性给出的一种视频编码器的基本工作流程图。其中,图1a以当前块为当前帧(当前图像)中第k个CU(如图1a所示的sk[x,y])为例进行举例说明,k为正整数,且k小于或等于当前帧包含的CU的总数量。sk[x,y]表示第k个CU中坐标为[x,y]的像素点(简称像素),x表示像素的横坐标,y表示像素的纵坐标;sk[x,y]经过运动补偿或者帧内预测等处理后可以获得预测信号,将预测信号和原始信号sk[x,y]进行相减,得到残差视频信号uk[x,y];然后对该残差视频信号uk[x,y]进行变换和量化处理。其中,量化处理输出的数据有A和B两个不同的去处:
A:量化处理输出的数据可以送给熵编码器进行熵编码,得到编码后的码流(即视频码流),并将该码流输出到一个缓冲器(buffer)中保存,等待传送出去。
B、量化处理输出的数据可以进行反量化和反变换处理,得到反变换后的残差视频信号u′k[x,y]。然后,将该反变换后的残差视频信号u′k[x,y]与预测信号相加得到新的预测信号并将新的预测信号送至当前图像的缓冲器中保存。然后,可将新的预测信号经过帧内预测处理,得到将新的预测信号经过环路滤波处理可以得到重建信号s′k[x,y],并将重建信号s′k[x,y]送至解码图像缓冲器中保存,以用于生成重建视频。重建信号s′k[x,y]经过运动补偿预测处理得到其中可以表示参考块,mx和my分别表示参考块的运动矢量的水平和竖直分量。
在一些实施例中,主流的视频编码标准如HEVC,VVC,AVS3,AV1,AV2均采用基于块的混合编码框架,可将原始的视频数据分成一系列的编码块,结合预测,变换和熵编码等视频编码方法,实现视频数据的压缩。其中,运动补偿是视频编码常用的一类预测方法,运动补偿基于视频内容在时域或空域的冗余特性,从已编码的参考块中导出当前块的预测值。这类预测方法包括:帧间预测、帧内块复制预测、帧内串复制预测等,在具体的编码实现中,可能单独或组合使用这些预测方法。对于使用了这些预测方法的编码块,通常需要在视频码流中显式或隐式的编码一个或多个二维的位移矢量,位移矢量用于指示当前块(或当前块的同位块)相对它的一个或多个参考块的位移。
需要注意的是,在不同的预测模式下及不同的实现,位移矢量可能有不同的名称,本文统一按照以下方式进行描述:1)帧间预测中的位移矢量称为运动矢量(Motion Vector,简称MV);2)帧内块复制预测中的位移矢量称为块矢量(Block Vector,简称BV);3)帧内串复制预测中的位移矢量称为串矢量(String Vector,简称SV)。以下以帧间预测为例,对帧间预测相关的技术进行介绍。
帧间预测:帧间预测利用视频时间域的相关性,使用邻近已编码图像像素预测当前图像的像素,以达到有效去除视频时域冗余的目的,能够有效节省编码残差数据的比特。如图1b所示,图1b为本申请实施例提供的一种帧间预测的示意图。其中,在图1b中,P为当前帧,Pr为参考帧,B为当前块,Br是B的参考块。B’与B在图像中的坐标位置相同(即B’是B的同位块),Br坐标为(xr,yr),B’坐标为(x,y)。当前块与其参考块之间的位移,称为运动向量(MV),即:MV=(xr-x,yr-y)。
其中,考虑到时域或空域邻近块具有较强的相关性,可以采用MV预测技术进一步减少编码MV所需要的比特。在H.265/HEVC中,帧间预测包含Merge(合并)和AMVP(Advanced Motion Vector Prediction,高级运动向量预测)两种MV预测技术,Merge模式会为当前PU(Prediction Unit,预测单元)建立一个MV候选列表,其中存在5个候选MV(及其对应的参考图像)。遍历这5个候选MV,选取率失真代价最小的作为最优MV。若编解码器依照相同的方式建立候选列表,则编码器只需要传输最优MV在候选列表中的索引即可。在AV1,AV2中使用了一种称为动态运动矢量预测(Dynamic motion vector prediction,DMVP)的技术对MV进行预测。
为了取得更好的预测效果,目前主流的视频编码标准中均允许使用多参考帧进行帧间预测。在AV1标准和目前正在研发的AOM下一代标准AV2中包含一种称为复合预测(compound prediction)的预测模式。该复合预测模式允许当前块使用两个参考帧进行帧间预测,并对帧
间预测值进行加权组合导出当前块的预测值,或使用一个参考帧导出的帧间预测值和使用当前帧导出的帧内预测值进行加权组合从而导出当前块的预测值。其中,当前块是指正在被编码(或解码)的编码块。在本申请实施例中,帧间预测值和帧内预测值后续均称为参考预测值。以下是复合预测过程中导出当前块的预测值的公式:
P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y))/2
P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y))/2
其中,P(x,y)为当前块的预测值;P0(x,y)和P1(x,y)为当前块(x,y)对应的两个参考预测值,w(x,y)为应用在第一个参考预测值P0(x,y)上的权重。
可选地,在视频编码中,为了降低加权预测的复杂度,通常会使用整型的计算代替浮点型的计算,以下是采用整型计算方式导出当前块的预测值的公式:
P(x,y)=(w(x,y)·P0(x,y)+(64-w(x,y))·P1(x,y)+32)>>6
P(x,y)=(w(x,y)·P0(x,y)+(64-w(x,y))·P1(x,y)+32)>>6
其中,权重值w(x,y)和参考预测值P0(x,y)、P1(x,y)均为整型,且使用右移操作来代替除法,“>>6”表示右移6位,通过右移6位可以整除64,32为用于四舍五入加上的偏置。
按照目前的视频编码标准,复合预测采用一种特殊的加权模式,即P0和P1具有相等的权重值,且不同位置所对应的参考预测值对应的权重均被设置固定值。具体的公式如下:
P(x,y)=(32×P0(x,y)+32×P1(x,y)+32)>>6
P(x,y)=(32×P0(x,y)+32×P1(x,y)+32)>>6
二、视频解码
在解码侧,对于每一个CU,在获得视频码流后,一方面,先对视频码流进行熵解码,获得各种预测编码模式的信息及量化后的变换系数,然后对各个变换系数经过反量化及反变换,得到残差视频信号。另一方面,根据已知的预测编码模式的信息,可获得该CU对应的预测信号(后续称为预测值),并将残差视频信号和预测信号进行相加,得到重建视频信号,该重建视频信号可用于重建该CU对应的解码图像。最后,该重建视频信号需要经过环路滤波的操作,产生最终的输出信号。
基于上述相关描述,本申请实施例提供了一种视频处理方案,该视频处理方案可应用于使用了复合预测(或基于多参考帧的加权预测)的视频编码器或者视频压缩产品中。该视频处理方案的大致原理如下:
在编码端:可对视频帧包含的CU使用复合预测,得到CU的N个参考预测值,N为大于1的整数;并根据N个参考预测值的重要性为CU自适应选择合适的权重进行加权预测,得到CU的预测值,所谓加权预测是指采用自适应选择的权重对CU的N个参考预测值进行加权预测处理。然后根据CU的预测值对视频进行编码处理,生成视频码流,并将该视频码流发送到解码端。
在解码端:在对视频码流中的CU进行解码时,可根据预测编码模式的信息确定对视频码流中的CU进行复合预测,并根据N个参考预测值的重要性为CU自适应选择合适的权重进行加权预测;然后基于自适应选择的权重对N个参考预测值进行加权预测,从而得到该CU的预测值,利用CU的预测值重建该CU对应的解码图像。
如前述,目前的视频编码标准规定在复合预测中使用相等的权重值对不同参考块导出的参考预测值进行加权预测,但实际应用中,不同的参考块导出的参考预测值可能具有不相等的重要性,采用相等的权重值无法体现出预测参考值的重要性差异,在此情况下使用现有标准会影响预测准确性;本申请实施例对目前的视频编码标准进行了改进,在复合预测中充分考虑不同参考块导出的参考预测值的重要性,允许在复合预测中基于各参考预测值的重要性为CU自适应选择合适的权重进行加权预测,拓展了视频编码标准中加权预测方式,可以提高CU的预测准确性,从而提高编码性能。
接下来对本申请实施例所提供的视频处理系统进行相关阐述。请参见图2,图2为本申请实施例提供的一种视频处理系统的架构示意图。该视频处理系统20可包括编码设备201和解码设备202。编码设备201位于编码端,解码设备位于解码端。编码设备201可以是终端,也可以是服务器,解码设备202可以是终端,也可以是服务器,编码设备201和解码设备202之间可以建立通信连接。其中,终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、
智能音箱、智能手表、车载终端、智能电视等,但并不局限于此。服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN(Content Delivery Network,内容分发网络)、以及大数据和人工智能平台等基础云计算服务的云服务器。
(1)对于编码设备201:
编码设备201可以获取待编码的视频,该视频可以通过摄像设备拍摄或计算机生成获取得到。摄像设备可以是设置于编码设备201中的硬件组件,例如,摄像设备可以是设置于终端中的普通摄像头、立体摄像头、以及光场摄像头等等。摄像设备也可以是指与编码设备201相连接的硬件装置,例如与服务器相连接的摄像头等。
其中,一个视频包括一个或多个视频帧,编码设备201可将每个视频帧划分成一个或多个CU,并对每个CU进行编码。在对任意CU进行编码时,可对正在编码的CU(后续称为当前块)进行复合预测,得到当前块的N个参考预测值,并综合考量加权预测处理时所消耗的码率、当前块在编码过程中的质量损失等因素确定各个参考预测值的重要性,进而基于各参考预测值的重要性为当前块自适应选择合适的目标权重组,该目标权重组中可包括一个或多个权重值;然后,采用目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值。其中,当前块的预测值可以理解为当前块对应的预测信号,当前块的预测值可以用于重建当前块对应的解码图像。
其中,当前块的N个参考预测值是从当前块的N个参考块中导出得到的;一个参考预测值对应一个参考块。参考块所在的视频帧为参考帧,当前块所在的视频帧为当前帧。N个参考块与当前块之间的位置关系可包括但不限于以下任一种:①N个参考块分别位于N个参考帧中,且N个参考帧与当前帧属于视频码流中不同的视频帧;②N个参考块位于同一个参考帧中,且同一个参考帧与当前帧属于视频码流中不同的视频帧;③N个参考块中存在一个或多个参考块位于当前帧中,N个参考块中剩余参考块位于一个或多个参考帧中,且一个或多个参考帧与当前帧属于视频码流中不同的视频帧;④N个参考块与当前块均位于当前帧中。可见,本申请实施例的复合预测的预测模式,既包括帧间预测模式,即允许使用至少两个参考帧进行帧间预测;也包含组合预测模式,即允许使用至少一个参考帧进行帧间预测,同时允许使用当前帧进行帧内预测;还包括帧内预测模式,即允许使用当前帧进行帧内预测。
对应于复合预测的不同预测模式,当前块的N个参考预测值的导出方式可以包括以下任一种:①当前块的N个参考预测值是使用当前块的N个参考块分别进行帧间预测后导出得到的,此时该N个参考预测值均可称为帧间预测值。②当前块的N个参考预测值中,至少一个参考预测值是使用当前块的N个参考块中的至少一个参考块进行帧间预测后导出得到的,此部分参考预测值可称为帧间预测值;剩余的参考预测值是使用N个参考块中的剩余参考块进行帧内预测后导出得到的,此部分参考预测值可称为帧内预测值。
然后,编码设备201基于视频帧包含的CU的预测值对视频进行变换编码、量化,熵编码等操作,得到视频码流,并将该视频码流发送给解码设备202,以使解码设备202对视频码流进行解码处理。
(2)对于解码设备202:
解码设备202可在接收编码设备201发送的视频码流后,对视频码流进行解码处理,重建该视频码流对应的视频。具体的,一方面,解码设备202可对视频码流进行熵解码,得到视频码流中的各CU的预测模式以及量化后的变换系数,并根据当前块(即正在被解码的CU)的预测模式对当前块进行复合预测,得到当前块的N个参考预测值;并确定当前块是否允许使用自适应加权预测。
若确定当前块允许使用自适应加权预测,则可根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表,并从目标权重列表中为当前块确定用于加权预测的目标权重组,该目标权重组中包含一个或多个权重值。然后,采用目标权重组中的权重值直接对
N个参考预测值进行加权预测处理,得到当前块的预测值。若确定当前块不允许使用自适应加权预测,则可按照现有视频编码标准对N个参考预测值进行加权预测处理,例如采用相等权重值对各个参考预测值进行加权预测处理,得到当前块的预测值。
另一方面,解码设备202对量化后的变换系数进行反量化以及反变换,得到当前块的残差信号值,并根据该当前块的预测值和残差信号值进行叠加,得到该当前块的重建值,然后根据该重建值重建当前块对应的解码图像。其中,解码出来的解码图像既可作为其他CU进行解码时所使用的参考图像,又可用于重建视频。
在本申请实施例中,在视频的编解码过程中均使用复合预测,并且在复合预测中充分考虑不同参考块导出的参考预测值的重要性,允许在复合预测中基于各参考预测值的重要性为CU自适应选择合适的权重进行加权预测,拓展了视频编码标准中加权预测方式,可以提高CU的预测准确性,从而提高编解码性能。
接下来对本申请实施例所提供的视频处理方法进行相关阐述。请参阅图3,图3为本申请实施例提供的一种视频处理方法的流程示意图。该视频处理方法可由上述视频处理系统中的解码设备执行,本实施例中所描述的视频处理方法可以包括以下步骤S301-S303:
S301、对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,当前块是指视频码流中正在被解码的编码块,N为大于1的整数;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频码流中针对当前块进行解码时所参考的编码块;参考预测值与参考块之间一一对应。
视频码流中包含一个或多个视频帧,每个视频帧中可以包含一个或多个编码块。在对视频码流进行解码时,解码设备可以从视频码流中获取编码块,解码设备可以将当前待进行解码处理的编码块作为当前块。上述N个参考预测值可以是从N个参考块中导出得到的,一个参考预测值对应一个参考块,即N个参考预测值是根据N个参考块得到的,参考预测值与参考块一一对应,N个参考块是在视频码流中针对当前块进行解码时所参考的编码块。在本申请实施例中,参考块所在的视频帧可以为参考帧,当前块所在的视频帧为当前帧。
其中,N个参考块与当前块之间的位置关系包括以下任一种:①N个参考块分别位于N个参考帧中,且N个参考帧与当前帧可以属于视频码流中不同的视频帧。例如,N=2,两个参考块中的一个参考块位于参考帧一,另外一个参考块位于参考帧二,参考帧一、参考帧二和当前帧属于视频码流中不同的视频帧。②N个参考块位于同一个参考帧中,且同一个参考帧与当前帧可以属于视频码流中不同的视频帧。例如,N=2,两个参考块均位于参考帧一,参考帧一和当前帧属于视频码流中不同的视频帧。③N个参考块中存在一个或多个参考块位于当前帧中,N个参考块中剩余参考块位于一个或多个参考帧中,且一个或多个参考帧与当前帧属于视频码流中不同的视频帧。例如,N=4,4个参考块分别为:参考块1、参考块2、参考块3、参考块4;参考块1位于当前帧中,剩余的参考块2、参考块3、参考块4均位于参考帧一中,且该参考帧一与当前帧属于视频码流中的不同视频帧;又例如,参考块1和参考块2位于当前帧中,剩余的参考块3可以位于参考帧一,参考块4位于一个参考帧二中,且参考帧一、参考帧二与当前帧属于视频码流中的不同视频帧。④N个参考块与当前块均位于当前帧中,例如,N=2,两个参考块均位于当前帧。
根据上述①-④所示的N个参考块与当前块之间的位置关系可知,本申请实施例的复合预测的预测模式,既包括帧间预测模式,即允许使用至少两个参考帧进行帧间预测;也可包含组合预测模式,即允许使用至少一个参考帧进行帧间预测,同时允许使用当前帧进行帧内预测;还包括帧内预测模式,即允许使用当前帧进行帧内预测。
对应于复合预测的不同预测模式,当前块的N个参考预测值的导出方式可以包括以下任一种:①当前块的N个参考预测值是使用当前块的N个参考块分别进行帧间预测后导出得到的,此时该N个参考预测值均可称为帧间预测值。②当前块的N个参考预测值中,至少一个参考预测值是使用当前块的N个参考块中的至少一个参考块进行帧间预测后导出得到的,此部分参考预测值可称为帧间预测值;剩余的参考预测值是使用N个参考块中的剩余参考块进
行帧内预测后导出得到的,此部分参考预测值可称为帧内预测值。本实施例中,N个参考块可以来源于不同的视频帧,具体可以来源于当前块所在的当前帧或来源于不同的参考帧,从而能够适用于不同的复合预测场景,确保在不同场景中复合预测的编解码性能。
在一个实施例中,在执行步骤S302之前,解码设备可先判断当前块是否满足自适应加权预测的条件,即确定自适应加权预测的条件,若当前块满足自适应加权预测的条件,则执行步骤S302;解码设备通过判断当前块是否满足自适应加权预测的条件,可以实现自适应选择权重进行加权预测,进一步提高预测准确性,从而提高编码性能。
其中,当前块满足自适应加权预测的条件包括以下至少一种:
a)当前块所属的帧序列的序列头中包含第一指示字段,且第一指示字段指示帧序列中的编码块均允许使用自适应加权预测;其中,帧序列是指由视频码流中多个视频帧按序构成的序列。应理解的是,在帧序列的序列头中包含第一指示字段时,该第一指示字段用于指示整个帧序列中包含的所有编码块均允许使用自适应加权预测。
作为一种实现方式,第一指示字段可以表示为seq_acp_flag,该第一指示字段可根据取值来指示帧序列中的编码块是否允许使用自适应加权预测,若上述第一指示字段为第一预设值(如1),则指示帧序列中的所有编码块均允许使用自适应加权预测,即可确定当前块满足自适应加权预测条件;若上述第一指示字段为第二预设值(如0),则指示帧序列中的所有编码块均不允许使用自适应加权预测,即可确定当前块不满足自适应加权预测条件。
b)当前块所属的当前片(slice)的片头中包含第二指示字段,且第二指示字段指示当前片中的编码块允许使用自适应加权预测,其中,一个视频帧可以被划分为多个图像分片(slice),每个图像分片包含一个或多个编码块;当前片是指当前块所属的图像分片,也即是正在被解码的图像分片。应理解的是,在当前片的片头中包含第二指示字段时,该第二指示字段可以用于指示当前片中包含的所有编码块均允许使用自适应加权预测。
作为一种实现方式,第二指示字段可以表示为slice_acp_flag,该第二指示字段可根据取值来指示当前片中的编码块是否允许使用自适应加权预测,若上述第二指示字段为第一预设值(如1),则指示当前片中的所有编码块均允许使用自适应加权预测,即可确定当前块满足自适应加权预测条件;若上述第二指示字段为第二预设值(如0),则指示当前片中的所有编码块均不允许使用自适应加权预测,即可确定当前块不满足自适应加权预测条件。
c)当前块所在的当前帧的帧头中包含第三指示字段,且第三指示字段指示当前帧中的编码块允许使用自适应加权预测。应理解的是,在当前帧的帧头中包含第三指示字段时,该第三指示字段可以用于指示当前帧中包含的所有编码块均允许使用自适应加权预测。
作为一种实现方式,第三指示字段可表示为pic_acp_flag,该第三指示字段根据取值来指示当前帧是否允许使用自适应加权预测,若上述第三指示字段为第一预设值(如1),则指示当前帧中的编码块允许使用自适应加权预测,即可确定当前块满足自适应加权预测条件;若第三指示字段为第二预设值(如0),则可以指示当前帧中的编码块不允许使用自适应加权预测,即可确定当前块不满足自适应加权预测条件。
d)在复合预测过程中,当前块使用至少两个参考帧进行帧间预测。
e)在复合预测过程中,当前块使用至少一个参考帧进行帧间预测,并使用当前帧进行帧内预测。例如,当前块使用参考帧一进行帧间预测,并使用当前帧进行帧内预测,那么可确定当前块满足自适应加权预测的条件。
f)当前块的运动类型为指定的运动类型,如当前块的运动类型为simple_translation(简单的平衡),那么确定当前块满足自适应加权预测的条件。
g)当前块使用预设的运动矢量预测模式,例如,当前块使用预设的运动矢量预测模式为NEAR_NEARMV,那么可确定当前块满足自适应加权预测的条件。
在AV1、AV2标准中使用了一种称为动态运动矢量预测(Dynamic motion vector prediction)的技术对MV进行预测,可以通过当前帧中的空间相邻块或参考帧中的时域相邻块来预测MV。对于单参考帧间预测,每个参考帧都有单独的预测MV列表;对于复合帧间预测,不同
参考帧对应的预测mv列表组成预测MV组列表,并允许使用多种预测MV模式,如NEAR_NEARMV、NEAR_NEWMV、NEW_NEAR_MV、NEW_NEWMV、GLOBAL_GLOBALMV、JOINT_NEWMV等。其中:
NEAR_NEARMV:表示两个参考帧对应的MV即为预测MV组中的MV;
NEAR_NEWMV:第一个MV为预测MV组中的第一个预测mv,第二个MV根据视频码流中解码得到的MVD与预测MV组中的第二个预测MV导出。其中,MVD(MotionVector Difference)是指对当前MV和预测MV(Motion Vector Prediction,MVP,)的差值。
NEW_NEARMV:第一个MV根据视频码流中解码得到的MVD与预测MV组中的第一个预测MV导出,第二个MV为预测MV组中的第二个预测MV。
NEW_NEWMV:第一个MV根据视频码流中解码得到的MVD1与预测MV组中的第一个预测MV导出,第二个MV根据视频码流中解码得到的MVD2与预测MV组中的第二个预测MV导出。
GLOBAL_GLOBALMV:根据每帧级全局运动信息导出MV。
NEW_NEWMV:类似于NEW_NEWMV,但视频码流中只包含一个MVD,另一个MVD根据参考帧之间的信息导出。
本申请实施例中预设的运动矢量预测模式可以为上述的:NEAR_NEARMV、NEAR_NEWMV、NEW_NEARMV、NEW_NEWMV、GLOBAL_GLOBALMV、JOINT_MV中的一种或多种。需要说明的是,预设的运动矢量预测模式并不限制于以上的MV预测模式,对于其他标准来说,如H.265和H.266,也可以结合merge、AMVP来确定运动矢量预测模式。
h)当前块使用预设的插值滤波器。视频编码过程中通常会包含很多的编码工具,包括多种类型的插值滤波器,例如:线性插值滤波器、cic(积分-梳状级联)插值滤波器,等等。本申请实施例中预设的插值滤波器可以是该多种类型的插值滤波器中的任一种,例如预设的插值滤波器可以是线性插值滤波器,当当前块使用线性插值滤波器时,可确定当前块满足自适应加权预测的条件。
i)当前块不使用特定的编码工具,如当前块不使用基于光流的运动矢量优化方法,那么可确定当前块满足自适应加权预测的条件。其中,视频编码标准如AV2,H.266中允许使用基于光流的运动矢量优化方法,该方法是一种基于光流方程推导对运动矢量进行细化的方法。
j)当前块在复合预测过程中所使用的参考帧满足特定条件,那么可确定当前块满足自适应加权预测的条件。其中,特定条件包括以下一种或多种(即特定条件可以包含①和②中的一种或多种):①在复合预测中所使用的参考帧与当前帧之间在视频码流中的方位关系满足预设关系;其中,方位关系满足预设关系包括以下任一种:所使用的参考帧均位于当前帧之前;所使用的参考帧均位于当前帧之后;所使用的部分参考帧位于当前帧之前,且其余参考帧位于当前帧之后。视频码流中包含多个视频帧,任意视频帧在视频中对应有帧显示时间,该方位关系实际上可理解为帧显示时间先后顺序,例如,所使用的参考帧均位于当前帧之前可以理解为:参考帧的帧显示时间早于当前帧的帧显示时间;所使用的参考帧均位于当前帧之后可以理解为:参考帧的帧显示时间晚于当前帧的帧显示时间。
②在复合预测中所使用的参考帧对应的参考预测值之间的重要性差异的绝对值大于或等于预设阈值,此时可确定当前块满足自适应加权预测的条件;其中,预设阈值可根据需求设置。作为一种实现方式,参考帧对应的参考预测值的重要性可由重要性度量值来衡量,此时在复合预测中所使用的参考帧对应的参考预测值之间的重要性差异可以根据所使用的参考帧对应的参考预测值之间的重要性度量值来确定。例如,N=2,在复合预测中,两个参考预测值分别为:参考帧一对应的参考预测值1以及参考帧二对应的参考预测值2;设参考预测值1的重要性度量值为D0,参考预测值2的重要性度量值为D1,那么,两个参考帧对应的参考预测值之间的重要性差异为:参考预测值1的重要性度量值D0和参考预测值2的重要性度量值D1之间的差值D0-D1,即两个参考预测值之间的重要性差异的绝对值为:ΔD=abs(D0-D1),
abs()表示取绝对值。
应理解是,上述当前块满足自适应加权预测的条件可以单独使用或者组合起来使用,例如,当前块满足自适应加权预测的条件可以包括:当前块所属的帧序列的序列头中包含第一指示字段,且第一指示字段指示帧序列中的编码块允许使用自适应加权预测,以及当前块的运动类型为指定的运动类型。又例如,当前块所在的当前帧的帧头中包含第三指示字段,且第三指示字段指示当前帧中的编码块允许使用自适应加权预测和当前块使用预设的运动矢量预测模式;等等;本申请对此不作任何限定。本实施例中,支持解码设备灵活配置自适应加权预测的条件中的各条件项,能够适用于不同的复合预测场景,确保在不同场景中复合预测的编解码性能。
S302、根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组;重要性用于表征相应参考预测值对当前块的解码性能的影响程度。
参考预测值的重要性可依据加权预测处理时所消耗的码率、当前块在编码过程中的质量损失等因素来综合确定,例如:当当前块使用某个参考预测值进行加权预测时,码率消耗明显增加,说明该参考预测值对降低码率消耗上的帮助不够明显,其重要性较低;再如:当当前块使用某个参考预测值进行加权预测时,质量损失明显增大,说明该参考预测值对减小质量损失的帮助较小,其重要性较低。根据参考预测值的重要性,可以确定参考预测值对当前块的解码性能的影响程度。目标权重组中包含一个或多个权重值,这些权重值会在加权预测过程中作用到各个参考预测值上,如果某个参考预测值的重要性较低,那么该参考预测值对应目标权重组中取值较小的权重值,而某个参考预测值的重要性较高,那么该参考预测值对应目标权重组中取值较大的权重值。也就是说,目标权重组是在考虑各参考预测值对码率消耗、质量损失等因素的影响的基础上所选择的,使得加权预测过程的综合代价(即码率消耗的代价、质量损失的代价、码率消耗和质量损失的代价)较小、编解码性能较优的一组权重。
在一个实施例中,视频码流中包含一个或多个权重列表,每个权重列表中包含一个或多个权重组,一个权重组中包含一个或多个权重值;每个权重组中包含的权重值的数量允许相同或不同,每个权重组中包含的权重值的取值允许相同或不同,每个权重组中包含的权重值的顺序允许相同或不同。以下为权重列表的一个示例:
i、权重列表1表示为:{2,4,6,8,10,12,14}/16。这表明权重列表1中包含7个权重组,分别为:权重组1:{2}/16;权重组2:{4}/16;权重组3:{6}/16,以此类推,权重组7:{14}/16。
ii、权重列表2表示为:{14,8,4,12,2}/16。这表明权重列表2中包含5个权重组,分别为:权重组1:{14}/16;权重组2:{8}/16;以此类推,权重组5:{2}/16。
iii、权重列表3表示为:{4,{8,8},12}/16。这表明权重列表3中包含3个权重组,分别为:权重组1:{4}/16;权重组2:{8,8}/16;权重组3:{12}/16。
通过上述示例可以看出:(1)每个权重列表中包含的权重组的数量允许不同,如权重列表1中的权重组的数量为7,权重列表2中的权重组的数量为5,权重列表1中的权重组数量为3。(2)每个权重组包含的权重值的数量允许相同,例如权重列表1中的各个权重组包含的权重值的数量均为1。(3)每个权重组包含的权重值的数据也允许不同,例如权重列表3中的权重组1包含的权重值的数量为1,但权重列表3中的权重组2包含的权重值的数量为2。(4)每个权重组中的权重值允许相同,例如:权重列表1中的权重组1包含权重值2/16,权重列表2中的权重组1也包含权重值2/16。(5)每个权重组中的权重值也允许不同,例如权重列表1中的权重组1包含权重值2/16,但权重列表1中的权重组2包含权重值4/16。可以理解的是,一个权重组提供的各个权重值之和应当等于1,上述示例中,每个权重组中虽然仅包含一个权重值,但各权重值均小于1,因此每个权重组隐含还包含另一个权重值,也即是在具体应用中,每个权重组实际提供两个权重值,这两个权重值之和为1;例如,权重列表1中的权重组1仅包含权重值2/16,但在具体应用中,该权重组1实际提供权重值2/16和权重值14/16共两个权重值。再如:权重列表3中的权重组3仅包含权重值12/16,但在具体应用中,该权重组3实际提供权重值12/16和权重值4/16共两个权重值。可见,本申请实施例中,当
权重列表中的权重组包含的各权重值之和小于1时,该权重组隐含的权重值可通过计算得到。
以下为权重列表的另一个示例:
i、权重列表4中包含4个权重组,分别为权重组1:{2,14}/16;权重组2:{4,12}/16;权重组3:{6,10}/16;权重组4:{8,8}}/16。
ii、权重列表5包含2个权重组,分别为权重组1:{4,12}/16;权重组2:{10,6}/16。
通过上述示例可以看出:(1)权重列表中的权重组包含的全部权重值之和等于1。(2)每个权重组中包含的权重值的顺序允许不同,例如权重列表4中的权重组3和权重列表5中的权重组2虽然包含相同的权重值,但各权重值的顺序是不同的。
在一个实施例中,步骤S302可以包括步骤s31-s32:
s31、根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表。
其中,解码设备根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表时,可以综合权重列表的数量以及各个参考预测值的重要性度量值实现,其具体实现方式可包括如下几种方法:
方法1:当视频码流中权重列表的数量仅为一个时,解码设备可将视频码流中这唯一一个权重列表直接确定为目标权重列表。
方法2:当视频码流中存在多个权重列表时,解码设备可引入参考预测值的重要性度量值,并根据N个参考预测值的重要性度量值从多个权重列表中确定目标权重列表。
(1)根据N个参考预测值的重要性度量值之间的差值的绝对值来确定目标权重列表。
当视频码流中的权重列表的数量为M+1时,M为大于等于1的正整数,M+1组权重列表可分别记为{w_list1,w_list2……w_listM+1},一个权重列表对应一个阈值区间,即阈值区间的数量也为M+1。作为一种实现方式,解码设备可直接根据需求设置M+1个阈值区间。作为另一种实现方式,解码设备可获取M个阈值,并根据M个阈值划分M+1个阈值区间,如获取M个阈值,分别为T1,T2,…,TM;然后解码设备根据M个阈值划分为M+1个阈值区间,分别为[0,T1]、(T1,T2],(T2,T3],…,(TM,+∞);其中,可将[0,T1]对应w_list1,(T1,T2]对应w_list2,依次类推。T为大于等于0的整数。
解码设备可获取N个参考预测值的重要性度量值,并计算N个参考预测值之间的重要性差异,任意两个参考预测值之间的重要性差异采用任意两个参考预测值的重要性度量值之间的差值进行衡量。然后,解码设备可以确定N个参考预测值之间的重要性差异的绝对值所在的阈值区间;并将N个参考预测值之间的重要性差异的绝对值所在的阈值区间对应的权重列表确定为目标权重列表。其中,设任意两个参考预测值的重要性度量值分别为D0和D1,该任意两个参考预测值之间的重要性差异表示为D0-D1,那么该任意两个参考预测值的重要性差异的绝对值表示为ΔD=abs(D0-D1)。需要说明的是,重要性度量值是用于衡量重要性程度的指标,但衡量基准可以存在多种情况,例如,衡量基准可以是重要性度量值越大代表重要性程度越高;再如:衡量基准也可以是重要性度量值越小代表重要性程度越高;对此衡量基准,本申请并不进行限定。
在一种实施方式中,如果N=2,即只包含两个参考预测值,该两个参考预测值的重要性度量值分别为D0和D1,那么,解码设备将ΔD=abs(D0-D1)所在的阈值区间所对应的权重列表直接确定为目标权重列表。例如:当视频码流中的权重列表的数量为2时,该两个权重列表分别记为{w_list1,w_list2},其中,w_list1为{8,12,14}/16,w_list2为{12,8,4}/16,设w_list1对应的阈值区间为[0,1],w_list2对应的阈值区间为(1,+∞);那么,当ΔD=abs(D0-D1)小于等于1时,所在的阈值区间为[0,1],则将w_list1确定为目标权重列表;否则,当ΔD=abs(D0-D1)大于1时,所在的阈值区间为(1,+∞),则将w_list2确定为目标权重列表。
在另一种实施方式中,如果N>2,解码设备可分别计算任意两个参考预测值之间的重要性差异的绝对值,解码设备分别找出各绝对值所在阈值区间对应的权重列表,解码设备将对应数量最多的权重列表确定为目标权重列表。例如N=3,三个参考预测值的重要性度量值分别为D0、D1和D2,那么解码设备可分别计算任意两个参考预测值之间的重要性差异的绝对
值,即解码设备分别计算ΔD=abs(D0-D1)、ΔD'=abs(D1-D2)和ΔD”=abs(D0-D1),解码设备再分别判断ΔD、ΔD'和ΔD”分别所在阈值区间对应的权重列表,如果三者中存在二者或二者以上对应相同的权重列表,则解码设备将该相同的权重列表确定为目标权重列表。
在另一种实施方式中,如果N>2,可分别计算任意两个参考预测值之间的重要性差异的绝对值,解码设备再找出各绝对值中的最大值,解码设备将最大值所在阈值区间对应的权重列表确定为目标权重列表。例如上述N=3的示例中,计算得到ΔD、ΔD'和ΔD”之后,解码设备找出ΔD、ΔD'和ΔD”中的最大值,假设最大值为ΔD',则解码设备将ΔD'所在阈值区间对应的权重列表确定为目标权重列表。
在另一种实施方式中,如果N>2,解码设备可分别计算任意两个参考预测值之间的重要性差异的绝对值,解码设备再找出各绝对值中的最小值,解码设备将最小值所在阈值区间对应的权重列表确定为目标权重列表。例如上述N=3的示例中,计算得到ΔD、ΔD'和ΔD”之后,解码设备找出ΔD、ΔD'和ΔD”中的最小值,假设最小值为ΔD,则解码设备将ΔD所在阈值区间对应的权重列表确定为目标权重列表。
在另一种实施方式中,如果N>2,解码设备可分别计算任意两个参考预测值之间的重要性差异的绝对值,解码设备再计算各绝对值的平均值,解码设备将平均值所在阈值区间对应的权重列表确定为目标权重列表。例如上述N=3的示例中,计算得到ΔD、ΔD'和ΔD”之后,解码设备再计算ΔD、ΔD'和ΔD”中的平均值=(ΔD+ΔD'+ΔD”)/3,解码设备将平均值所在阈值区间对应的权重列表确定为目标权重列表。
需要说明的是,本申请实施例还可以利用N个参考预测值之间的重要性差异的绝对值的其他数值特性,例如各绝对值平方后的最大值、最小值、平均值等等;来确定目标权重列表,本申请并不对此进行限定。本实施例中,解码设备通过N个参考预测值的重要性度量值之间的差异所处阈值区间,确定目标权重列表,从而能够确定出适用于针对当前块进行重建的权重值,有利于提高当前块的预测准确性,从而能够提高编解码性能。
(2)比对参考预测值的重要性度量值之间的大小来确定目标权重列表。
当前块的N个参考预测值中可以包括第一参考预测值和第二参考预测值,视频码流中可以包括第一权重列表和第二权重列表。解码设备可比较第一参考预测值的重要性度量值与第二参考预测值的重要性度量值之间的大小;若确定第一参考预测值的重要性度量值大于第二参考预测值的重要性度量值,则将第一权重列表确定为目标权重列表;若确定第一参考预测值的重要性度量值小于或等于第二参考预测值的重要性度量值,则解码设备将第二权重列表确定为目标权重列表。
例如,上述第一参考预测值的重要性度量值为D0,第二参考预测值的重要性度量值为D1,上述第一权重列表为w_list1,第二权重列表为w_list2;解码设备比较D0和D1的大小;若D0>D1,则解码设备可将第一权重列表w_list1确定为目标权重列表;若D0≤D1,则解码设备可将第一权重列表w_list2确定为目标权重列表。
可选地,第一权重列表中的权重值与第二权重列表中的权重值相反,即w_list2[x]=1-w_list1[x],x表示权重列表中的权重值,如w_list1={0.2,0.4},w_list2[x]=1-w_list1[x],即w_list2[x]={0.8,0.6},第一权重列表中与第二权重列表中处于相同次序的权重值之和为1。另外,第一权重列表中的权重值与第二权重列表中的权重值也可以分别进行设置。
本实施例中,解码设备根据两个参考预测值之间的数值大小关系,确定相应的权重列表作为针对当前块的目标权重列表,能够提高目标权重列表的针对性,有利于提高当前块的预测准确性,从而能够提高编解码性能。
(3)利用数学符号函数、参考预测值的重要性度量值来确定目标权重列表。
当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;视频码流中包括第一权重列表、第二权重列表和第三权重列表。解码设备可调用数学符号函数对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值之间的差值进行处理,得到符号值。若上述符号值为第一预设值(如-1),则将第一权重列表确定为目标权重列表;若上述符号值为
第二预设值(如0),则将第二权重列表确定为目标权重列表;若上述符号值为第三预设值(如1),则将第三权重列表确定为目标权重列表。其中,上述第一权重列表、第二权重列表和第三权重列表分别为不同的权重列表;或者,第一权重列表、第二权重列表和第三权重列表中允许两个权重列表为相同的权重列表。
其中,上述第一参考预测值的重要性度量值为D0,第二参考预测值的重要性度量值为D1,调用数据符号函数对D0与D1之间的差值进行处理,得到符号值,即符号值=sign(D0-D1),sign()表示数据符号函数。解码设备根据两个参考预测值之间的数值大小关系,从三个权重列表中确定相应的权重列表作为针对当前块的目标权重列表,能够提高目标权重列表的针对性,有利于提高当前块的预测准确性,从而能够提高编解码性能。
(4)上述方式(1)(2)(3)可单独使用,也可将方式(1)(2)(3)组合来确定目标权重列表。作为一种实现方式,设有M+1个权重列表,针对任意两个参考预测值,解码设备可以先采用方式(1)通过该两个参考预测值之间的重要性差异的绝对值所在阈值区间确定出候选权重列表,然后再采用方式(2)来比较第一参考预测值的重要性度量值和第二参考预测值的重要性度量值的大小,若确定第一参考预测值的重要性度量值大于第二参考预测值的重要性度量值,则解码设备将候选权重列表直接确定为目标权重列表;若确定第一参考预测值的重要性度量值小于或等于第二参考预测值的重要性度量值,则解码设备将与候选权重列表中的权重值相反的权重值所对应的权重列表确定为目标权重列表。
例如,第一参考预测值的重要性度量值为D0,第二参考预测值的重要性度量阈值为D1;采用方式(1)可以确定候选权重列表w_list1。如果D0>D1时,则解码设备将权重列表w_list1确定为目标权重列表;如果D0≤D1,则解码设备将权重列表w_list2确定为目标权重列表;其中,w_list2中的权重值与w_list1的权重值相反,即w_list2[x]=1-w_list1[x]。
作为另一种实现方式,设视频码流中存在3*(M+1)个权重列表,即一个阈值区间可对应3个权重列表;针对任意两个参考预测值,解码设备可以先采用方式(1)可以确定第一参考预测值和第二参考预测值之间的重要性差异的绝对值所在的阈值区间,并将该阈值区间对应的3个权重列表均确定为候选权重列表,即候选权重列表可包含第一权重列表、第二权重列表、第三权重列表。然后解码设备再采用方式(2)调用数学符号函数对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值之间的差值进行处理,得到符号值。若符号值为第一预设值,则解码设备将第一权重列表确定为目标权重列表;若符号值为第二预设值,则解码设备将第二权重列表确定为目标权重列表;若符号值为第三预设值,则解码设备将第三权重列表确定为目标权重列表。
例如,采用方式(1)确定3个权重列表{w_list1,w_list2,w_list3}作为候选权重列表,然后调用数学符号函数对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值之间的差值进行处理,得到符号值。当符号值为-1时,解码设备将w_list1确定为目标权重列表;当符号值为0时,解码设备将w_list2确定为目标权重列表;当符号值为1时,解码设备将w_list3确定为目标权重列表。
N个参考预测值中的任一个表示为参考预测值i,参考预测值i是从参考块i导出得到的,参考块i所在的视频帧为参考帧i;i为整数且i小于等于N;当前块所在的视频帧为当前帧;参考预测值i的重要性度量值可以采用以下任一种方法确定:
方法一:根据当前帧在视频码流中的帧显示顺序(Picture Order Count,POC)及参考帧i在视频码流中的帧显示顺序计算得到。具体的,解码设备可计算当前帧在视频码流中的帧显示顺序及参考帧i在视频码流中的帧显示顺序之间的差值,并将该差值的绝对值作为参考预测值i的重要性度量值。例如,设当前帧在视频码流中的帧显示序号表示为cur_poc,参考帧i在视频码流中的帧显示顺序为ref_poc;参考预测值i的重要性度量值D=abs(cur_poc-ref_poc),其中,abs()表示取绝对值。
方法二:根据当前帧在视频码流中的帧显示顺序、参考帧i在视频码流中的帧显示顺序及质量度量Q计算得到,其中,质量度量Q可以根据多种情况确定,本申请并不对此进行限
定,具体地,参考帧i的质量度量Q可根据当前块的量化信息导出,例如:质量度量Q可以设置为参考帧i的base_qindex(基础量化索引),任意参考帧的base_qindex可以不同或者可以相同。在另一种实现中,参考帧i的质量度量Q还可根据其他编码信息导出,例如:参考帧i的质量度量Q可以依据参考帧i中已编码CU与当前帧中的已编码CU之间的编码信息差异导出。
作为一种实现方式,解码设备可计算当前帧在视频码流中的帧显示顺序及参考帧i在视频码流中的帧显示顺序之间的差值;然后,采用目标函数,根据上述差值、质量度量Q以及重要性度量值列表确定参考预测值i的重要性度量值。
其中,目标函数为:D=f(cur_poc-ref_poc)+Q,D表示参考预测值i的重要性度量值,f(x)为递增函数,x=cur_poc-ref_poc,cur_poc表示当前帧在视频码流中的帧显示序号,ref_poc表示参考帧i在视频码流中的帧显示顺序。上述重要性度量值列表中包含f(x)与参考重要性度量值之间的对应关系,如表1所示:
表1
需要说明的是,上述f(x)的函数表达式仅为举例,本申请实施例允许f(x)的函数表达式存在变化,即本申请实施例并不限定f(x)的具体表达形式。
在一个实施例中,上述参考帧i的重要性度量值可根据参考帧i与当前帧之间的方位关系以及质量度量Q计算得到。作为一种实现方式,解码设备可建立参考方位关系和参考重要性度量值之间的对应关系,例如,若参考方位关系为参考帧在当前帧之前,则参考重要性度量值可以对应第一取值;若参考方位关系为参考帧在当前帧之后,则参考重要性度量值可以对应第二取值。然后,解码设备可根据参考帧i对应的参考重要性度量值和质量度量Q计算得到参考预测值i的重要性度量值。
方法三:根据方法一的计算结果和方法二的计算结果计算参考帧i的重要度量分数,并按照从小到大的顺序对N个参考预测值对应的参考帧的重要度量分数进行排序,将参考帧i在排序中的索引确定为参考预测值i的重要性度量值。
作为一种实现方式,通过上述方法一可计算得到参考预测值i的第一重要性度量值,以及上述方法二可计算得到参考预测值i的第二重要性度量值,然后根据第一重要性度量值和第二重要性度量值计算参考帧i的重要度量分数(如score),然后按照从小到大的顺序对N个参考预测值对应的参考帧的重要度量分数进行排序,并将参考帧i在排序中的索引确定为参考预测值i的重要性度量值。
例如,参考预测值1对应的参考帧一的重要度量分数为20,参考预测值2对应的参考帧二的重要度量分数为30,参考预测值3对应的参考帧三的重要度量分数为40;然后,按照从小到大的顺序对3个参考预测值对应的参考帧的重要度量分数进行排序,排序结果为:参考帧一、参考帧二、参考帧三;其中,参考帧一在排序中的索引为1,那么参考预测值1的重要性度量值为1;参考帧二在排序中的索引为2,那么参考预测值2的重要性度量值为2;参考帧三在排序中的索引为3,那么参考预测值3的重要性度量值为3。
其中,根据第一重要性度量值和第二重要性度量值计算参考帧i的重要度量分数可以有如下方式:①对第一重要性度量值和第二重要性度量值进行加权求和,得到参考预测值i的重要性度量值。②对第一重要性度量值和第二重要性度量值进行平均处理,得到参考预测值i的重要性度量值。
方法四:为了获取比较准确的重要性度量值,解码设备可基于参考预测值i的预测模式对方法一、方法二或方法三的计算结果进行调整,得到参考预测值i的重要性度量值;其中,参考预测值i的预测模式包括以下任一种:帧间预测模式、帧内预测模式。作为一种实现方式,可按照调整函数对方法一、方法二或方法三的计算结果进行调整,得到参考预测值i的重要性度量值。调整函数例如可以为D’=g(D)=a*D+b;其中,D’表示参考预测值i的重
要性度量值;D为上述方法一、方法二或方法三的计算结果,a和b可根据预测模式进行自适应设置。例如,以下为具体示例出的3个例子:
a)若参考预测值i的预测模式为帧间预测,则a=1,b=0;
b)若参考预测值i的预测模式为帧内预测,则a=2,b=0;
c)若参考预测值i的预测模式为帧内预测,则a=0,b=160。
应理解的是,在实际过程中,可根据需求采用上述方法一至方法四中的任一种方法确定参考预测值的重要性度量值,本申请对此不做限定。解码设备可以根据各种不同方法针对确定重要性度量值,可以确保重要性度量值的可靠性,从而有利于提高当前块的预测准确性,从而能够提高编解码性能。
s32、从目标权重列表中选择用于加权预测的目标权重组。
其中,解码设备从目标权重列表中选择目标权重组,可以分为以下两种情况:
(1)当目标权重列表中包含的权重组数等于1时,此时无需从视频码流中进行解码目标权重组的索引,直接将目标权重列表中的权重组作为用于加权预测的目标权重组。
(2)当目标权重列表中包含的权重组数大于1时,即在目标权重列表中包含多个权重组。例如,目标权重列表表示为{{2,14},{4,12},{6,10},{8,8}}/16,在该目标权重列表中包含4个权重组。此时,在视频码流中可以指示当前块进行加权预测时的目标权重组的索引,该目标权重组的索引使用截断一元码进行二值化编码的编码方式,或者使用多符号的熵编码方式。其中,截断一元码是在已知待编码的语法元素的最大值Max的情况下。假设待编码符号为x:如果0<x<Max,x二值化采用一元码的方式;如果x=Max,x二值化的二进制串全部由1组成,长度为Max。然后,解码设备需要从视频码流中解码出用于加权预测的目标权重组的索引,并按照目标权重组的索引从目标权重列表中选择目标权重组。其中,目标权重组的索引可以指示在目标权重列表中的位置;例如上述示例中的目标权重列表包含4个权重组;解码设备从视频码流中解码出目标权重组的索引为2,那么根据该目标权重组的索引确定在目标权重列表中的位置,即处于目标权重列表中的第二个位置(即为权重组2),那么确定出的目标权重组为{4,12}/16。解码设备可以根据从所述视频码流中解码出索引以确定目标权重组,可以针对确定出目标权重组,提高目标权重组选择的处理效率,从而提高解码效率。
本实施例中,解码设备根据N个参考预测值的重要性确定目标权重列表,并从目标权重列表中选择用于加权预测的目标权重组,通过对权重列表进行筛选,可以选择合适的权重组针对当前块进行解码重建,能够提高当前块的预测准确性,从而提高编解码性能。
S303、基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像。
目标权重组实际提供的权重值的数量应当与参考预测值的数量相对应。例如,参考预测值的数量为N,相对应的,目标权重组实际提供的权重值的数量也为N。例如,N=3,3个参考预测值分别为参考预测值1、参考预测值2和参考预测值3,目标权重组中可以包含3个权重值,分别为权重值1、权重值2和权重值3;其中,参考预测值1对应权重值1,参考预测值2对应权重值2,参考预测值3对应权重值3。上述例子中,目标权重组中也可以仅包含2个权重值,分别为权重值1和权重值2,该权重值1与权重值2之和小于1,那么还可通过计算得到该目标权重组中隐含的权重值3=1-权重值1-权重值2。
在一个实施例中,基于目标权重对N个参考预测值进行加权预测处理,得到当前块的预测值的方式有以下两种方式(1)和(2):
(1)解码设备可采用目标权重组中的权重值分别对N个参考预测值进行加权求和处理,得到所述当前块的预测值。其中,当前块的预测值P(x,y)可以为:
P(x,y)=(w1·P0(x,y)+w2·P1(x,y)+…+wn·PN-1(x,y))/N
P(x,y)=(w1·P0(x,y)+w2·P1(x,y)+…+wn·PN-1(x,y))/N
其中,P(x,y)为当前块的预测值,P0(x,y)、P1(x,y)……PN-1(x,y)分别表示N个参考预测值;w1表示当前块(x,y)对应的第一个参考预测值对应的权重值,w2表示当前块(x,y)对应的第2个参考预测值对应的权重值,以此类推,wn表示当前块(x,y)对应的第N个参考预测
值对应的权重值。
在一个实施例中,当N=2时,即当前块使用两个参考预测值进行加权预测,此时,目标权重组中包含的权重值的数量为一个,即包含隐含权重值,当前块的预测值P(x,y)可以为:
P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y))/2。
P(x,y)=(w(x,y)·P0(x,y)+(1-w(x,y))·P1(x,y))/2。
其中,P(x,y)为当前块的预测值,P0(x,y)和P1(x,y)为当前块(x,y)对应的两个参考预测值,w(x,y)为应用在第一个参考预测值P0(x,y)上的目标权重组中的权重值,1-w(x,y)为隐含权重值,并应用在第二个参考预测值P1(x,y)上。
(2)在视频编码中,考虑预测值计算的复杂度,可使用右移操作来代替除法实现整型计算,并按照整型计算的形式采用目标权重组中的权重值分别对N个参考预测值进行加权处理,得到当前块的预测值,通过整型计算的形式可以在一定程度上降低预测值计算的复杂度。
例如,以参考预测值的数量为2(即N=2)、目标权重组中的权重值的数量为1为例,解码设备可按照整型计算的形式采用目标权重组中的权重值分别对2个参考预测值进行加权处理,得到当前块的预测值。此时,当前块的预测值P(x,y)可以为:
P(x,y)=(w(x,y)×P0(x,y)+(16-w(x,y))×P1(x,y)+8)>>4
P(x,y)=(w(x,y)×P0(x,y)+(16-w(x,y))×P1(x,y)+8)>>4
其中,“>>4”表示右移4位,即通过右移4位可以使得加权处理过程的数据能整除16,8为用于四舍五入加上的偏置。上述P0(x,y)、P1(x,y)、w(x,y)、P(x,y)均为整型;P(x,y)为当前块的预测值,P0(x,y)和P1(x,y)为当前块(x,y)对应的两个参考预测值,w(x,y)为应用在第一个预测值P0(x,y)上的权重值(即目标权重组中的权重值)。
又例如,“>>6”表示右移6位,即通过右移6位可以使得加权处理过程的数据能整除64,32为用于四舍五入加上的偏置。此时,当前块的预测值P(x,y)可以为:
P(x,y)=(w·P0(x,y)+(64-w)·P1(x,y)+32)>>6
P(x,y)=(w·P0(x,y)+(64-w)·P1(x,y)+32)>>6
本实施例中,解码设备通过目标权重组中的权重值分别对N个参考预测值进行加权求和处理,或采用整型计算的形式进行加权求和处理,以得到当前块的预测值,可以确保预测值的准确性,从而有利于提高针对当前块的解码性能。
在一个实施例中,在得到当前块的预测值之后,可根据当前块的残差视频信号和该预测值进行叠加处理,得到当前块的重建值,基于该当前块的重建值重建当前块对应的解码图像。一方面,当前块对应的解码图像可以作为其他编码块进行加权预测时的参考图像,另一方面,当前块对应的解码图像也可用于重建当前块的当前帧,最终可根据已重建的多个视频帧重建视频。
在本申请实施例中,解码设备对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,当前块是指视频码流中正在被解码的编码块,N为大于1的整数;解码设备根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组;基于目标权重组对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像,在视频的解码过程中使用复合预测,并且在复合预测中充分考虑参考预测值的重要性,实现在复合预测中基于各参考预测值的重要性为当前块自适应选择合适的权重进行加权预测,可以提高当前块的预测准确性,从而提高编解码性能。
请参阅图4,图4为本申请实施例提供的另一种视频处理方法的流程示意图。该视频处理方法可由视频处理系统中的编码设备执行,本实施例中所描述的视频处理方法可以包括以下步骤S401-S405:
S401、对视频中的当前帧进行划分处理,得到当前块。当前块是指视频中正在被编码设备编码的编码块。其中,视频可以包括一个或多个视频帧,当前帧是指正在被编码的视频帧。编码设备可以对视频中的当前帧进行划分,得到一个或多个编码块,而当前块是指当前帧中的一个或多个编码块中任意一个正在被编码的编码块。
S402、对当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频中针对当前块进行编码时所参考的编码块;参考预测值与参考块之间一一对应。
上述N个参考预测值可以是从N个参考块中导出得到的,一个参考预测值对应一个参考块。在本申请实施例中,参考块所在的视频帧可以为参考帧,当前块所在的视频帧为当前帧。其中,N个参考块与当前块之间的位置关系包括以下任一种:①N个参考块分别位于N个参考帧中,且N个参考帧与当前帧可以属于视频中不同的视频帧。②N个参考块位于同一个参考帧中,且同一个参考帧与当前帧可以属于视频中不同的视频帧。③N个参考块中存在一个或多个参考块位于当前帧中,N个参考块中剩余参考块位于一个或多个参考帧中,且一个或多个参考帧与当前帧属于视频中不同的视频帧。④N个参考块与当前块均位于当前帧中。
根据上述①-④所示的N个参考块与当前块之间的位置关系可知,本申请实施例的复合预测的预测模式,既包括帧间预测模式,即允许使用至少两个参考帧进行帧间预测;也可包含组合预测模式,即允许使用至少一个参考帧进行帧间预测,同时允许使用当前帧进行帧内预测;还包括帧内预测模式,即允许使用当前帧进行帧内预测。
对应于复合预测的不同模式,编码设备对当前块进行复合预测,得到当前块的N个参考预测值可以包括以下任一种:①在对当前块进行复合预测的过程中,当前块使用N个参考块进行帧间预测,得到当前块的N个参考预测值,此时,当前块的N个参考预测值是使用当前块的N个参考块分别进行帧间预测后导出得到的。②在对当前块进行复合预测的过程中,当前块使用N个参考预测值中至少一个参考预测值进行帧间预测以及使用N个参考块中的剩余参考块进行帧内预测,从而得到N个参考预测值。此时,N个参考预测值中部分参考预测值是使用当前块的N个参考块中的至少一个参考块进行帧间预测后导出得到的;剩余的参考预测值是使用N个参考块中的剩余参考块进行帧内预测后导出得到的。
S403、根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组;重要性用于表征相应参考预测值对当前块的编码性能的影响程度。
参考预测值的重要性可依据加权预测处理时所消耗的码率、当前块在编码过程中的质量损失等因素来综合确定,例如:当当前块使用某个参考预测值进行加权预测时,码率消耗明显增加,说明该参考预测值对降低码率消耗上的帮助不够明显,其重要性较低;再如:当当前块使用某个参考预测值进行加权预测时,质量损失明显增大,说明该参考预测值对减小质量损失的帮助较小,其重要性较低。目标权重组中包含一个或多个权重值,这些权重值会在加权预测过程中作用到各个参考预测值上,如果某个参考预测值的重要性较低,那么该参考预测值对应目标权重组中取值较小的权重值,而某个参考预测值的重要性较高,那么该参考预测值对应目标权重组中取值较大的权重值。也就是说,目标权重组是在考虑各参考预测值对码率消耗、质量损失等因素的影响的基础上所选择的,使得加权预测过程的综合代价(即码率消耗的代价、质量损失的代价、码率消耗和质量损失的代价)较小、编解码性能较优的一组权重。
在一种实现中,编码设备采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值;或者,基于目标权重组中的权重值对N个参考预测值进行加权预测处理,使得当前块在编码过程中的质量损失小于预设损失阈值;或者,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值,并且基于目标权重组中的权重值对N个参考预测值进行加权预测处理,使得当前块在编码过程中的质量损失小于预设损失阈值。其中,预设码率阈值和预设损失阈值可以根据实际需要预先进行设定,如可以根据历史编解码记录统计分析出相应的码率阈值和损失阈值,以使加权预测过程的综合代价较小。本实施例中,编码设备可以根据预设码率阈值和预设损失阈值中的至少一项,约束目标权重组的选择,可以根据实际需要选择适合的目标权重组,有利于降低编码代价,提高编解码性能。
在一个实施例中,在编码过程中存在一个或多个权重列表,可理解为:一个或多个权重列表均可在编码过程中被使用,每个权重列表中包含一个或多个权重组,每个权重组中包含一个或多个权重值,每个权重组包含的权重值的数量允许相同或不同,且每个权重组中包含的权重值的取值允许相同或不同。此时,根据N个参考预测值的重要性,为当前块确定用于
加权预测的目标权重组可以包括步骤s41-s42:
s41、根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表。
其中,根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表的具体实现方式可包括如下几种方法:
方法1:当在编码过程中存在一个权重列表时,可将该权重列表直接作为目标权重列表。
方法2:当在编码过程中存在多个权重列表时,可引入参考预测值的重要性度量值,并根据N个参考预测值的重要性度量值从多个权重列表中确定目标权重列表。
(1)根据N个参考预测值的重要性度量值之间的差值的绝对值来确定目标权重列表。
当在编码过程中存在的权重列表的数量为M+1时,M为大于等于1的正整数,一个权重列表对应一个阈值区间,即阈值区间的数量也为M+1。然后,编码设备可获取N个参考预测值的重要性度量值,并计算N个参考预测值之间的重要性差异,任意两个参考预测值之间的重要性差异采用任意两个参考预测值的重要性度量值之间的差值进行衡量;具体的,可计算任意两个参考预测值之间的重要性度量之间的差值,并将任意两个参考预测值的重要性度量值之间的差值作为任意两个参考预测值之间的重要性差异;然后,确定N个参考预测值之间的重要性差异的绝对值所在的阈值区间;并将N个参考预测值之间的重要性差异的绝对值所在的阈值区间对应的权重列表确定为目标权重列表。
(2)比对参考预测值的重要性度量值之间的大小来确定目标权重列表。
当前块的N个参考预测值中可以包括第一参考预测值和第二参考预测值,在编码过程中存在的权重列表中包括第一权重列表和第二权重列表。编码设备可比较第一参考预测值的重要性度量值与第二参考预测值的重要性度量值之间的大小;若确定第一参考预测值的重要性度量值大于第二参考预测值的重要性度量值,则将第一权重列表确定为目标权重列表;若确定第一参考预测值的重要性度量值小于或等于第二参考预测值的重要性度量值,则将第二权重列表确定为目标权重列表。
可选地,第一权重列表中的权重值与第二权重列表中的权重值相反。
(3)利用数学符号函数、参考预测值的重要性度量值来确定目标权重列表。
当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;在编码过程中存在的权重列表中包括第一权重列表、第二权重列表和第三权重列表。编码设备可调用数学符号函数对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值之间的差值进行处理,得到符号值。若上述符号值为第一预设值(如-1),则将第一权重列表确定为目标权重列表;若上述符号值为第二预设值(如0),则将第二权重列表确定为目标权重列表;若上述符号值为第三预设值(如1),则将第三权重列表确定为目标权重列表。
(4)上述方式(1)(2)(3)可单独使用,也可将方式(1)(2)(3)组合来确定目标权重列表。作为一种实现方式,设有M+1个权重列表,采用方式(1)可以将重要性差异的绝对值所在的阈值区间对应的权重列表确定为候选权重列表,然后,N个参考预测值中包含第一参考预测值和第二预测值,采用方式(2)比对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值的大小,若确定第一参考预测值的重要性度量值大于第二参考预测值的重要性度量值,则将候选权重列表直接确定为目标权重列表;若确定第一参考预测值的重要性度量值小于或等于第二参考预测值的重要性度量值,则将与候选权重列表中的权重值相反的权重值所对应的权重列表确定为目标权重列表。
其中,N个参考预测值中的任一个表示为参考预测值i,参考预测值i是从参考块i导出得到的,参考块i所在的视频帧为参考帧i;i为整数且i小于等于N;当前块所在的视频帧为当前帧;参考预测值i的重要性度量值可以采用以下任一种方法确定:
方法一:根据当前帧在视频中的帧显示顺序(POC)及参考帧i在视频中的帧显示顺序计算得到。具体的,编码设备可计算当前帧在视频中的帧显示顺序及参考帧i在视频中的帧显示顺序之间的差值,并将该差值的绝对值作为参考预测值i的重要性度量值。例如,设当前帧在视频中的帧显示序号表示为cur_poc,参考帧i在视频中的帧显示顺序为ref_poc;参考预
测值i的重要性度量值D=abs(cur_poc-ref_poc),其中,abs()表示取绝对值。
方法二:根据当前帧在视频中的帧显示顺序、参考帧i在视频中的帧显示顺序及质量度量Q计算得到。其中,质量度量Q可以根据多种情况确定,本申请并不对此进行限定,具体地,参考帧i的质量度量Q可根据当前块的量化信息导出,例如,质量度量Q可以设置为参考帧i的base_qindex(基础量化索引)任意参考帧的base_qindex可以不同或者可以相同。在另一种实现中,参考帧i的质量度量Q还可根据其他编码信息导出,例如:参考帧i的质量度量Q可以依据参考帧i中已编码CU与当前帧中的已编码CU之间的编码信息差异导出。
作为一种实现方式,编码设备可计算当前帧在视频中的帧显示顺序及参考帧i在视频中的帧显示顺序之间的差值;然后,采用目标函数,根据上述差值、质量度量Q以及重要性度量值列表确定参考预测值i的重要性度量值。
在一个实施例中,上述参考帧i的重要性度量值可根据参考帧i与当前帧之间的方位关系及质量度量Q计算得到。作为一种实现方式,可建立参考方位关系和参考重要性度量值之间的对应关系,例如,若参考方位关系为参考帧在当前帧之前,则参考重要性度量值可以对应第一取值;若参考方位关系为参考帧在当前帧之后,则参考重要性度量值可以对应第二取值。然后,编码设备可根据参考帧i对应的参考重要性度量值和质量度量Q计算得到参考预测值i的重要性度量值。
方法三:根据方法一的计算结果和方法二的计算结果计算参考帧i的重要度量分数,并按照从小到大的顺序对N个参考预测值对应的参考帧的重要度量分数进行排序,将参考帧i在排序中的索引确定为参考预测值i的重要性度量值。
作为一种实现方式,通过上述方法一可计算得到参考预测值i的第一重要性度量值,以及上述方法二可计算得到参考预测值i的第二重要性度量值,然后根据第一重要性度量值和第二重要性度量值计算参考帧i的重要度量分数(如score),然后按照从小到大的顺序对N个参考预测值对应的参考帧的重要度量分数进行排序,并将参考帧i在排序中的索引确定为参考预测值i的重要性度量值。
其中,根据第一重要性度量值和第二重要性度量值计算参考帧i的重要度量分数可以有如下方式:①对第一重要性度量值和第二重要性度量值进行加权求和,得到参考预测值i的重要性度量值。②对第一重要性度量值和第二重要性度量值进行平均处理,得到参考预测值i的重要性度量值。
方法四:为了获取比较准确的重要性度量值,可基于参考预测值i的预测模式对方法一、方法二或方法三的计算结果进行调整,得到参考预测值i的重要性度量值;其中,参考预测值i的预测模式包括以下任一种:帧间预测模式、帧内预测模式。
应理解的是,在实际过程中,可根据需求采用上述方法一至方法四中的任一种方法确定参考预测值的重要性度量值,本申请对此不做限定。
s42、从目标权重列表中选择用于加权预测的目标权重组。
其中,目标权重列表中包含一个或多个权重组。
(1)当目标权重列表中包含的权重组数等于1时,此时直接将目标权重列表中的权重组作为用于加权预测的目标权重组。
(2)目标权重列表中包含多个权重组,从目标权重列表中选择用于加权预测的目标权重组。
在进行加权预测处理时,可能会出现码率消耗的情况,还可能会出现编码块在编码过程中产生质量损失的情况,因此编码设备可先获取采用目标权重列表中的各个权重组分别对N个参考预测值进行加权预测处理时的编码性能,然后将目标权重列表中编码性能最优的权重组确定为目标权重组。作为一种实现方式,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是目标权重列表中所有权重组对应的消耗码率中的最小值;或者,基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是目标权重列表中所有权重组对应的质量损失中的最小值;或者,采用
目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是目标权重列表中所有权重组对应的消耗码率中的最小值,并且基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是目标权重列表中所有权重组对应的质量损失中的最小值。编码设备可以将编码性能最优的权重组确定为目标权重组,按照目标权重组针对当前进行编码可以最优的编码性能,从而提高了视频的编码性能。
应理解的是,在编码侧选择用于加权预测的目标权重组时,需要不断从目标权重列表中尝试各权重组,从而获取用于加权预测的目标权重组。而在解码侧,则无需进行不断尝试,而是在视频码流中会指示使用的目标权重组的索引,解码侧只需要从视频码流中解码出目标权重组的索引,并根据目标权重组的索引从目标权重列表中找到目标权重组即可。编码设备根据N个参考预测值的重要性确定目标权重列表,并从目标权重列表中选择用于加权预测的目标权重组,通过对权重列表进行筛选,可以选择合适的权重组针对当前块进行重建,能够提高当前块的预测准确性,从而提高编码性能。
S404、基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值。
其中,步骤S404的具体实现方式可参见上述步骤S303的具体实现方式。
S405、基于当前块的预测值对视频进行编码生成视频码流。编码设备可基于当前块的预测值、目标权重组的索引对视频进行编码生成视频码流,其中,基于当前块的预测值、目标权重组的索引对视频进行编码生成视频码流可参见上述相应编码描述。
在本申请实施例中,编码设备对视频中的当前帧进行划分处理,得到当前块,当前块是指视频中正在被编码的编码块;对当前块进行复合预测,得到当前块的N个参考预测值;根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组;基于目标权重组对N个参考预测值进行加权预测处理,得到当前块的预测值;基于当前块的预测值对视频进行编码生成视频码流。在视频的编码过程中使用复合预测,并且在复合预测中导出的参考预测值的重要性,实现在复合预测中基于各参考预测值的重要性为当前块自适应选择合适的权重进行加权预测,可以提高当前块的预测准确性,从而提高编解码性能。
请参见图5,图5是本申请实施例提供的一种视频处理装置的结构示意图,该视频处理装置可以设置于本申请实施例提供的计算机设备中,计算机设备可以是上述方法实施例中提及的解码设备。图5所示的视频处理装置可以是运行于计算机设备中的一个计算机可读指令(包括程序代码),该视频处理装置可以用于执行图3所示的方法实施例中的部分或全部步骤。请参见图5,该视频处理装置可以包括如下单元:
处理单元501,用于对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;当前块是视频码流中待进行解码处理的编码块;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频码流中针对当前块进行解码时所参考的编码块;参考预测值与参考块之间一一对应;
确定单元502,用于根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的解码性能的影响程度;及
处理单元501,还用于基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像。
在一个实施例中,参考块所在的视频帧为参考帧,当前块所在的视频帧为当前帧;N个参考块与当前块之间的位置关系包括以下任一种:N个参考块分别位于N个参考帧中,且N个参考帧与当前帧属于视频码流中不同的视频帧;N个参考块位于同一个参考帧中,且同一个参考帧与当前帧属于视频码流中不同的视频帧;N个参考块中存在一个或多个参考块位于当前帧中,N个参考块中剩余参考块位于一个或多个参考帧中,且一个或多个参考帧与当前帧属于视频码流中不同的视频帧;及N个参考块与当前块均位于当前帧中。
在一个实施例中,处理单元501,还用于:确定自适应加权预测的条件;及若当前块满足
自适应加权预测的条件,则根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组。
在一个实施例中,当前块满足自适应加权预测的条件包括以下至少一种:当前块所属的帧序列的序列头中包含第一指示字段,且第一指示字段指示帧序列中的编码块允许使用自适应加权预测;帧序列,是由视频码流中各视频帧构成的序列;当前块所属的当前片的片头中包含第二指示字段,且第二指示字段指示当前片中的编码块允许使用自适应加权预测;当前片是当前块所属的图像分片,图像分片是由当前块所在的当前帧划分得到的;所述当前块所在的当前帧的帧头中包含第三指示字段,且所述第三指示字段指示所述当前帧中的编码块允许使用自适应加权预测;复合预测过程中,当前块使用至少两个参考帧进行帧间预测;复合预测过程中,当前块使用至少一个参考帧进行帧间预测,并使用当前帧进行帧内预测;当前块的运动类型为指定的运动类型;当前块使用预设的运动矢量预测模式;当前块使用预设的插值滤波器;当前块不使用特定的编码工具;及当前块在复合预测过程中所使用的参考帧满足特定条件,特定条件包括以下一种或多种:所使用的参考帧与当前帧之间在视频码流中的方位关系满足预设关系;所使用的参考帧对应的参考预测值之间的重要性差异的绝对值大于或等于预设阈值;其中,方位关系满足预设关系包括以下任一种:所使用的参考帧均位于当前帧之前;所使用的参考帧均位于当前帧之后;所使用的部分参考帧位于当前帧之前,且其余参考帧位于当前帧之后。
在一个实施例中,视频码流中包含一个或多个权重列表,每个权重列表中包含一个或多个权重组,每个权重组中包含一个或多个权重值,每个权重组中的权重值的数量允许相同或不同,且每个权重组中包含的权重值的取值允许相同或不同;确定单元502,可具体用于:
根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表;及从目标权重列表中选择用于加权预测的目标权重组。
在一个实施例中,视频码流中权重列表的数量为M+1个,一个权重列表对应一个阈值区间,M为大于等于1的正整数;确定单元502,可具体用于:获取N个参考预测值的重要性度量值,并计算N个参考预测值之间的重要性差异,任意两个参考预测值之间的重要性差异采用任意两个参考预测值的重要性度量值之间的差值进行衡量;确定N个参考预测值之间的重要性差异的绝对值所在的阈值区间;及将N个参考预测值之间的重要性差异的绝对值所在的阈值区间对应的权重列表确定为目标权重列表。
在一个实施例中,当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;视频码流中包括第一权重列表和第二权重列表;确定单元502,可具体用于:比较第一参考预测值的重要性度量值与第二参考预测值的重要性度量值之间的大小;若第一参考预测值的重要性度量值大于第二参考预测值的重要性度量值,则将第一权重列表确定为目标权重列表;及若第一参考预测值的重要性度量值小于或等于第二参考预测值的重要性度量值,则将第二权重列表确定为目标权重列表;其中,第一权重列表中与第二权重列表中处于相同次序的权重值之和为1。
在一个实施例中,当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;视频码流中包括第一权重列表、第二权重列表和第三权重列表;确定单元502,可具体用于:调用数学符号函数对第一参考预测值的重要性度量值和第二参考预测值的重要性度量值之间的差值进行处理,得到符号值;若符号值为第一预设值,则将第一权重列表确定为目标权重列表;若符号值为第二预设值,则将第二权重列表确定为目标权重列表;及若符号值为第三预设值,则将第三权重列表确定为目标权重列表;其中,第一权重列表、第二权重列表和第三权重列表分别为不同的权重列表;或者,第一权重列表、第二权重列表和第三权重列表中允许两个权重列表为相同的权重列表。
在一个实施例中,N个参考预测值中的任一个表示为参考预测值i,参考预测值i是从参考块i导出得到的,参考块i所在的视频帧为参考帧i;i为整数且i小于等于N;当前块所在的视频帧为当前帧;参考预测值i的重要性度量值采用以下任一种方法确定:方法一:根据
当前帧在视频码流中的帧显示顺序及参考帧i在视频码流中的帧显示顺序计算得到;方法二:根据参考帧i在视频码流中的帧显示顺序及质量度量Q、当前帧在视频码流中的帧显示顺序计算得到;方法三:根据方法一的计算结果和方法二的计算结果计算参考帧i的重要度量分数,按照从小到大的顺序对N个参考预测值对应的参考帧的重要度量分数进行排序,将参考帧i在排序中的索引确定为参考预测值i的重要性度量值;及方法四:基于参考预测值i的预测模式对方法一、方法二或方法三的计算结果进行调整,得到参考预测值i的重要性度量值;其中,参考预测值i的预测模式包括以下任一种:帧间预测模式、帧内预测模式。
在一个实施例中,目标权重列表中包含的权重组数大于1;确定单元502,可具体用于:从视频码流中解码出用于加权预测的目标权重组的索引;及按照目标权重组的索引,从目标权重列表中选择目标权重组;其中,目标权重组的索引使用截断一元码进行二值化编码的编码方式,或者使用多符号的熵编码方式。
在一个实施例中,处理单元501,可具体用于:采用目标权重组中的权重值分别对N个参考预测值进行加权求和处理,得到当前块的预测值;或者,按照整型计算的形式采用目标权重组中的权重值分别对N个参考预测值进行加权处理,得到当前块的预测值。
在本申请实施例中,对视频码流中的当前块进行复合预测,得到当前块的N个参考预测值,当前块是指视频码流中正在被解码的编码块,根据N个参考预测值的重要性,为当前块自适应选择用于加权预测的目标权重组;基于目标权重组对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像。在视频的解码过程中使用复合预测,并且在复合预测中充分考虑参考预测值的重要性,实现在复合预测中基于各参考预测值的重要性为当前块确定合适的权重进行加权预测,可以提高当前块的预测准确性,从而提高编解码性能。
请参见图6,图6是本申请实施例提供的一种视频处理装置的结构示意图,该视频处理装置可以设置于本申请实施例提供的计算机设备中,计算机设备可以是上述方法实施例中提及的编码设备。图6所示的视频处理装置可以是运行于计算机设备中的一个计算机可读指令(包括程序代码),该视频处理装置可以用于执行图4所示的方法实施例中的部分或全部步骤。请参见图6,该视频处理装置可以包括如下单元:
处理单元601,用于对视频中的当前帧进行划分处理,得到当前块;
处理单元601,还用于对当前块进行复合预测,得到当前块的N个参考预测值,N为大于1的整数;N个参考预测值是从当前块的N个参考块中导出得到的;N个参考块是视频中针对当前块进行编码时所参考的编码块;参考预测值与参考块之间一一对应;
确定单元602,用于根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组,目标权重组包含一个或多个权重值;重要性用于表征相应参考预测值对当前块的编码性能的影响程度;
处理单元601,还用于基于目标权重组中的权重值对N个参考预测值进行加权预测处理,得到当前块的预测值,当前块的预测值用于重建当前块对应的解码图像;及
处理单元601,还用于基于当前块的预测值对视频进行编码生成视频码流。
在一个实施例中,确定单元602,可具体用于:根据N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表;每个权重列表中包含一个或多个权重组,每个权重组中包含一个或多个权重值,每个权重组中包含的权重值的数量允许相同或不同,且每个权重组中包含的权重值的取值允许相同或不同;及从目标权重列表中选择用于加权预测的目标权重组。
在一个实施例,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值;或者,基于目标权重组中的权重值对N个参考预测值进行加权预测处理,使得当前块在编码过程中的质量损失小于预设损失阈值;或者,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值,并且基于目标权重组中的权重值对N个参考预测值进行加权预测处理,使得当前块在编码过程中的
质量损失小于预设损失阈值。
在一个实施例中,目标权重组是目标权重列表中编码性能最优的权重组;其中,编码性能最优包括:采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是目标权重列表中所有权重组对应的消耗码率中的最小值;或者,基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是目标权重列表中所有权重组对应的质量损失中的最小值;或者,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是目标权重列表中所有权重组对应的消耗码率中的最小值,并且基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是目标权重列表中所有权重组对应的质量损失中的最小值。
在本申请实施例中,对视频中的当前帧进行划分处理,得到当前块,当前块是指视频中正在被编码的编码块;对当前块进行复合预测,得到当前块的N个参考预测值;根据N个参考预测值的重要性,为当前块确定用于加权预测的目标权重组;基于目标权重组对N个参考预测值进行加权预测处理,得到当前块的预测值;基于当前块的预测值对视频进行编码生成视频码流。在视频的编码过程中使用复合预测,并且在充分考虑参考预测值的重要性,实现在复合预测中基于各参考预测值的重要性为当前块确定合适的权重进行加权预测,可以提高当前块的预测准确性,从而提高编解码性能。
进一步地,本申请实施例还提供了一种计算机设备的结构示意图,该计算机设备的结构示意图可参见图7;该计算机设备可以是上述编码设备或者解码设备;该计算机设备可以包括:处理器701、输入设备702,输出设备703和存储器704。上述处理器701、输入设备702、输出设备703和存储器704通过总线连接。存储器704用于存储计算机可读指令,计算机可读指令包括程序指令,处理器701用于执行存储器704存储的程序指令。
当计算机设备为上述解码设备时,在本申请实施例中,处理器701通过运行存储器704中的可执行程序代码,执行如上述解码设备执行的与解码相关的视频处理方法的各步骤。
可选地,当计算机设备为上述编码设备时,在本申请实施例中,处理器701通过运行存储器704中的可执行程序代码,执行上述编码设备执行的与编码相关的视频处理方法的各步骤。
此外,这里需要指出的是:本申请实施例还提供了一种计算机可读存储介质,且计算机可读存储介质中存储有计算机可读指令,且该计算机可读指令包括程序指令,当处理器执行上述程序指令时,能够执行前文图3和图4所对应实施例中的方法,因此,这里将不再进行赘述。对于本申请所涉及的计算机可读存储介质实施例中未披露的技术细节,请参照本申请方法实施例的描述。作为示例,程序指令可以被部署在一个计算机设备上,或者在位于一个地点的多个计算机设备上执行,又或者,在分布在多个地点且通过通信网络互连的多个计算机设备上执行。
根据本申请的一个方面,提供了一种计算机程序产品,该计算机程序产品包括计算机可读指令,该计算机可读指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机可读指令,处理器执行该计算机可读指令,使得该计算机设备可以执行前文图3和图4所对应实施例中的方法,因此,这里将不再进行赘述。本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机可读指令来指令相关的硬件来完成,所述的计算机可读指令可存储于一计算机可读取存储介质中,该程序在执行时,可包括如上述各方法的实施例的流程。其中,所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)或随机存储记忆体(Random Access Memory,RAM)等。
以上所揭露的仅为本申请一种较佳实施例而已,当然不能以此来限定本申请之权利范围,本领域普通技术人员可以理解实现上述实施例的全部或部分流程,并依本申请权利要求所作的等同变化,仍属于发明所涵盖的范围。
Claims (20)
- 一种视频处理方法,由计算机设备执行,其特征在于,包括:对视频码流中的当前块进行复合预测,得到所述当前块的N个参考预测值,N为大于1的整数;所述当前块是所述视频码流中待进行解码处理的编码块;所述N个参考预测值是从所述当前块的N个参考块中导出得到的;所述N个参考块是所述视频码流中针对所述当前块进行解码时所参考的编码块;所述参考预测值与所述参考块之间一一对应;根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,所述目标权重组包含一个或多个权重值;所述重要性用于表征相应参考预测值对所述当前块的解码性能的影响程度;及基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,得到所述当前块的预测值,所述当前块的预测值用于重建所述当前块对应的解码图像。
- 如权利要求1所述的方法,其特征在于,参考块所在的视频帧为参考帧,所述当前块所在的视频帧为当前帧;所述N个参考块与所述当前块之间的位置关系包括以下任一种:所述N个参考块分别位于N个参考帧中,且所述N个参考帧与所述当前帧属于所述视频码流中不同的视频帧;所述N个参考块位于同一个参考帧中,且所述同一个参考帧与所述当前帧属于所述视频码流中不同的视频帧;所述N个参考块中存在一个或多个参考块位于所述当前帧中,所述N个参考块中剩余参考块位于一个或多个参考帧中,且所述一个或多个参考帧与所述当前帧属于所述视频码流中不同的视频帧;及所述N个参考块与所述当前块均位于所述当前帧中。
- 如权利要求1所述的方法,其特征在于,所述方法还包括:确定自适应加权预测的条件;及若所述当前块满足所述自适应加权预测的条件,则执行根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组的步骤。
- 如权利要求3所述的方法,其特征在于,所述当前块满足所述自适应加权预测的条件包括以下至少一种:所述当前块所属的帧序列的序列头中包含第一指示字段,且所述第一指示字段指示所述帧序列中的编码块允许使用自适应加权预测;所述帧序列,是由所述视频码流中各视频帧构成的序列;所述当前块所属的当前片的片头中包含第二指示字段,且所述第二指示字段指示所述当前片中的编码块允许使用自适应加权预测;所述当前片是所述当前块所属的图像分片,所述图像分片是由所述当前块所在的当前帧划分得到的;所述当前块所在的当前帧的帧头中包含第三指示字段,且所述第三指示字段指示所述当前帧中的编码块允许使用自适应加权预测;所述复合预测过程中,所述当前块使用至少两个参考帧进行帧间预测;所述复合预测过程中,所述当前块使用至少一个参考帧进行帧间预测,并使用所述当前帧进行帧内预测;所述当前块的运动类型为指定的运动类型;所述当前块使用预设的运动矢量预测模式;所述当前块使用预设的插值滤波器;所述当前块不使用特定的编码工具;及所述当前块在所述复合预测过程中所使用的参考帧满足特定条件,所述特定条件包括以下一种或多种:所使用的参考帧与所述当前帧之间在所述视频码流中的方位关系满足预设关系;所使用的参考帧对应的参考预测值之间的重要性差异的绝对值大于或等于预设阈值;其中,所述方位关系满足预设关系包括以下任一种:所使用的参考帧均位于所述当前帧之前;所使用的参考帧均位于所述当前帧之后;所使用的部分参考帧位于所述当前帧之前,且其余参考帧位于所述当前帧之后。
- 如权利要求1所述的方法,其特征在于,所述视频码流中包含一个或多个权重列表,每个权重列表中包含一个或多个权重组,每个权重组中包含一个或多个权重值,每个权重组中包含的权重值的数量允许相同或不同,且每个权重组中包含的权重值的取值允许相同或不同;所述根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,包括:根据所述N个参考预测值的重要性,从所述一个或多个权重列表中确定目标权重列表;及从所述目标权重列表中选择用于加权预测的目标权重组。
- 如权利要求5所述的方法,其特征在于,所述视频码流中权重列表的数量为M+1个,一个权重列表对应一个阈值区间,M为大于等于1的正整数;所述根据所述N个参考预测值的重要性,从所述一个或多个权重列表中确定目标权重列表,包括:获取所述N个参考预测值的重要性度量值,并计算所述N个参考预测值之间的重要性差异,任意两个参考预测值之间的重要性差异采用所述任意两个参考预测值的重要性度量值之间的差值进行衡量;确定所述N个参考预测值之间的重要性差异的绝对值所在的阈值区间;及将所述N个参考预测值之间的重要性差异的绝对值所在的阈值区间对应的权重列表确定为目标权重列表。
- 如权利要求5所述的方法,其特征在于,所述当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;所述视频码流中包括第一权重列表和第二权重列表;所述根据所述N个参考预测值的重要性,从所述一个或多个权重列表中确定目标权重列表,包括:比较所述第一参考预测值的重要性度量值与所述第二参考预测值的重要性度量值之间的大小;若所述第一参考预测值的重要性度量值大于所述第二参考预测值的重要性度量值,则将所述第一权重列表确定为目标权重列表;及若所述第一参考预测值的重要性度量值小于或等于所述第二参考预测值的重要性度量值,则将所述第二权重列表确定为目标权重列表;其中,所述第一权重列表中与所述第二权重列表中处于相同次序的权重值之和为1。
- 如权利要求5所述的方法,其特征在于,所述当前块的N个参考预测值中包括第一参考预测值和第二参考预测值;所述视频码流中包括第一权重列表、第二权重列表和第三权重列表;所述根据所述N个参考预测值的重要性,从所述一个或多个权重列表中确定目标权重列表,包括:调用数学符号函数对所述第一参考预测值的重要性度量值和所述第二参考预测值的重要性度量值之间的差值进行处理,得到符号值;若所述符号值为第一预设值,则将所述第一权重列表确定为目标权重列表;若所述符号值为第二预设值,则将所述第二权重列表确定为目标权重列表;及若所述符号值为第三预设值,则将所述第三权重列表确定为目标权重列表;其中,所述第一权重列表、所述第二权重列表和所述第三权重列表分别为不同的权重列表;或者,所述第一权重列表、所述第二权重列表和所述第三权重列表中允许两个权重列表为相同的权重列表。
- 如权利要求6-8任一项所述的方法,其特征在于,所述N个参考预测值中的任一个表示为参考预测值i,所述参考预测值i是从参考块i导出得到的,所述参考块i所在的视频帧为参考帧i;i为整数且i小于等于N;所述当前块所在的视频帧为当前帧;所述参考预测值i的重要性度量值采用以下任一种方法确定:方法一:根据所述当前帧在所述视频码流中的帧显示顺序及所述参考帧i在所述视频码流中的帧显示顺序计算得到;方法二:根据所述参考帧i在所述视频码流中的帧显示顺序及质量度量Q、所述当前帧在所述视频码流中的帧显示顺序计算得到;所述质量度量Q根据所述当前块的量化信息或所述参考帧i的编码信息确定;方法三:根据方法一的计算结果和方法二的计算结果计算所述参考帧i的重要度量分数,按照从小到大的顺序对所述N个参考预测值对应的参考帧的重要度量分数进行排序,将所述参考帧i在所述排序中的索引确定为所述参考预测值i的重要性度量值;及方法四:基于所述参考预测值i的预测模式对方法一、方法二或方法三的计算结果进行调整,得到所述参考预测值i的重要性度量值;其中,所述参考预测值i的预测模式包括以下任一种:帧间预测模式、帧内预测模式。
- 如权利要求5所述的方法,其特征在于,所述目标权重列表中包含的权重组数大于1;所述从所述目标权重列表中选择用于加权预测的目标权重组,包括:从所述视频码流中解码出用于加权预测的目标权重组的索引;及按照所述目标权重组的索引,从所述目标权重列表中选择所述目标权重组;其中,所述目标权重组的索引使用截断一元码进行二值化编码的编码方式,或者使用多符号的熵编码方式。
- 如权利要求1所述的方法,其特征在于,所述基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,得到所述当前块的预测值,包括:采用所述目标权重组中的权重值分别对所述N个参考预测值进行加权求和处理,得到所述当前块的预测值;或者,按照整型计算的形式采用所述目标权重组中的权重值分别对所述N个参考预测值进行加权处理,得到所述当前块的预测值。
- 一种视频处理方法,由计算机设备执行,其特征在于,包括:对视频中的当前帧进行划分处理,得到当前块;对所述当前块进行复合预测,得到所述当前块的N个参考预测值,N为大于1的整数;所述N个参考预测值是从所述当前块的N个参考块中导出得到的;所述N个参考块是所述视频中针对所述当前块进行编码时所参考的编码块;所述参考预测值与所述参考块之间一一对应;根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,所述目标权重组包含一个或多个权重值;所述重要性用于表征相应参考预测值对所述当前块的编码性能的影响程度;基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,得到所述当前块的预测值,所述当前块的预测值用于重建所述当前块对应的解码图像;及基于所述当前块的预测值对所述视频进行编码生成视频码流。
- 如权利要求12所述的方法,其特征在于,所述根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,包括:根据所述N个参考预测值的重要性,从一个或多个权重列表中确定目标权重列表;每个权重列表中包含一个或多个权重组,每个权重组中包含一个或多个权重值,每个权重组中包含的权重值的数量允许相同或不同,且每个权重组中包含的权重值的取值允许相同或不同;及从所述目标权重列表中选择用于加权预测的目标权重组。
- 如权利要求13所述的方法,其特征在于,采用所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值;或者,基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,使得所述当前块在编码过程中的质量损失小于预设损失阈值;或者,采用所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理时所消耗的码率小于预设码率阈值,并且基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,使得所述当前块在编码过程中的质量损失小于预设损失阈值。
- 如权利要求13所述的方法,其特征在于,所述目标权重组是所述目标权重列表中编码性能最优的权重组;其中,所述编码性能最优包括:采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是所述目标权重列表中所有权重组对应的消耗码率中的最小值;或者,基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是所述目标权重列表中所有权重组对应的质量损失中的最小值;或者,采用目标权重组中的权重值对N个参考预测值进行加权预测处理时所消耗的码率是所述目标权重列表中所有权重组对应的消耗码率中的最小值,并且基于目标权重组中的权重值对N个参考预测值进行加权预测处理时对应的当前块在编码过程中的质量损失是所述目标权重列表中所有权重组对应的质量损失中的最小值。
- 一种视频处理装置,其特征在于,包括:处理单元,用于对视频码流中的当前块进行复合预测,得到所述当前块的N个参考预测值,N为大于1的整数;所述当前块是所述视频码流中待进行解码处理的编码块;所述N个参考预测值是从所述当前块的N个参考块中导出得到的;所述N个参考块是所述视频码流中针对所述当前块进行解码时所参考的编码块;所述参考预测值与所述参考块之间一一对应;确定单元,用于根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,所述目标权重组包含一个或多个权重值;所述重要性用于表征相应参考预测值对所述当前块的解码性能的影响程度;及所述处理单元,还用于基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,得到所述当前块的预测值,所述当前块的预测值用于重建所述当前块对应的解码图像。
- 一种视频处理装置,其特征在于,包括:处理单元,用于对视频中的当前帧进行划分处理,得到当前块;所述处理单元,还用于对所述当前块进行复合预测,得到所述当前块的N个参考预测值,N为大于1的整数;所述N个参考预测值是从所述当前块的N个参考块中导出得到的;所述N个参考块是所述视频中针对所述当前块进行编码时所参考的编码块;所述参考预测值与所述参考块之间一一对应;确定单元,用于根据所述N个参考预测值的重要性,为所述当前块确定用于加权预测的目标权重组,所述目标权重组包含一个或多个权重值;所述重要性用于表征相应参考预测值对所述当前块的编码性能的影响程度;所述处理单元,还用于基于所述目标权重组中的权重值对所述N个参考预测值进行加权预测处理,得到所述当前块的预测值,所述当前块的预测值用于重建所述当前块对应的解码图像;及所述处理单元,还用于基于所述当前块的预测值对所述视频进行编码生成视频码流。
- 一种计算机设备,其特征在于,包括:处理器,适于执行计算机可读指令;计算机可读存储介质,所述计算机可读存储介质中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时,执行如权利要求1-15任一项所述的视频处理方法。
- 一种计算机可读存储介质,其特征在于,所述计算机存储介质存储有计算机可读指令,所述计算机可读指令被处理器执行时,执行如权利要求1-15任一项所述的视频处理方法。
- 一种计算机程序产品,其特征在于,所述计算机程序产品包括计算机可读指令,所述计算机可读指令被处理器执行时实现如权利要求1-15中任一项所述的视频处理方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211230593.1A CN115633168A (zh) | 2022-09-30 | 2022-09-30 | 一种视频处理方法及相关设备 |
CN202211230593.1 | 2022-09-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024066649A1 true WO2024066649A1 (zh) | 2024-04-04 |
Family
ID=84904297
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/106374 WO2024066649A1 (zh) | 2022-09-30 | 2023-07-07 | 一种视频处理方法及相关设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN115633168A (zh) |
WO (1) | WO2024066649A1 (zh) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115633168A (zh) * | 2022-09-30 | 2023-01-20 | 腾讯科技(深圳)有限公司 | 一种视频处理方法及相关设备 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102307303A (zh) * | 2011-08-24 | 2012-01-04 | 北京航空航天大学 | 一种基于三进制表示的图像预测编码方法 |
CN102695061A (zh) * | 2011-03-20 | 2012-09-26 | 华为技术有限公司 | 一种权重因子的确定方法和装置,以及一种帧内加权预测方法和装置 |
KR102140398B1 (ko) * | 2019-05-27 | 2020-07-31 | 한국항공대학교산학협력단 | 영상 처리 장치 및 영상 처리 방법 |
CN113099229A (zh) * | 2021-02-22 | 2021-07-09 | 浙江大华技术股份有限公司 | 块划分方法、帧间预测方法、视频编码方法及相关装置 |
CN113452997A (zh) * | 2020-03-25 | 2021-09-28 | 杭州海康威视数字技术股份有限公司 | 一种编解码方法、装置及其设备 |
CN114885160A (zh) * | 2021-02-05 | 2022-08-09 | 腾讯科技(深圳)有限公司 | 视频编解码方法、装置、计算机可读介质及电子设备 |
CN115633168A (zh) * | 2022-09-30 | 2023-01-20 | 腾讯科技(深圳)有限公司 | 一种视频处理方法及相关设备 |
-
2022
- 2022-09-30 CN CN202211230593.1A patent/CN115633168A/zh active Pending
-
2023
- 2023-07-07 WO PCT/CN2023/106374 patent/WO2024066649A1/zh unknown
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102695061A (zh) * | 2011-03-20 | 2012-09-26 | 华为技术有限公司 | 一种权重因子的确定方法和装置,以及一种帧内加权预测方法和装置 |
CN102307303A (zh) * | 2011-08-24 | 2012-01-04 | 北京航空航天大学 | 一种基于三进制表示的图像预测编码方法 |
KR102140398B1 (ko) * | 2019-05-27 | 2020-07-31 | 한국항공대학교산학협력단 | 영상 처리 장치 및 영상 처리 방법 |
CN113452997A (zh) * | 2020-03-25 | 2021-09-28 | 杭州海康威视数字技术股份有限公司 | 一种编解码方法、装置及其设备 |
CN114885160A (zh) * | 2021-02-05 | 2022-08-09 | 腾讯科技(深圳)有限公司 | 视频编解码方法、装置、计算机可读介质及电子设备 |
CN113099229A (zh) * | 2021-02-22 | 2021-07-09 | 浙江大华技术股份有限公司 | 块划分方法、帧间预测方法、视频编码方法及相关装置 |
CN115633168A (zh) * | 2022-09-30 | 2023-01-20 | 腾讯科技(深圳)有限公司 | 一种视频处理方法及相关设备 |
Also Published As
Publication number | Publication date |
---|---|
CN115633168A (zh) | 2023-01-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11523133B2 (en) | Image encoding method using a skip mode, and a device using the method | |
CN111742554B (zh) | 具有视频数据自适应量化的视频编码或解码方法和装置 | |
KR101647376B1 (ko) | 비디오 신호 처리 방법 및 장치 | |
US20210368200A1 (en) | Methods and devices for selectively applying bi-directional optical flow and decoder-side motion vector refinement for video coding | |
JP7425118B2 (ja) | ビデオ復号化方法、およびプログラム | |
US20210235091A1 (en) | Method and apparatus for determining reference picture set of image | |
US20130022117A1 (en) | Temporal block merge mode | |
KR20130004173A (ko) | 비디오 부호화 방법 및 복호화 방법과 이를 이용한 장치 | |
KR20100087600A (ko) | 보간 필터를 적응적으로 사용하여 영상을 부호화 및 복호화하는 방법 및 장치 | |
CN114424567B (zh) | 使用基于矩阵的帧内预测进行组合的帧间-帧内预测的方法和装置 | |
WO2024066649A1 (zh) | 一种视频处理方法及相关设备 | |
KR20100079037A (ko) | 비디오 영상 부호화 방법 및 장치 | |
KR20150086793A (ko) | 변환 깊이 기반 dct 사이즈 결정 장치 및 방법 | |
CN111010578B (zh) | 一种帧内帧间联合预测的方法、装置以及存储介质 | |
CN110944179B (zh) | 一种视频数据解码方法及装置 | |
US11064201B2 (en) | Method and device for image encoding/decoding based on effective transmission of differential quantization parameter | |
US20240155134A1 (en) | Method and apparatus for video coding using improved cross-component linear model prediction | |
KR100657277B1 (ko) | 매크로블록 타입 결정 방법 | |
US20230199196A1 (en) | Methods and Apparatuses of Frequency Domain Mode Decision in Video Encoding Systems | |
WO2013005966A2 (ko) | 비디오 부호화 방법 및 복호화 방법과 이를 이용한 장치 | |
CN113709454A (zh) | 视频编码方法、模型训练方法及相关装置 | |
Yang et al. | A Fast Inter Mode Decision Algorithm for HEVC Encode System based on Spatial and Temporal Correlations | |
JP2024099849A (ja) | ビデオ符号化のための双方向オプティカルフローおよびデコーダ側動きベクトル補正を選択的に適用する方法およびデバイス | |
KR20180046217A (ko) | 계산-율-왜곡을 최적화하는 방법 및 장치, 그리고 이를 이용해 영상 프레임을 부호화하는 방법 및 장치 | |
KR20130050534A (ko) | 하다마드 변환을 이용한 부호화 방법 및 이러한 방법을 사용하는 장치 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23869896 Country of ref document: EP Kind code of ref document: A1 |