WO2021203394A1 - 环路滤波的方法与装置 - Google Patents

环路滤波的方法与装置 Download PDF

Info

Publication number
WO2021203394A1
WO2021203394A1 PCT/CN2020/084071 CN2020084071W WO2021203394A1 WO 2021203394 A1 WO2021203394 A1 WO 2021203394A1 CN 2020084071 W CN2020084071 W CN 2020084071W WO 2021203394 A1 WO2021203394 A1 WO 2021203394A1
Authority
WO
WIPO (PCT)
Prior art keywords
ccalf
filter coefficients
alf
filter
image block
Prior art date
Application number
PCT/CN2020/084071
Other languages
English (en)
French (fr)
Inventor
马思伟
孟学苇
郑萧桢
王苫社
Original Assignee
北京大学
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学, 深圳市大疆创新科技有限公司 filed Critical 北京大学
Priority to CN202080004298.7A priority Critical patent/CN112514401A/zh
Priority to PCT/CN2020/084071 priority patent/WO2021203394A1/zh
Publication of WO2021203394A1 publication Critical patent/WO2021203394A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness

Definitions

  • the present invention relates to the technical field of digital video coding, and more specifically, to a method and device for loop filtering.
  • the video coding compression process includes: block division, prediction, transformation, quantization, and entropy coding processes to form a hybrid video coding framework.
  • the video coding and decoding technology standards have gradually formed.
  • some mainstream video coding and decoding standards include: international video coding standards H.264/MPEG-AVC, H. 265/MEPG-HEVC, the domestic audio and video coding standard AVS2, and the H.266/VVC international standard and AVS3 domestic standard that are being developed.
  • the loop filter includes deblocking filter (DBF), which is adaptive Sample compensation filter (Sample Adaptive Offset, SAO) and Adaptive Loop Filter (Adaptive Loop Filter, ALF). Among them, the filtering process still has room for improvement.
  • DPF deblocking filter
  • SAO sample Adaptive Offset
  • ALF Adaptive Loop Filter
  • the present invention provides a method and device for loop filtering. Compared with the prior art, the complexity of loop filtering can be reduced and the filtering effect can be improved.
  • a method for loop filtering including:
  • the luminance reconstruction pixel value before the adaptive loop filter ALF the luminance reconstruction pixel value before the adaptive loop filter ALF, the chrominance reconstruction pixel value after ALF, and the chrominance reconstruction pixel value after the cross-component adaptive loop filter CCALF Distortion of each set of filter coefficients in multiple sets of candidate filter coefficients;
  • a method of loop filtering including:
  • a method for loop filtering including:
  • a device for loop filtering including: a memory for storing codes;
  • the processor is configured to execute the code stored in the memory to perform the following operations:
  • the luminance reconstruction pixel value before the adaptive loop filter ALF the luminance reconstruction pixel value before the adaptive loop filter ALF, the chrominance reconstruction pixel value after ALF, and the chrominance reconstruction pixel value after the cross-component adaptive loop filter CCALF Distortion of each set of filter coefficients in multiple sets of candidate filter coefficients;
  • a loop filtering device including:
  • Memory used to store code
  • the processor is configured to execute the code stored in the memory to perform the following operations:
  • a loop filtering device including:
  • Memory used to store code
  • the processor is configured to execute the code stored in the memory to perform the following operations:
  • Fig. 1 is a structural diagram of a technical solution applying an embodiment of the present application.
  • Fig. 2 is a schematic diagram of a video coding framework according to an embodiment of the present application.
  • Fig. 3 is a schematic diagram of a video decoding framework according to an embodiment of the present application.
  • Fig. 4 is a schematic diagram of a Wiener filter according to an embodiment of the present application.
  • Fig. 5a is a schematic diagram of an ALF filter according to an embodiment of the present application.
  • Fig. 5b is a schematic diagram of another ALF filter according to an embodiment of the present application.
  • Fig. 6 is a schematic flowchart of a loop filtering method according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the shape of a CCALF filter according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a loop filtering method according to another embodiment of the present application.
  • Fig. 9 is a schematic flowchart of a loop filtering device according to another embodiment of the present application.
  • the embodiments of the present application can be applied to standard or non-standard image or video encoders.
  • the encoder of the VVC standard For example, the encoder of the VVC standard.
  • the size of the sequence number of each process does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not correspond to the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • Fig. 1 is a structural diagram of a technical solution applying an embodiment of the present application.
  • the system 100 can receive the data 102 to be processed, process the data 102 to be processed, and generate processed data 108.
  • the system 100 may receive the data to be encoded and encode the data to be encoded to generate encoded data, or the system 100 may receive the data to be decoded and decode the data to be decoded to generate decoded data.
  • the components in the system 100 may be implemented by one or more processors.
  • the processor may be a processor in a computing device or a processor in a mobile device (such as a drone).
  • the processor may be any type of processor, which is not limited in the embodiment of the present invention.
  • the processor may include an encoder, a decoder, or a codec, etc.
  • One or more memories may also be included in the system 100.
  • the memory can be used to store instructions and data, for example, computer-executable instructions that implement the technical solutions of the embodiments of the present invention, to-be-processed data 102, processed data 108, and so on.
  • the memory may be any type of memory, which is not limited in the embodiment of the present invention.
  • Fig. 2 is a schematic diagram of a video coding framework 2 according to an embodiment of the present application.
  • each frame of the video to be encoded is encoded in turn.
  • the current coded frame mainly undergoes processing such as prediction (Prediction), transformation (Transform), quantization (Quantization), and entropy coding (Entropy Coding), and finally the bit stream of the current coded frame is output.
  • the decoding process usually decodes the received code stream according to the inverse process of the above process to recover the video frame information before decoding.
  • the video encoding framework 2 includes an encoding control module 201, which is used to perform decision-making control actions and parameter selection in the encoding process.
  • the encoding control module 201 controls the parameters used in transformation, quantization, inverse quantization, and inverse transformation, controls the selection of intra-frame or inter-frame modes, and parameter control of motion estimation and filtering, and
  • the control parameters of the encoding control module 201 will also be input to the entropy encoding module, and the encoding will be performed to form a part of the encoded bitstream.
  • the frame to be coded is divided 202 processing, specifically, it is firstly divided into slices, and then divided into blocks.
  • the frame to be encoded is divided into a plurality of non-overlapping largest coding tree units (CTUs), and each CTU can also be divided into quad-tree, or binary tree, or tri-tree.
  • the method is iteratively divided into a series of smaller coding units (Coding Unit, CU).
  • the CU may also include a prediction unit (Prediction Unit, PU) and a transformation unit (Transform Unit, TU) associated with it.
  • PU is the basic unit of prediction
  • TU is the basic unit of transformation and quantization.
  • the PU and TU are respectively obtained by dividing into one or more blocks on the basis of the CU, where one PU includes multiple prediction blocks (PB) and related syntax elements.
  • the PU and TU may be the same, or obtained by the CU through different division methods.
  • at least two of the CU, PU, and TU are the same.
  • CU, PU, and TU are not distinguished, and prediction, quantization, and transformation are all performed in units of CU.
  • the CTU, CU, or other data units formed are all referred to as coding blocks in the following.
  • the data unit for video encoding may be a frame, a slice, a coding tree unit, a coding unit, a coding block, or any group of the above.
  • the size of the data unit can vary.
  • a prediction process is performed to remove the spatial and temporal redundant information of the current frame to be encoded.
  • predictive coding methods include intra-frame prediction and inter-frame prediction.
  • Intra-frame prediction uses only the reconstructed information in the current frame to predict the current coding block
  • inter-frame prediction uses the information in other previously reconstructed frames (also called reference frames) to predict the current coding block.
  • Make predictions Specifically, in the embodiment of the present application, the encoding control module 201 is used to make a decision to select intra prediction or inter prediction.
  • the process of intra-frame prediction 203 includes obtaining the reconstructed block of the coded neighboring block around the current coding block as a reference block, and based on the pixel value of the reference block, the prediction mode method is used to calculate the predicted value to generate the predicted block , Subtracting the corresponding pixel values of the current coding block and the prediction block to obtain the residual of the current coding block, the residual of the current coding block is transformed 204, quantized 205, and entropy coding 210 to form the code stream of the current coding block. Further, all the coded blocks of the frame to be coded currently form a part of the coded stream of the frame to be coded after undergoing the above-mentioned coding process. In addition, the control and reference data generated in the intra-frame prediction 203 are also encoded by the entropy encoding 210 to form a part of the encoded bitstream.
  • the transform 204 is used to remove the correlation of the residual of the image block, so as to improve the coding efficiency.
  • two-dimensional discrete cosine transform (DCT) transformation and two-dimensional discrete sine transform (DST) transformation are usually adopted.
  • DCT discrete cosine transform
  • DST two-dimensional discrete sine transform
  • the residual of the block to be coded is The information is respectively multiplied by an N ⁇ M transformation matrix and its transposed matrix, and the transformation coefficient of the current coding block is obtained after the multiplication.
  • the quantization 205 is used to further improve the compression efficiency.
  • the transform coefficients can be quantized to obtain the quantized coefficients, and then the quantized coefficients are entropy-encoded 210 to obtain the residual code stream of the current coding block, wherein the entropy coding method includes But it is not limited to content adaptive binary arithmetic coding (Context Adaptive Binary Arithmetic Coding, CABAC) entropy coding.
  • CABAC Context Adaptive Binary Arithmetic Coding
  • the coded neighboring block in the intra prediction 203 process is: the neighboring block that has been coded before the current coding block is coded, and the residual generated in the coding process of the neighboring block is transformed 204, quantized 205, After inverse quantization 206 and inverse transform 207, the reconstructed block is obtained by adding the prediction block of the neighboring block.
  • the inverse quantization 206 and the inverse transformation 207 are the inverse processes of the quantization 206 and the transformation 204, which are used to restore the residual data before the quantization and transformation.
  • the inter prediction process includes motion estimation 208 and motion compensation 209. Specifically, the motion estimation is performed 208 according to the reference frame image in the reconstructed video frame, and the image block most similar to the current encoding block is searched for in one or more reference frame images according to a certain matching criterion as a matching block.
  • the relative displacement with the current coding block is the motion vector (Motion Vector, MV) of the current block to be coded.
  • Motion Compensation is performed 209 on the frame to be coded based on the motion vector and the reference frame to obtain the predicted value of the frame to be coded.
  • the original value of the pixel of the frame to be coded is subtracted from the corresponding predicted value to obtain the residual of the frame to be coded.
  • the residual of the current frame to be encoded is transformed 204, quantized 205, and entropy encoding 210 to form a part of the encoded bitstream of the frame to be encoded.
  • the control and reference data generated in the motion compensation 209 are also encoded by the entropy encoding 210 to form a part of the encoded bitstream.
  • the reconstructed video frame is a video frame obtained after filtering 211.
  • the filtering 211 is used to reduce compression distortion such as blocking effects and ringing effects generated in the encoding process.
  • the reconstructed video frame is used to provide a reference frame for inter-frame prediction; in the decoding process, the reconstructed video frame is output as the final decoded video after post-processing.
  • the filtering 211 includes at least one of the following filtering techniques: deblocking DB filtering, adaptive sample compensation offset SAO filtering, adaptive loop filtering ALF, cross-component ALF (Cross-Component ALF, CCALF).
  • ALF is set after DB and/or SAO.
  • the luminance component before ALF is used to filter the chrominance component after ALF.
  • the filter parameters in the process of filtering 211 are also transmitted to the entropy coding for coding, forming a part of the coded bitstream.
  • Fig. 3 is a schematic diagram of a video decoding framework 3 according to an embodiment of the present application.
  • video decoding executes operation steps corresponding to video encoding.
  • entropy decoding 301 is used to obtain one or more data information of residual data, prediction syntax, intra-frame prediction syntax, motion compensation syntax, and filtering syntax in the encoded bitstream.
  • the residual data undergoes inverse quantization 302 and inverse transformation 303 to obtain original residual data information.
  • the reconstructed image block in the current frame is used to construct prediction information according to the intra-frame prediction method; if it is inter-frame prediction, according to the decoded motion compensation syntax, Determine the reference block in the reconstructed image to obtain the prediction information; then, superimpose the prediction information and the residual information, and filter 311 to obtain the reconstructed video frame. After the reconstructed video frame undergoes post-processing 306, the decoded video is obtained .
  • the filter 311 may be the same as the filter 211 in FIG. 2, and includes at least one of the following: deblocking DB filter, adaptive sample compensation offset SAO filter, adaptive loop filter ALF, cross-component ALF (Cross-Component ALF, CCALF).
  • deblocking DB filter adaptive sample compensation offset SAO filter
  • adaptive loop filter ALF adaptive loop filter ALF
  • cross-component ALF Cross-component ALF
  • CCALF Cross-component ALF
  • the DB filter is used to process pixels on the boundary between the prediction unit PU and the transformation unit TU, and a low-pass filter obtained by training is used to perform nonlinear weighting of boundary pixels, thereby reducing blocking effects.
  • SAO filtering uses the coding block in the frame image as a unit to classify the pixel values in the coding block, and add compensation values to each type of pixel. Different coding blocks use different filtering forms and different The different types of pixel compensation values in the encoding block are different, so that the reconstructed frame image is closer to the original frame image, and the ringing effect is avoided.
  • ALF filtering is a Wiener filtering process.
  • filter coefficients are calculated for filtering, which is mainly used to minimize the mean square between the reconstructed frame image and the original frame image.
  • Error Magnetic-square Error, MSE
  • MSE mean-square Error
  • a pixel signal in the currently encoded original encoding frame is X
  • the reconstructed pixel signal after encoding, DB filtering and SAO filtering Is Y
  • the noise or distortion introduced by Y in this process is e
  • the reconstructed pixel signal is filtered by the filter coefficient f in the Wiener filter to form an ALF reconstructed signal So that the ALF reconstructs the signal
  • the mean square error with the original pixel signal is the smallest, and f is obtained as the ALF filter coefficient.
  • the calculation formula of f is as follows:
  • a filter composed of a set of ALF filter coefficients is shown in Figures 5a and 5b, with 13 filter coefficients distributed symmetrically from C0 to C12, and the filter length L is 7. ; Or there are 7 filter coefficients distributed symmetrically from C0 to C6, and the filter length L is 5.
  • the filter shown in Fig. 5a is also called a 7*7 filter, which is suitable for encoding frame brightness components
  • the filter shown in Fig. 5b is also called a 5*5 filter, which is suitable for encoding frame colors. Degree component.
  • the filter composed of the coefficients of the ALF filter may also be a filter of other forms, for example, a filter form such as a symmetrical distribution and a filter length of 9; This is not limited.
  • the weighted average of surrounding pixels is used to obtain the result after the current point filtering, that is, the corresponding pixel in the ALF reconstructed image frame .
  • the pixel I (x, y) in the reconstructed image frame is the current pixel to be filtered
  • (x, y) is the position coordinate of the current pixel to be filtered in the encoding frame
  • the filter coefficient of the filter center corresponds to it
  • the other filter coefficients in the filter correspond to the pixels around I(x, y) one by one.
  • the filter coefficient value in the filter is the weight value.
  • the filter coefficient value in the filter is multiplied by the corresponding pixel point.
  • the value obtained by adding and averaging is the filtered pixel value O(x, y) of the current pixel I(x, y) to be filtered.
  • the specific calculation formula is as follows:
  • w (i, j) represents any filter coefficient in the filter
  • (i, j) represents the relative position of the filter coefficient in the filter from the center point
  • i and j are both less than L/2 and greater than -L/ An integer of 2, where L is the length of the filter.
  • L is the length of the filter.
  • the filter coefficient C12 at the center of the filter is represented as w(0,0)
  • the filter coefficient C6 above C12 is represented as w(0,1)
  • the filter coefficient C11 to the right of C12 It is expressed as w(1, 0).
  • each pixel in the reconstructed image frame is filtered in turn to obtain the filtered ALF reconstructed image frame.
  • the filter coefficient w(i, j) of the filter is an integer between [-1, 1).
  • the filter coefficient w(i,j) is enlarged by 128 times and then rounded to obtain w'(i,j), w'(i,j) is [- 128, 128).
  • encoding and transmitting the amplified w'(i,j) is easy to implement hardware encoding and decoding, and the amplified w'(i,j) is used for filtering to obtain the calculation formula of O(x,y) as follows:
  • the filter is no longer directly used as a weight, and a weighted average of multiple pixels is used to obtain the filtered result.
  • nonlinear parameter factors are introduced to optimize the filtering effect.
  • the nonlinear ALF filter is used to filter I(x, y) to obtain the calculation formula of O’(x, y) as follows:
  • the filter coefficient w(i, j) of the filter is an integer between [-1, 1).
  • k(i,j) represents the loop filter ALF correction clip parameter, which is also referred to as the correction parameter or the clip parameter hereinafter, and each filter coefficient w(i,j) will correspond to A clip parameter.
  • the clip parameter selects one from ⁇ 1024, 181, 32, 6 ⁇ .
  • the clip parameter selects one from ⁇ 1024, 161, 25, 4 ⁇ , and each The index corresponding to the clip parameter, that is, the clip index parameter is written into the code stream. If the clip parameter is 1024, the clip index parameter 0 must be written into the code stream. Similarly, if it is 181, 1 must be written into the code stream. Therefore, you can see the clip of the coded frame brightness classification and the coded frame chroma classification.
  • the index parameters are all integers between 0 and 3.
  • the encoding frame of the luminance Y component can correspond to 25 sets of filters at most, and the encoding frame of the chrominance UV component corresponds to a set of filters.
  • the pixel category may be a category corresponding to the luminance Y component, but the embodiment of the present application is not limited to this, and the pixel category may also be a category corresponding to other components or all components.
  • the following takes the classification and division and ALF filtering of the coded frame of the luminance Y component as an example for description.
  • the reconstructed image frame after DB filtering and SAO filtering is divided into a plurality of 4*4 pixel blocks. Classify the multiple 4*4 blocks.
  • each 4*4 block can be classified according to the Laplace direction:
  • D is the Laplace direction, It is the result of the fine classification after the direction D (Direction) classification, There are many ways to obtain, but here is just the result of sub-categorization.
  • the calculation method of direction D is as follows. First, calculate the Laplacian gradient of the current 4*4 block in different directions. The calculation formula is:
  • i and j are the coordinates of the upper left pixel of the current 4*4 block.
  • R(k,l) represents the reconstructed pixel value at the (k,l) position in the 4*4 block.
  • V k,l represents the vertical Laplacian gradient of the pixel at the (k,l) coordinate in the 4*4 block.
  • H k,l represents the Laplacian gradient of the pixel at the (k,l) coordinate in the 4*4 block in the horizontal direction.
  • D1 k,l represents the Laplacian gradient of the pixel at the (k,l) coordinate in the 4*4 block in the direction of 135 degrees.
  • D2 k,l represents the 45-degree Laplacian gradient of the pixel at the (k,l) coordinate in the 4*4 block.
  • the calculated g v represents the Laplacian gradient of the current 4*4 block in the vertical direction.
  • g h represents the Laplacian gradient of the current 4*4 block in the horizontal direction.
  • g d1 represents the Laplacian gradient of the current 4*4 block in the direction of 135 degrees.
  • g d2 represents the Laplacian gradient of the current 4*4 block in the direction of 45 degrees.
  • R h, v represents the ratio of the Laplacian gradient in the horizontal and vertical directions.
  • R d0, d1 represent the ratio of the Laplacian gradient in the 45 and 135 directions.
  • t1 and t2 represent preset thresholds.
  • the value range of C is an integer between 0-24.
  • at most 4*4 blocks in one frame of image are divided into 25 categories.
  • each type of 4*4 block has a set of ALF filter coefficients, where N is an integer between 1-25.
  • the number of classifications can be classified into any other number in addition to 25 types, which is not limited in the embodiment of the present application.
  • ALF filtering can be divided into frame-based ALF, block-based ALF and quad-tree-based ALF.
  • frame-based ALF is to use a set of filter coefficients to filter the entire frame
  • block-based ALF is to divide the coded frame into image blocks of equal size, and determine whether to perform ALF filtering on the image block, based on quadtree ALF
  • the coding frame is divided into image blocks of different sizes based on the quadtree division method, and it is judged whether to perform ALF filtering.
  • the frame-based ALF calculation is simple, but the filtering effect is not good, and the quad-tree-based ALF calculation is more complicated. Therefore, in some standards or technologies, such as the latest VVC standard under study, its reference software VTM uses block-based ALF.
  • VTM Take the block-based ALF in VTM as an example.
  • a coded frame has a frame-level ALF filter flag and a block-level ALF filter flag.
  • the block level may be a CTU, a CU, or an image block in other division modes, which is not limited in the embodiment of the present application.
  • the CTU level ALF filter flag bit is used as an example for illustration below.
  • the frame-level ALF filter flag indicates that ALF filtering is not performed
  • the CTU-level ALF filter flag in the encoded frame is not identified.
  • the frame-level ALF filter flag indicates that ALF filtering is performed
  • the CTU-level ALF in the encoded frame is not identified.
  • the filtering flag bit indicates whether the current CTU performs ALF filtering.
  • the coded frame includes Z CTUs
  • the method for calculating N groups of ALF filter coefficients in the coded frame is as follows: whether the Z CTUs in the coded frame are combined with ALF filtering, and for each combination mode, the calculation is obtained.
  • the calculation method of the i-th group of ALF in each group of ALF filter coefficients is: in the current CTU combination mode, the i-th type of pixels in the CTU undergoing ALF filtering are calculated for f, and the other CTUs not undergoing ALF filtering are calculated For the i-th type of pixels, no f calculation is performed, and the i-th group of ALF coefficients in the current combination mode are calculated. It should be understood that in different combinations, the N groups of ALF filter coefficients obtained by calculation may be different from each other.
  • the frame-level ALF flag of the coded frame is identified as performing ALF filtering, and the CTU-level ALF flag in turn indicates whether to perform ALF filtering in the CTU data . For example, when the flag is marked as 0, it means that ALF filtering is not performed, and when the flag is marked as 1, it means that ALF filtering is performed.
  • the coded frame is not subjected to ALF filtering, and the frame-level ALF flag bit of the coded frame is marked as not performing ALF filtering. At this time, the ALF flag bit of the CTU level is not identified.
  • the ALF in the embodiments of the present application is not only applicable to the VVC standard, but also applicable to other technical solutions or standards using block-based ALF.
  • Cross component ALF Cross component ALF (Cross Component ALF, CCALF)
  • CCALF is used to adjust the chrominance component by using the value of the luminance component to improve the quality of the chrominance component.
  • the current block includes a luminance component and a chrominance component, where the chrominance component includes a first chrominance component (for example, Cb in FIG. 6) and a second chrominance component (for example, Cr in FIG. 6).
  • the luminance component is filtered through SAO and ALF in sequence.
  • the first chrominance component is filtered through SAO and ALF in sequence.
  • the second chrominance component is filtered through SAO and ALF in sequence.
  • a CCALF filter is also used to perform CCALF on the chrominance components.
  • the shape of the CCALF filter may be as shown in FIG. 7.
  • the CCALF filter adopts a 3x4 diamond shape with a total of 8 coefficients.
  • the position of the mark 2 is the current pixel of the first chroma component or the second chroma component, and the weighted average of the surrounding 7 points is used to obtain the filtered result of the pixel at the position of the middle mark 2.
  • the first chrominance component and the second chrominance component can be filtered by the same or different target filters from the same CCALF filter, or they can be filtered separately. Select the target filter from different dial CCALF filters for filtering.
  • the total number of CCALF filters used by the current image needs to be written into the code stream, where the total number of CCALF filters may include the total number of CCALF filters for the first chrominance component and/or the total number of CCALF filters for the second chrominance component.
  • the total number of CCALF filters for the first chroma component is the same as the total number of CCALF filters for the second chroma component, or the first chroma component and the second chroma component can select the target filter from the same dial CCALF filter , Can be indicated by the total number of only one CCALF filter.
  • the index of the target filter selected by the current block is also encoded into the code stream.
  • the indexes of the target filters respectively selected by the first chrominance component and the second chrominance component are the same or different
  • the indexes of the target filters of the two chrominance components may be encoded into the code stream respectively.
  • only one index may be encoded into the bitstream, and the index is used to indicate the two chrominance components The target filter.
  • the CCALF filter coefficient is determined according to the adaptation parameter set (Adaptation Parameter Sets, APS).
  • the adaptation parameter set Adaptation Parameter Sets, APS
  • APS 0 APS 1 ,...APS N
  • N is an integer greater than 1.
  • n is any integer not greater than N
  • the APS n contains multiple sets of filter coefficients, called f 0 , f 1 ,...f L , and L is an integer greater than 1.
  • the number of filter coefficients included in different APSs may be the same or different.
  • CCALF is performed on the current frame by using the filter coefficients, and the distortion of the current frame is calculated according to the result of CCALF.
  • the RD cost is calculated according to the decision result of each APS, and the best APS is selected according to the RD cost result.
  • step S601 the available CCALF APS of the current frame is determined, referred to as APS 0 , APS 1 , ... APS N , where N is an integer greater than 1.
  • n is any integer not greater than N
  • the APS n includes multiple sets of filter coefficients, called f 0 , f 1 ,...f L , and L is an integer greater than 1.
  • the number of filter coefficients included in different APSs may be the same or different. Determine the L sets of filter coefficients in the first APS (that is, APS 0 ).
  • step S603 the distortion caused when each set of filter coefficients in APS 0 is used for CCALF is calculated. Step S603 is repeated until the distortion caused by CCALF on all filter coefficients in APS 0 is calculated separately.
  • step S603 the following formula may be used to calculate the distortion:
  • o(k), l(k), c(k) and y(k) are the original chrominance pixel value, the luminance reconstruction pixel value before ALF, the chrominance reconstruction pixel value after ALF and the CCALF
  • N is the number of taps of the CCALF filter. In an example, N can take the value 8.
  • the distortion can be estimated by the following formula:
  • ⁇ , ⁇ > represents the inner product
  • K represents the area where filtering is turned on
  • R ll (k) is the autocorrelation matrix of l(k)
  • R co (k) is the ratio of c(k) and o(k) Cross-correlation vector.
  • step S604 according to the distortion result calculated in step S603 each time, it is determined whether each CTU should turn on CCALF, and for the CTU that decides to turn on CCALF, the filter coefficient used by the CTU is also decided.
  • step S605 the RD cost of the decision result in step S604 is calculated. At this point, the RD cost of APS 0 is obtained. Then, step 602 to step 605 are repeated to obtain the RD cost of APS 1. Repeat this until the RD cost of each APS is calculated.
  • step S603 uses a new calculation method to calculate distortion, there is no need to use each set of filters in each APS to perform CCALF on the CTU to obtain the results, which can reduce the amount of calculation and access to the image cache, which is beneficial to software and hardware. the design of.
  • a filter is selected from the preset filters and the filter coefficient of the filter is calculated as the CCALF filter coefficient of the CTU. Specifically, determine multiple available CCALF filters for the current CTU, initialize the coefficients of the filter for each CCALF filter, and use the original chrominance value of the current CTU, the luminance reconstruction value before ALF, and after ALF Calculate the covariance of the current CTU based on the chrominance reconstruction value. Then the variance of each CTU is used to calculate the variance of the current frame, and the coefficients of the filter are trained according to the variance of the current frame.
  • the original chrominance value of the current CTU, the luminance reconstruction value before ALF, and the chrominance reconstruction value after ALF need to be used continuously.
  • Access the image buffer (buffer) to read the data This is not conducive to the design of the hardware.
  • the image cache is generally located off-chip and in the external storage. Compared to accessing the data in the on-chip memory, accessing the data in the off-chip memory consumes more energy and has a greater delay.
  • step S801 when determining the CCALF filter coefficient of the CTU, first calculate the variance of the CTU level. For example, the original chrominance value of the current CTU, the luminance reconstruction value before ALF, and the chrominance reconstruction value after ALF may be used to calculate the variance of the current CTU.
  • step S802 multiple available CCALF filters for the current CTU are determined, and the coefficients of each CCALF filter are initialized.
  • step S803 on the basis of the initialization result of step S802, one of the CCALF filter coefficients is trained.
  • the CTU-level variance calculated in step S801 is directly used to calculate the frame-level variance.
  • step S804 the coefficient of the corresponding CCALF filter is calculated according to the frame-level variance obtained in step S803.
  • step S805 the distortion is calculated according to the calculated CCALF filter coefficient.
  • the method for calculating the distortion may adopt the method for calculating the distortion in step S603 in the embodiment shown in FIG. 6.
  • step S806 it is determined whether to enable CCALF for each CTU according to the calculation result of step S805, and for the CTU that decides to enable CCALF, the filter coefficient used by the CTU is also determined.
  • step S807 the RD cost of the decision result in step S806 is calculated. So far, the RD cost of the first CCALF filter is obtained. Then repeat steps 803 to 807 to calculate the RD cost of the second CCALF filter. In this loop, when the RDCost of the current loop does not decrease compared with the previous loop, or the number of loops reaches the threshold, the loop is terminated.
  • the chrominance reconstruction pixel values after ALF used in the training process of the CCALF filter coefficients can be replaced with the chrominance reconstruction pixel values before ALF, so that the training process of the CCALF filter coefficients and ALF cannot be processed in parallel in the encoder/decoder.
  • FIG. 9 is a schematic block diagram of another device 30 for loop filtering on the encoding end according to an embodiment of the present application.
  • the device 30 for loop filtering is a device for loop filtering on the video encoding end.
  • the loop filter The filtering device 20 may correspond to the method 100 of loop filtering.
  • the loop filtering device 30 includes: a processor 31 and a memory 32;
  • the memory 32 may be used to store programs, and the processor 31 may be used to execute the programs stored in the memory.
  • the processor 31 may be used to perform the following operations:
  • the luminance reconstruction pixel value before the adaptive loop filter ALF the luminance reconstruction pixel value before the adaptive loop filter ALF, the chrominance reconstruction pixel value after ALF, and the chrominance reconstruction pixel value after the cross-component adaptive loop filter CCALF Distortion of each set of filter coefficients in multiple sets of candidate filter coefficients;
  • the processor 31 may be used to perform the following operations:
  • the processor 31 may be used to perform the following operations:
  • An embodiment of the present application also provides an electronic device, which may include the loop filtering apparatus of the various embodiments of the present application described above.
  • processors of the embodiment of the present application may be an integrated circuit chip with signal processing capability.
  • steps of the foregoing method embodiments can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software.
  • the above-mentioned processors include but are not limited to the following: general-purpose processors, central processing units (CPUs), digital signal processors (digital signal processors, DSP), application specific integrated circuits (ASICs), ready-made programmable gate arrays Field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPUs central processing units
  • DSP digital signal processors
  • ASICs application specific integrated circuits
  • FPGA Field programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
  • the memory of the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), and electrically available Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • the volatile memory may be random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • static random access memory static random access memory
  • dynamic RAM dynamic RAM
  • DRAM dynamic random access memory
  • synchronous dynamic random access memory synchronous DRAM, SDRAM
  • double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM
  • enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
  • synchronous connection dynamic random access memory serial DRAM, SLDRAM
  • direct rambus RAM direct rambus RAM
  • the embodiment of the present application also proposes a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions.
  • the instructions When the instructions are included in a portable electronic device that includes multiple application programs When executed, the portable electronic device can be made to execute the method of the embodiment shown in FIG. 6 to FIG. 9.
  • the embodiment of the present application also proposes a computer program, which includes instructions.
  • the computer program When the computer program is executed by a computer, the computer can execute the method of the embodiments shown in FIG. 6 to FIG. 9.
  • An embodiment of the present application also provides a chip that includes an input and output interface, at least one processor, at least one memory, and a bus.
  • the at least one memory is used to store instructions
  • the at least one processor is used to call the at least one memory. Instructions to execute the method of the embodiment shown in FIGS. 6-9.
  • the disclosed system, device, and method can be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

提供环路滤波的方法与装置，通过优化编解码环路滤波过程中的编码方式，降低环路滤波的复杂度，提高编解码性能。一种环路滤波的方法包括：确定图像的多套滤波器系数；根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。

Description

环路滤波的方法与装置
版权申明
本专利文件披露的内容包含受版权保护的材料。该版权为版权所有人所有。版权所有人不反对任何人复制专与商标局的官方记录和档案中所存在的该专利文件或者该专利披露。
技术领域
本发明涉及数字视频编码技术领域，并且更为具体地，涉及一种环路滤波的方法与装置。
背景技术
目前，为了减少视频存储和传输所占用的带宽，需要对视频数据进行编码压缩处理。目前常用的编码技术中，视频的编码压缩处理过程包括：块划分、预测、变换、量化和熵编码过程，形成一个混合视频编码框架。在该混合视频编码框架的基础上，经过几十年的发展，逐渐形成了视频编解码技术标准，目前主流的一些视频编解码标准包括：国际视频编码标准H.264/MPEG-AVC、H.265/MEPG-HEVC、国内音视频编码标准AVS2，以及正在制定的H.266/VVC国际标准和AVS3国内标准。
在块划分、预测、变换、量化和熵编码的编码过程中，由于量化的存在，解码重构视频中会存在块效应、振铃效应等压缩失真，同时，帧间预测模式中，重构视频中的压缩失真会影响后续图像的编码质量。因此，为了降低压缩失真，通过编解码结构框架中引入环路滤波(in loop filter)技术，提高当前解码图像质量以及为后续编码图像提供高质量的参考图像，提高压缩效率。
在目前正在制定的通用视频编码(Versatile Video Coding，VVC)标准以及部分高性能视频编码(High Efficiency Video Coding，HEVC)标准中，环路滤波器包括去块滤波(deblocking filter，DBF)，自适应样值补偿滤波(Sample Adaptive Offset，SAO)以及自适应环路滤波(Adaptive Loop Filter，ALF)。其中，滤波的过程仍有改进的空间。
发明内容
本发明提供一种环路滤波的方法与装置，相对于现有技术，可以降低环路滤波的复杂度，提高滤波的效果。
第一方面，提供了一种环路滤波的方法，包括：
确定图像的多套滤波器系数；
根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
第二方面，提供了一种环路滤波的方法，包括：
计算图像块级方差covariance；
确定图像的多套CCALF滤波器，以及分别初始化所述多套CCALF滤波器系数；
根据所述图像块级方差分别训练所述多套CCALF滤波器系数；
分别计算所述多套CCALF滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
第三方面，提供了一种环路滤波的方法，包括：
根据当前图像块经ALF前的色度重建像素值训练所述当前图像块的CCALF滤波器系数；
根据训练得到的所述当前图像块的CCALF滤波器系数对所述当前图像块进行CCALF；
对所述当前图像块进行ALF，其中，所述当前图像块的CCALF和ALF是并行处理的。
第四方面，提供了一种环路滤波的装置，包括：存储器，用于存储代码；
处理器，用于执行所述存储器中存储的代码，以执行如下操作：
确定图像的多套滤波器系数；
根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
第五方面，提供了一种环路滤波的装置，包括：
存储器，用于存储代码；
处理器，用于执行所述存储器中存储的代码，以执行如下操作：
计算图像块级方差covariance；
确定图像的多套CCALF滤波器，以及分别初始化所述多套CCALF滤波器系数；
根据所述图像块级方差分别训练所述多套CCALF滤波器系数；
分别计算所述多套CCALF滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
第六方面，提供了一种环路滤波的装置，包括：
存储器，用于存储代码；
处理器，用于执行所述存储器中存储的代码，以执行如下操作：
根据当前图像块经ALF前的色度重建像素值训练所述当前图像块的CCALF滤波器系数；
根据训练得到的所述当前图像块的CCALF滤波器系数对所述当前图像块进行CCALF；
对所述当前图像块进行ALF，其中，所述当前图像块的CCALF和ALF是并行处理的。
附图说明
图1是应用本申请实施例的技术方案的架构图。
图2是根据本申请实施例的视频编码框架示意图。
图3是根据本申请实施例的视频解码框架示意图。
图4是根据本申请实施例的维纳滤波器示意图。
图5a是根据本申请实施例的一种ALF滤波器示意图。
图5b是根据本申请实施例的另一种ALF滤波器示意图。
图6是本申请一个实施例的环路滤波的方法的示意性流程图。
图7是本申请一个实施例的CCALF滤波器的形状示意图。
图8是本申请另一个实施例的环路滤波的方法的示意性流程图。
图9是本申请另一个实施例的环路滤波的装置的示意性流程图。
具体实施方式
下面将结合附图，对本申请实施例中的技术方案进行描述。
本申请实施例可适用于标准或非标准的图像或视频编码器。例如，VVC标准的编码器。
应理解，本文中的具体的例子只是为了帮助本领域技术人员更好地理解本申请实施例，而非限制本申请实施例的范围。
还应理解，本申请实施例中的公式只是一种示例，而非限制本申请实施例的范围，各公式可以进行变形，这些变形也应属于本申请保护的范围。
还应理解，在本申请的各种实施例中，各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
还应理解，本说明书中描述的各种实施方式，既可以单独实施，也可以组合实施，本申请实施例对此并不限定。
除非另有说明，本申请实施例所使用的所有技术和科学术语与本申请的技术领域的技术人员通常理解的含义相同。本申请中所使用的术语只是为了描述具体的实施例的目的，不是旨在限制本申请的范围。本申请所使用的术语“和/或”包括一个或多个相关的所列项的任意的和所有的组合。
图1是应用本申请实施例的技术方案的架构图。
如图1所示，系统100可以接收待处理数据102，对待处理数据102进行处理，产生处理后的数据108。例如，系统100可以接收待编码数据，对待编码数据进行编码以产生编码后的数据，或者，系统100可以接收待解码数据，对待解码数据进行解码以产生解码后的数据。在一些实施例中，系统100中的部件可以由一个或多个处理器实现，该处理器可以是计算设备中的处理器，也可以是移动设备(例如无人机)中的处理器。该处理器可以为任意种类的处理器，本发明实施例对此不做限定。在一些可能的设计中，该处理器可以包括编码器、解码器或编解码器等。系统100中还可以包括一个或多个存储器。该存储器可用于存储指令和数据，例如，实现本发明实施例的技术方案的计算机可执行指令、待处理数据102、处理后的数据108等。该存储器可以为任意种类的存储器，本发明实施例对此也不做限定。
图2是根据本申请实施例的视频编码框架2示意图。如图2所示，在接 收待编码视频后，从待编码视频的第一帧开始，依次对待编码视频中的每一帧进行编码。其中，当前编码帧主要经过：预测(Prediction)、变换(Transform)、量化(Quantization)和熵编码(Entropy Coding)等处理，最终输出当前编码帧的码流。对应的，解码过程通常是按照上述过程的逆过程对接收到的码流进行解码，以恢复出解码前的视频帧信息。
具体地，如图2所示，所述视频编码框架2中包括一个编码控制模块201，用于进行编码过程中的决策控制动作，以及参数的选择。例如，如图2所示，编码控制模块201控制变换、量化、反量化、反变换的中用到的参数，控制进行帧内或者帧间模式的选择，以及运动估计和滤波的参数控制，且编码控制模块201的控制参数也将输入至熵编码模块中，进行编码形成编码码流中的一部分。
对当前待编码帧开始编码时，对待编码帧进行划分202处理，具体地，首先对其进行条带(slice)划分，再进行块划分。可选地，在一个示例中，待编码帧划分为多个互不重叠的最大的编码树单元(Coding Tree Unit，CTU)，各CTU还可以分别按四叉树、或二叉树、或三叉树的方式迭代划分为一系列更小的编码单元(Coding Unit，CU)，一些示例中，CU还可以包含与之相关联的预测单元(Prediction Unit，PU)和变换单元(Transform Unit，TU)，其中PU为预测的基本单元，TU为变换和量化的基本单元。一些示例中，PU和TU分别是在CU的基础上划分成一个或多个块得到的，其中一个PU包含多个预测块(Prediction Block，PB)以及相关语法元素。一些示例中，PU和TU可以是相同的，或者，是由CU通过不同的划分方法得到的。一些示例中，CU、PU和TU中的至少两种是相同的，例如，不区分CU、PU和TU，全部是以CU为单位进行预测、量化和变换。为方便描述，下文中将CTU、CU或者其它形成的数据单元均称为编码块。
应理解，在本申请实施例中，视频编码针对的数据单元可以为帧，条带，编码树单元，编码单元，编码块或以上任一种的组。在不同的实施例中，数据单元的大小可以变化。
具体地，如图2所示，待编码帧划分为多个编码块后，进行预测过程，用于去除当前待编码帧的空域和时域冗余信息。当前比较常用的预测编码方法包括帧内预测和帧间预测两种方法。帧内预测仅利用本帧图像中己重建的 信息对当前编码块进行预测，而帧间预测会利用到之前已经重建过的其它帧图像(也被称作参考帧)中的信息对当前编码块进行预测。具体地，在本申请实施例中，编码控制模块201用于决策选择帧内预测或者帧间预测。
当选择帧内预测模式时，帧内预测203的过程包括获取当前编码块周围已编码相邻块的重建块作为参考块，基于该参考块的像素值，采用预测模式方法计算预测值生成预测块，将当前编码块与预测块的相应像素值相减得到当前编码块的残差，当前编码块的残差经过变换204、量化205以及熵编码210后形成当前编码块的码流。进一步的，当前待编码帧的全部编码块经过上述编码过程后，形成待编码帧的编码码流中的一部分。此外，帧内预测203中产生的控制和参考数据也经过熵编码210编码，形成编码码流中的一部分。
具体地，变换204用于去除图像块的残差的相关性，以便提高编码效率。对于当前编码块残差数据的变换通常采用二维离散余弦变换(Discrete Cosine Transform，DCT)变换和二维离散正弦变换(Discrete Sine Transform，DST)变换，例如在编码端将待编码块的残差信息分别与一个N×M的变换矩阵及其转置矩阵相乘，相乘之后得到当前编码块的变换系数。
在产生变换系数之后用量化205进一步提高压缩效率，变换系数经量化可以得到量化后的系数，然后将量化后的系数进行熵编码210得到当前编码块的残差码流，其中，熵编码方法包括但不限于内容自适应二进制算术编码(Context Adaptive Binary Arithmetic Coding，CABAC)熵编码。
具体地，帧内预测203过程中的已编码相邻块为：当前编码块编码之前，已进行编码的相邻块，该相邻块的编码过程中产生的残差经过变换204、量化205、反量化206、和反变换207后，与该相邻块的预测块相加得到的重建块。对应的，反量化206和反变换207为量化206和变换204的逆过程，用于恢复量化和变换前的残差数据。
如图2所示，当选择帧间预测模式时，帧间预测过程包括运动估计208和运动补偿209。具体地，根据重建视频帧中的参考帧图像进行运动估计208，在一张或多张参考帧图像中根据一定的匹配准则搜索到与当前编码块最相似的图像块为匹配块，该匹配块与当前编码块的相对位移即为当前待编码块的运动矢量(Motion Vector，MV)。对待编码帧中的全部编码块进行运动估计之后，基于运动矢量和参考帧对当前待编码帧进行运动补偿209，获得当 前待编码帧的预测值。将该待编码帧像素的原始值与对应的预测值相减得到待编码帧的残差。当前待编码帧的残差经过变换204、量化205以及熵编码210后形成待编码帧的编码码流中的一部分。此外，运动补偿209中产生的控制和参考数据也经过熵编码210编码，形成编码码流中的一部分。
其中，如图2所示，重建视频帧为经过滤波211之后得到视频帧。滤波211用于减少编码过程中产生的块效应和振铃效应等压缩失真。在编码过程中，重建视频帧用于为帧间预测提供参考帧；在解码过程中，重建视频帧经过后处理后输出为最终的解码视频。在本申请实施例中，滤波211包括以下滤波技术中的至少一种：去块DB滤波、自适应样值补偿偏移SAO滤波、自适应环路滤波ALF、跨分量ALF(Cross-Component ALF，CCALF)。在一个示例中，ALF设置于DB和/或SAO之后。在一个示例中，采用经ALF前的亮度分量对经ALF之后的色度分量进行滤波。滤波211过程中的滤波参数同样被传送至熵编码中进行编码，形成编码码流中的一部分。
图3是根据本申请实施例的视频解码框架3示意图。如图3所示，视频解码执行与视频编码相对应的操作步骤。首先利用熵解码301得到编码码流中的残差数据、预测语法、帧内预测语法、运动补偿语法以及滤波语法中的一种或多种数据信息。其中，残差数据经过反量化302和反变换303得到原始残差数据信息。此外，根据预测语法确定当前解码块使用帧内预测还是帧间预测。如果是帧内预测304，则根据解码得到的帧内预测语法，利用当前帧中已重建图像块按照帧内预测方法构建预测信息；如果是帧间预测，则根据解码得到的运动补偿语法，在已重建的图像中确定参考块，得到预测信息；接下来，再将预测信息与残差信息进行叠加，并经过滤波311操作便可以得到重建视频帧，重建视频帧经过后处理306后得到解码视频。
具体地，在本申请实施例中，所述滤波311可以与图2中的滤波211相同，包括以下至少一种：去块DB滤波、自适应样值补偿偏移SAO滤波、自适应环路滤波ALF、跨分量ALF(Cross-Component ALF，CCALF)。其中，滤波311中的滤波参数和控制参数可以通过对编码码流进行熵解码得到，基于得到的滤波参数和控制参数分别进行滤波。
一个示例中，DB滤波用于处理预测单元PU和变换单元TU边界的像素，利用训练得到的低通滤波器进行边界像素的非线性加权，从而减少块效应。 一个示例中，SAO滤波以帧图像中的编码块为单元，用于对编码块内的像素值进行分类，并为每一类像素加上补偿值，不同的编码块采用不同的滤波形式且不同的编码块中不同类的像素补偿值不同，使得重构帧图像更接近原始帧图像，避免振铃效应。一个示例中，ALF滤波为一种维纳滤波(wiener filtering)过程，根据维纳滤波的原理，计算出滤波系数进行滤波，主要用于最小化重构帧图像与原始帧图像之间的均方误差(Mean-square Error，MSE)，从而进一步改善重建帧的图像质量，提高运动估计和运动补偿的准确度并有效的提高整个编码系统的编码效率，但是与此同时，ALF滤波的复杂度高、运算耗时，在实际应用过程中存在一定的缺陷。
为方便理解，下面结合图4、图5a和图5b对ALF滤波过程的一个示例进行描述。
ALF滤波系数计算原理
首先，根据维纳滤波原理，说明计算ALF滤波系数的计算方法，如图4所示，当前编码的原始编码帧中的一个像素信号为X，经过编码以及DB滤波和SAO滤波后的重建像素信号为Y，在此过程中Y引入的噪声或者失真为e，重建像素信号经过维纳滤波器中的滤波系数f滤波后，形成ALF重建信号
Figure PCTCN2020084071-appb-000001
使得该ALF重建信号
Figure PCTCN2020084071-appb-000002
与原始像素信号的均方误差最小，得到f即为ALF滤波系数，具体地，f的计算公式如下：
Figure PCTCN2020084071-appb-000003
可选地，在一种可能的实施方式中，一组ALF滤波系数组成的滤波器如图5a和图5b所示，具有C0～C12呈对称分布的13个滤波系数，滤波器长度L为7；或者具有C0～C6呈对称分布的7个滤波系数，滤波器长度L为5。可选地，图5a所示的滤波器也称为7*7的滤波器，适用于编码帧亮度分量，图5b所示的滤波器也称为5*5的滤波器，适用于编码帧色度分量。
应理解，在本申请实施例中，所述ALF滤波器系数组成的滤波器还可以为其它形式的滤波器，例如具有呈对称分布，滤波器长度为9等滤波器形式，本申请实施例对此不做限定。
可选地，在一种线性ALF滤波过程中，对于重建图像帧中的待滤波像素点，使用周围像素点的加权平均来得到当前点滤波之后的结果，即ALF重建图像帧中对应的像素点。具体地，重建图像帧中像素点I(x，y)为当前待 滤波像素点，(x，y)为当前待滤波像素点在编码帧中的位置坐标，滤波器中心的滤波系数与其对应，滤波器中其它滤波系数与I(x，y)周围的像素点一一对应，滤波器中的滤波系数值即为权值，将滤波器中的滤波系数值与对应的像素点相乘后相加，再平均得到的数值即为当前待滤波像素点I(x，y)滤波后的像素值O(x，y)，具体的计算公式如下：
Figure PCTCN2020084071-appb-000004
其中，w(i，j)表示滤波器中的任意一个滤波系数，(i，j)表示滤波器中的滤波系数距离中心点的相对位置，i和j均为小于L/2大于-L/2的整数，其中L为滤波器的长度。例如，如图5a中的滤波器所示，滤波器中心的滤波系数C12表示为w(0，0)，C12上方的滤波系数C6表示为w(0，1)，C12右方的滤波系数C11表示为w(1，0)。
按照此方式，依次对重建图像帧中的每个像素点进行滤波，得到滤波后的ALF重建图像帧。
可选地，在一种可能的实现方式中，所述滤波器的滤波系数w(i，j)为[-1，1)之间的整数。
可选地，在一种可能的实施方式中，对所述滤波器系数w(i，j)放大128倍后取整得到w’(i，j)，w’(i，j)为[-128，128)之间的整数。具体地，对放大后的w’(i，j)进行编码传输易于硬件编解码实现，且采用放大后的w’(i，j)进行滤波得到O(x，y)的计算公式如下：
Figure PCTCN2020084071-appb-000005
可选地，在另一种非线性ALF滤波过程中，不再直接采用滤波器作为权值，将多个像素点的加权平均得到滤波之后的结果。而是引入非线性参数因子，优化滤波效果，具体地，采用非线性ALF滤波对I(x，y)进行滤波计算得到O’(x，y)的计算公式如下：
Figure PCTCN2020084071-appb-000006
其中，所述滤波器的滤波系数w(i，j)为[-1，1)之间的整数。K(d，b)是一个修正(clip)的操作，K(d，b)＝min(b，max(-b，d))。
具体地，在K(d，b)clip操作中，k(i，j)代表环路滤波ALF修正clip参数，下文也简称修正参数或者clip参数，每一个滤波系数w(i，j)都会对应一个clip的参数。对于编码帧亮度分量，clip参数从{1024，181，32，6}中选择一个，对于编码帧色度分量，clip参数从{1024，161，25，4}中选择一个，且需要将每一个clip参数对应的索引，即修正(clip)索引参数写入码流。如果clip参数是1024，就要将clip索引参数0写入码流，同理，如果是181，就要将1写入码流，因此可以看出编码帧亮度分类和编码帧色度分类的clip索引参数均为0～3之间的整数。
像素分类划分
其次，若对一个像素点计算一组对应的ALF滤波系数，其计算复杂度大且耗时，并且如果将每一个像素点的ALF系数都写入码流，会到来巨大的开销，因此，需要对重建图像中的像素点进行分类划分，每一类像素点采用同一组ALF滤波系数(一种滤波器)，这样能够减少计算复杂度，提高编码效率。
可选地，像素分类的方式可以有很多种。例如，可以只对像素的亮度Y分量进行分类，对色度UV分量不进行分类。例如，对亮度Y分量划分为25类，色度UV分量不划分，只有一类。换言之，对于一帧图像来讲，亮度Y分量的编码帧最多可以对应25组滤波器，色度UV分量的编码帧对应一组滤波器。
应理解，在本申请实施例中，像素类别可以是对应于亮度Y分量的类别，但本申请实施例对此并不限定，像素类别也可以是对应于其他分量或所有分量的类别。为方便描述，下文以对亮度Y分量的编码帧进行分类划分和ALF滤波为例进行说明。
可选地，在一种可能的实施方式中，将经过DB滤波和SAO滤波后的重建图像帧划分为多个4*4像素的块。将该多个4*4的块进行分类。
例如，每一个4*4的块都可以根据拉普拉斯(Laplace)方向进行分类：
Figure PCTCN2020084071-appb-000007
C代表像素块所属类别。D为拉普拉斯方向，
Figure PCTCN2020084071-appb-000008
是进行方向D(Direction)分类之后的细分类结果，
Figure PCTCN2020084071-appb-000009
的获取可以有多种方式，这里只是代表细分类的结果。
方向D的计算方式如下，首先，计算当前4*4块在不同方向上的拉普拉斯梯度，计算公式为：
Figure PCTCN2020084071-appb-000010
Figure PCTCN2020084071-appb-000011
Figure PCTCN2020084071-appb-000012
Figure PCTCN2020084071-appb-000013
其中，i和j是当前4*4块的左上像素点的坐标。
R(k，l)代表4*4块中位于(k，l)位置的重构像素值。V k，l代表4*4块中位于(k，l)坐标的像素点在竖直方向的拉普拉斯梯度。H k，l代表4*4块中位于(k，l)坐标的像素点在水平方向拉普拉斯梯度。D1 k，l代表4*4块中位于(k，l)坐标的像素点在135度方向拉普拉斯梯度。D2 k，l代表4*4块中位于(k，l)坐标的像素点在45度拉普拉斯梯度。
对应的，计算得到的g v代表当前4*4块在竖直方向的拉普拉斯梯度。g h代表当前4*4块在水平方向的拉普拉斯梯度。g d1代表当前4*4块在135度方向的拉普拉斯梯度。g d2代表当前4*4块在45度方向的拉普拉斯梯度。
然后，根据四个方向上拉普拉斯梯度的极值比，判断方向D，具体计算公式如下：
Figure PCTCN2020084071-appb-000014
Figure PCTCN2020084071-appb-000015
Figure PCTCN2020084071-appb-000016
Figure PCTCN2020084071-appb-000017
其中，
Figure PCTCN2020084071-appb-000018
代表水平、竖直方向拉普拉斯梯度值的最大值。
Figure PCTCN2020084071-appb-000019
代表水平、竖直方向拉普拉斯梯度值的最小值。
Figure PCTCN2020084071-appb-000020
代表45、135方向拉普拉斯梯度值的最大值。
Figure PCTCN2020084071-appb-000021
代表45、135方向拉普拉斯梯度值的最小值。R h，v代表水平、竖直方向拉普拉斯梯度的比值。R d0，d1代表45、135方向拉普拉斯 梯度的比值。
如果
Figure PCTCN2020084071-appb-000022
而且
Figure PCTCN2020084071-appb-000023
D设置为0。
如果
Figure PCTCN2020084071-appb-000024
而且
Figure PCTCN2020084071-appb-000025
D设置为1。
如果
Figure PCTCN2020084071-appb-000026
而且
Figure PCTCN2020084071-appb-000027
D设置为2。
如果
Figure PCTCN2020084071-appb-000028
而且
Figure PCTCN2020084071-appb-000029
D设置为3。
如果
Figure PCTCN2020084071-appb-000030
而且
Figure PCTCN2020084071-appb-000031
D设置为4。
t1和t2代表预先设置的阈值。
可选地，在一种可能的实施方式中，
Figure PCTCN2020084071-appb-000032
的计算方式如下，
Figure PCTCN2020084071-appb-000033
将A量化得到0～4之间的整数，得到
Figure PCTCN2020084071-appb-000034
因此，综合D和A的取值，C的取值范围为0～24之间的整数，在本申请实施例中，最多将一帧图像中的4*4块划分为25类。
可选地，在一种可能的实施方式中，编码帧中具有N类4*4块，每一类4*4块具有一组ALF滤波系数，其中，N为1～25之间的整数。
应理解，在本申请实施例中，对整帧图像除了可以划分为多个4*4的块以外，还可以为划分为其它像素大小的块，例如，划分为多个8*8或者16*16大小的块，本申请实施例对此不做限制。
还应理解，在本申请实施例中，除了上述根据拉普拉斯(Laplace)方向进行分类外，还可以采用其他的分类方法对块进行分类，本申请实施例对此不做限定。
还应理解，在本申请实施例中，分类数量除了25种外，还可以分类为其它任意数量，本申请实施例对此也不做限定。
基于块的ALF滤波
ALF滤波可以分为基于帧的ALF，基于块的ALF以及基于四叉树的ALF。其中，基于帧的ALF为采用一组滤波系数对整帧进行滤波，基于块的ALF为将编码帧划分为大小相等的图像块，对图像块进行判断是否进行ALF滤波，基于四叉树的ALF则是基于四叉树的划分方式将编码帧划分为大小不同的图像块，进行判断是否进行ALF滤波。其中，基于帧的ALF计算简单，但滤波效果不佳，而基于四叉树的ALF计算复杂度较高，因此，在一些标 准或技术中，例如最新正在研究制定的VVC标准中，其参考软件VTM采用的是基于块的ALF。
以VTM中基于块的ALF作为示例说明。在VTM中，编码帧具有帧级ALF滤波标志位，且具有块级ALF滤波标志位。可选地，该块级可以为CTU，CU或者其它划分方式的图像块，本申请实施例对此不做限定，为方便描述，下文以CTU级ALF滤波标志位进行举例说明。
具体地，当帧级ALF滤波标志位标识不进行ALF滤波时，则不标识编码帧中CTU级ALF滤波标志位，当帧级ALF滤波标志位标识进行ALF滤波时，对编码帧中CTU级ALF滤波标志位进行标识表示当前CTU是否进行ALF滤波。
可选地，编码帧中包括Z个CTU，计算编码帧中的N组ALF滤波系数的方法如下：对编码帧中Z个CTU是否进行ALF滤波进行组合，针对每一种组合方式，计算得到该方式下的N组ALF滤波系数和编码帧的率失真代价(Rate-distortion Cost，RD Cost)。其中，每一组ALF滤波系数中的第i组ALF的计算方式为：在当前CTU组合方式下，将进行ALF滤波的CTU中的第i类像素进行f计算，其它不进行ALF滤波的CTU中的第i类像素则不进行f计算，计算得到当前组合方式下的第i组ALF系数。应理解，不同组合方式下，计算得到的N组ALF滤波系数可能互不相同。
比较多个组合方式下的RD Cost，其中，RD Cost最小的组合方式确定为最终的组合方式。且在该组合方式下计算得到的N组ALF滤波系数为适应性最优的ALF滤波系数。
当RD Cost最小的组合方式为Z个CTU中至少一个CTU进行ALF滤波时，编码帧的帧级ALF标志位标识为进行ALF滤波，CTU级的ALF标志位依次在CTU数据中标识是否进行ALF滤波。例如，标识位标识为0时，表示不进行ALF滤波，标识位标识为1时，表示进行ALF滤波。
特别地，当RD Cost最小的组合方式为Z个CTU均不进行ALF滤波时，此时，编码帧不进行ALF滤波，将编码帧的帧级ALF标志位标识为不进行ALF滤波。此时，CTU级的ALF标志位不进行标识。
应理解，本申请实施例中的ALF不仅适用于VVC标准中，还适应用于其它采用基于块的ALF技术方案或者标准中。
跨分量ALF(Cross Component ALF，CCALF)
一个示例中，CCALF用于利用亮度分量的数值来对色度分量进行调整，提升色度分量质量。为方便理解，下面结合图6对CCALF和ALF过程的一个示例进行描述。当前块包括亮度分量和色度分量，其中色度分量包括第一色度分量(例如图6中的Cb)和第二色度分量(例如图6中的Cr)。
亮度分量依次经过SAO和ALF进行滤波。第一色度分量依次经过SAO和ALF进行滤波。第二色度分量依次经过SAO和ALF进行滤波。另外，还采用CCALF滤波器对色度分量进行CCALF。
在一个示例中，CCALF滤波器的形状可以如图7所示。该CCALF滤波器中采用3x4菱形，共8个系数。图中标识2所在位置为当前的第一色度分量或者第二色度分量的像素点，使用周围的7个点的加权平均得到中间标识2所在位置像素点滤波之后的结果。
一帧图像中总共可以有多套滤波器，其中，第一色度分量和第二色度分量可以从同一拨CCALF滤波器中分别选择相同或不同的目标滤波器进行滤波，或者，也可以分别从不同拨CCALF滤波器中各自选择出目标滤波器进行滤波。
当前图像使用的CCALF滤波器的总数需要写入码流，其中，该CCALF滤波器的总数可以包括第一色度分量的CCALF滤波器总数和/或第二色度分量的CCALF滤波器总数。在第一色度分量的CCALF滤波器总数和第二色度分量的CCALF滤波器总数相同，或者第一色度分量和第二色度分量可以从同一拨CCALF滤波器中选择目标滤波器的情况中，可以仅用一个CCALF滤波器的总数来指示。
对于当前块，还将该当前块所选择的目标滤波器的索引编码到码流中。在第一色度分量和第二色度分量分别所选择的目标滤波器的索引相同或不同的情况中，可以分别将该两个色度分量的目标滤波器的索引编码到码流中。或者，在第一色度分量和第二色度分量分别所选择的目标滤波器的索引相同情况中，可以将仅将一个索引编码到码流中，该索引用来指示该两个色度分量的目标滤波器。
在一个示例中，在确定CTU的CCALF滤波器系数时，根据自适应参数集(Adaptation Parameter Sets，APS)中确定该CCALF滤波器系数。
确定当前帧多个可用的CCALF APS，称为APS 0，APS 1，...APS N，N为大于1的整数。其中，每一个APS n，n为不大于N的任意整数，该APS n包含多套滤波器系数，称为f 0，f 1，...f L，L为大于1的整数。不同的APS中所包含的滤波器系数的数量可以相同也可以不同。
对每一个APS中的每一套滤波器系数，分别采用该滤波器系数对当前帧进行CCALF，并根据CCALF的结果来计算当前帧的失真(distortion)。在利用每套APS计算出当前帧的失真后，根据每套APS计算出的失真结果分别决策当前帧中的每个CTU是否开启CCALF，以及在确定开启CCALF的情况下该CTU的CCALF滤波器系数。然后再根据每个APS的决策结果计算RD cost，根据RD cost结果从中选择出最优的APS。
然而，在该示例中，由于存在多个APS，以及每个APS中包含多套滤波器系数，导致在确定CTU的CCALF滤波器系数的过程中需要进行多次CCALF。例如，在存在8个可用APS以及每个APS包含4套滤波器系数的情况下，一个CTU需要进行32次CCALF。
可选的，在另一个示例中，如图6所示，步骤S601中，确定当前帧的可用的CCALF APS，称为APS 0，APS 1，...APS N，N为大于1的整数。
步骤S602中，对于每一个APS n，n为不大于N的任意整数，该APS n包含多套滤波器系数，称为f 0，f 1，...f L，L为大于1的整数。不同的APS中所包含的滤波器系数的数量可以相同也可以不同。确定第一个APS(也即APS 0)中的L套滤波器系数。
步骤S603中，计算采用APS 0中每套滤波器系数进行CCALF时所造成的失真。重复步骤S603，直至分别计算APS 0中所有滤波器系数进行CCALF后所造成的失真。
其中，在步骤S603中，可采用以下公式来计算失真：
Figure PCTCN2020084071-appb-000035
其中，o(k)，l(k)，c(k)和y(k)分别为色度原始像素值、经ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过CCALF的色度重建像素值。N是CCALF滤波器抽头的数量。一个示例中，N可以取值为8。滤波 器系数f＝(w 0，w 1，...，w N-1) T，是根据开启了滤波的CTU计算得到的。失真可通过以下公式估计得到：
Figure PCTCN2020084071-appb-000036
Figure PCTCN2020084071-appb-000037
Figure PCTCN2020084071-appb-000038
其中，<·，·>表示内积，K表示开启了滤波的区域，R ll(k)是l(k)的自相关矩阵，R co(k)是c(k)和o(k)的互相关向量。
步骤S604中，根据每一次步骤S603所计算的失真结果来决策各CTU是否开启CCALF，对于决定开启CCALF的CTU，还决策该CTU所采用的滤波器系数。
步骤S605中，计算步骤S604中的决策结果的RD cost。至此，得到APS 0的RD cost。然后重复步骤602至步骤605，得到APS 1的RD cost。以此循环，直至计算出每个APS的RD cost。
本示例中，由于步骤S603采用新的计算方式来计算失真，无需采用每个APS中的每套滤波器分别对CTU进行CCALF得到结果，可以减少计算量以及对图像缓存的访问，有利于软硬件的设计。
在一个示例中，在确定CTU的CCALF滤波器系数时，从预设滤波器中选择一个滤波器并计算该滤波器的滤波器系数作为该CTU的CCALF滤波器系数。具体的，确定当前CTU多个可用的CCALF滤波器，对每一个CCALF滤波器，对该滤波器的系数初始化，并利用当前CTU的原始色度值、进行ALF之前的亮度重建值以及进行ALF之后的色度重建值计算当前CTU的方差(covariance)。然后利用各CTU的方差计算当前帧的方差，并根据该当前帧的方差来对滤波器的系数进行训练。
然而，在该示例中，在训练每一个CCALF滤波器的系数时，由于需要用到当前CTU的原始色度值、进行ALF之前的亮度重建值以及进行ALF之后的色度重建值，需要不停访问图像的缓存(buffer)去读取数据。这不利于硬件的设计。而且，由于片上存储器非常有限，图像缓存一般是位于片 下并位于在外部存储中，相比访问片上存储器的数据，访问片下存储器的数据需要消耗更多的能量以及会有更大的延迟。
可选的，在另一示例中，如图8所示，步骤S801中，在确定CTU的CCALF滤波器系数时，首先计算CTU级的方差。例如，可利用当前CTU的原始色度值、进行ALF之前的亮度重建值以及进行ALF之后的色度重建值计算当前CTU的方差。
步骤S802中，确定当前CTU多个可用的CCALF滤波器，对每一个CCALF滤波器的系数进行初始化。
步骤S803中，在步骤S802的初始化结果的基础上，对其中一个CCALF滤波器系数进行训练。训练每一个CCALF滤波器的系数时，直接采用步骤S801计算好的CTU级的方差来计算帧级的方差。
步骤S804中，根据步骤S803得到的帧级方差计算对应的CCALF滤波器的系数。
步骤S805中，根据计算出的CCALF滤波器系数计算失真。可选的，该计算失真的方法可以采用图6所示实施例中步骤S603中计算失真的方法。
步骤S806中，根据步骤S805的计算结果来决策各CTU是否开启CCALF，对于决定开启CCALF的CTU，还决策该CTU所采用的滤波器系数。
步骤S807中，计算步骤S806中的决策结果的RD cost。至此，得到第一CCALF滤波器的RD cost。然后重复步骤803至步骤807，计算第二个CCALF滤波器的RD cost。以此循环，当当前循环和上一次循环相比RDCost没有减小，或者循环次数达到阈值时，就中止循环。
在一些示例中，在CCALF滤波器系数的训练过程中，需要采用经过ALF之后的色度重建像素值，这导致CCALF滤波器系数的训练过程和ALF无法在编码器/解码器中并行处理。因此，可选的，可以将CCALF滤波器系数的训练过程中所采用的经ALF之后的色度重建像素值替换成经ALF之前的色度重建像素值，以使得CCALF滤波器系数的训练过程和ALF无法在编码器/解码器中并行处理。
图9是本申请实施例的另一种编码端的环路滤波的装置30的示意性框图，该环路滤波的装置30为视频编码端中的环路滤波的装置，可选地，该 环路滤波的装置20可以对应于环路滤波的方法100。
如图9所述，所述环路滤波的装置30包括：处理器31和存储器32；
存储器32可用于存储程序，处理器31可用于执行所述存储器中存储的程序。
一个示例中，处理器31可用于执行如下操作：
确定图像的多套滤波器系数；
根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
应理解，装置实施例与方法实施例相互对应，类似的描述可以参照方法实施例。
一个示例中，处理器31可用于执行如下操作：
计算图像块级方差covariance；
确定图像的多套CCALF滤波器，以及分别初始化所述多套CCALF滤波器系数；
根据所述图像块级方差分别训练所述多套CCALF滤波器系数；
分别计算所述多套CCALF滤波器系数的失真；
根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
一个示例中，处理器31可用于执行如下操作：
根据当前图像块经ALF前的色度重建像素值训练所述当前图像块的CCALF滤波器系数；
根据训练得到的所述当前图像块的CCALF滤波器系数对所述当前图像块进行CCALF；
对所述当前图像块进行ALF，其中，所述当前图像块的CCALF和ALF是并行处理的。
本申请实施例还提供了一种电子设备，该电子设备可以包括上述本申请各种实施例的环路滤波的装置。
应理解，本申请实施例的处理器可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬 件的集成逻辑电路或者软件形式的指令完成。上述的处理器包括但不限于以下各种：通用处理器，中央处理器CPU、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
可以理解，本申请实施例的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
本申请实施例还提出了一种计算机可读存储介质，该计算机可读存储介质存储一个或多个程序，该一个或多个程序包括指令，该指令当被包括多个应用程序的便携式电子设备执行时，能够使该便携式电子设备执行图6至图 9所示实施例的方法。
本申请实施例还提出了一种计算机程序，该计算机程序包括指令，当该计算机程序被计算机执行时，使得计算机可以执行图6至图9所示实施例的方法。
本申请实施例还提供了一种芯片，该芯片包括输入输出接口、至少一个处理器、至少一个存储器和总线，该至少一个存储器用于存储指令，该至少一个处理器用于调用该至少一个存储器中的指令，以执行图6至图9所示实施例的方法。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应所述理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者所述技术方案的部分可以以软件产品的形式体现出来，所述计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。

Claims (8)

  1. 一种环路滤波的方法，其特征在于，包括：
    确定图像的多套滤波器系数；
    根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；
    根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
  2. 根据权利权要1所述的环路滤波的方法，其特征在于，
    所述多套候选滤波器系数位于所述图像的自适应参数集Adaptation parameter set中。
  3. 一种环路滤波的方法，其特征在于，包括：
    计算图像块级方差covariance；
    确定图像的多套CCALF滤波器，以及分别初始化所述多套CCALF滤波器系数；
    根据所述图像块级方差分别训练所述多套CCALF滤波器系数；
    分别计算所述多套CCALF滤波器系数的失真；
    根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
  4. 一种环路滤波的方法，其特征在于，包括：
    根据当前图像块经ALF前的色度重建像素值训练所述当前图像块的CCALF滤波器系数；
    根据训练得到的所述当前图像块的CCALF滤波器系数对所述当前图像块进行CCALF；
    对所述当前图像块进行ALF，其中，所述当前图像块的CCALF和ALF是并行处理的。
  5. 一种环路滤波的装置，其特征在于，包括：
    存储器，用于存储代码；
    处理器，用于执行所述存储器中存储的代码，以执行如下操作：
    确定图像的多套滤波器系数；
    根据色度原始像素值、经自适应环路滤波ALF前的亮度重建像素值、经ALF后的色度重建像素值和经过跨分量自适应环路滤波CCALF的色度重 建像素值分别计算所述多套候选滤波器系数中每套滤波器系数的失真；
    根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
  6. 根据权利权要5所述的环路滤波的装置，其特征在于，
    所述多套候选滤波器系数位于所述图像的自适应参数集Adaptation parameter set中。
  7. 一种环路滤波的装置，其特征在于，包括：
    存储器，用于存储代码；
    处理器，用于执行所述存储器中存储的代码，以执行如下操作：
    计算图像块级方差covariance；
    确定图像的多套CCALF滤波器，以及分别初始化所述多套CCALF滤波器系数；
    根据所述图像块级方差分别训练所述多套CCALF滤波器系数；
    分别计算所述多套CCALF滤波器系数的失真；
    根据所述每套滤波器系数的失真决策当前图像块是否开启CCALF。
  8. 一种环路滤波的装置，其特征在于，包括：
    存储器，用于存储代码；
    处理器，用于执行所述存储器中存储的代码，以执行如下操作：
    根据当前图像块经ALF前的色度重建像素值训练所述当前图像块的CCALF滤波器系数；
    根据训练得到的所述当前图像块的CCALF滤波器系数对所述当前图像块进行CCALF；
    对所述当前图像块进行ALF，其中，所述当前图像块的CCALF和ALF是并行处理的。
PCT/CN2020/084071 2020-04-09 2020-04-09 环路滤波的方法与装置 WO2021203394A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080004298.7A CN112514401A (zh) 2020-04-09 2020-04-09 环路滤波的方法与装置
PCT/CN2020/084071 WO2021203394A1 (zh) 2020-04-09 2020-04-09 环路滤波的方法与装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/084071 WO2021203394A1 (zh) 2020-04-09 2020-04-09 环路滤波的方法与装置

Publications (1)

Publication Number Publication Date
WO2021203394A1 true WO2021203394A1 (zh) 2021-10-14

Family

ID=74953138

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/084071 WO2021203394A1 (zh) 2020-04-09 2020-04-09 环路滤波的方法与装置

Country Status (2)

Country Link
CN (1) CN112514401A (zh)
WO (1) WO2021203394A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024088003A1 (en) * 2022-10-24 2024-05-02 Mediatek Inc. Method and apparatus of position-aware reconstruction in in-loop filtering

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113489977B (zh) * 2021-07-02 2022-12-06 浙江大华技术股份有限公司 环路滤波方法、视频/图像编解码方法及相关装置
CN114640847B (zh) * 2021-06-30 2023-03-24 杭州海康威视数字技术股份有限公司 编解码方法、装置及其设备
CN114125445B (zh) * 2021-06-30 2023-03-24 杭州海康威视数字技术股份有限公司 解码方法、装置、设备及机器可读存储介质
CN114025160B (zh) * 2021-11-26 2023-06-02 成都爱奇艺智能创新科技有限公司 一种视频滤波方法、装置、电子设备及存储介质
CN115348448B (zh) * 2022-10-19 2023-02-17 北京达佳互联信息技术有限公司 滤波器训练方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104683819A (zh) * 2015-01-31 2015-06-03 北京大学 一种自适应环路滤波方法及装置
CN105659601A (zh) * 2013-10-11 2016-06-08 索尼公司 图像处理装置和图像处理方法
WO2019026807A1 (en) * 2017-08-03 2019-02-07 Sharp Kabushiki Kaisha SYSTEMS AND METHODS FOR PARTITIONING VIDEO BLOCKS IN INTER VIDEO PREDICTION WAFER
CN109691102A (zh) * 2016-08-31 2019-04-26 高通股份有限公司 跨分量滤波器
WO2019089695A1 (en) * 2017-11-01 2019-05-09 Vid Scale, Inc. Methods for simplifying adaptive loop filter in video coding

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9813738B2 (en) * 2010-10-05 2017-11-07 Hfi Innovation Inc. Method and apparatus of adaptive loop filtering
US20130113880A1 (en) * 2011-11-08 2013-05-09 Jie Zhao High Efficiency Video Coding (HEVC) Adaptive Loop Filter
TWI652935B (zh) * 2012-09-28 2019-03-01 Ｖｉｄ衡器股份有限公司 視訊編碼方法及裝置
JP6530751B2 (ja) * 2014-06-30 2019-06-12 ソニーセミコンダクタソリューションズ株式会社 画像処理装置、撮像装置、情報処理装置、画像処理方法およびプログラム
KR20170071594A (ko) * 2014-10-28 2017-06-23 미디어텍 싱가폴 피티이. 엘티디. 비디오 코딩을 위한 가이드된 크로스-컴포넌트 예측 방법
CN104735450B (zh) * 2015-02-26 2019-06-07 北京大学 一种在视频编解码中进行自适应环路滤波的方法及装置
CN105357538B (zh) * 2015-11-26 2020-01-03 广东中星微电子有限公司 自适应环路滤波方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105659601A (zh) * 2013-10-11 2016-06-08 索尼公司 图像处理装置和图像处理方法
CN104683819A (zh) * 2015-01-31 2015-06-03 北京大学 一种自适应环路滤波方法及装置
CN109691102A (zh) * 2016-08-31 2019-04-26 高通股份有限公司 跨分量滤波器
WO2019026807A1 (en) * 2017-08-03 2019-02-07 Sharp Kabushiki Kaisha SYSTEMS AND METHODS FOR PARTITIONING VIDEO BLOCKS IN INTER VIDEO PREDICTION WAFER
WO2019089695A1 (en) * 2017-11-01 2019-05-09 Vid Scale, Inc. Methods for simplifying adaptive loop filter in video coding

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024088003A1 (en) * 2022-10-24 2024-05-02 Mediatek Inc. Method and apparatus of position-aware reconstruction in in-loop filtering

Also Published As

Publication number Publication date
CN112514401A (zh) 2021-03-16

Similar Documents

Publication Publication Date Title
WO2021203394A1 (zh) 环路滤波的方法与装置
WO2021134706A1 (zh) 环路滤波的方法与装置
TWI737137B (zh) 視訊編碼之非線性適應性迴圈濾波方法和裝置
WO2020258055A1 (zh) 环路滤波的方法与装置
WO2021163862A1 (zh) 视频编码的方法与装置
WO2020182102A1 (zh) 进行编码和解码的方法、解码端和编码端
CN115552905A (zh) 用于图像和视频编码的基于全局跳过连接的cnn滤波器
CN116916036A (zh) 视频压缩方法、装置及系统
WO2022116246A1 (zh) 帧间预测方法、视频编解码方法、装置及介质
WO2023082107A1 (zh) 解码方法、编码方法、解码器、编码器和编解码系统
WO2022178686A1 (zh) 编解码方法、编解码设备、编解码系统以及计算机可读存储介质
CN116982262A (zh) 视频编码中依赖性量化的状态转换
WO2020258052A1 (zh) 图像分量预测方法、装置及计算机存储介质
WO2021056220A1 (zh) 视频编解码的方法与装置
WO2023141970A1 (zh) 解码方法、编码方法、解码器、编码器和编解码系统
WO2024007116A1 (zh) 解码方法、编码方法、解码器以及编码器
WO2023019407A1 (zh) 帧间预测方法、编码器、解码器以及存储介质
WO2023245544A1 (zh) 编解码方法、码流、编码器、解码器以及存储介质
WO2023240618A1 (zh) 一种滤波方法、解码器、编码器及计算机可读存储介质
WO2023231008A1 (zh) 编解码方法、编码器、解码器以及存储介质
WO2023197179A1 (zh) 解码方法、编码方法、解码器以及编码器
WO2023193253A1 (zh) 解码方法、编码方法、解码器以及编码器
WO2023193254A1 (zh) 解码方法、编码方法、解码器以及编码器
WO2023197181A1 (zh) 解码方法、编码方法、解码器以及编码器
WO2020258053A1 (zh) 图像分量预测方法、装置及计算机存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20930285

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20930285

Country of ref document: EP

Kind code of ref document: A1