WO2021128265A1

WO2021128265A1 - Filtering method and device

Info

Publication number: WO2021128265A1
Application number: PCT/CN2019/129144
Authority: WO
Inventors: 徐陆航; 朱建清; 数井君彦
Original assignee: 富士通株式会社; 徐陆航; 朱建清
Priority date: 2019-12-27
Filing date: 2019-12-27
Publication date: 2021-07-01

Abstract

Embodiments of the present application provide a filtering method and device. The method comprises: determining a quantization scale for each of multiple sets of filter coefficients of a cross-component adaptive loop filter (CC-ALF); performing a quantization operation on each set of filter coefficients using the quantization scale corresponding thereto, so as to obtain a quantization result for each set of filter coefficients; and using the quantization results of respective sets of filter coefficients to perform a filtering operation on luma information of an input image, so as to obtain chroma compensation information. In the present application, respective filter coefficients in each set of filter coefficients are quantized using a single quantization scale, and then luma information of an input image is filtered on the basis of the quantization results, so as to obtain chroma compensation information, thus improving coding and decoding efficiency.

Description

Filtering method and device

Technical field

This application relates to the technical field of video coding and decoding and the technical field of image compression.

Background technique

The loop filter is a series of filters in the current VCC (Versatile Video Coding) draft, which can improve subjective quality and coding efficiency at the same time. Now, a new tool called Cross Component Adaptive Loop Filter (CC-ALF) is under core experimental research. Figure 1 shows the structure of the CC-ALF in the adaptive loop filter, and Figure 2 shows the shape of each set of filter coefficients of the current CC-ALF.

As shown in Figure 1, the principle of CC-ALF is to use the edge information in the luminance information (Luma) to compensate the chroma (Chroma), because the chroma information may be lost during compression. As shown in Figure 2, CC-ALF currently supports 4 groups of 3×4 diamonds and 8 coefficient asymmetric filters in each frame, which is calculated by the encoder. Each coefficient is limited to 6 bits (from -32 to 31).

On the encoder side, the calculated original filter coefficients are in decimal form (double type number). They need to be quantized into 6-bit integers. Figure 3 shows a quantification method, including:

Operation 301: Obtain the original filter coefficient in decimal form, and let C _i denote the original filter coefficient in the CC-ALF filter, i∈[1,8];

Operation 302: C′ _i =round(C _i ×128);

Operation 303: _{Limit C′ i} to the range of [-32, 31] to obtain C″ _i ; here, use C″ _{i to} represent the quantization coefficient limited to a 6-bit integer:

Operation 304: When filtering the pixels, Pi' _chroma = Pi _chroma + round(∑C″ _i × Pi _luma ) ÷ 128, where Pi' _chroma is the chrominance information obtained by filtering the chrominance information of the input image , Round(∑C″ _i ×Pi _luma )÷128 is the chrominance compensation information obtained by filtering the luminance information of the input image.

It should be noted that the above introduction to the technical background is only for the convenience of a clear and complete description of the technical solutions of the present application, and to facilitate the understanding of those skilled in the art. It should not be considered that the above technical solutions are well-known to those skilled in the art just because these solutions are described in the background art part of this application.

Summary of the invention

The inventor of the present application found that using this process for quantization of all filter coefficients may cause a problem that some coefficients in a small range will be quantized to very low precision. For example, the original filter coefficient of the filter is 0.03, that is, C _i =0.03. According to the above process, it will be quantized to an integer 4, that is, C′ _i =4. Since 4 is in the range of [-32, 31], then C" _i =C' _i =4, and since the 6-bit range can support the maximum value of 31, this will result in a relatively large loss of accuracy.

In order to solve the above-mentioned problems or solve other similar problems, embodiments of the present application provide a filtering method and device, which modify the current CC-ALF filtering process and use an adaptive scale to quantize the CC-ALF filter coefficients. In order to make full use of the bit range that needs to be quantized, the precision loss is reduced and the coding efficiency is improved.

According to a first aspect of the embodiments of the present application, there is provided a filtering method, wherein the filtering method includes:

Determine a quantization scale for each set of filter coefficients in the multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF);

Perform a quantization operation on each set of filter coefficients using the quantization scale corresponding to each set of filter coefficients to obtain the quantization result of each set of filter coefficients;

The quantization result of each set of filter coefficients is used to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information.

According to a second aspect of the embodiments of the present application, there is provided a filtering device, wherein the filtering device includes:

A determining unit, which determines a quantization scale for each set of filter coefficients in the multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF);

A quantization unit, which uses the quantization scale corresponding to each group of filter coefficients to perform a quantization operation on each group of filter coefficients to obtain a quantization result of each group of filter coefficients;

The filtering unit uses the quantization result of each set of filter coefficients to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information.

According to a third aspect of the embodiments of the present application, there is provided a filtering method, wherein the filtering method includes:

Obtain the quantization scale from the received code stream;

The luminance information of the input image is filtered by using the quantization scale, and the chrominance compensation information is output.

According to a fourth aspect of the embodiments of the present application, there is provided a filtering device, wherein the filtering device includes:

A reading unit, which obtains the quantization scale from the received code stream;

The filtering unit uses the quantization scale to perform a filtering operation on the luminance information of the input image, and output chrominance compensation information.

According to a fifth aspect of the embodiments of the present application, there is provided an image encoder, wherein the image encoder includes:

Luma filter (SAO Luma), which performs a filtering operation on an input image and outputs the brightness information of the input image;

An adaptive loop filter (CC-ALF Cb/Cr), which performs a filtering operation on the luminance information of the input image, and outputs chrominance compensation information; and

Chroma filter (ALF Chroma), which performs a filtering operation on the input image, and outputs the chroma information of the input image;

Wherein, the adaptive loop filter (CC-ALF Cb/Cr) is configured to determine one filter coefficient for each of the multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF) Quantization scale; use the quantization scale corresponding to each set of filter coefficients to quantize each set of filter coefficients to obtain the quantization result of each set of filter coefficients; use the quantization result of each set of filter coefficients to filter the brightness information of the input image to obtain the color The chrominance compensation information is used to compensate the chrominance information of the input image.

According to a sixth aspect of the embodiments of the present application, there is provided an image decoder, wherein the image decoder includes:

Wherein, the adaptive loop filter (CC-ALF Cb/Cr) is configured to obtain a quantization scale from the received code stream, use the quantization scale to filter the brightness information of the input image, and output color The chrominance compensation information is used to compensate the chrominance information of the input image.

According to other aspects of the embodiments of the present application, a computer-readable program is provided, wherein when the program is executed in an image processing device, the program causes the image processing device to execute the aforementioned first aspect or third aspect. The method described.

According to other aspects of the embodiments of the present application, there is provided a storage medium storing a computer readable program, wherein the computer readable program causes an image processing device to execute the method described in the first aspect or the third aspect.

One of the beneficial effects of the embodiments of the present application is that an adaptive quantization scale (Scale) is used in the quantization of CC-ALF coefficients to obtain higher precision and improve coding efficiency.

With reference to the following description and drawings, specific implementations of the present application are disclosed in detail, and the ways in which the principles of the present application can be adopted are indicated. It should be understood that the scope of the embodiments of the present application is not limited thereby. Within the scope of the terms of the appended claims, the implementation of the present application includes many changes, modifications and equivalents.

Features described and/or shown for one embodiment can be used in one or more other embodiments in the same or similar manner, combined with features in other embodiments, or substituted for features in other embodiments .

It should be emphasized that the term "comprising/comprising" when used herein refers to the existence of a feature, a whole, a step or a component, but does not exclude the existence or addition of one or more other features, a whole, a step or a component.

Description of the drawings

The elements and features described in one drawing or one embodiment of the embodiment of the present application may be combined with the elements and features shown in one or more other drawings or embodiments. In addition, in the drawings, similar reference numerals indicate corresponding parts in several drawings, and may be used to indicate corresponding parts used in more than one embodiment.

The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the embodiments of the present application, and together with the text description, explain the principle of the present application. Obviously, the drawings in the following description are only some embodiments of the application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the attached picture:

Figure 1 is a schematic diagram of the position of the cross-component adaptive loop filter (CC-ALF) in the encoder;

Figure 2 is a schematic diagram of each set of filter coefficients of a cross-component adaptive loop filter (CC-ALF);

Fig. 3 is a schematic diagram of a conventional cross-component adaptive loop filter (CC-ALF) filtering method;

FIG. 4 is a schematic diagram of a filtering method according to an embodiment of the present application;

FIG. 5 is another schematic diagram of the filtering method according to an embodiment of the present application;

FIG. 6 is another schematic diagram of the filtering method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a filtering method according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a filtering device according to an embodiment of the present application;

FIG. 9 is another schematic diagram of the filtering device of the embodiment of the present application;

Fig. 10 is a schematic diagram of an image encoder according to an embodiment of the present application;

Fig. 11 is a schematic diagram of an image decoder according to an embodiment of the present application;

Fig. 12 is a schematic diagram of an image processing device according to an embodiment of the present application.

Detailed ways

With reference to the drawings, the foregoing and other features of this application will become apparent through the following description. In the specification and drawings, specific implementations of the application are specifically disclosed, which indicate some implementations in which the principles of the application can be adopted. It should be understood that the application is not limited to the described implementations. On the contrary, the present application is not limited to the described implementations. The application includes all modifications, variations and equivalents falling within the scope of the appended claims.

In the embodiments of this application, the terms "first", "second", etc. are used to distinguish different elements from the terms, but they do not indicate the spatial arrangement or chronological order of these elements. These elements should not be used by these terms. Limited. The term "and/or" includes any and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having" and the like refer to the existence of the stated features, elements, elements or components, but do not exclude the presence or addition of one or more other features, elements, elements or components.

In the embodiments of this application, the singular forms "a", "the", etc. include plural forms, which should be broadly understood as "a" or "a type" rather than being limited to the meaning of "a"; in addition, the term "so" "Said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "based on" should be understood as "based at least in part on...", and the term "based on" should be understood as "based at least in part on...", unless the context clearly dictates otherwise.

Various implementations of the embodiments of the present application will be described below with reference to the accompanying drawings. These implementation manners are only exemplary, and not a limitation of the present application.

The first aspect of the embodiment

The embodiment of the present application provides a filtering method, which is applied to one side of an encoder. FIG. 4 is a schematic diagram of the filtering method of the embodiment of the present application. As shown in FIG. 4, the filtering method of the embodiment of the present application includes:

401: Determine a quantization scale for each set of filter coefficients in multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF);

402: Use the quantization scale corresponding to each group of filter coefficients to perform a quantization operation on each group of filter coefficients to obtain a quantization result of each group of filter coefficients;

403: Use the quantization result of each set of filter coefficients to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information.

The embodiment of the present application modifies the current CC-ALF filtering process and uses an adaptive scale to quantize the CC-ALF filter coefficients, which can make full use of the bit range that needs to be quantized, reduce precision loss, and improve coding efficiency.

In the embodiment of the present application, the cross-component adaptive loop filter (CC-ALF) has multiple sets of filter coefficients, such as four sets, and each set of filter coefficients has multiple filter coefficients, such as 8, but the application is not limited to this The CC-ALF may also have other number of groups of filter coefficients, and each group of filter coefficients may also include other numbers of filter coefficients.

In 401, determining the quantization scale may be to obtain multiple original filter coefficients of each group of filter coefficients of CC-ALF, and calculate the quantization scale corresponding to each group of filter coefficients according to the maximum value of the multiple original filter coefficients. Here, the maximum value among the multiple original filter coefficients refers to the original filter coefficient with the largest absolute value among the multiple original filter coefficients.

In some embodiments, according to the number of bits representing the quantization scale, the quantization scale may also be limited to a range that can be represented by the number of bits. For example, a range (referred to as the first range) is determined according to the number of bits representing the quantization scale, and the quantization scale is limited to the first range to obtain the restricted quantization scale.

In 402, the quantization operation may be performed by using the quantization scale corresponding to each group of filter coefficients to perform a quantization operation on each original filter coefficient of each group of filter coefficients to obtain a quantization result of each original filter coefficient of each group of filter coefficients.

In some embodiments, according to the number of bits representing the quantization result, the quantization result may also be limited to a range that can be represented by the number of bits. For example, a range (referred to as a second range) is determined according to the number of bits representing the quantization result, and the quantization result is limited to the second range to obtain the limited quantization result.

In 403, the filtering operation can be performed on the luminance information of the input image according to the quantization result of each group of filter coefficients and the quantization scale corresponding to each group of filter coefficients to obtain chrominance compensation information, which is used as the cross-component self The output of the adaptive loop filter (CC-ALF).

In some embodiments, the brightness information of the input image can be obtained by filtering the input image through the SAO Luma filter (SAO Luma). For the specific processing process of the SAO Luma filter, reference may be made to the related technology. Go into details. The embodiment of the present application uses the edge information in the luminance information to obtain chrominance compensation information.

In the embodiment of the present application, the value of the aforementioned quantization scale (Scale Value) can also be stored (be signaled to) in the adaptation parameter set (Adaption Parameter Set, APS) of the bin file of the video sequence, that is, encoded into the code stream , So that the decoder side reads the Scale Value from the received code stream, and uses the Scale Value to filter the pixels of the input image.

Fig. 5 is another schematic diagram of the filtering method of an embodiment of the present application. Fig. 5 shows the processing procedure for each set of filter coefficients. Each set of filter coefficients contains 8 coefficients, the quantization scale is represented by 2 bits, and the quantization result (Quantized filter coefficients) are represented by 6 bits as an example, but the application is not limited to this. As mentioned above, each set of filter coefficients can also contain other numbers of coefficients, and the quantization scale can also be represented by 3 bits or other numbers. It is expressed in bits, and the quantization result can also be expressed in other numbers of bits. As shown in Figure 5, the method includes:

501: Obtain raw coefficients in decimal form, and use C _{i to} represent the coefficients in the CC-ALF filter, and i∈[1,8];

502: Find the maximum value of the original coefficients, which is represented _{by C max;}

503: Calculate the quantization scale Scale;

In some embodiments,

For example, Scale=floor(2.6)=2;

504: Limit the quantization scale Scale to the range [0,3] to obtain Scale'. Therefore, the quantization scale can be represented by 2 bits;

505: Use the quantization scale to quantize the original filter coefficients to obtain a quantization result C′ _i ;

In some embodiments, C′ _i =round(C _i ×128×2 ^Scale′ );

506: _{Limit C′ i} to the range [-32,31] to obtain C″ _i , thus, the quantized filter coefficient can be represented by 6 bits;

_{507: Obtain chrominance information Pi' chroma} when filtering the pixels of the input image.

In some embodiments, Pi' _chroma = Pi _chroma + round (∑C " _i × Pi _luma ) ÷ 128 ÷ 2 ^Scale' , and the above-mentioned Pi _chroma is obtained by filtering the input image through a chroma filter (ALF Chroma) In the obtained chromaticity information, the above-mentioned round (ΣC" _i ×Pi _luma )÷128÷2 ^Scale' is chromaticity compensation information.

Fig. 6 is another schematic diagram of the filtering method according to an embodiment of the present application. Fig. 6 shows the processing process for each set of filter coefficients, still assuming that each set of filter coefficients contains 8 coefficients, the quantization scale is represented by 2 bits, and the quantization The result (quantized filter coefficient) is represented by 6 bits as an example, but the application is not limited to this. As mentioned above, each set of filter coefficients can also contain other numbers of coefficients, and the quantization scale can also be represented by 3 bits or other numbers. The quantization result can also be expressed by other numbers of bits. As shown in Figure 6, the method includes:

601: Obtain raw coefficients in decimal form, and use C _{i to} represent the coefficients in the CC-ALF filter, and i∈[1,8];

602: Find the maximum value of the original coefficients, which is represented _{by C max;}

603: Calculate the quantization scale Scale;

In some embodiments,

604: Limit the quantization scale Scale to the range [0,3] to obtain Scale'. Therefore, the quantization scale can be represented by 2 bits;

605: Use the quantization scale to quantize the original filter coefficients to obtain a quantization result C′ _i ;

In some embodiments, C′ _i =round(C _i ×32×2 ^Scale′ );

606: _{Limit C′ i} to the range [-32,31] to obtain C″ _i , thus, the quantized filter coefficient can be represented by 6 bits;

_{607: Obtain chrominance information Pi' chroma} when filtering the pixels of the input image.

In some embodiments, Pi' _chroma = Pi _chroma + round (∑C " _i × Pi _luma ) ÷ 32 ÷ 2 ^Scale' , and the above-mentioned Pi _chroma is obtained by filtering the input image through a chroma filter (ALF Chroma) In the obtained chromaticity information, the above-mentioned round (ΣC" _i ×Pi _luma )÷32÷2 ^Scale' is chromaticity compensation information.

In the examples of FIG. 5 and FIG. 6, the aforementioned 2-bit Scale value is obtained based on the maximum filter coefficient of each filter (that is, each group of filter coefficients), so it is adaptive. As mentioned earlier, the Scale value can be saved in the APS of the bin file of the video sequence and compiled into the transmitted code stream. However, this application is not limited to this, and the aforementioned Scale value may also be other bit depths (3 bits as described above), or stored in other locations of the bin file of the video sequence.

It is worth noting that the above figures 4-6 only schematically illustrate the embodiments of the present application, but the present application is not limited thereto. For example, the order of execution among various operations can be appropriately adjusted, and some other operations can be added or some operations can be reduced. Those skilled in the art can make appropriate modifications based on the foregoing content, and are not limited to the description of the foregoing FIGS. 4-6.

It is worth noting that the above only describes the steps or processes related to the application, but the application is not limited thereto. The method may also include other steps or processes. For the specific content of these steps or processes, reference may be made to related technologies.

According to the embodiment of the present application, an adaptive quantization scale (Scale) is used in the quantization of CC-ALF coefficients, thereby obtaining higher accuracy and improving coding efficiency.

The second aspect of the embodiment

The embodiment of the present application provides a filtering method, which is applied to one side of the decoder, and corresponds to the method of the first aspect of the embodiment, and the same content will not be repeated. FIG. 7 is a schematic diagram of the filtering method of the embodiment of the present application. As shown in FIG. 7, the filtering method of the embodiment of the present application includes:

701: Obtain the quantization scale Scale from the received code stream;

702: Filter the pixels of the input image by using the quantization scale Scale, and output chrominance compensation information, where the chrominance compensation information is used to compensate the chrominance information of the input image.

According to the embodiment of the present application, an adaptive quantization scale (Scale) is used in the quantization of CC-ALF coefficients, thereby obtaining higher accuracy and improving decoding efficiency.

The third aspect of the embodiment

The embodiment of the present application provides a filtering device, which corresponds to the method of the first aspect of the embodiment, and the description of the same content is not repeated.

FIG. 8 is a schematic diagram of a filtering device of an embodiment of the present application. As shown in FIG. 8, the filtering device 800 of an embodiment of the present application includes: a determining unit 801, a quantizing unit 802, and a filtering unit 803. Each set of filter coefficients in the multiple sets of filter coefficients of the channel filter (CC-ALF) determines a quantization scale; the quantization unit 802 uses the quantization scale corresponding to each set of filter coefficients to perform a quantization operation on each set of filter coefficients to obtain each set of filter coefficients. The quantization result of the coefficient; the filtering unit 803 uses the quantization result of each set of filter coefficients to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information.

_{In some embodiments, the determining unit 801 obtains multiple original filter coefficients (C i} , i ∈ [1, 8]) of each set of filter coefficients of the cross-component adaptive loop filter (CC-ALF), according to The maximum value (C _max ) of the plurality of original filter coefficients (C _i ) is calculated for the quantization scale (Scale) corresponding to each group of filter coefficients.

In some embodiments, the determining unit 801 further determines the first range according to the number of bits representing the quantization scale, limits the quantization scale (Scale) to the first range, and obtains the restricted quantization scale (Scale').

In some embodiments, the maximum value C _max in the plurality of original filter coefficients is the original filter coefficient with the largest absolute value among _{the plurality of original filter coefficients C i.}

In some embodiments, the quantization unit 802 uses the quantization scale (Scale) corresponding to each group of filter coefficients to _{perform a quantization operation on each original filter coefficient (C i} ) of each group of filter coefficients to obtain each original filter coefficient of each group of filter coefficients. results quantized coefficients (C _i) of (C _'i).

In some embodiments, the quantization unit 802 further determines the second range according to the number of bits representing the quantization result, limits the quantization result (C′ _i ) to the second range, and obtains the restricted quantization result (C″ _i ) .

In some embodiments, the filtering unit 803 _{performs a filtering operation on the brightness information of the input image according to the quantization result (C" i} ) of each set of filter coefficients and the quantization scale (Scale') corresponding to each set of filter coefficients to obtain chroma compensation. Information as the output of the cross-component adaptive loop filter (CC-ALF).

In some embodiments, for each set of filter coefficients:

The determining unit 801 calculates the quantization scale according to the following formula:

The quantization unit 802 calculates the quantization result according to the following formula: C′ _i =round(C _i ×128×2 ^Scale′ );

The filtering unit 803 calculates the chromaticity compensation information according to the following formula: round(∑C″ _i ×Pi _luma )÷128÷2 ^Scale' ;

Among them, scale is the quantization scale, Scale′ is the limited quantization scale, C _max is the maximum value among multiple original filter coefficients (C _i _{) in a set of filter coefficients, C i} is the original filter coefficient, and _C'i is For the quantization result of the original filter coefficient, C" _i is the quantization result after restriction, and Pi _luma is the brightness information of the input image.

In some embodiments, for each set of filter coefficients:

The quantization unit 802 calculates the quantization result (C′ _i ) according to the following formula: C′ _i =round(C _i ×32×2 ^Scale′ );

The filtering unit 803 calculates the chromaticity compensation information according to the following formula: round(∑C″ _i ×Pi _luma )÷32÷2 ^Scale' ;

In some embodiments, as shown in FIG. 8, the apparatus 800 further includes:

The processing unit 804 saves the quantization scale in an adaptive parameter set (APS).

Fourth aspect of the embodiment

The embodiment of the present application provides a filtering device, which corresponds to the method of the second aspect of the embodiment, and the same contents will not be repeated.

FIG. 9 is a schematic diagram of a filtering device according to an embodiment of the present application. As shown in FIG. 9, the filtering device 900 according to an embodiment of the present application includes: a reading unit 901 and a filtering unit 902. The reading unit 901 obtains from the received code stream The quantization scale Scale; the filtering unit 902 uses the quantization scale Scale to filter the pixels of the input image, and output chrominance compensation information, which is used to compensate the chrominance information of the input image.

Fifth aspect of the embodiment

An embodiment of the present application provides an image encoder. FIG. 10 is a schematic diagram of an image encoder according to an embodiment of the present application. As shown in FIG. 10, the image encoder 1000 according to an embodiment of the present application includes: a luminance filter (SAO Luma) 1001, adaptive loop filter 1002, and chroma filter (ALF Chroma) 1003. Luminance filter (SAO Luma) 1001 performs filtering operations on the input image, and outputs the brightness information of the input image; adaptive loop filter 1002 performs a filtering operation on the brightness information of the input image, and outputs chrominance compensation information; a chroma filter (ALF Chroma) 1003 performs a filtering operation on the input image, and outputs the chroma information of the input image.

In the embodiment of the present application, the aforementioned chrominance compensation information is used to compensate the chrominance information of the input image, and the adaptive loop filter 1002 is configured to implement the method of the first aspect of the embodiment, It may include the device of the third aspect of the embodiment, the content of which is incorporated here, and will not be repeated here.

In the embodiment of the present application, the above-mentioned adaptive loop filter 1002 may include a cross-component adaptive loop filter (CC-ALF Cb) for outputting chrominance compensation information for blue, and may also include a cross-component adaptive loop filter (CC-ALF Cb) for outputting red The cross-component adaptive loop filter (CC-ALF Cr) of the chrominance compensation information, CC-ALF Cb and CC-ALF Cr can respectively implement the method of the first aspect of the embodiment. The description is omitted here.

In the embodiment of the present application, the processing of the luminance filter (SAO Luma) 1001 and the chrominance filter (ALF Chroma) 1003 is not limited, and the prior art can be referred to for details.

The embodiment of the present application also provides an image encoder, the image encoder includes a memory and a processor, the memory stores a computer program, and the processor is configured to execute the foregoing computer program to implement the method of the first aspect of the embodiment. The content is merged here, so I won’t repeat it here.

It is worth noting that the above only describes the components related to the application, but the application is not limited to this. The encoder can also include other components, such as the luminance adaptive loop filter (ALF Luma) as shown in Figure 1, the blue chrominance filter (SAO Cb), the red chrominance filter (SAO Cr), etc. For the specific content of these components, you can refer to related technologies.

Sixth aspect of the embodiment

The embodiment of the present application also provides an image decoder. FIG. 11 is a schematic diagram of the image decoder of the embodiment of the present application. As shown in FIG. 11, the image decoder 1100 of the embodiment of the present application also includes: a luminance filter (SAO Luma) 1101, adaptive loop filter 1102, and chroma filter (ALF Chroma) 1103. Luma filter (SAO Luma) 1101 performs filtering operations on the input image, and outputs the brightness information of the input image; adaptive loop The filter 1102 performs a filtering operation on the brightness information of the input image and outputs chroma compensation information; the chroma filter (ALF Chroma) 1103 performs a filtering operation on the input image, and outputs the chroma information of the input image.

In this embodiment of the application, unlike the image encoder 1000 in the fifth aspect of the embodiment, the adaptive loop filter (CC-ALF Cb/Cr) 1102 is used to obtain the quantization scale from the received code stream. , Using the quantization scale to perform a filtering operation on the luminance information of the input image, and output chrominance compensation information, where the chrominance compensation information is used to compensate the chrominance information of the input image. In addition, the adaptive loop filter 1102 is configured to implement the method of the second aspect of the embodiment, and may include the device of the fourth aspect of the embodiment, the content of which is incorporated herein, and will not be repeated here.

In the embodiment of the present application, the above-mentioned adaptive loop filter 1102 may include a cross-component adaptive loop filter (CC-ALF Cb) for outputting chrominance compensation information for blue, and may also include a cross-component adaptive loop filter (CC-ALF Cb) for outputting red The cross-component adaptive loop filter (CC-ALF Cr) of the chrominance compensation information, CC-ALF Cb and CC-ALF Cr can respectively implement the method of the second aspect of the embodiment. The description is omitted here.

In the embodiment of the present application, the processing of the luminance filter (SAO Luma) 1101 and the chrominance filter (ALF Chroma) 1103 is not limited, and the prior art can be referred to for details.

An embodiment of the present application further provides an image decoder, including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the above computer program to implement the method of the second aspect of the embodiment, which The content is merged here, so I won’t repeat it here.

It is worth noting that the above only describes the components related to the application, but the application is not limited to this. The decoder may also include other components. For details, please refer to related technologies.

Seventh aspect of the embodiment

The seventh aspect of the embodiments of the present application provides an encoding and decoding system, including an image encoder and an image decoder, and the image encoder is configured to: address multiple sets of cross-component adaptive loop filters (CC-ALF) Each group of filter coefficients in the filter coefficients determines a quantization scale; uses the quantization scale corresponding to each group of filter coefficients to quantize each group of filter coefficients to obtain the quantization result of each group of filter coefficients; use the quantization result of each group of filter coefficients to The luminance information of the input image is filtered to obtain the chrominance compensation information; the image decoder is configured to: obtain a quantization scale from the received code stream, use the quantization scale to perform a filtering operation on the luminance information of the input image, and output Chrominance compensation information, where the chrominance compensation information is used to compensate the chrominance information of the input image.

For the specific content of the image encoder, refer to the description of the fifth aspect of the embodiment, and for the specific content of the image decoder, refer to the description of the sixth aspect of the embodiment.

Eighth aspect of the embodiment

An embodiment of the present application provides an image processing device that includes the image encoder according to the fifth aspect of the embodiments, or includes the image decoder according to the sixth aspect of the embodiments, the content of which is combined with this.

Fig. 12 is a schematic diagram of an image processing device according to an embodiment of the present application. As shown in FIG. 12, the image processing device 1200 may include: a central processing unit (CPU) 1201 and a memory 1202; the memory 1202 is coupled to the central processing unit 1201. The memory 1202 can store various data; in addition, it also stores information processing programs, which are executed under the control of the central processing unit 1201.

In some embodiments, the filtering device of the third aspect or the fourth aspect of the embodiment may be integrated into the central processing unit 1201. Wherein, the central processing unit 1201 may be configured to implement the method described in the first aspect or the second aspect of the embodiment.

In some embodiments, the filtering device of the third aspect or the fourth aspect of the embodiment may be configured separately from the central processing unit 1201. For example, the filtering device of the third aspect or the fourth aspect of the embodiment may be configured to be connected to the central processing unit 1201. The chip connected to 1201 realizes the function of the filter device of the third aspect or the fourth aspect of the embodiment through the control of the central processing unit 1201.

In addition, as shown in FIG. 12, the image processing device may further include: an input/output (I/O) device 1203, a display 1204, etc.; wherein the functions of the above-mentioned components are similar to those in the prior art, and will not be repeated here. It is worth noting that the image processing device does not have to include all the components shown in FIG. 12; in addition, the image processing device may also include components not shown in FIG. 12, and reference may be made to the prior art.

The embodiment of the present application provides a computer-readable program, wherein when the program is executed in an image processing device, the program causes the image processing device to execute the method described in the first aspect or the second aspect of the embodiment .

The embodiment of the present application provides a storage medium storing a computer readable program, wherein the computer readable program causes an image processing device to execute the method described in the first aspect or the second aspect of the embodiment.

The above devices and methods of this application can be implemented by hardware, or can be implemented by hardware combined with software. This application relates to such a computer-readable program. When the program is executed by a logic component, the logic component can realize the above-mentioned device or constituent component, or the logic component can realize the above-mentioned various methods. Or operation. This application also relates to storage media used to store the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

The method/device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams and/or one or more combinations of the functional block diagrams shown in the figure may correspond to each software module of the computer program flow or each hardware module. These software modules can respectively correspond to the operations shown in the figure. These hardware modules can be implemented by solidifying these software modules by using a field programmable gate array (FPGA), for example.

The software module can be located in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. A storage medium may be coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be a component of the processor. The processor and the storage medium may be located in the ASIC. The software module can be stored in the memory of the mobile terminal, or can be stored in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a larger-capacity MEGA-SIM card or a large-capacity flash memory device, the software module can be stored in the MEGA-SIM card or a large-capacity flash memory device.

One or more of the functional blocks described in the drawings and/or one or more combinations of the functional blocks can be implemented as general-purpose processors, digital signal processors (DSPs) for performing the functions described in this application. ), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof. One or more of the functional blocks described in the drawings and/or one or more combinations of the functional blocks can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, or multiple micro-processing Processor, one or more microprocessors in communication with the DSP, or any other such configuration.

The application is described above in conjunction with specific implementations, but it should be clear to those skilled in the art that these descriptions are all exemplary and do not limit the scope of protection of the application. Those skilled in the art can make various variations and modifications to the application according to the spirit and principle of the application, and these variations and modifications are also within the scope of the application.

Regarding the implementation including the above examples, the following supplementary notes are also disclosed:

1. A filtering method, wherein the method includes:

2. The method according to appendix 1, wherein determining the quantization scale includes:

_{Acquiring multiple original filter coefficients (C i} , i∈[1,8]) of each set of filter coefficients of the cross-component adaptive loop filter (CC-ALF);

Calculate the quantization scale (Scale) corresponding to each group of filter coefficients according to the maximum value (C _max ) of the plurality of original filter coefficients (C _{i ).}

3. The method according to appendix 2, wherein determining the quantization scale further includes:

Determine the first range according to the number of bits representing the quantization scale;

The quantization scale (Scale) is restricted to the first range to obtain a restricted quantization scale (Scale').

4. The method according to appendix 2, wherein the maximum value C _max of the plurality of original filter coefficients is the original filter coefficient with the largest absolute value among _{the plurality of original filter coefficients C i.}

5. The method according to appendix 1, wherein the quantization operation includes:

Use the quantization scale (Scale) corresponding to each set of filter coefficients to quantize each original filter coefficient (C _i ) of each set of filter coefficients to obtain the quantization result _{of each original filter coefficient (C i) of each set of filter coefficients (} C′ _i ).

6. The method according to appendix 5, wherein performing the quantization operation further includes:

The second range is determined according to the number of bits representing the quantization result, and the quantization result (C' _i ) is limited to the second range to obtain the limited quantization result (C" _i ).

7. The method according to Supplement 1, wherein the filtering operation includes:

According to the quantization result (C" _i ) of each set of filter coefficients and the quantization scale (Scale') corresponding to each set of filter coefficients, the luminance information of the input image is filtered to obtain chrominance compensation information, which is used as the cross-component adaptation The output of the loop filter (CC-ALF).

8. The method according to Supplement 1, wherein, for each set of filter coefficients:

The quantization scale is calculated according to the following formula:

Calculate the quantization result according to the following formula: C′ _i =round(C _i ×128×2 ^Scale′ );

Calculate the chromaticity compensation information according to the following formula: round(∑C″ _i ×Pi _luma )÷128÷2 ^Scale' ;

Among them, scale is the quantization scale, Scale′ is the limited quantization scale, C _max is the maximum value among multiple original filter coefficients (C _i _{) in a set of filter coefficients, C i} is the original filter coefficient, and _C'i is As for the quantization result of the original filter coefficient, C" _i is the limited quantization result, and Pi _luma is the brightness information of the input image.

9. The method according to appendix 1, wherein, for each set of filter coefficients:

The quantization scale is calculated according to the following formula:

Calculate the quantization result (C′ _i ) according to the following formula: C′ _i =round(C _i ×32×2 ^Scale′ );

Calculate the chromaticity compensation information according to the following formula: round(∑C″ _i ×Pi _luma )÷32÷2 ^Scale' ;

10. The method according to Supplement 1, wherein the method further includes:

The quantization scale is stored in the adaptive parameter set (APS).

11. An image encoder comprising a memory and a processor, the memory storing a computer program, and the processor is configured to execute the above computer program to implement the following method:

Use the quantization scale corresponding to each group of filter coefficients to quantize each group of filter coefficients to obtain the quantization result of each group of filter coefficients;

12. An image decoder, comprising a memory and a processor, the memory storing a computer program, and the processor is configured to execute the above computer program to implement the following method:

Obtain the quantization scale from the received code stream, use the quantization scale to filter the luminance information of the input image, and output chrominance compensation information. The chrominance compensation information is used to perform the chrominance information of the input image. make up.

13. An encoding and decoding system, including an image encoder and an image decoder,

The image encoder is configured to: determine a quantization scale for each group of filter coefficients in a plurality of sets of filter coefficients of a cross-component adaptive loop filter (CC-ALF); use a pair of quantization scales corresponding to each group of filter coefficients Perform a quantization operation on each set of filter coefficients to obtain a quantization result of each set of filter coefficients; use the quantization result of each set of filter coefficients to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information;

The image decoder is configured to: obtain a quantization scale from the received code stream, use the quantization scale to perform a filtering operation on the luminance information of the input image, and output chrominance compensation information. The chrominance compensation information is used to The chrominance information of the input image is compensated.

Claims

A filtering device, characterized in that the device comprises:

A determining unit, which determines a quantization scale for each set of filter coefficients in the multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF);

A quantization unit, which uses the quantization scale corresponding to each group of filter coefficients to perform a quantization operation on each group of filter coefficients to obtain a quantization result of each group of filter coefficients;

The filtering unit uses the quantization result of each set of filter coefficients to perform a filtering operation on the luminance information of the input image to obtain chrominance compensation information.
The apparatus according to claim 1, wherein the determining unit obtains a plurality of original filter coefficients (C i , i∈[1) of each set of filter coefficients of the cross-component adaptive loop filter (CC-ALF) , 8]), calculate the quantization scale (Scale) corresponding to each group of filter coefficients according to the maximum value (C max ) of the plurality of original filter coefficients (C i ).
3. The device according to claim 2, wherein the determining unit further determines the first range according to the number of bits representing the quantization scale, and limits the quantization scale (Scale) to the first range to obtain the restricted quantization Scale (Scale').
3. The apparatus according to claim 2, wherein the maximum value C max of the plurality of original filter coefficients is the original filter coefficient with the largest absolute value among the plurality of original filter coefficients C i.
The device according to claim 1, wherein the quantization unit uses the quantization scale (Scale) corresponding to each set of filter coefficients to perform a quantization operation on each original filter coefficient (C i) of each set of filter coefficients to obtain each set of filter coefficients. The quantization result (C′ i ) of each original filter coefficient (C i) of the coefficient.
5. The apparatus according to claim 5, wherein the quantization unit further determines the second range according to the number of bits representing the quantization result, and limits the quantization result (C′ i ) to the second range to obtain the restricted quantization Result (C″ i ).
The device according to claim 1, wherein the filtering unit filters the brightness information of the input image according to the quantization result (C" i ) of each set of filter coefficients and the quantization scale (Scale') corresponding to each set of filter coefficients Operation, chrominance compensation information is obtained as the output of the cross-component adaptive loop filter (CC-ALF).
The device according to claim 1, wherein for each set of filter coefficients:

The determining unit calculates the quantization scale according to the following formula:

The quantization unit calculates the quantization result according to the following formula: C′ i =round(C i ×128×2 Scale′ );

The filtering unit calculates the chromaticity compensation information according to the following formula: round(∑C″ i ×Pi luma )÷128÷2 Scale' ;

Among them, scale is the quantization scale, Scale′ is the limited quantization scale, C max is the maximum value among multiple original filter coefficients (C i ) in a set of filter coefficients, C i is the original filter coefficient, and C'i is For the quantization result of the original filter coefficient, C" i is the quantization result after restriction, and Pi luma is the brightness information of the input image.
The device according to claim 1, wherein for each set of filter coefficients:

The determining unit calculates the quantization scale according to the following formula:

The quantization unit calculates the quantization result (C′ i ) according to the following formula: C′ i =round(C i ×32×2 Scale′ );

The filtering unit calculates the chromaticity compensation information according to the following formula: round(∑C″ i ×Pi luma )÷32÷2 Scale' ;

Among them, scale is the quantization scale, Scale′ is the limited quantization scale, C max is the maximum value among multiple original filter coefficients (C i ) in a set of filter coefficients, C i is the original filter coefficient, and C'i is For the quantization result of the original filter coefficient, C" i is the quantization result after restriction, and Pi luma is the brightness information of the input image.
The device according to claim 1, wherein the device further comprises:

The processing unit saves the quantization scale in an adaptive parameter set (APS).
An image encoder, including:

Luma filter (SAO Luma), which performs a filtering operation on an input image and outputs the brightness information of the input image;

An adaptive loop filter (CC-ALF Cb/Cr), which performs a filtering operation on the luminance information of the input image, and outputs chrominance compensation information; and

Chroma filter (ALF Chroma), which performs a filtering operation on the input image, and outputs the chroma information of the input image;

Wherein, the adaptive loop filter (CC-ALF Cb/Cr) is configured to determine one filter coefficient for each of the multiple sets of filter coefficients of the cross-component adaptive loop filter (CC-ALF) Quantization scale; use the quantization scale corresponding to each set of filter coefficients to quantize each set of filter coefficients to obtain the quantization result of each set of filter coefficients; use the quantization result of each set of filter coefficients to filter the brightness information of the input image to obtain the color The chrominance compensation information is used to compensate the chrominance information of the input image.
An image decoder, including:

Luma filter (SAO Luma), which performs a filtering operation on an input image and outputs the brightness information of the input image;

An adaptive loop filter (CC-ALF Cb/Cr), which performs a filtering operation on the luminance information of the input image, and outputs chrominance compensation information; and

Chroma filter (ALF Chroma), which performs a filtering operation on the input image, and outputs the chroma information of the input image;

Wherein, the adaptive loop filter (CC-ALF Cb/Cr) is configured to obtain a quantization scale from the received code stream, use the quantization scale to filter the brightness information of the input image, and output color The chrominance compensation information is used to compensate the chrominance information of the input image.