CN115988201B - Method, apparatus, electronic device and storage medium for encoding film grain - Google Patents
Method, apparatus, electronic device and storage medium for encoding film grain Download PDFInfo
- Publication number
- CN115988201B CN115988201B CN202310239786.1A CN202310239786A CN115988201B CN 115988201 B CN115988201 B CN 115988201B CN 202310239786 A CN202310239786 A CN 202310239786A CN 115988201 B CN115988201 B CN 115988201B
- Authority
- CN
- China
- Prior art keywords
- current
- coding unit
- unit
- distortion
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000009466 transformation Effects 0.000 claims description 46
- 238000004364 calculation method Methods 0.000 claims description 28
- 238000012937 correction Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 230000001131 transforming effect Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006835 compression Effects 0.000 description 6
- 238000007906 compression Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Images
Abstract
The invention discloses a method, a device, an electronic device and a storage medium for encoding film grain, wherein the method comprises the following steps: calculating coding cost by a reconstruction coding unit according to pixel domain distortion and subjective distortion of an original coding unit and the reconstruction coding unit after the reconstruction coding unit; a predictive coding mode is then selected based on the coding cost. The invention realizes coding under the condition of preserving film grain, and simultaneously avoids the problems of uneven film grain, inconsistent definition of different frames and the like which occur when the film grain is rebuilt by the traditional method.
Description
Technical Field
The present invention relates to the field of image processing technology, and in particular, to a method, apparatus, electronic device, and storage medium for encoding film grain.
Background
Conventional film grain is formed during the development of motion picture images. Film grain, while a random noise, its presence can make video content more realistic and natural. In particular in the motion picture industry, there is an artistic need for controlling grain texture, which is considered part of the motion picture authoring process and intent, and moderate grain texture can help improve subjective perception and can also be used to mask coding artifacts resulting from compression. However, with the widespread use of digital cameras, digital image video does not have film grain due to the imaging principles. The viewer, while watching an image video without film grain, will create a sensation that, although the rendered picture is clear, it appears to lack realism. Thus, in order to provide texture to the presented content, the content producer may wish to preserve the original film grain in the image video and even actively add film grain to the image video content to create unique cinema characteristics.
However, film grain is also inherently a high frequency noise and has a different temporal randomness than other types of noise, with irregular distribution and size. This randomness makes it difficult for conventional coding techniques to compress them efficiently, and prediction becomes very difficult, and the accuracy of motion estimation is reduced. Film grain is not well preserved in current video coding standards. The original image video is filtered and lossy compressed during the encoding process, and the encoder suppresses such high frequency signals as film grain as noise and cannot be reconstructed when decoding. If film grain is preserved by lossless compression of the image video, a huge code stream is consumed, and the practical applicability is lacking.
The current approach to film grain coding is to first remove the film grain, then encode the video with the film grain removed, model the film grain at the same time, and finally re-synthesize the film grain during decoding by modeling parameters. The new generation of coding standard AV1 writes the film grain processing tool to the standard for the first time, which also works by first removing the film grain by denoising and estimating film grain parameters that will be sent with the compressed video stream to the decoder. After decoding, the film grain is then synthesized and added to the reconstructed video frame. However, such processing can have the problem of inadequate uniformity of reconstructed film grain and inconsistent sharpness of different frames, particularly when processing video that has been compressed, where it is difficult to reconstruct the film grain again. The above problems are largely avoided if the film grain can be directly subjected to retained compression. Also, due to the randomness of the film grain, direct retention compression of the film grain presents challenges to image video compression algorithms.
Disclosure of Invention
In view of the foregoing problems with existing methods, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for encoding film grain.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, embodiments of the present invention provide a method of encoding film grain, comprising:
and traversing the coding prediction mode for the current original coding unit to obtain the current reconstruction coding unit.
And calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
And calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
And selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
In a second aspect, embodiments of the present invention provide an apparatus for encoding film grain, comprising:
and the coding module is used for traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
And the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
And the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
And the mode selection module is used for selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
In a third aspect, embodiments of the present invention also provide an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of encoding film grain according to the first aspect when the computer program is executed.
In a fourth aspect, embodiments of the present invention also provide a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of encoding film grain according to the first aspect.
From the above technical solutions, the present application realizes encoding while preserving film grain, and avoids the problems of insufficient uniformity of film grain, inconsistent sharpness of different frames, etc. that occur when film grain is reconstructed by conventional methods.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other drawings can be obtained from these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an example diagram of CTU partition CUs in an HEVC video coding standard.
Fig. 2 is an exemplary diagram of a CU divided into PUs and TUs in the HEVC video coding standard.
Fig. 3 is a flow chart of a film grain encoding method according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of h.265/HEVC prediction modes.
Fig. 5 is a schematic structural diagram of a film grain encoding apparatus according to an embodiment of the present invention.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following describes the embodiments of the present invention further with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.
For a better understanding of the following examples, the basic concepts will be explained first.
Video is made up of a sequence of images. The video frame is a sequence of images in the video, and each image in the video can be regarded as a frame. The current video frame is the video frame that currently needs to be encoded. The current mainstream video Coding standard generally adopts a hybrid Coding framework, including h.264/AVC and h.265/HEVC, in the video Coding process, the current video frame is subjected to block division Coding, taking h.265/HEVC as an example, as shown in fig. 1, in the h.265/HEVC, a quadtree division mode is adopted to divide the video frame in the video into Coding Tree Units (CTUs), the size of the Coding Tree is generally set to 64×64, each CTU can be further uniformly divided into 4 Coding units (Coding units, CUs), and one CTU can be recursively divided into 4 small CUs according to the quadtree structure. As shown in fig. 2, each CU may include one or more Prediction Units (PUs) of different sizes, and further each PU may further include a plurality of Transform Units (TUs), where quantization is performed on TUs, typically by dividing TUs into a plurality of coefficient sets (Coefficient Group, CG), and performing quantization on each CG, i.e., quantization Unit.
The transformation of the image is to transform the spatial domain signal to the frequency domain signal, effectively removing the correlation of the signal and concentrating most of the energy into the low frequency region. Common transformation methods such as DCT transformation, DST transformation, hadmard transformation, etc. The transformation compresses the energy of the matrix into the first element, known as the Direct Current (DC) coefficient. The remaining coefficients are referred to as Alternating Current (AC) coefficients. After transformation, the transform coefficient energy is mainly concentrated in the upper left region. The coefficients in this region are very significant and tend to concentrate image information that is sensitive to the human eye. After quantization of the transform coefficients, the quantized coefficients in the upper left region are significant, while the quantized coefficients in the lower right region are mostly zero. Quantization is one of the sources of distortion from video/image compression, and the zero quantized coefficients tend to be compressed high frequency signals that include film grain.
Fig. 3 is a flowchart of a method for encoding film grain according to an embodiment of the present invention, and as shown in fig. 3, the method for encoding film grain according to an embodiment of the present invention specifically includes the following:
and step 1, traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
In this step, it is understood that the mainstream coding standards are currently based on a hybrid video coding framework. The hybrid framework mainly comprises links such as prediction (prediction), transformation (transformation), quantization (quantization), entropy coding (entropy coding) and the like. The prediction link is to generate a prediction pixel corresponding to an original pixel of the current coding unit by using a reconstructed pixel of the coded region. The coding unit where the original pixel is located is the current original coding unit, and the coding unit where the predicted pixel is located is the reconstructed coding unit. The prediction mode includes two major types, i.e., intra prediction (intra prediction) and inter prediction (inter prediction). The specific coding prediction modes may vary from video standard to video standard. As shown in fig. 4, taking h.265/HEVC as an example, 35 intra prediction modes are specified in HEVC, including Planar mode, DC mode, and 33 angle modes. One or more coding prediction modes can be set in practical application.
And 2, calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
The calculating the pixel domain distortion of the current original coding unit and the current reconstructed coding unit may be obtained by calculating an absolute error sum (Sum of Absolute Difference, SAD), a hadamard transform absolute error sum (Sum of Absolute Transformed Difference, SATD), a difference square sum (Sum of Squared Difference, SSD), an average absolute error (Mean Absolute Difference, MAD), an average square error (Mean Squared Difference, MSD), and the like.
The calculating subjective distortion of the current original coding unit and the current reconstruction coding unit comprises the following steps:
in one embodiment, subjective distortion of the current original coding unit and the current reconstructed coding unit may be represented by an energy difference between the current original coding unit and the current reconstructed coding unit, including:
and carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit.
And transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit.
Wherein the transformation includes, but is not limited to: DCT transform, hadnard transform, DST transform.
And calculating the energy difference between the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transform unit, the distortion is subjective distortion, and abs is an absolute function.
Further, in order to measure the subjective distortion between the current original encoding unit and the current reconstructed encoding unit more accurately, the method for calculating the subjective distortion may be further modified as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon AC series for the reconstruction transformation unitThe number, the distriction is subjective distortion, abs is absolute function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon For the number of non-zero transform coefficients in the current reconstructed coding unit.
In another embodiment, the subjective distortion of the current original encoding unit and the current reconstruction encoding unit may be represented by a pixel variance difference value of the current original encoding unit and the current reconstruction encoding unit, and the calculation formula is as follows:
wherein X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current recoding unit, N is the number of pixels in the current coding unit, the distortion is subjective distortion, and abs is an absolute value function.
And step 3, calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
The calculation formula of the coding cost is as follows:
wherein D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate.
Further, the higher the subjective distortion between the current original coding unit and the current reconstructed coding unit, the greater the probability that film grain is removed. To avoid film grain removal, subjective distortion is weighted and the coding cost is corrected as follows:
where k is a correction factor and k is greater than 1, d is pixel domain distortion, distortion is subjective distortion, λ is lagrangian multiplier, and R is the current coding rate.
Further, for flat areas, film grain tends to be weaker and the human eye has a relatively low sensitivity to film grain in such areas. In order to preserve the film grain more accurately, the calculation of the encoding cost can also be modified according to the degree of film grain.
The correction of the coding cost calculation according to the intensity degree of the film grain comprises the following steps:
and judging the intensity degree of the film grain of the current original coding unit.
Correcting the calculation of the coding cost according to the judgment result:
if the current original coding unit is judged to be a region with weaker film grain, the calculation formula of the coding cost is as follows:
wherein m is a correction factor and is smaller than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate.
If the current original coding unit is judged to be a region with stronger film grain degree, the calculation formula of the coding cost is as follows:
where k is a correction factor and k is greater than 1, d is pixel domain distortion, distortion is subjective distortion, λ is lagrangian multiplier, and R is the current coding rate.
The judging the film grain intensity of the current original coding unit comprises the following steps:
calculating pixel variance of the current original coding unit:
wherein X is the pixel value in the current original coding unit, X' is the average value of the pixels in the current original coding unit, and N is the number of the pixels in the coding unit.
And presetting a pixel variance threshold T of the current original coding unit.
And if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is a region with weaker film grain degree, otherwise, judging that the current original coding unit is a region with stronger film grain degree.
And 4, selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
Fig. 5 shows a schematic structural diagram of an apparatus for encoding film grain according to an embodiment of the present invention, as shown in fig. 5:
and the coding module is used for traversing the coding prediction mode of the current original coding unit to obtain the current reconstruction coding unit.
And the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit.
The calculating the pixel domain distortion of the current original coding unit and the current reconstructed coding unit may be obtained by calculating an absolute error sum (Sum of Absolute Difference, SAD), a hadamard transform absolute error sum (Sum of Absolute Transformed Difference, SATD), a difference square sum (Sum of Squared Difference, SSD), an average absolute error (Mean Absolute Difference, MAD), an average square error (Mean Squared Difference, MSD), and the like.
The calculating subjective distortion of the current original coding unit and the current reconstruction coding unit comprises the following steps:
in one embodiment, subjective distortion of the current original coding unit and the current reconstructed coding unit may be represented by an energy difference between the current original coding unit and the current reconstructed coding unit, including:
and carrying out block division on the current original coding unit to obtain an original transformation unit, and carrying out block division on the current reconstruction coding unit to obtain a reconstruction transformation unit.
And transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit.
Wherein the transformation includes, but is not limited to: DCT transform, hadnard transform, DST transform.
And calculating the energy difference between the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transform unit, the distortion is subjective distortion, and abs is an absolute function.
Further, in order to measure the subjective distortion between the current original encoding unit and the current reconstructed encoding unit more accurately, the method for calculating the subjective distortion may be further modified as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transformation unit, the distortion is subjective distortion, abs is absolute function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon For the number of non-zero transform coefficients in the current reconstructed coding unit.
In another embodiment, the subjective distortion of the current original encoding unit and the current reconstruction encoding unit may be represented by a pixel variance difference value of the current original encoding unit and the current reconstruction encoding unit, and the calculation formula is as follows:
wherein X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current recoding unit, N is the number of pixels in the current coding unit, the distortion is subjective distortion, and abs is an absolute value function.
And the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion.
The calculation formula of the coding cost is as follows:
wherein D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate.
Further, the higher the subjective distortion between the current original coding unit and the current reconstructed coding unit, the greater the probability that film grain is removed. To avoid film grain removal, subjective distortion is weighted and the coding cost is corrected as follows:
where k is a correction factor and k is greater than 1, d is pixel domain distortion, distortion is subjective distortion, λ is lagrangian multiplier, and R is the current coding rate.
Further, for flat areas, film grain tends to be weaker and the human eye has a relatively low sensitivity to film grain in such areas. In order to preserve the film grain more accurately, the calculation of the encoding cost can also be modified according to the degree of film grain.
The correction of the coding cost calculation according to the intensity degree of the film grain comprises the following steps:
and judging the intensity degree of the film grain of the current original coding unit.
Correcting the calculation of the coding cost according to the judgment result:
if the current original coding unit is judged to be a region with weaker film grain, the calculation formula of the coding cost is as follows:
wherein m is a correction factor and is smaller than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate.
If the current original coding unit is judged to be a region with stronger film grain degree, the calculation formula of the coding cost is as follows:
where k is a correction factor and k is greater than 1, d is pixel domain distortion, distortion is subjective distortion, λ is lagrangian multiplier, and R is the current coding rate.
The judging the film grain intensity of the current original coding unit comprises the following steps:
calculating pixel variance of the current original coding unit:
wherein X is the pixel value in the current original coding unit, X' is the average value of the pixels in the current original coding unit, and N is the number of the pixels in the coding unit.
And presetting a pixel variance threshold T of the current original coding unit.
And if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is a region with weaker film grain degree, otherwise, judging that the current original coding unit is a region with stronger film grain degree.
And the mode selection module is used for selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
Based on the same inventive concept, a further embodiment of the present invention provides an electronic device, as shown in fig. 6, including: a processor 601, a memory 602, a communication interface 603 and a communication bus 604.
Wherein the processor 601, the memory 602, and the communication interface 603 complete communication with each other through the communication bus 604; the communication interface 603 is used to implement information transmission between devices.
The processor 601 is operative to invoke a computer program in the memory 602 which when executed implements all the steps of the method of encoding film grain described above.
Based on the same inventive concept, a further embodiment of the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements all the steps of the above-described method of encoding film grain.
The logic instructions in the memory described above may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Moreover, in embodiments of the present invention, relational terms such as "first" and "second", and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. In the description of the present specification, a description of "in one embodiment," "in another embodiment," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments. Furthermore, the various embodiments described in this specification, as well as the features of the various embodiments, can be combined and combined by one skilled in the art without contradiction.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (16)
1. A method of encoding film grain comprising:
traversing the coding prediction mode for the current original coding unit to obtain a current reconstruction coding unit;
calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit;
calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion;
when the coding cost of the current coding unit is calculated according to the pixel domain distortion and the subjective distortion, correcting the calculation of the coding cost according to the intensity of film grains, including:
judging the intensity degree of film grains of the current original coding unit;
correcting the calculation of the coding cost according to the judgment result:
if the current original coding unit is judged to be a region with weaker film grain, the calculation formula of the coding cost is as follows:
wherein m is a correction factor and is smaller than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate;
if the current original coding unit is judged to be a region with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor and k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate;
and selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
2. The method of encoding film grain of claim 1, wherein said calculating pixel domain distortion of said current original coding unit and said current reconstructed coding unit uses an algorithm comprising:
absolute error sum, hadamard transform absolute error sum, difference square sum, average absolute error, average square error.
3. The method of encoding film grain of claim 1, wherein said calculating subjective distortion of said current original coding unit and said current reconstructed coding unit comprises:
performing block division on the current original coding unit to obtain an original transformation unit, and performing block division on the current reconstruction coding unit to obtain a reconstruction transformation unit;
transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit;
and calculating subjective distortion of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transform unit, the distortion is subjective distortion, and abs is an absolute function.
4. The method of encoding film grain of claim 3, wherein said calculating subjective distortion of said current original encoding unit and said current reconstructed encoding unit from AC coefficients of said original transform unit and AC coefficients of said reconstructed transform unit is formulated as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transformation unit, the distortion is subjective distortion, abs is absolute function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon For the number of non-zero transform coefficients in the current reconstructed coding unit.
5. The method of encoding film grain of claim 3, wherein the transforming comprises: DCT transform, hadnard transform, DST transform.
6. The method of encoding film grain of claim 1, wherein said calculating subjective distortion of said current original coding unit and said current reconstructed coding unit further comprises:
wherein ,
for the pixel variance of the current original coding unit,
for the pixel variance of the current reconstruction coding unit, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current recoding unit, N is the number of pixels in the current coding unit, the distortion is subjective distortion, and abs is an absolute function.
7. The method of encoding film grain of claim 1, wherein said determining the degree of film grain of the current original encoding unit comprises:
calculating pixel variance of the current original coding unit:
wherein X is the pixel value in the current original coding unit, X' is the average value of the pixels in the current original coding unit, and N is the number of the pixels in the coding unit;
presetting a pixel variance threshold T of the current original coding unit;
and if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is a region with weaker film grain degree, otherwise, judging that the current original coding unit is a region with stronger film grain degree.
8. An apparatus for encoding film grain comprising:
the coding module is used for traversing the coding prediction mode of the current original coding unit to obtain a current reconstruction coding unit;
the first calculation module is used for calculating pixel domain distortion and subjective distortion of the current original coding unit and the current reconstruction coding unit;
the second calculation module is used for calculating the coding cost of the current coding unit according to the pixel domain distortion and the subjective distortion;
when the coding cost of the current coding unit is calculated according to the pixel domain distortion and the subjective distortion, correcting the calculation of the coding cost according to the intensity of film grains, including:
judging the intensity degree of film grains of the current original coding unit;
correcting the calculation of the coding cost according to the judgment result:
if the current original coding unit is judged to be a region with weaker film grain, the calculation formula of the coding cost is as follows:
wherein m is a correction factor and is smaller than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate;
if the current original coding unit is judged to be a region with stronger film grain degree, the calculation formula of the coding cost is as follows:
wherein k is a correction factor and k is greater than 1, D is pixel domain distortion, distortion is subjective distortion, lambda is Lagrangian multiplier, and R is current coding rate;
and the mode selection module is used for selecting a coding prediction mode corresponding to the minimum coding cost to code according to the coding cost.
9. The apparatus for encoding film grain of claim 8, wherein said calculating pixel domain distortion of said current original encoding unit and said current reconstructed encoding unit uses an algorithm comprising:
absolute error sum, hadamard transform absolute error sum, difference square sum, average absolute error, average square error.
10. The apparatus for encoding film grain of claim 8, wherein said calculating subjective distortion of said current original coding unit and said current reconstructed coding unit comprises:
performing block division on the current original coding unit to obtain an original transformation unit, and performing block division on the current reconstruction coding unit to obtain a reconstruction transformation unit;
transforming the original transformation unit to obtain an AC coefficient of the original transformation unit, and transforming the reconstruction transformation unit to obtain an AC coefficient of the reconstruction transformation unit;
and calculating subjective distortion of the current original coding unit and the current reconstruction coding unit according to the AC coefficient of the original transformation unit and the AC coefficient of the reconstruction transformation unit, wherein the formula is as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transform unit, the distortion is subjective distortion, and abs is an absolute function.
11. The apparatus for encoding film grain of claim 10, wherein said calculating subjective distortion of said current original encoding unit and said current reconstructed encoding unit from AC coefficients of said original transform unit and AC coefficients of said reconstructed transform unit is formulated as follows:
wherein ,Esrc AC for the energy value of the current original coding unit src AC coefficients for the original transform unit; e (E) recon AC for the energy value of the current reconstruction coding unit recon For the AC coefficients of the reconstruction transformation unit, the distortion is subjective distortion, abs is absolute function, N scr For the number of non-zero transform coefficients of the current original coding unit, N recon For the number of non-zero transform coefficients in the current reconstructed coding unit.
12. The apparatus for encoding film grain of claim 10, wherein the transforming comprises: DCT transform, hadnard transform, DST transform.
13. The apparatus for encoding film grain of claim 8, wherein said calculating subjective distortion of said current original coding unit and said current reconstructed coding unit further comprises:
wherein ,
for the pixel variance of the current original coding unit,
for the pixel variance of the current reconstruction coding unit, X is the pixel value in the current original coding unit, X 'is the pixel average value in the current original coding unit, Y is the pixel value in the current reconstruction coding unit, Y' is the pixel average value in the current recoding unit, N is the number of pixels in the current coding unit, the distortion is subjective distortion, and abs is an absolute function.
14. The apparatus for encoding film grain of claim 8, wherein said determining the degree of film grain of said current original encoding unit comprises:
calculating pixel variance of the current original coding unit:
wherein X is the pixel value in the current original coding unit, X' is the average value of the pixels in the current original coding unit, and N is the number of the pixels in the coding unit;
presetting a pixel variance threshold T of the current original coding unit;
and if the pixel variance of the current original coding unit is smaller than the variance threshold T, judging that the current original coding unit is a region with weaker film grain degree, otherwise, judging that the current original coding unit is a region with stronger film grain degree.
15. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of encoding film grain of any one of claims 1 to 7 when the computer program is executed.
16. A non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of encoding film grain of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310239786.1A CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310239786.1A CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115988201A CN115988201A (en) | 2023-04-18 |
| CN115988201B true CN115988201B (en) | 2023-05-30 |
Family
ID=85961138
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310239786.1A Active CN115988201B (en) | 2023-03-14 | 2023-03-14 | Method, apparatus, electronic device and storage medium for encoding film grain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115988201B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101883280A (en) * | 2009-05-07 | 2010-11-10 | 香港科技大学 | A kind of video coding-decoding method and system of recovering noise |
| CN115174898A (en) * | 2022-06-27 | 2022-10-11 | 福州大学 | Rate distortion optimization method based on visual perception |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634600B (en) * | 2012-08-22 | 2016-09-28 | 中国科学院声学研究所 | A kind of Video Encoding Mode system of selection based on SSIM evaluation, system |
| US10277907B2 (en) * | 2012-10-25 | 2019-04-30 | Integrated Device Technology, Inc. | Rate-distortion optimizers and optimization techniques including joint optimization of multiple color components |
| US20160142716A1 (en) * | 2014-11-17 | 2016-05-19 | Vixs Systems, Inc. | Video coder with simplified rate distortion optimization and methods for use therewith |
| CN107548556A (en) * | 2015-04-21 | 2018-01-05 | Vid拓展公司 | Video coding based on artistic intent |
| CN109982082B (en) * | 2019-05-05 | 2022-11-15 | 山东大学 | HEVC multi-distortion criterion rate-distortion optimization method based on local texture characteristics |
| CN112381744B (en) * | 2020-10-27 | 2022-10-11 | 杭州微帧信息科技有限公司 | Adaptive pretreatment method for AV1 synthetic film grain |
-
2023
- 2023-03-14 CN CN202310239786.1A patent/CN115988201B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101883280A (en) * | 2009-05-07 | 2010-11-10 | 香港科技大学 | A kind of video coding-decoding method and system of recovering noise |
| CN115174898A (en) * | 2022-06-27 | 2022-10-11 | 福州大学 | Rate distortion optimization method based on visual perception |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115988201A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11589041B2 (en) | Method and apparatus of neural network based processing in video coding | |
| KR102287947B1 (en) | Apparatus and method for performing artificial intelligence encoding and artificial intelligence decoding of image | |
| CN108886621B (en) | Non-local self-adaptive loop filtering method | |
| CN111819854B (en) | Method and apparatus for coordinating multi-sign bit concealment and residual sign prediction | |
| US10469876B2 (en) | Non-local adaptive loop filter combining multiple denoising technologies and grouping image patches in parallel | |
| WO2021139770A1 (en) | Signaling quantization related parameters | |
| CN103782598A (en) | Fast encoding method for lossless coding | |
| US8594189B1 (en) | Apparatus and method for coding video using consistent regions and resolution scaling | |
| CN110944179B (en) | Video data decoding method and device | |
| CN116916036A (en) | Video compression method, device and system | |
| US20240064309A1 (en) | Residual coding method and device, video coding method and device, and storage medium | |
| EP3759917A1 (en) | A spectrally adaptive noise filling tool (sanft) for perceptual transform coding of still and moving images | |
| JP2009153226A (en) | Decoding method | |
| CN112001854A (en) | Method for repairing coded image and related system and device | |
| CN115988201B (en) | Method, apparatus, electronic device and storage medium for encoding film grain | |
| CN113709459B (en) | Intra-frame prediction method, device and computer storage medium | |
| KR20130078320A (en) | Method and device for encoding depth image | |
| CN115280772A (en) | Dual standard block partitioning heuristic for lossy compression | |
| CN116248895B (en) | Video cloud transcoding method and system for virtual reality panorama roaming | |
| US20230044603A1 (en) | Apparatus and method for applying artificial intelligence-based filtering to image | |
| US20200413105A1 (en) | Metric driven adaptive processing for video encoders | |
| WO2023102290A1 (en) | Method and apparatus for chart based mesh compression | |
| WO2020000487A1 (en) | Transformation method, inverse transformation method and apparatus | |
| Cheng et al. | JND-based Wyner-Ziv Video Coding. | |
| US20200329232A1 (en) | Method and device for encoding or decoding video signal by using correlation of respective frequency components in original block and prediction block |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |