CN113079374B - Image encoding method and apparatus - Google Patents
Image encoding method and apparatus Download PDFInfo
- Publication number
- CN113079374B CN113079374B CN202110237593.3A CN202110237593A CN113079374B CN 113079374 B CN113079374 B CN 113079374B CN 202110237593 A CN202110237593 A CN 202110237593A CN 113079374 B CN113079374 B CN 113079374B
- Authority
- CN
- China
- Prior art keywords
- coding unit
- texture
- texture intensity
- intensity
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 abstract description 15
- 238000005192 partition Methods 0.000 description 11
- 238000000638 solvent extraction Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000005477 standard model Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses an image coding method and device. An image encoding method comprising: acquiring an image to be coded; in the process of segmenting the image to be coded to obtain a plurality of coding units, corresponding to any one coding unit, calculating a brightness gradient value of the coding unit; calculating texture intensities of the encoding units in multiple directions according to the brightness gradient values; judging whether the conditions for not continuously segmenting the coding unit are met or not according to the texture intensities in the multiple directions; if so, stopping continuously dividing the coding unit. The method and the device stop the division of the coding unit in time, improve the coding efficiency and reduce the coding calculation amount.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to an image encoding method and apparatus.
Background
With the rapid development of information technology, the communication mode of people is changed greatly, and the vision is used as the fastest and most efficient mode for acquiring information, so that the real-time video transmission is widely applied in life. However, the data size of the original video is very large, and it must be compressed and encoded before transmission in a limited channel. H.265/HEVC is formally released by JCT-VC in 2013 as a new generation international video coding standard, although the compression efficiency is improved by 50% compared with the H.264/AVC standard of the previous generation, the computational complexity of the HEVC is also greatly improved, and the video coding time is increased. In the process of encoding an image frame, the image frame is often divided into one layer or multiple layers. The coding units are obtained, and before each coding unit is divided, rate distortion needs to be calculated through prediction. If each image frame is divided into 4 layers to obtain the minimum coding units, 35 prediction modes are used, and if 35 prediction modes are used for predicting and calculating the rate distortion for each minimum coding unit, a large amount of calculation resources are consumed.
Disclosure of Invention
It is a primary objective of the present application to provide an image encoding method and apparatus to solve the above problems in the prior art.
In order to achieve the above object, according to an aspect of the present application, there is provided an image encoding method including:
acquiring an image to be coded;
in the process of segmenting the image to be coded to obtain a plurality of coding units, corresponding to any one coding unit, calculating the brightness gradient value of the coding unit;
calculating texture intensities of the coding units in multiple directions according to the brightness gradient values;
judging whether the texture intensity in the multiple directions meets the condition that the coding unit is not continuously segmented or not according to the texture intensity in the multiple directions;
if so, stopping continuously dividing the coding unit.
In some embodiments, determining whether the condition for not continuing to partition the coding unit according to the texture strengths of the plurality of directions is met includes:
if the maximum value of the texture intensity of each direction is less than a predetermined first threshold value Th1 and has directivity; or,
if the difference between the texture intensity of each direction and zero is smaller than a predetermined second threshold Th2 and has similarity.
In some embodiments, the plurality of directions is four directions; determining to have directionality, comprising:
if the difference between the texture intensity in the first direction and the texture intensity in the second direction, the difference between the texture intensity in the third direction and the texture intensity in the fourth direction are respectively larger than the preset direction difference threshold value;
in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction, the difference value of any two texture intensities is smaller than a preset similarity difference value threshold value;
the directivity is determined to be the first direction.
In some embodiments, the directionality is determined using the following formula:
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity for the third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; td is a first proportional threshold.
In some embodiments, determining that there is a similarity comprises: and if the difference value of the texture strength of any two directions is less than a preset similarity difference value threshold value, determining that the two directions have similarity.
In some embodiments, the similarity is calculated using the following formula:
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the grain strength of the second direction;
V 3 texture intensity for the third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; ts is the second proportional threshold.
In some embodiments, the method further comprises:
judging whether the conditions for continuously dividing the coding unit are met or not according to the texture intensities in the multiple directions;
if so, continuing to divide the coding unit.
In some embodiments, if neither the condition for continuing the partitioning of the coding unit nor the condition for stopping the partitioning of the coding unit is met, the method further comprises:
continuously dividing the coding unit to obtain a plurality of sub-coding units;
calculating the sum of rate-distortion costs of a plurality of sub-coding units;
if the sum of the rate distortion costs of a plurality of sub-coding units is less than or equal to the rate distortion cost of the coding unit, continuing to divide the coding unit;
and stopping continuously dividing the coding unit if the sum of the rate distortion costs of the plurality of sub-coding units is larger than the rate distortion cost of the coding unit.
In some embodiments, the method further comprises: determining a prediction mode group of the coding unit;
performing error prediction on the coding unit according to the prediction mode group to obtain the rate distortion cost of the coding unit;
the determining the prediction mode group of the coding unit specifically includes:
determining a minimum texture intensity value of the texture intensity values of the plurality of directions of the coding unit;
and determining the direction corresponding to the minimum texture intensity value and the prediction mode group corresponding to the direction.
In some embodiments, the plurality of directions are evenly distributed over 360 degrees.
According to a second aspect of the present application, the present invention also proposes an image encoding device comprising:
the image acquisition module is used for acquiring an image to be coded;
the segmentation module is used for segmenting an image to be coded; after the judgment module determines that the condition of not continuously dividing the coding unit is met, the coding unit is stopped to be continuously divided;
the computing module is used for computing the brightness gradient value of the coding unit corresponding to any one coding unit in the process of obtaining a plurality of coding units by segmenting the image to be coded by the segmenting module; and calculating texture intensities of a plurality of directions of the coding unit according to the brightness gradient values;
and the judging module is used for judging whether the condition that the coding unit is not continuously segmented is met according to the texture intensities in the directions.
In one embodiment, the determining module is further configured to determine whether the maximum value of the texture intensity of each direction is less than a predetermined first threshold Th1 and has directionality; or,
if the difference between the texture intensity of each direction and zero is smaller than a predetermined second threshold Th2 and has similarity.
In one embodiment, the determining module is further configured to determine whether the difference between the texture intensity in the first direction and the texture intensity in the second direction, the texture intensity in the third direction, and the texture intensity in the fourth direction is greater than a predetermined direction difference threshold;
in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction, the difference value of any two texture intensities is smaller than a preset similarity difference value threshold value;
the directivity is determined to be the first direction.
In one embodiment, the determining module is further configured to,
wherein, V 1 Strength of grain in a first direction；
V 2 Is the grain strength of the second direction;
V 3 texture intensity in a third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; td is a first proportional threshold.
In one embodiment, the determining module is further configured to determine that there is similarity if the difference between the texture intensities of any two directions is smaller than a predetermined similarity difference threshold.
In one embodiment, the determining module is further configured to calculate the similarity using the following formula:
wherein, V 1 Is the grain strength in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity for the third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; ts is the second proportional threshold.
In one embodiment, the determining module is further configured to determine whether a condition for continuing to partition the coding unit is met according to the texture intensities of the multiple directions; if so, the coding unit is continuously divided.
In one embodiment, the determining module is further configured to, if neither the condition for continuing the partitioning of the coding unit nor the condition for stopping the partitioning of the coding unit is met; continuously dividing the coding unit to obtain a plurality of sub coding units;
calculating the sum of rate-distortion costs of a plurality of sub-coding units;
if the sum of the rate distortion costs of a plurality of sub-coding units is less than or equal to the rate distortion cost of the coding unit, continuing to divide the coding unit;
and if the sum of the rate distortion costs of the sub-coding units is larger than the rate distortion cost of the coding unit, stopping continuously dividing the coding unit.
In some embodiments, the method further comprises determining a prediction mode group of the coding unit;
the rate distortion cost determining module is used for carrying out error prediction on the coding unit according to the prediction mode group to obtain the rate distortion cost of the coding unit;
the prediction mode group determination module is further configured to determine a minimum texture intensity value among texture intensity values of a plurality of directions of the coding unit;
and determining the direction corresponding to the minimum texture intensity value and the prediction mode group corresponding to the direction. In order to achieve the above object, according to a third aspect of the present application, there is provided an electronic apparatus; comprising at least one processor and at least one memory; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform any of the methods described above.
According to a fourth aspect of the present application, there is provided a computer readable storage medium having one or more program instructions embodied therein for performing the method of any of the above.
In the technical scheme of the invention, in the process of segmenting the image to be coded to obtain a plurality of coding units, the brightness gradient value of the coding unit is calculated corresponding to any one coding unit; calculating texture intensities of the coding units in multiple directions according to the brightness gradient values; judging whether the texture intensity in the multiple directions meets the condition that the coding unit is not continuously segmented or not according to the texture intensity in the multiple directions; if so, stopping continuously dividing the coding unit. Therefore, the processes of unnecessary sub-coding unit division and rate distortion cost calculation can be omitted, the coding efficiency can be improved, and the coding calculation amount can be reduced.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the application and to enable other features, objects, and advantages of the application to be more apparent. The drawings and the description of the exemplary embodiments of the present application are provided for explaining the present application and do not constitute an undue limitation on the present application. In the drawings:
FIG. 1 is a flow chart of a method of image encoding according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a Sobel operator according to an embodiment of the present application;
FIG. 3A is a diagram illustrating a coding unit partitioning method according to an embodiment of the present application;
FIG. 3B is a diagram illustrating another coding unit partitioning method according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating comparison between results obtained after coding tests are performed on different video sequences and results obtained after coding according to the HEVC standard according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an image encoding device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
In the process of coding an image, the image needs to be divided into multiple layers, generally including 4 layers, if the image is divided into 4 layers, 35 prediction modes are adopted for each minimum coding unit to calculate the rate distortion cost, and the calculation amount is too large, which causes serious waste of calculation resources.
Based on this, the present application proposes an image encoding method, see the flowchart of an image encoding method shown in fig. 1; an image encoding method comprising:
step S102, acquiring an image to be coded;
illustratively, an original YUV420 image with a resolution of 832 × 480 in the HEVC standard test sequence is selected, with a pixel depth of 8 bits.
Reading video image frames, and analyzing image information including width, height, brightness component Y, chrominance component U and V of the image.
Step S104, in the process of obtaining a plurality of coding units by dividing the image to be coded, corresponding to any one coding unit, calculating the brightness gradient value of the coding unit;
illustratively, the luminance component Y of the read image is processed. Calculating the brightness gradient value in the corresponding direction of each pixel point except for the boundary in the image by utilizing the Sobel operators in the four directions, wherein the Sobel operators refer to an attached diagram 2; the calculation method is shown in formulas (1) to (4):
x is the abscissa of the pixel point;
y is the ordinate of the pixel point;
p x+1，y-1 the brightness of a pixel point with the coordinate of (x +1, y-1);
p x，y-1 the brightness of a pixel point with the coordinate of (x, y-1);
p x-1，y-1 the brightness of the pixel point with the coordinate of (x-1, y-1);
p x-1，y+1 the brightness of a pixel point with the coordinate of (x-1, y + 1);
p x+1，y+1 the brightness of the pixel point with the coordinate (x +1, y + 1);
p x，y+1 the brightness of a pixel point with the coordinate of (x, y + 1);
The luminance gradient value of the pixel point on the boundary is set to 0.
Step S106, calculating the texture intensity of the coding unit in multiple directions according to the brightness gradient value;
wherein the number n of the plurality of directions is evenly distributed within a range of 360 degrees. If n is equal to 2, then 180 degrees in both directions. If n is equal to 4, the separation angle between any two adjacent directions is 90 degrees, and if n is equal to 8, the separation angle between any two adjacent directions is 45 degrees.
Preferably, n is equal to 4, the plurality of directions comprising: horizontal, vertical, 45 degree and 135 degree.
Illustratively, all the coding units (corresponding to the partition depths of 0,1,2,3, respectively) with sizes of 64 × 64, 32 × 32, 16 × 16, 8 × 8 and the prediction units with sizes of 4 × 4 pixels in the image are texture intensity calculated and stored, and the calculation method is as shown in formulas (5) to (8):
wherein N is the number of pixel points in the coding unit;
V h the grain strength in the horizontal direction;
V v the grain strength in the vertical direction;
V 45° grain strength in the 45 degree direction;
V 135° the strength of the texture is 135 degrees.
Step S108, judging whether the conditions for not continuing to divide the coding unit are met according to the texture intensities in the multiple directions; if yes, go to step S110;
before the above step S108, encoding is performed according to the HEVC standard test model HM, and an image frame is divided into several coding tree units that are mutually irrelevant, with a maximum coding unit of 64 × 64 pixels as a basic unit, as shown in fig. 3A.
The test image resolution as employed in this example is 832 × 480, 480 is not an integer multiple of 64, so that the 64 × 64 effective maximum coding unit is not set at the region boundary of 448 to 480 lines, but still divided according to the coding tree unit mode. Recursive partitioning encoding of each coding tree unit is started. See fig. 3B. The depths include 0,1,2 and 3;
the depth of the coding units 8,9, 10 is 1, and the size is 32 × 32;
the depth of the coding units 5, 6,7 is 2 and the size is 16 × 16;
the coding units 1,2,3,4 have a depth of 3 and a size of 8 × 8.
When step S108 is executed, specifically, the prediction unit corresponding to one coding unit is sequentially selected, and whether the coding unit is divided is determined according to the calculated texture intensities in the four directions and the texture intensities in the four directions of the four sub-coding units of the coding unit.
Step S110, stopping the continuous division of the coding unit.
In the method, in the process of coding and dividing the image frame, whether the condition of no longer continuously dividing the coding unit is met or not is judged for each coding unit according to the texture intensity in multiple directions; if so, stopping continuously dividing the coding unit. Therefore, the processes of unnecessary sub-coding unit division and rate distortion cost calculation can be omitted, the coding efficiency can be improved, and the coding calculation amount can be reduced.
In some embodiments, in step S108, determining whether the condition for not continuing to partition the coding unit is met according to the texture intensities of the multiple directions includes the following two specific conditions:
a condition is that if the maximum value of the texture intensity of each direction is smaller than a predetermined first threshold Th1 and has directivity;
another condition is that if the difference between the texture intensity of each direction and zero is smaller than a predetermined second threshold Th2 and has similarity.
Specifically, the plurality of directions are four directions; determining to have directionality, and adopting the following steps:
if the difference between the texture intensity in the first direction and the texture intensity in the second direction, the difference between the texture intensity in the third direction and the texture intensity in the fourth direction are respectively larger than a preset direction difference threshold value;
in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction, the difference value of any two texture intensities is smaller than a preset similarity difference value threshold value;
the directivity is determined to be the first direction.
In some embodiments, directionality is determined using the following formula:
wherein, V 1 Is the first of the coding unitTexture intensity in one direction;
V 2 a texture strength of a second direction of the coding unit;
V 3 texture strength of a third direction of the coding unit;
V 4 the texture strength of the fourth direction of the coding unit;
x and y are variables; td is a first proportional threshold.
In some embodiments, the plurality of directions are four directions, specifically, the first direction is a horizontal direction, the second direction is a vertical direction, the third direction is a 45-degree direction, and the fourth direction is a 135-degree direction;
the directionality is determined using the following formula:
wherein, V h The grain strength in the horizontal direction;
V v the grain strength in the vertical direction;
V 45° grain strength in the 45 degree direction;
V 135° texture intensity in the 135 degree direction;
x and y are variables; td is the difference threshold.
In some embodiments, there are similarities, including: similarity exists if the difference in the texture intensities of any two directions is less than a predetermined similarity difference threshold.
Specifically, the similarity is calculated by using the following formula:
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity in a third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; ts is the second proportional threshold.
In some embodiments, the plurality of directions includes a first direction, a second direction, a third direction, and a fourth direction; the first direction is a horizontal direction, the second direction is a vertical direction, the third direction is a 45-degree direction, and the fourth direction is a 135-degree direction;
similarity was calculated using the following formula:
wherein, V h The grain strength in the horizontal direction;
V v the grain strength in the vertical direction;
V 45° grain strength in the 45 degree direction;
V 135° the strength of the texture is 135 degrees.
For the case of continuing to partition the coding unit, in some embodiments, determining whether a condition for continuing to partition the coding unit is met according to the texture strengths of the multiple directions; if so, the coding unit is continuously divided. After the division is continued, the cost of each divided sub-coding unit is calculated, and the sum of the coding costs of all the sub-coding units is calculated. But need not be compared to the original coding unit.
Specifically, the method comprises the following steps:
step S202, determining whether the texture intensities in the four directions of the coding unit satisfy one of the following conditions:
(1) The texture intensity values in the four directions are very large, the minimum value is larger than a threshold Th3, and the whole is very complex;
(2) The texture strength values in four directions are moderate, the maximum value is larger than the threshold Th4, but the texture strength of the sub-coding units in the same direction is greatly different.
If yes, executing step S204;
step S204, determining that the current coding unit needs to be divided, directly dividing the current coding unit into four sub-coding units for predictive coding respectively without predicting the current coding unit, and executing step S108 on the sub-coding units.
Wherein, the directionality of the sub-coding units can be calculated by adopting the following formula:
The above embodiments describe in detail the case where the partition is not continued for the coding unit and continues to be divided, and for the case in question, in some embodiments, if neither the condition for continuing the partition for the coding unit nor the condition for stopping the partition for the coding unit is met, the following steps are adopted:
step S302, continuously dividing the coding unit to obtain a plurality of sub-coding units;
step S304, calculating the sum of rate-distortion costs of a plurality of sub-coding units; the method specifically comprises the following steps:
for any one sub-coding unit, determining the minimum texture intensity in the texture intensities of a plurality of directions;
determining the direction corresponding to the minimum texture intensity value; and the prediction mode group corresponding to the direction:
error prediction is carried out on the sub-coding units by adopting the prediction mode group to obtain rate distortion cost;
step S306, judging whether the sum of the rate distortion costs of a plurality of sub coding units is less than or equal to the rate distortion cost of the coding unit;
step S308, if the sum of the rate-distortion costs of the multiple sub-coding units is less than or equal to the rate-distortion cost of the coding unit, continuing to divide the coding unit;
and step S310, if the sum of the rate distortion costs of the plurality of sub-coding units is greater than the rate distortion cost of the coding unit, stopping continuously dividing the coding unit.
In some embodiments, for a coding unit with a size of 8 × 8, the division of the coding unit cannot be continued, but the corresponding prediction unit may be divided into 4 sub-prediction units, and only step S108 is performed. Through tests, the obtained threshold values Th1, th2, th3, th4, td, ts and Tc have different values for the coding units with different sizes, and the values are as follows:
the size is 64 × 64: th1=4, th2=2, th3=2, th4=1.1, td =4, ts =1.6, tc =1.8;
dimensions 32 × 32: th1=8, th2=4, th3=6, th4=5.1, td =3.5, ts =1.8, tc =2.75;
the size is 16 × 16: th1=6, th2=6, th3=16, th4=10, td =4, ts =2, tc =5;
the size is 8 × 8: th1=8, th2=8, th3=18, th4=8, td =4, ts =2, tc =8.
It is worth emphasizing that for any coding unit, whether the coding unit is divided, not divided or pending, the prediction mode of the coding unit needs to be determined, and the prediction mode is not only used for calculating the rate distortion cost, but also used for coding. In some embodiments, therefore, the method further comprises:
determining a prediction mode group of the coding unit;
performing error prediction on the coding unit according to the prediction mode group to obtain the rate distortion cost of the coding unit;
the determining the prediction mode group of the coding unit specifically includes:
determining a minimum texture intensity value of the texture intensity values of the plurality of directions of the coding unit;
and determining the direction corresponding to the minimum texture intensity value and the prediction mode group corresponding to the direction.
Specifically, the method comprises the following steps:
step S402, determining the minimum texture intensity value in the texture intensity values of the coding unit in multiple directions;
step S404, determining a direction corresponding to the minimum texture intensity value and a prediction mode group corresponding to the direction;
specifically, after the pre-decision of the coding unit division is made, the prediction value calculation for the corresponding prediction unit is started. When the best prediction mode is selected, a coarse mode decision is made for the 35 prediction modes. Based on the directionality, 35 prediction modes (0-34) are divided into 4 groups of 11 prediction modes, and the prediction mode numbers are classified as follows:
illustratively, the plurality of directions includes a horizontal direction, a vertical direction, a 45 degree direction, and a 135 degree direction. 35 prediction modes (0-34) are divided into 4 groups, each group comprises 11 prediction modes, and the number of the prediction modes is classified as follows:
horizontal direction mode group: {0,1,6,7,8,9, 10, 11, 12, 13, 14};
vertical direction mode group: {0,1, 22, 23, 24, 25, 26, 27, 28, 29, 30};
45-degree directional pattern group: {0,1,2,3,4,5, 30, 31, 32, 33, 34};
135 degree directional mode set: {0,1, 14, 15, 16, 17, 18, 19, 20,2l,22};
in a specific implementation, the corresponding mode set may be selected according to a direction in which the texture of the prediction unit has the strongest directionality (the smallest texture intensity value).
For example, if the horizontal texture strength value of the current coding unit is the smallest, i.e. the horizontal directivity is the strongest, the horizontal mode set is selected: {0,1,6,7,8,9, 10, 11, 12, 13, 14}.
The present application makes coarse mode decisions for only 11 prediction modes. After the rough mode decision, a final decision of a prediction mode is made according to a rate distortion optimization method of an HEVC standard model, and coding cost is calculated. After a complete encoding process is finished, the encoded HEVC data stream is written into a file or transmitted to a network.
Step S406, adopting the prediction mode group to carry out error prediction on the coding unit to obtain a rate distortion cost;
specifically, the prediction mode set may be used to perform error prediction on the coding unit to obtain a coding distortion value; and calculating rate distortion cost according to the coding distortion value. The invention discloses an HEVC intra-frame compression acceleration method based on a Sobel operator. Illustratively, the directions are horizontal, vertical, 45 °, and 135 °. Carrying out encoding tree recursive division on each maximum encoding unit according to a quadtree model, and calculating texture intensity in four directions according to gradient information of all pixels in the encoding unit during each division; comparing the texture intensity of the coding unit in four directions, comparing the texture intensity of the corresponding sub-coding unit, and determining whether to terminate the recursive division of the coding unit in advance, thereby reducing a large amount of unnecessary calculation; dividing a coding unit into a prediction unit and a transformation unit; when the prediction unit selects the prediction mode, finding out the strongest texture direction according to the texture intensity information of the prediction unit; in the course of rough mode decision, 11 modes in the direction with the strongest texture intensity are selected from the 35 modes to calculate predicted values, and the calculation time is shortened.
The HEVC intra-frame compression acceleration method based on the Sobel operator is simple in overall calculation mode. And calculating texture intensity values by using a Sobel operator in the second step, and using the brightness information of the 3 multiplied by 3 area around the pixel point, so that the texture calculation is more accurate, and the detection of the edge of the area is more facilitated.
The texture intensity of the coding unit and the texture intensity of the sub-coding units in four directions are compared and judged, the segmentation strategy of a part of coding units is determined in advance, unnecessary recursive coding operation is reduced, the time of intra-frame coding is greatly shortened, and the influence on the coding quality and the coding rate of the image is small.
By grouping the prediction modes, the range of rough mode decision is reduced, a part of encoding time can be shortened, and the influence on the encoding quality and the encoding code rate of the image is very small.
As shown in fig. 4, when the results of the coding test performed on different video sequences are compared with the results of the coding performed by the HEVC standard, it can be seen that the intra-frame coding time is reduced by 49.39% on average, while the image quality is not affected substantially, and the coding rate is increased by only 1.08%.
Comparing, wherein Δ BR is the change of coding code rate, Δ PSNR is the change of peak signal-to-noise ratio, the larger the peak signal-to-noise ratio is, the better the image quality is, and Δ T is the change of coding time;
it should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
In a second aspect, the present invention also provides an image encoding apparatus, as shown in fig. 5, comprising:
an image obtaining module 51, configured to obtain an image to be encoded;
a segmentation module 52, configured to segment the image to be encoded; after the judgment module determines that the condition of not continuously dividing the coding unit is met, the coding unit is stopped to be continuously divided;
a calculating module 53, configured to calculate a luminance gradient value of each coding unit corresponding to any one coding unit in a process that the segmentation module segments the image to be coded to obtain multiple coding units; and calculating the texture intensity of the coding unit in multiple directions according to the brightness gradient value;
and a determining module 54, configured to determine whether a condition for not continuing to partition the coding unit is met according to the texture intensities in the multiple directions.
In one embodiment, the determining module 54 is further configured to, if the maximum value of the texture intensity of each direction is smaller than a predetermined first threshold Th1 and has directionality; or,
if the difference between the texture intensity of each direction and zero is smaller than a predetermined second threshold Th2 and has similarity.
In one embodiment, the determining module 54 is further configured to, if the difference between the texture strength in the first direction and the texture strength in the second direction, the texture strength in the third direction, and the texture strength in the fourth direction is greater than the predetermined direction difference threshold;
the difference value of any two texture intensities in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction is smaller than a preset similar difference value threshold value;
the directivity is determined to be the first direction.
In one embodiment, the determination module 54 is further configured to,
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity for the third direction;
V 4 texture intensity in the fourth direction;
x and y are variables; td is a first proportional threshold.
In one embodiment, the determining module 54 is further configured to determine that there is similarity if the difference between the texture strengths of any two directions is less than a predetermined similarity difference threshold.
In one embodiment, the determining module 54 is further configured to calculate the similarity by using the following formula:
wherein, V 1 Is the grain strength in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity in a third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; ts is the second scaling threshold.
In one embodiment, the determining module 54 is further configured to determine whether a condition for continuing to partition the coding unit is met according to the texture intensities of the multiple directions; if so, the coding unit is continuously divided.
In one embodiment, the determining module 54 is further configured to, if neither the condition for continuing the partitioning of the coding unit nor the condition for stopping the partitioning of the coding unit is met;
continuously dividing the coding unit to obtain a plurality of sub-coding units;
calculating the sum of rate-distortion costs of a plurality of sub-coding units;
if the sum of the rate distortion costs of a plurality of sub-coding units is less than or equal to the rate distortion cost of the coding unit, continuing to divide the coding unit;
and if the sum of the rate distortion costs of the plurality of sub-coding units is larger than the rate distortion cost of the current coding unit, stopping continuously dividing the coding unit.
In some embodiments, further comprising a prediction mode group determination module to determine a prediction mode group of the coding unit;
a rate distortion cost determination module, configured to perform error prediction on the coding unit according to the prediction mode group to obtain a rate distortion cost of the coding unit;
the prediction mode group determination module is further configured to determine a minimum texture intensity value among texture intensity values of a plurality of directions of the coding unit;
and determining the direction corresponding to the minimum texture intensity value and the prediction mode group corresponding to the direction. According to a third aspect of the present application, there is provided an image encoding apparatus; comprising at least one processor and at least one memory; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform any of the methods described above.
In a fourth aspect, the present application also proposes a computer-readable storage medium having embodied therein one or more program instructions for executing the method of any one of the above.
In an embodiment of the invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (5)
1. An image encoding method, comprising:
acquiring an image to be coded;
in the process of segmenting the image to be coded to obtain a plurality of coding units, corresponding to any one coding unit, calculating a brightness gradient value of the coding unit;
calculating texture intensities of the coding units in multiple directions according to the brightness gradient values;
judging whether the conditions for continuously dividing the coding unit are met according to the texture intensities in the multiple directions;
if so, stopping continuously dividing the coding unit;
judging whether the condition that the coding unit is not continuously segmented is met according to the texture intensities of the multiple directions, wherein the judging comprises the following steps:
if the maximum value of the texture intensity of each direction is less than a predetermined first threshold value Th1 and has directivity; or,
if the difference between the texture intensity of each direction and zero is less than a predetermined second threshold Th2 and has similarity;
the plurality of directions are four directions; determining to have directionality, including:
if the difference between the texture intensity in the first direction and the texture intensity in the second direction, the difference between the texture intensity in the third direction and the texture intensity in the fourth direction are respectively larger than a preset direction difference threshold value;
the difference value of any two texture intensities in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction is smaller than a preset similar difference value threshold value;
determining the directivity to be a first direction;
the directionality is determined using the following formula:
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity for the third direction;
V 4 texture intensity in the fourth direction;
x and y are variables; td is a first proportional threshold;
determining similarity includes: if the difference value of the texture intensities of any two directions is smaller than a preset similarity difference threshold value, determining that the two directions have similarity;
similarity was calculated using the following formula:
wherein, V 1 Is the grain strength in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity in a third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; ts is the second scaling threshold.
2. The image encoding method of claim 1, wherein the method further comprises:
judging whether the conditions for continuously dividing the coding unit are met or not according to the texture intensities in the multiple directions;
if so, the coding unit is continuously divided.
3. The image encoding method according to claim 2, wherein if neither a condition for continuing the segmentation for the encoding unit nor a condition for stopping the segmentation for the encoding unit is satisfied, the method further comprises:
continuously dividing the coding unit to obtain a plurality of sub coding units;
calculating the sum of rate-distortion costs of a plurality of sub-coding units;
if the sum of the rate distortion costs of a plurality of sub-coding units is less than or equal to the rate distortion cost of the coding unit, continuing to divide the coding unit;
and stopping continuously dividing the coding unit if the sum of the rate distortion costs of the plurality of sub-coding units is larger than the rate distortion cost of the coding unit.
4. The image encoding method of claim 1, wherein the method further comprises: determining a prediction mode group of the coding unit;
performing error prediction on the coding unit according to the prediction mode group to obtain the rate distortion cost of the coding unit;
the determining the prediction mode group of the coding unit specifically includes:
determining a minimum texture intensity value of the texture intensity values of the plurality of directions of the coding unit;
and determining the direction corresponding to the minimum texture intensity value and the prediction mode group corresponding to the direction.
5. An image encoding device characterized by comprising:
the image acquisition module is used for acquiring an image to be coded;
the segmentation module is used for segmenting the image to be encoded; if the judgment module determines that the condition of not continuously dividing the coding unit is met, the coding unit is stopped to be continuously divided;
the computing module is used for computing the brightness gradient value of the coding unit corresponding to any one coding unit in the process of obtaining a plurality of coding units by segmenting the image to be coded by the segmenting module; and calculating the texture intensity of the coding unit in multiple directions according to the brightness gradient value;
the judging module is used for judging whether the condition that the coding unit is not continuously segmented is met according to the texture intensities in the multiple directions;
judging whether the condition that the coding unit is not continuously segmented is met according to the texture intensities of the multiple directions, wherein the judging comprises the following steps:
if the maximum value of the texture intensity of each direction is less than a predetermined first threshold value Th1 and has directivity; or,
if the difference between the texture intensity of each direction and zero is less than a predetermined second threshold Th2 and has similarity;
the plurality of directions are four directions; determining to have directionality, comprising:
if the difference between the texture intensity in the first direction and the texture intensity in the second direction, the difference between the texture intensity in the third direction and the texture intensity in the fourth direction are respectively larger than the preset direction difference threshold value;
in the texture intensity in the second direction, the texture intensity in the third direction and the texture intensity in the fourth direction, the difference value of any two texture intensities is smaller than a preset similarity difference value threshold value;
determining the directivity to be a first direction;
the directionality is determined using the following formula:
wherein, V 1 Is the grain strength in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity in a third direction;
V 4 texture intensity for the fourth direction;
x and y are variables; td is a first proportional threshold;
determining similarity includes: if the difference value of the texture intensities of any two directions is smaller than a preset similarity difference threshold value, determining that the two directions have similarity;
similarity was calculated using the following formula:
wherein, V 1 Is the texture intensity in the first direction;
V 2 is the texture intensity in the second direction;
V 3 texture intensity for the third direction;
V 4 in a fourth directionTexture strength;
x and y are variables; ts is the second proportional threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110237593.3A CN113079374B (en) | 2021-03-03 | 2021-03-03 | Image encoding method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110237593.3A CN113079374B (en) | 2021-03-03 | 2021-03-03 | Image encoding method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113079374A CN113079374A (en) | 2021-07-06 |
| CN113079374B true CN113079374B (en) | 2023-01-10 |
Family
ID=76609814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110237593.3A Active CN113079374B (en) | 2021-03-03 | 2021-03-03 | Image encoding method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113079374B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297781A (en) * | 2013-06-07 | 2013-09-11 | 安科智慧城市技术（中国）有限公司 | High efficiency video coding (HEVC) intraframe coding method, device and system based on texture direction |
| CN103517069A (en) * | 2013-09-25 | 2014-01-15 | 北京航空航天大学 | HEVC intra-frame prediction quick mode selection method based on texture analysis |
| WO2015123806A1 (en) * | 2014-02-18 | 2015-08-27 | Mediatek Singapore Pte. Ltd. | Methods for depth based block partitioning |
| CN105491385A (en) * | 2015-12-02 | 2016-04-13 | 同济大学 | Quick deciding method for division of HEVC intraframe coding units |
| CN108322747A (en) * | 2018-01-05 | 2018-07-24 | 中国软件与技术服务股份有限公司 | A kind of coding unit Partitioning optimization method towards ultra high-definition video |
| CN111770340A (en) * | 2020-07-23 | 2020-10-13 | 腾讯科技（深圳）有限公司 | Video encoding method, device, equipment and storage medium |
-
2021
- 2021-03-03 CN CN202110237593.3A patent/CN113079374B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297781A (en) * | 2013-06-07 | 2013-09-11 | 安科智慧城市技术（中国）有限公司 | High efficiency video coding (HEVC) intraframe coding method, device and system based on texture direction |
| CN103517069A (en) * | 2013-09-25 | 2014-01-15 | 北京航空航天大学 | HEVC intra-frame prediction quick mode selection method based on texture analysis |
| WO2015123806A1 (en) * | 2014-02-18 | 2015-08-27 | Mediatek Singapore Pte. Ltd. | Methods for depth based block partitioning |
| CN105491385A (en) * | 2015-12-02 | 2016-04-13 | 同济大学 | Quick deciding method for division of HEVC intraframe coding units |
| CN108322747A (en) * | 2018-01-05 | 2018-07-24 | 中国软件与技术服务股份有限公司 | A kind of coding unit Partitioning optimization method towards ultra high-definition video |
| CN111770340A (en) * | 2020-07-23 | 2020-10-13 | 腾讯科技（深圳）有限公司 | Video encoding method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113079374A (en) | 2021-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101437195B1 (en) | Detecting block artifacts in coded image and video | |
| US10313692B2 (en) | Visual perception characteristics-combining hierarchical video coding method | |
| CN108184115B (en) | HEVC intra-frame CU partition and PU prediction mode selection method and system | |
| CN103404137B (en) | The method and apparatus of effective sample adaptive equalization | |
| CN109068142B (en) | 360-degree video intra-frame prediction rapid decision-making method, device, coder-decoder and medium | |
| CN110650338B (en) | Method, system and storage medium for dividing multifunctional video coding frame | |
| CN105120292A (en) | Video coding intra-frame prediction method based on image texture features | |
| CN102395030A (en) | Motion analysis method based on video compression code stream, code stream conversion method and apparatus thereof | |
| CN102917225A (en) | Method for quickly selecting HEVC (high-efficiency video coding) inframe coding units | |
| CN110324617B (en) | Image processing method and device | |
| US20240031576A1 (en) | Method and apparatus for video predictive coding | |
| US8363978B2 (en) | System and method for block edge location with varying block sizes and offsets in compressed digital video | |
| CN117156133B (en) | Inter-frame prediction mode selection method and device for video coding | |
| CN110213584A (en) | Coding unit classification method and coding unit sorting device based on Texture complication | |
| CN103888764A (en) | Self-adaptation compensation system and method for video compression distortion | |
| CN104992419A (en) | Super pixel Gaussian filtering pre-processing method based on JND factor | |
| US8891609B2 (en) | System and method for measuring blockiness level in compressed digital video | |
| CN112087624A (en) | Coding management method based on high-efficiency video coding | |
| CN103702131A (en) | Pattern-preprocessing-based intraframe coding optimization method and system | |
| CN106878754B (en) | A kind of 3D video depth image method for choosing frame inner forecast mode | |
| CN113079374B (en) | Image encoding method and apparatus | |
| CN111246221B (en) | AVS3 intra-frame rapid division method, system and storage medium | |
| CN103905818B (en) | Method for rapidly determining inter-frame prediction mode in HEVC standard based on Hough conversion | |
| CN116260968A (en) | Encoding method, apparatus, device and storage medium for reducing video encoding complexity | |
| CN114827606A (en) | Quick decision-making method for coding unit division |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |