CN110213584A - Coding unit classification method and coding unit sorting device based on Texture complication - Google Patents
Coding unit classification method and coding unit sorting device based on Texture complication Download PDFInfo
- Publication number
- CN110213584A CN110213584A CN201910592999.6A CN201910592999A CN110213584A CN 110213584 A CN110213584 A CN 110213584A CN 201910592999 A CN201910592999 A CN 201910592999A CN 110213584 A CN110213584 A CN 110213584A
- Authority
- CN
- China
- Prior art keywords
- coding unit
- coding
- complexity
- model
- texture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/96—Tree coding, e.g. quad-tree coding
Abstract
The invention discloses a kind of coding unit classification method and coding unit sorting device based on Texture complication, the coding unit classification method include: that the complexity of coding unit is obtained using the image texture complexity model pre-seted；According to the complexity, the corresponding threshold value of the coding unit using the adaptive threshold model output pre-seted classifies to the coding unit.Embodiment provided by the invention is directed to existing issue, according to the relationship of coding tree unit distinguishing hierarchy and image texture complexity, using non-linear relation existing between threshold value and coding depth and quantization parameter, the coding unit Accurate classification based on image texture complexity is realized；It is directed to complex texture region simultaneously, encoded information is combined with convolutional neural networks, image texture information is realized and carries out accurate description, to improve the precision of texture complex region coding unit level prediction, it the scramble time for saving HEVC standard encoder, is with a wide range of applications.
Description
Technical field
The present invention relates to multimedia coding techniques fields, more particularly to a kind of coding unit based on Texture complication point
Class method and coding unit sorting device.
Background technique
Vision is the main path of human perception and the understanding external world, and the U.S.'s red Rett of experimental psychology scholar is pulled through greatly
Amount experiment confirms that the mankind obtain the 83% of information and both are from vision.And video is increasingly becoming people as the main carriers of visual information
Live in indispensable a part, with the development of internet, smart phone and multimedia technology, the presentation mode of video
More diversified, people also promote the experience of video perception therewith.In terms of it is mainly reflected in following two: being video first
Data volume sharply increases, and is primarily referred to as the continuous improvement of spatial resolution here, and nowadays HD video is gradually popularized, currently
The trend of development is (3840 × 2160) 4K, the ultra high-definition video of even 8K (7680 × 4320)；Secondly, video acquisition approach
Diversification, the propagation of video has been no longer limited to the modes such as radio and television, and nowadays mobile Internet has become video transmission
Main path, prediction address is issued according to cisco, arrives the year two thousand twenty, annual global mobile data flow is up to 366.8EB
(1,000,000,000 GB of 1EB ≈), wherein video flow will be more than 75%, bring very big burden to the storage and transmission of vision signal.
Based on above-mentioned analysis, have become problem urgent and real at present for the research of video image compression technology.
Summary of the invention
At least one to solve the above-mentioned problems, first aspect present invention provides a kind of coding list based on Texture complication
First classification method, comprising:
S11: the complexity of coding unit is obtained using the image texture complexity model pre-seted；
S12: according to the complexity, the coding unit using the adaptive threshold model output pre-seted is corresponding
Threshold value classifies to the coding unit.
Further, the complexity that the image texture complexity model that the utilization pre-sets obtains coding unit is further
Include:
S111: the standard deviation of the coding unit is calculated；
S112: described image Texture complication model obtains the complexity of coding unit according to the index bed.
Further, described according to the complexity, utilize the coding of the adaptive threshold model output pre-seted
The corresponding threshold value of unit carries out classification to the coding unit:
S121: the adaptive threshold model exports corresponding according to the coding depth and quantization parameter of the coding unit
Threshold value；
S122: judging whether the complexity is less than the threshold value, and the coding unit is smooth grain area if being less than
Domain simultaneously terminates the coding unit classification method, is otherwise complex texture region.
Further, the adaptive threshold model are as follows:
Thr=f (QP) × g (depth)；
Wherein, Thr is Texture complication threshold value, and QP is quantization parameter, and depth is the depth of current coded unit.
Further, the volume of the adaptive threshold model output pre-seted is utilized according to the complexity described
After the corresponding threshold value of code unit classifies to the coding unit, the method also includes:
S13: according to the coding depth of the coding unit, using convolutional neural networks model to the complex texture region
Coding unit carry out level prediction.
Further, the convolutional neural networks model includes input layer, convolutional layer, articulamentum and full articulamentum, wherein
The input layer, for determining classifier according to the coding depth of the coding unit；
The convolutional layer, for convolution kernel to be arranged according to the coding depth of the coding unit and exports two groups of characteristic patterns
Picture；
The articulamentum, for connecting two groups of characteristic images and being converted to feature vector；
The full articulamentum exports level for calculating the spatial correlation of the coding unit, and by activation primitive
Prediction.
Further,
The classifier is one of 64*64,32*32 or 16*16；
And/or
The convolutional layer includes the convolution kernel of 32*32 and 4*4；
And/or
The convolutional layer includes the first convolutional layer and the second convolutional layer, when the coding unit is 64*64, the convolution
Layer further includes pond layer；
And/or
Full articulamentum includes two hidden layers and an output layer；
And/or
The convolutional neural networks model further includes loss function.
Further, in the coding depth according to the coding unit, using convolutional neural networks model to described
After the coding unit in complex texture region carries out level prediction, the method also includes:
S14: judging whether the coding unit of the level prediction output needs to divide, and is to return to S11, otherwise jumps to
S15；
S15: whether the coding unit for judging the level prediction output is minimum coding unit, otherwise returns to S11, is then
Terminate the coding unit classification method.
Second aspect of the present invention provides a kind of coding unit sorting device based on Texture complication, including complicated dynamic behaviour
Device and sorter, wherein
The complicated dynamic behaviour device is configured to obtain coding unit using the image texture complexity model pre-seted
Complexity；
The sorter is configured to utilize the institute of the adaptive threshold model output pre-seted according to the complexity
The corresponding threshold value of coding unit is stated to classify to the coding unit.
It further, further include convolutional neural networks prediction meanss, the first judgment means and the second judgment means, wherein
The convolutional neural networks prediction meanss, are configured to the coding depth according to the coding unit, utilize convolution mind
Level prediction is carried out through coding unit of the network model to the complex texture region；
First judgment means are configured to judge whether the coding unit of the level prediction output needs to divide, are
Then otherwise output is exported to the complicated dynamic behaviour device to the second judgment means；
Second judgment means are configured to judge whether the coding unit of the level prediction output is minimum code list
Otherwise member is exported to the complicated dynamic behaviour device, be to export the coding unit.
Beneficial effects of the present invention are as follows:
The present invention formulates a kind of coding unit classification method and volume based on Texture complication for problem existing at present
Code unit sorting device, it is deep using threshold value and coding according to the relationship of coding tree unit distinguishing hierarchy and image texture complexity
Existing non-linear relation between degree and quantization parameter, realizes the coding unit Accurate classification based on image texture complexity；Together
When be directed to complex texture region, encoded information is combined with convolutional neural networks, realize image texture information carry out accurately
Description, so that the precision of texture complex region coding unit level prediction is improved, when saving the coding of HEVC standard encoder
Between, with good application prospect.
Detailed description of the invention
Specific embodiments of the present invention will be described in further detail with reference to the accompanying drawing.
Fig. 1 shows the flow chart of coding unit classification method described in one embodiment of the present of invention；
Fig. 2 shows the relational graphs of quantization parameter, coding depth described in one embodiment of the present of invention and threshold value；
Fig. 3 shows three-layer classification decision-making technique schematic diagram described in one embodiment in the present invention；
Fig. 4 shows convolutional neural networks structural block diagram described in one embodiment in the present invention；
Fig. 5 shows convolutional neural networks structural schematic diagram described in one embodiment in the present invention；
Fig. 6 shows convolutional neural networks structural schematic diagram described in another embodiment in the present invention；
Fig. 7 shows convolutional neural networks structural schematic diagram described in the further embodiment in the present invention；
Fig. 8 shows adjacent encoder cell schematics described in one embodiment in the present invention；
Fig. 9 shows the structural block diagram of coding unit sorting device described in one embodiment in the present invention.
Specific embodiment
In order to illustrate more clearly of the present invention, the present invention is done further below with reference to preferred embodiments and drawings
It is bright.Similar component is indicated in attached drawing with identical appended drawing reference.It will be appreciated by those skilled in the art that institute is specific below
The content of description is illustrative and be not restrictive, and should not be limited the scope of the invention with this.
In recent years, with the continuous promotion of Internet technology and terminal processes processing capacity, to existing Video coding mark
H.264/AVC, standard is put forward new requirements.In order to better meet in wisdom traffic, safety monitoring and Medical Image Processing etc.
The application in field.In April, 2010, in the joint efforts of Video Coding Experts Group (VCEG) and Motion Picture Experts Group (MPEG)
Under, Video coding joint group (JCT-VC) has been built, the research and development and system of tissue a new generation video encoding standard H.265/HEVC are responsible for
It is fixed.And in 2013, by International Telecommunication Union-telecommunication standardization group (ITU-T) and International Organization for standardization/International Power committee member
Meeting (ISO/IEC) is formally issued.As video encoding standard of new generation, the target of HEVC is in H.264/AVC high
On the basis of profile, by using a variety of advanced coding techniques, its compression efficiency is made to be doubled, i.e., in video matter
Under the same conditions, memory space shared by compressed video code flow is 50% originally to amount.
However, the promotion of HEVC compression efficiency is to sacrifice the time complexity of coding side as cost, this is for video
Application is a huge challenge in real time.HEVC standard identifying code distance applies about poor 5000 speed in real time.
Therefore, under the premise of guaranteeing coding quality, the coding rate of HEVC standard is further promoted, for accelerating HEVC
Standard improves video coding technique in the application level of every field, applies valence with important in the extensive use of industrial circle
Value and social effect.
Present inventor study and find for intraframe coding tree dividing elements problem: intraframe coding conduct
One of several functional modules the most time-consuming, coding tree unit pass through recursive traversal institute using quad-tree structure in HEVC standard
There is partition mode, brings the gain of more coding distortion performances, but this necessarily leads to sharply increasing for encoder complexity, no
Conducive to the real-time application of HEVC standard；Simultaneously as the algorithm uses same treatment mode to non-homogeneous texture region, that is, use
The same feature extracting method coding unit different to texture complexity degree is analyzed, and causes to consume the plenty of time.Therefore,
The application proposes, under the premise of guaranteeing coding efficiency, removes partition mode in the frame of some redundancies, reduces coding tree unit and draws
The encoder complexity divided, optimizes its coding rate.
Based on above-mentioned discovery, as shown in Figure 1, one embodiment of the application provides a kind of volume based on Texture complication
Code unit classification method, comprising: S11: the complexity of coding unit is obtained using the image texture complexity model pre-seted；
S12: according to the complexity, using the corresponding threshold value of the coding unit of the adaptive threshold model output pre-seted to institute
Coding unit is stated to classify.
As shown in Figure 1, following the steps below description in a specific example:
S11: the complexity of coding unit is obtained using the image texture complexity model pre-seted.Further comprise:
S111: the standard deviation of the coding unit is calculated.
In view of the problem of coding tree unit divides in the prior art, the division of coding tree unit and the complexity of picture material
Degree has very strong relationship.In the present embodiment, biggish encoding block is used to smooth region, passes through the transformation of large coded block
So that energy is more concentrated, and reduce the bit number for being used to mark block type；Similarly, the more complicated region of texture is then used
More accurate predicted value is obtained compared with lower Item unit.Therefore, use the standard deviation of coding unit commenting as coding unit
Estimate parameter, first calculates the standard deviation of the coding unit.Specifically, standard deviation can intuitively reflect each picture in coding tree unit
The standard deviation of dispersion degree between element, usual smooth region is smaller, and the standard deviation of complex region is larger, so the present embodiment is sharp
The complexity of current coded unit texture is described with standard deviation:
Wherein, SD is the standard deviation of current coded unit, and N is the size of current coded unit, and P (x, y) is (x, y) coordinate
The pixel value at place.
It is worth noting that those skilled in the art should select suitable assessment parameter characterization according to concrete application scene
Coding unit, details are not described herein.
S112: described image Texture complication model obtains the complexity of coding unit according to the index bed.
In the present embodiment, it will calculate and be obtained in the pre-set image texture complexity model of standard deviation input obtained
The complexity of the coding unit.
S12: according to the complexity, the coding unit using the adaptive threshold model output pre-seted is corresponding
Threshold value classifies to the coding unit.
Further comprise:
S121: the adaptive threshold model exports corresponding according to the coding depth and quantization parameter of the coding unit
Threshold value.
In the prior art, the building of threshold value is for the prediction based on traditional didactic coding tree unit partitioning algorithm
Precision influences greatly, so the building of threshold value is vital for coding unit classification problem.Due to influencing threshold value building
Factor is more, such as coding depth, quantization parameter QP, image content complexity etc., so existing algorithm still remains threshold value
The problem of model inadequate robust.Based on this, the application has found to draw based on heuristic coding tree unit by the analysis of many experiments
There are certain non-linear relations between the threshold value and coding depth and quantization parameter of point algorithm.
Therefore, in an alternative embodiment, Shandong is constructed using the threshold value and coding depth, the relationship of quantization parameter
Stick more preferably adaptive threshold model, the adaptive threshold model are as follows:
Thr=f (QP) × g (depth)；
Wherein, Thr is Texture complication threshold value, and QP is quantization parameter, and depth is the depth of current coded unit.
Specifically, as shown in Fig. 2, when one timing (the application is set as 95%) of judgement precision, the judgement precision is this
The ratio of the calculated forecast level of embodiment and the forecast level of HM16.0 encoder output, threshold value Thr is with quantization parameter
Reduction and reduce, the two shows nonlinear relationship, that is, shows in the case where precision is certain, with subtracting for quantization parameter
Small, the optimal dividing mode of coding tree unit uses, the then setting of the threshold value also higher and higher compared with the ratio of lower Item unit
Also should accordingly reduce, to increase the decision probability in complex texture region.
Meanwhile the coding depth of coding unit equally be influence coding unit size decision an important factor for one of, from volume
From the point of view of the statistical law of the coding depth selection of code unit, with the increase of coding depth, the quantity of coding unit also constantly increases
Add.Based on this, the application adjusts using the statistical law found out and corrects the essence of the adaptive threshold model based on quantization parameter
Degree, such as when coding depth is 0, coding unit is divided into the coding unit of 32 × 32 sizes according to statistical law, therefore
Reduce threshold value so that more coding units enter phase judgement, rather than stops dividing herein.Therefore, the present invention utilizes
The adaptive threshold model of coding depth and quantization parameter building.
Thr=f (QP) × g (depth)
Wherein, Thr is Texture complication threshold value, and QP is quantization parameter, and depth is the depth of current coded unit.
S122: judging whether the complexity is less than the threshold value, and the coding unit is smooth grain area if being less than
Domain simultaneously terminates the coding unit classification method, is otherwise complex texture region.
In the present embodiment, classified according to the threshold value for calculating acquisition to the coding unit, when the coding unit
Complexity be less than adaptive threshold model calculate threshold value, then show the coding unit be smooth grain region, do not have to pair
The smooth grain region is divided；And the complexity for working as the coding unit is greater than the threshold value that adaptive threshold model calculates,
Then show that the coding unit is complex texture region, needs that the complex texture region is continued to divide.
In view of the complex texture region that tradition can not be handled very well based on didactic coding tree unit partitioning algorithm,
In an alternative embodiment, the institute of the adaptive threshold model output pre-seted is utilized according to the complexity described
It states after the corresponding threshold value of coding unit classifies to the coding unit, the method also includes: S13: according to the volume
It is pre- to carry out level using coding unit of the convolutional neural networks model to the complex texture region for the coding depth of code unit
It surveys.
In the specific implementation of the present embodiment, the partition problem of a coding tree unit is modeled as one two points by the present invention
Class problem directly determines current coded unit using convolutional neural networks, carries out to the belonging kinds of coding unit pre-
It surveys, to avoid rate-distortion optimization process.
In view of the coding tree unit division of HEVC is in optimized selection with quaternary tree layering in the prior art, in this reality
The difference of the number of plies according to locating for current coded unit in example is applied, establishes three classifiers respectively.It is illustrated in figure 3 coding unit
Three-layer classification decision is divided, wherein the classifier is respectively the S of 64*641(CU64×64), the S of 32*322(CU32×32) and 16*
16 S3(CU16×16), while requiring the brightness value that volume inputs the coding unit of product neural network model to normalize to [0,1]
Between.
Specifically,
Wherein,It is convolutional neural networks to the predicted value of current coded unit division result y, l is current coded unit
The place number of plies, Sl(CU) classifier for being l layers.
It is worth noting that 8 × 8 coding unit can not continue to divide in the present embodiment, so not needing to classify again.
Have for the neural network model shortage existing in the prior art for coding unit prediction to encoded information
The effect problem lower using caused network model precision of prediction, the present embodiment are constructed according to different coding depth in conjunction with volume
The model of the layering convolutional neural networks of code information.
As shown in figure 4, convolutional neural networks model includes input layer, convolutional layer, articulamentum and full articulamentum, wherein described
Input layer, for determining classifier according to the coding depth of the coding unit；The convolutional layer, for single according to the coding
The coding depth setting convolution kernel of member simultaneously exports two groups of characteristic images；The articulamentum, for connecting two groups of characteristic images
And be converted to feature vector；The full articulamentum, for calculating the spatial correlation of the coding unit, and passes through activation primitive
Export level prediction.
It is as illustrated in figs. 5-7 three kinds of different convolutional neural networks models.
Specifically, the input layer is directed to different coding depths, the coding unit of convolutional neural networks model will be inputted
It is divided into 64 × 64,32 × 32 or 16 × 16 according to size, and is handled using corresponding classifier.
Size of the convolutional layer according to current coded unit, sets the big of level coding unit to be divided for convolution kernel
It is small, to realize the convolution nuclear design based on coding depth, the convolution kernel for example including 32*32 and 4*4, while convolution kernel being moved
Dynamic step-length is set as the size of current convolution kernel.
In a specific example, the present embodiment devises two kinds of different size of volumes for 64 × 64 coding unit
Product core Kernel, respectively 32 × 32 (Kernel1-2) and 4 × 4 (Kernel1-1With Kernel2-1), it is big for design 32 × 32
Small core mainly considers the partition structure of coding tree unit, is equivalent to four 32 × 32 of 64 × 64 coding unit
Sub-block does 4 convolution operations, to extract the texture information of this 4 sub-blocks, and is then mainly using the convolution kernel of 4 × 4 sizes
Consider the enhancing to image texture details, and using the core of 4 × 4 sizes without filling to current coded unit
(Padding) it operates.It is worth noting that the step-length of convolution kernel movement is set as the size of current convolution kernel, it can be without covering
To current coded unit carry out convolution, to reduce computation complexity.
In addition, the present embodiment is in the first convolutional layer (Layer for 64 × 64 coding unit1-1) and the second convolutional layer
(Layer3-1) intermediate it joined a pond layer (Layer2-1), the present embodiment in this way may be used using the pondization operation being maximized
To reduce the offset that convolutional layer parameter error causes estimation mean value, the texture information as much as possible for retaining image.For 32 ×
32 and 16 × 16 coding unit, the setting of convolutional layer is similar with above-mentioned 64 × 64 coding unit, only convolution kernel
Size is different.It is worth noting that the convolutional neural networks of both coding units for 32 × 32 and 16 × 16
In do not have pondization operation.
Further, the articulamentum, for connecting two groups of characteristic images and being converted to feature vector；It is described to connect entirely
Layer is connect, exports level prediction for calculating the spatial correlation of the coding unit, and by activation primitive.
In a specific example, after convolutional layer, two groups of characteristic patterns (feature map), every group of packet are produced
128 2 × 2 characteristic patterns are included, the present embodiment using articulamentum by two groups of characteristic patterns by connecting together and being converted into one
Feature vector.Finally, joined three full articulamentums, including two hidden layers, an output layer.Based on convolutional neural networks
Its core of coding unit partitioning algorithm is that the feature of current coded unit is extracted by convolution operation, and there is no consider present encoding
The spatial correlation of unit reduces the correlation between coding unit although coding unit is extended to 64 × 64 by HEVC,
But for lesser encoding block, can still be obtained using the feature based on spatial correlation as the judgment basis whether divided compared with
Good precision, and its feature extraction hardly needs and introduces additional computation complexity, is suitable as supplemental characteristic and volume is added
Product neural network, helps the training of network model.Based on above-mentioned analysis, the present invention two in convolutional neural networks model it is complete
It joined a spatial correlation feature in connection hidden layer, the calculation method of spatial correlation is shown below.
In formula, correlation indicates spatial correlation, depth (A), depth (B) and depth (C) presentation code block
A, the coding depth of B and C, as shown in Figure 8.
In the present embodiment, the convolutional layer with entirely connect hidden layer be mainly use line rectification function (ReLU) as
Activation primitive, and for output layer, target output is two classification problems, i.e., " divides " or " not dividing ", so this hair
It is bright mainly to use sigmoid function.
In the present embodiment, the convolutional neural networks model further includes loss function.In other words, coding unit divides
It is two classification problems in question essence, wherein cross entropy is loss function, and quantization parameter QP draws coding tree unit
Divide result that there is larger impact, since the increase of quantization parameter QP can cause the increase of distortion, then when quantization parameter QP increases
Ratio using large code unit is higher and higher, and rate-distortion optimization can reduce code flow, to reduce population rate distortion
The result of cost, i.e. rate distortion can be more likely to use biggish coding unit.Therefore quantization parameter QP can be to loss function
Preferable impulse is played, for example, we can use quantization parameter if the true value of current sample is " division "
The influence that QP divides coding tree unit, " exacerbation " or " mitigation " punishment are current to compile if current quantisation parameter QP is smaller
Code unit continues a possibility that dividing relatively more greatly, so the punishment of mitigation function is wanted, if instead current quantisation parameter
A possibility that QP is larger, then current coded unit is not subdivided is relatively bigger, so to increase the punishment of loss function.Therefore originally
Invention constructs weight coefficient using quantization parameter QP, so that loss function more efficiently removes the sample of fitting prediction error,
Improve the performance of network model.Specifically, the cross entropy loss function are as follows:
Wherein:For sample true value,For sample predictions value, m is number of samples, θ1With θ2For weight coefficient, QP is
Quantization parameter.
It is worth noting that in the present embodiment, the source of convolutional neural networks model training data is public data collection
CPIH includes: training data (1700 images), verify data (100 images), test data (200 figures in data set
Picture).
In the present embodiment, for the diversity of image resolution ratio in CPIH data set, in its data source RAISE
Original image (resolution ratio 4928 × 3264) has carried out down-sampling operation, increases three kinds of resolution ratio, respectively 2880 × 1920,
1536 × 1024 and 768 × 512.And the calibration for data label, experiment test platform are selected as the reference generation of HEVC standard
Code HM 16.0 encodes input data set and is demarcated using the default configuration of All Intra Main, and division mark is
" 1 ", division mark does not produce 12 Sub Data Sets, specific sample then according to coding depth and the difference of QP for " 0 "
Number is as shown in table 1.
It should be noted that in order to train more accurate convolutional neural networks prediction model, the present invention by texture compared with
It is excluded except the input of convolutional neural networks model for smooth coding unit, is no longer used to the training of network model.
The training data of the present invention of table 1 concentrates the number of samples for dividing and not dividing coding unit
In view of complex texture region has the case where repeatedly dividing, in an alternative embodiment, in the basis
The coding depth of the coding unit carries out layer using coding unit of the convolutional neural networks model to the complex texture region
After secondary prediction, the method also includes: S14: judging whether the coding unit of the level prediction output needs to divide, and is then
S11 is returned, S15 is otherwise jumped to；S15: whether the coding unit for judging the level prediction output is minimum coding unit, no
S11 is then returned, is, terminates the coding unit classification method.
So far, the division for the intraframe predictive coding tree unit classified based on Texture complication is completed, the present embodiment utilizes volume
The complexity of the standard deviation assessment coding unit of code unit, according to the non-of the coding depth of coding unit and quantization parameter and threshold value
Linear relationship classifies to coding unit by adaptive threshold, and complex texture region is carried out by convolutional neural networks
Further division meets the coding unit or minimum coding unit of threshold value, this method energy until original coding unit to be divided into
The accurate description for enough realizing image texture information improves the precision of texture complex region coding unit level prediction, effectively saves
The scramble time of HEVC standard encoder has preferable application value in engineering.
Corresponding with coding unit classification method provided by the above embodiment, one embodiment of the application also provides one kind
Coding unit sorting device is provided due to coding unit sorting device provided by the embodiments of the present application with above-mentioned several embodiments
Coding unit classification method is corresponding, therefore is also applied for coding unit classification provided in this embodiment in aforementioned embodiments and sets
It is standby, it is not described in detail in the present embodiment.
As shown in figure 9, one embodiment of the application also provides a kind of coding unit sorting device, including complicated dynamic behaviour
Device and sorter, wherein the complicated dynamic behaviour device, is configured to obtain using the image texture complexity model pre-seted
Take the complexity of coding unit；The sorter is configured to utilize the adaptive threshold mould pre-seted according to the complexity
The corresponding threshold value of the coding unit of type output classifies to the coding unit.
In an alternative embodiment, the coding unit sorting device further include convolutional neural networks prediction meanss,
First judgment means and the second judgment means, wherein the convolutional neural networks prediction meanss, are configured to single according to the coding
The coding depth of member carries out level prediction using coding unit of the convolutional neural networks model to the complex texture region；Institute
The first judgment means are stated, are configured to judge whether the coding unit of the level prediction output needs to divide, are then exported to institute
Complicated dynamic behaviour device is stated, is otherwise exported to the second judgment means；Second judgment means are configured to judge that the level is pre-
Whether the coding unit for surveying output is minimum coding unit, is otherwise exported to the complicated dynamic behaviour device, is described in then output
Coding unit.
The present invention formulates a kind of coding unit classification method and volume based on Texture complication for problem existing at present
Code unit sorting device, it is deep using threshold value and coding according to the relationship of coding tree unit distinguishing hierarchy and image texture complexity
Existing non-linear relation between degree and quantization parameter, realizes the coding unit Accurate classification based on image texture complexity；Together
When be directed to complex texture region, encoded information is combined with convolutional neural networks, realize image texture information carry out accurately
Description, so that the precision of texture complex region coding unit level prediction is improved, when saving the coding of HEVC standard encoder
Between, with good application prospect.
Obviously, the above embodiment of the present invention be only to clearly illustrate example of the present invention, and not be pair
The restriction of embodiments of the present invention may be used also on the basis of the above description for those of ordinary skill in the art
To make other variations or changes in different ways, all embodiments can not be exhaustive here, it is all to belong to this hair
The obvious changes or variations that bright technical solution is extended out are still in the scope of protection of the present invention.
Claims (10)
1. a kind of coding unit classification method based on Texture complication characterized by comprising
S11: the complexity of coding unit is obtained using the image texture complexity model pre-seted；
S12: according to the complexity, the corresponding threshold value of the coding unit of the adaptive threshold model output pre-seted is utilized
Classify to the coding unit.
2. the method according to claim 1, wherein the image texture complexity model that the utilization pre-sets obtains
The complexity for obtaining coding unit further comprises:
S111: the standard deviation of the coding unit is calculated；
S112: described image Texture complication model obtains the complexity of coding unit according to the index bed.
3. adaptive using what is pre-seted the method according to claim 1, wherein described according to the complexity
The corresponding threshold value of the coding unit for answering threshold model to export carries out classification to the coding unit:
S121: the adaptive threshold model exports corresponding threshold according to the coding depth and quantization parameter of the coding unit
Value；
S122: judging whether the complexity is less than the threshold value, if be less than if the coding unit be smooth grain region simultaneously
Terminate the coding unit classification method, is otherwise complex texture region.
4. according to the method described in claim 3, it is characterized in that, the adaptive threshold model are as follows:
Thr=f (QP) × g (depth)；
Wherein, Thr is Texture complication threshold value, and QP is quantization parameter, and depth is the depth of current coded unit.
5. according to the method described in claim 3, it is characterized in that, described according to the complexity, using pre-set from
After the corresponding threshold value of the coding unit of adaptation threshold model output classifies to the coding unit, the method is also
Include:
S13: according to the coding depth of the coding unit, using convolutional neural networks model to the volume in the complex texture region
Code unit carries out level prediction.
6. according to the method described in claim 5, it is characterized in that, the convolutional neural networks model includes input layer, convolution
Layer, articulamentum and full articulamentum, wherein
The input layer, for determining classifier according to the coding depth of the coding unit；
The convolutional layer, for convolution kernel to be arranged according to the coding depth of the coding unit and exports two groups of characteristic images；
The articulamentum, for connecting two groups of characteristic images and being converted to feature vector；
The full articulamentum exports level prediction for calculating the spatial correlation of the coding unit, and by activation primitive.
7. according to the method described in claim 6, it is characterized in that,
The classifier is one of 64*64,32*32 or 16*16；
And/or
The convolutional layer includes the convolution kernel of 32*32 and 4*4；
And/or
The convolutional layer includes the first convolutional layer and the second convolutional layer, and when the coding unit is 64*64, the convolutional layer is also
Including pond layer；
And/or
Full articulamentum includes two hidden layers and an output layer；
And/or
The convolutional neural networks model further includes loss function.
8. according to the method described in claim 5, it is characterized in that, in the coding depth according to the coding unit, benefit
After carrying out level prediction with coding unit of the convolutional neural networks model to the complex texture region, the method is also wrapped
It includes:
S14: judging whether the coding unit of the level prediction output needs to divide, and is to return to S11, otherwise jumps to S15；
S15: whether the coding unit for judging the level prediction output is minimum coding unit, otherwise returns to S11, is, terminates
The coding unit classification method.
9. a kind of coding unit sorting device based on Texture complication, which is characterized in that including complicated dynamic behaviour device and divide
Class device, wherein
The complicated dynamic behaviour device is configured to obtain the complexity of coding unit using the image texture complexity model pre-seted
Degree；
The sorter is configured to utilize the volume of the adaptive threshold model output pre-seted according to the complexity
The corresponding threshold value of code unit classifies to the coding unit.
10. equipment according to claim 9, which is characterized in that further include convolutional neural networks prediction meanss, the first judgement
Device and the second judgment means, wherein
The convolutional neural networks prediction meanss, are configured to the coding depth according to the coding unit, utilize convolutional Neural net
Network model carries out level prediction to the coding unit in the complex texture region；
First judgment means are configured to judge whether the coding unit of the level prediction output needs to divide, are then defeated
Out to the complicated dynamic behaviour device, otherwise export to the second judgment means；
Second judgment means are configured to judge whether the coding unit of the level prediction output is minimum coding unit,
Otherwise it exports to the complicated dynamic behaviour device, is to export the coding unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910592999.6A CN110213584A (en) | 2019-07-03 | 2019-07-03 | Coding unit classification method and coding unit sorting device based on Texture complication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910592999.6A CN110213584A (en) | 2019-07-03 | 2019-07-03 | Coding unit classification method and coding unit sorting device based on Texture complication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110213584A true CN110213584A (en) | 2019-09-06 |
Family
ID=67795967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910592999.6A Pending CN110213584A (en) | 2019-07-03 | 2019-07-03 | Coding unit classification method and coding unit sorting device based on Texture complication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110213584A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429497A (en) * | 2020-03-20 | 2020-07-17 | 郑州轻工业大学 | Self-adaptive CU splitting decision method based on deep learning and multi-feature fusion |
| CN111479110A (en) * | 2020-04-15 | 2020-07-31 | 郑州轻工业大学 | Fast affine motion estimation method for H.266/VVC |
| CN111757110A (en) * | 2020-07-02 | 2020-10-09 | 中实燃气发展(西安)有限公司 | Video coding method, coding tree unit dividing method, system, device and readable storage medium |
| CN113139576A (en) * | 2021-03-22 | 2021-07-20 | 广东省科学院智能制造研究所 | Deep learning image classification method and system combining image complexity |
| CN114071189A (en) * | 2020-08-03 | 2022-02-18 | 纬创资通股份有限公司 | Video processing device and video streaming processing method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104796694A (en) * | 2015-04-30 | 2015-07-22 | 上海交通大学 | Intraframe video encoding optimization method based on video texture information |
| CN107071496A (en) * | 2017-05-14 | 2017-08-18 | 北京工业大学 | A kind of H.265/HEVC interframe encode unit depth fast selecting method |
| CN108174208A (en) * | 2018-02-12 | 2018-06-15 | 杭州电子科技大学 | A kind of efficient video coding method of feature based classification |
| CN109788296A (en) * | 2018-12-25 | 2019-05-21 | 中山大学 | Interframe encode dividing elements method, apparatus and storage medium for HEVC |
-
2019
- 2019-07-03 CN CN201910592999.6A patent/CN110213584A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104796694A (en) * | 2015-04-30 | 2015-07-22 | 上海交通大学 | Intraframe video encoding optimization method based on video texture information |
| CN107071496A (en) * | 2017-05-14 | 2017-08-18 | 北京工业大学 | A kind of H.265/HEVC interframe encode unit depth fast selecting method |
| CN108174208A (en) * | 2018-02-12 | 2018-06-15 | 杭州电子科技大学 | A kind of efficient video coding method of feature based classification |
| CN109788296A (en) * | 2018-12-25 | 2019-05-21 | 中山大学 | Interframe encode dividing elements method, apparatus and storage medium for HEVC |
Non-Patent Citations (2)
| Title |
|---|
| XINGANG LIU: "An adaptive mode decision algorithm based on video texture characteristics for HEVC intra prediction", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| YONGFEI ZHANG, ET AL.: "Texture-classification accelerated CNN scheme for fast intra CU partition in HEVC", 《2019 DATE COMPRESSION CONFERENCE(DCC)》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111429497A (en) * | 2020-03-20 | 2020-07-17 | 郑州轻工业大学 | Self-adaptive CU splitting decision method based on deep learning and multi-feature fusion |
| CN111429497B (en) * | 2020-03-20 | 2023-05-05 | 郑州轻工业大学 | Self-adaptive CU splitting decision method based on deep learning and multi-feature fusion |
| CN111479110A (en) * | 2020-04-15 | 2020-07-31 | 郑州轻工业大学 | Fast affine motion estimation method for H.266/VVC |
| CN111479110B (en) * | 2020-04-15 | 2022-12-13 | 郑州轻大产业技术研究院有限公司 | Fast affine motion estimation method for H.266/VVC |
| CN111757110A (en) * | 2020-07-02 | 2020-10-09 | 中实燃气发展(西安)有限公司 | Video coding method, coding tree unit dividing method, system, device and readable storage medium |
| CN114071189A (en) * | 2020-08-03 | 2022-02-18 | 纬创资通股份有限公司 | Video processing device and video streaming processing method |
| CN114071189B (en) * | 2020-08-03 | 2023-05-05 | 纬创资通股份有限公司 | Video processing device and video streaming processing method |
| CN113139576A (en) * | 2021-03-22 | 2021-07-20 | 广东省科学院智能制造研究所 | Deep learning image classification method and system combining image complexity |
| CN113139576B (en) * | 2021-03-22 | 2024-03-12 | 广东省科学院智能制造研究所 | Deep learning image classification method and system combining image complexity |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110213584A (en) | Coding unit classification method and coding unit sorting device based on Texture complication | |
| CN104796694B (en) | Optimization intraframe video coding method based on video texture information | |
| CN105532001B (en) | For using the difference vector based on depth to carry out interlayer coding method and coding/decoding method and equipment to video | |
| CN108632625A (en) | A kind of method for video coding, video encoding/decoding method and relevant device | |
| CN111355956B (en) | Deep learning-based rate distortion optimization rapid decision system and method in HEVC intra-frame coding | |
| CN104641640B (en) | For the method for video coding and video encoder and video encoding/decoding method and video decoding apparatus with signal transmission SAO parameters | |
| CN110087087A (en) | VVC interframe encode unit prediction mode shifts to an earlier date decision and block divides and shifts to an earlier date terminating method | |
| CN103546749B (en) | Method for optimizing HEVC (high efficiency video coding) residual coding by using residual coefficient distribution features and bayes theorem | |
| CN107566848A (en) | The method and device of encoding and decoding | |
| CN106464879A (en) | System and method for highly content adaptive quality restoration filtering for video coding | |
| CN110463203A (en) | Picture coding device, picture decoding apparatus, image encoding method and picture decoding method | |
| CN103067704B (en) | A kind of method for video coding of skipping in advance based on coding unit level and system | |
| CN109040764B (en) | HEVC screen content intra-frame rapid coding algorithm based on decision tree | |
| CN108924558A (en) | A kind of predictive encoding of video method neural network based | |
| CN104243997A (en) | Method for quality scalable HEVC (high efficiency video coding) | |
| CN114079779A (en) | Image processing method, intelligent terminal and storage medium | |
| CN108833925B (en) | Inter-frame prediction method based on deep neural network | |
| CN102420990B (en) | Multi-view video-oriented fast coding method | |
| CN109361920A (en) | A kind of interframe quick predict algorithm of the adaptive decision-making tree selection towards more scenes | |
| CN110049338A (en) | The quick inter-frame encoding methods of HEVC based on multilayer subseries | |
| Liu et al. | Fast depth intra coding based on depth edge classification network in 3D-HEVC | |
| CN106416256A (en) | Method and apparatus for encoding or decoding depth image | |
| CN105681803B (en) | A kind of HEVC video information hiding methods of large capacity | |
| CN106791876B (en) | A kind of depth map fast intra-frame predicting method based on 3D-HEVC | |
| CN108769696A (en) | A kind of DVC-HEVC video transcoding methods based on Fisher discriminates |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190906 |