CN102413330B - Texture-adaptive video coding/decoding system - Google Patents
Texture-adaptive video coding/decoding system Download PDFInfo
- Publication number
- CN102413330B CN102413330B CN201110388181.6A CN201110388181A CN102413330B CN 102413330 B CN102413330 B CN 102413330B CN 201110388181 A CN201110388181 A CN 201110388181A CN 102413330 B CN102413330 B CN 102413330B
- Authority
- CN
- China
- Prior art keywords
- texture
- information
- video
- adaption
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
本发明公开了一种纹理自适应视频编解码系统,纹理自适应视频编码系统包含视频编码器和编码端纹理分析器;纹理自适应视频解码系统包含视频解码器和解码端纹理分析器;本发明纹理自适应视频编解码系统将视频图像的纹理特征信息纳入到视频编解码系统中,以提高视频编码的压缩效率和主观质量。
The invention discloses a texture adaptive video encoding and decoding system. The texture adaptive video encoding system includes a video encoder and a texture analyzer at the encoding end; the texture adaptive video decoding system includes a video decoder and a texture analyzer at the decoding end; the present invention The texture adaptive video codec system incorporates the texture feature information of the video image into the video codec system to improve the compression efficiency and subjective quality of video coding.
Description
本申请是专利申请号为200710069093.3、发明名称为《纹理自适应视频编解码系统》、申请日为2007年6月12日的专利申请的分案申请。This application is a divisional application of the patent application No. 200710069093.3, the title of the invention is "Texture Adaptive Video Codec System", and the filing date is June 12, 2007.
技术领域technical field
本发明涉及信号处理与通信领域,特别地,涉及一种纹理自适应视频编码系统、一种纹理自适应视频解码系统和一种纹理自适应视频编解码系统。The present invention relates to the field of signal processing and communication, in particular to a texture adaptive video encoding system, a texture adaptive video decoding system and a texture adaptive video encoding and decoding system.
背景技术Background technique
当前视频编解码标准,比如ITU制定的H.261,H.263,H.26L和ISO的MPEG组织制定的MPEG-1,MPEG-2,MPEG-4,以及JVT制定的H.264/MPEG-AVC(简称H.264)和中国自主知识产权的视频编码标准AVS第二部分都是基于传统混合视频编解码框架。Current video codec standards, such as H.261, H.263, H.26L developed by ITU and MPEG-1, MPEG-2, MPEG-4 developed by ISO's MPEG organization, and H.264/MPEG- Both AVC (H.264 for short) and the second part of China's independent intellectual property video coding standard AVS are based on the traditional hybrid video codec framework.
视频编码的一个重要目的就是对视频信号进行压缩,减少视频信号的数据量,从而节约视频信号的存储空间和传输带宽。一方面,原始视频信号,数据量非常巨大,这是视频编码压缩的必要性所在;另一方面,原始视频信号存在大量的冗余信息,这是视频编码压缩的可能性所在。这些冗余信息可以分成空间冗余信息、时间冗余信息、数据冗余信息和视觉冗余信息。其中前面三种冗余信息仅仅只是考虑像素间统计概念上的冗余信息,总称统计冗余信息;视觉冗余信息更加侧重考虑人眼视觉系统的特性。视频编码要降低视频信号数据量,就需要降低视频信号中存在的各种冗余信息。传统混合视频编码框架是综合考虑预测编码、变换编码以及熵编码的视频编码框架,着力降低视频信号的统计冗余信息,传统混合视频编码框架有以下主要特点:An important purpose of video coding is to compress the video signal to reduce the data volume of the video signal, thereby saving the storage space and transmission bandwidth of the video signal. On the one hand, the original video signal has a huge amount of data, which is the necessity of video coding and compression; on the other hand, there is a large amount of redundant information in the original video signal, which is the possibility of video coding and compression. These redundant information can be divided into spatial redundant information, temporal redundant information, data redundant information and visual redundant information. Among them, the first three kinds of redundant information only consider the redundant information in the statistical concept between pixels, which are collectively called statistical redundant information; visual redundant information focuses more on the characteristics of the human visual system. In order to reduce the amount of video signal data in video coding, it is necessary to reduce various redundant information in the video signal. The traditional hybrid video coding framework is a video coding framework that comprehensively considers predictive coding, transform coding, and entropy coding, and focuses on reducing the statistical redundancy information of video signals. The traditional hybrid video coding framework has the following main features:
(1)利用预测编码降低时间冗余信息和空间冗余信息;(1) Use predictive coding to reduce temporal redundant information and spatial redundant information;
(2)利用变换编码进一步降低空间冗余信息;(2) Use transform coding to further reduce spatial redundancy information;
(3)利用熵编码降低数据冗余信息;(3) Use entropy coding to reduce data redundancy information;
预测编码包括帧内预测编码和帧间预测编码。用帧内预测编码技术压缩的视频帧,称为帧内编码帧(I帧)。帧内编码帧的编码过程如下:首先,将编码帧分成编码块(编码单元的一种形式);对编码块进行帧内预测,得到帧内预测的残差数据;接着对残差数据进行二维变换编码;然后在变换域中对变换系数进行量化;然后经过扫描将二维信号转换成一维信号;最后进行熵编码。用帧间预测编码技术压缩的视频帧,称为帧间编码帧(P帧、B帧)。帧间编码帧的编码过程如下:首先,将编码帧分成编码块;对编码块采用运动估计技术得到运动矢量和参考块(参考单元的一种形式);然后采用运动补偿技术,得到帧间预测后的残差数据;接着对残差数据进行二维变换编码;然后在变换域中对变换系数进行量化;然后经过扫描将二维信号转换成一维信号;最后进行熵编码。残差数据,也就是残差信号,相对于原始视频信号,空间冗余信息和时间冗余信息都减小了。如果空间冗余信息和时间冗余信息用数学上相关性来表示,则残差信号的空间相关性和时间相关性都比原始视频信息量小。然后对残差信号进行二维变换编码,进一步降低空间相关性,最后对变换系数进行量化和熵编码降低数据冗余信息。可见要继续提高视频编码的压缩效率,需要更精确的预测编码技术,进一步降低预测后残差信号的空间相关性和时间相关性;同时也需要更有效的变换编码技术,进一步降低空间相关性;同时在预测编码和变换编码后,设计相适应的扫描技术、量化技术和熵编码技术。Predictive coding includes intra-frame predictive coding and inter-frame predictive coding. A video frame compressed with intra-frame predictive coding technology is called an intra-coded frame (I frame). The encoding process of an intra-coded frame is as follows: first, the coded frame is divided into coding blocks (a form of coding unit); intra-frame prediction is performed on the coding block to obtain the residual data of intra-frame prediction; Then, the transform coefficients are quantized in the transform domain; then the two-dimensional signal is converted into a one-dimensional signal by scanning; finally, entropy coding is performed. Video frames compressed with interframe predictive coding technology are called interframe coded frames (P frames, B frames). The encoding process of an inter-frame coded frame is as follows: first, the coded frame is divided into coded blocks; the motion vector and the reference block (a form of reference unit) are obtained by using motion estimation technology for the coded block; and then the motion compensation technology is used to obtain inter-frame prediction The final residual data; then perform two-dimensional transform coding on the residual data; then quantize the transform coefficients in the transform domain; then convert the two-dimensional signal into a one-dimensional signal through scanning; finally perform entropy coding. The residual data, that is, the residual signal, has reduced spatial redundant information and temporal redundant information compared to the original video signal. If spatial redundant information and temporal redundant information are expressed by mathematical correlation, the spatial correlation and temporal correlation of the residual signal are smaller than the original video information. Then two-dimensional transform coding is performed on the residual signal to further reduce the spatial correlation, and finally the transform coefficients are quantized and entropy coded to reduce data redundancy information. It can be seen that in order to continue to improve the compression efficiency of video coding, more accurate predictive coding technology is needed to further reduce the spatial correlation and temporal correlation of the residual signal after prediction; at the same time, more effective transform coding technology is needed to further reduce the spatial correlation; At the same time, after predictive coding and transform coding, the corresponding scanning technology, quantization technology and entropy coding technology are designed.
上述基于传统混合视频编解码框架的视频编解码标准,虽然取得了很大的成功,但要进一步提高视频编码的压缩效率,传统混合视频编解码框架本身存在着瓶颈。研究结果显示,视频信号并非平稳信源,也就是说每个编码单元的特征是不尽相同的。然而,传统混合视频编解码框架中功能模块的设计却是建立在平稳视频信号的假设基础上,例如,其中预测编码模块、变换编码模块、量化模块、扫描模块等在对编码单元进行编码时,所采用的工作模式都是固定的:Although the above-mentioned video codec standard based on the traditional hybrid video codec framework has achieved great success, there is a bottleneck in the traditional hybrid video codec framework itself in order to further improve the compression efficiency of video coding. The research results show that the video signal is not a stationary source, which means that the characteristics of each coding unit are different. However, the design of functional modules in the traditional hybrid video codec framework is based on the assumption of a stationary video signal. For example, when the predictive coding module, transform coding module, quantization module, scanning module, etc. The working mode adopted is fixed:
(1)在帧间预测编码中预测补偿精确到亚像素时,需要采用插值技术对参考图像中的亚像素点进行构建。现行的基于传统混合视频编解码框架的视频标准都采用横向、纵向可分离的一维插值滤波器,对亚像素进行构建。插值滤波器的抽头个数和系数都是固定的,因此,采用的插值滤波器与被插值图像的内容无关。(1) When the prediction compensation is accurate to sub-pixel in inter-frame predictive coding, it is necessary to use interpolation technology to construct the sub-pixel points in the reference image. The current video standards based on the traditional hybrid video codec framework all use horizontally and vertically separable one-dimensional interpolation filters to construct sub-pixels. The number of taps and coefficients of the interpolation filter are fixed, so the interpolation filter used has nothing to do with the content of the interpolated image.
(2)变换编码模块广泛采用离散余弦变换(DCT)技术以及其近似变换技术整数余弦变换(ICT)技术。变换编码拟在降低空间相关性,将编码单元能量向少数几个变换系数集中,在DCT、ICT中就是向低频能量集中。变换矩阵都是固定的,因此,采用的变换与被变换图像的内容无关。(2) The transform coding module widely adopts discrete cosine transform (DCT) technology and its approximate transform technology integer cosine transform (ICT) technology. Transform coding intends to reduce the spatial correlation, and concentrate the energy of the coding unit to a few transform coefficients. In DCT and ICT, it is concentrated on low-frequency energy. The transformation matrices are all fixed, so the transformation applied is independent of the content of the image being transformed.
(3)量化模块是对变换系数的一种有损不可逆转编码模块。目前视频标准中采用的量化技术是对各个变换系数进行相同步长的标量量化或者通过量化矩阵加权对高频系数进行粗量化(利用人眼对高频信号不敏感特性)。可见,量化过程与被量化的图像内容无关。(3) The quantization module is a lossy irreversible coding module for transform coefficients. The quantization technology used in the current video standard is to perform scalar quantization with the same step length on each transform coefficient or to perform rough quantization on high-frequency coefficients through quantization matrix weighting (using the insensitivity of human eyes to high-frequency signals). It can be seen that the quantization process has nothing to do with the quantized image content.
(4)扫描模块是将二维信号转化成一维信号,具体就是把量化后的二维变换系数转化为run、level信号,以利于对run、level信号进行熵编码。目前视频编码标准中采用针对帧编码的zigzag扫描方式和针对场编码的纵向优先的alternative扫描方式,并且固定这两种扫描的扫描顺序,对于编码块的变换系数基本沿着左上到右下的顺序。可见,扫描顺序与被扫描的图像内容无关。(4) The scanning module converts two-dimensional signals into one-dimensional signals. Specifically, it converts quantized two-dimensional transformation coefficients into run and level signals, so as to facilitate entropy coding of run and level signals. The current video coding standard adopts the zigzag scanning method for frame coding and the vertical priority alternative scanning method for field coding, and the scanning order of these two scanning methods is fixed, and the transformation coefficients of coding blocks are basically in the order from upper left to lower right . It can be seen that the scanning order has nothing to do with the scanned image content.
近年来,为了进一步提高视频编码效率,涌现出一些新的编码技术,这些技术的共同点在于“自适应”,可以对每帧或者每个编码块选择不同的的编码方式(指某些功能模块选择相适应工作模式)。这些技术的自适应方法,有些基于率失真优化(RDO)技术实现,即通过RDO这种高复杂度的途径从几种候选的方法中选择一种在RD意义上最优的方法;有些基于统计的方法,采用“两遍”的思想,在第一遍结束后,利用第一遍的数据统计得到相适应的工作模式,然后用相适应的工作模式进行第二遍编码。In recent years, in order to further improve the efficiency of video coding, some new coding technologies have emerged. The commonality of these technologies is "adaptive", which can select different coding methods for each frame or each coding block (referring to certain functional modules. Select the appropriate working mode). Some of the adaptive methods of these technologies are based on the rate-distortion optimization (RDO) technology, that is, through the high-complexity approach of RDO, an optimal method in the sense of RD is selected from several candidate methods; some are based on statistical The method adopts the idea of "two passes". After the first pass, the data statistics of the first pass are used to obtain the corresponding working mode, and then the second pass of encoding is performed with the corresponding working mode.
另外有一种自适应变换技术,它基于神经网络的方法。最初设定一个初始变换模式,随着编码的进行,通过神经网络逐步训练新的变换模式,对接下来的编码块进行变换编码。In addition, there is an adaptive transformation technique, which is based on the method of neural network. An initial transformation mode is initially set, and as the encoding progresses, a new transformation mode is gradually trained through the neural network, and the next encoding block is transformed and encoded.
这些方法的“自适应”观点实际上是利用编码单元的局部特征不同的思想,但是它们的局部特征是模糊笼统的。The "adaptive" point of view of these methods is actually the idea of exploiting the different local characteristics of coding units, but their local characteristics are vague and general.
在前面分析和研究的基础上,为了突破传统混合视频编解码框架的瓶颈,本发明提出一种纹理自适应视频编解码系统,纹理自适应视频编解码系统包括纹理自适应视频编码系统和纹理自适应视频解码系统。纹理自适应视频编解码系统把视频图像的纹理特征(图像局部特征的一种)信息纳入到视频编解码系统中,以提高视频编码的压缩效率和主观质量。On the basis of the previous analysis and research, in order to break through the bottleneck of the traditional hybrid video coding and decoding framework, the present invention proposes a texture adaptive video coding and decoding system. The texture adaptive video coding and decoding system includes a texture adaptive video coding system and a texture Adapt to the video decoding system. The texture adaptive video codec system incorporates the texture feature (a kind of image local feature) information of the video image into the video codec system to improve the compression efficiency and subjective quality of video coding.
发明内容Contents of the invention
本发明的目的在于针对传统混合视频编解码框架的瓶颈,提出一种纹理自适应视频编解码系统。纹理自适应视频编解码系统包含纹理自适应视频编码系统和纹理自适应视频解码系统。The purpose of the present invention is to propose a texture adaptive video encoding and decoding system aiming at the bottleneck of the traditional hybrid video encoding and decoding framework. The texture adaptive video codec system includes a texture adaptive video coding system and a texture adaptive video decoding system.
本发明的目的是通过以下技术方案来实现的:一种纹理自适应视频编码系统,它包括视频编码器和编码端纹理分析器;视频编码器至少包括一个编码功能模块,以完成编码压缩;编码端纹理分析器用于进行纹理分析,以提取编码单元的纹理特征信息;视频编码器包括纹理自适应扫描模块,所述纹理自适应扫描模块根据编码端纹理分析器提取的纹理特征信息,选择一种相适应的扫描顺序。The purpose of the present invention is achieved through the following technical solutions: a texture adaptive video coding system, which includes a video encoder and an encoding end texture analyzer; the video encoder includes at least one encoding function module to complete encoding compression; encoding The end texture analyzer is used for texture analysis to extract the texture feature information of the coding unit; the video encoder includes a texture adaptive scanning module, and the texture adaptive scanning module selects a appropriate scan order.
进一步地:所述纹理特征信息包含纹理方向信息,或者所述纹理特征信息包含纹理方向信息和纹理强度信息。Further: the texture feature information includes texture direction information, or the texture feature information includes texture direction information and texture intensity information.
进一步地:所述的编码端纹理分析器的输入信号包含以下一种或多种:原始图像数据、编码单元的参考图像数据、编码功能模块输出数据。Further: the input signal of the texture analyzer at the coding end includes one or more of the following: original image data, reference image data of the coding unit, and output data of the coding function module.
进一步地:视频编码器还包括纹理自适应插值模块,所述纹理自适应插值模块根据编码端纹理分析器提取的纹理特征信息,选择一种相适应的插值方法,来构建亚像素点,其中,所述纹理特征信息是纹理方向信息。Further: the video encoder also includes a texture adaptive interpolation module, the texture adaptive interpolation module selects an appropriate interpolation method to construct sub-pixel points according to the texture feature information extracted by the texture analyzer at the encoding end, wherein, The texture feature information is texture direction information.
进一步地:纹理自适应插值模块和纹理自适应扫描模块使用同种的或不同种的编码单元纹理特征信息进行控制。Further: the texture adaptive interpolation module and the texture adaptive scanning module use the same or different texture feature information of the coding unit for control.
一种纹理自适应视频解码系统,它包括视频解码器和解码端纹理分析器;视频解码器至少包括一个解码功能模块,以完成解码重建;解码端纹理分析器用于进行纹理分析,以提取解码单元的纹理特征信息;视频解码器包括纹理自适应扫描模块,所述纹理自适应扫描模块根据解码端纹理分析器提取的纹理特征信息,选择一种相适应的反扫描顺序。A texture adaptive video decoding system, which includes a video decoder and a texture analyzer at a decoding end; the video decoder includes at least one decoding function module to complete decoding and reconstruction; the texture analyzer at the decoding end is used for texture analysis to extract decoding units The texture feature information; the video decoder includes a texture adaptive scanning module, and the texture adaptive scanning module selects an appropriate inverse scanning order according to the texture feature information extracted by the texture analyzer at the decoding end.
进一步地:所述的纹理特征信息包含纹理方向信息,或者所述纹理特征信息包含纹理方向信息和纹理强度信息。Further: the texture feature information includes texture direction information, or the texture feature information includes texture direction information and texture intensity information.
进一步地:所述的解码端纹理分析器的输入信号包含以下一种或多种:参考图像数据、解码功能模块输出数据。Further: the input signal of the texture analyzer at the decoding end includes one or more of the following: reference image data, and output data of a decoding function module.
进一步地:视频解码器还包括纹理自适应插值模块,所述纹理自适应插值模块根据解码端纹理分析器提取的纹理特征信息,选择一种相适应的插值方法,来构建亚像素点,其中,所述纹理特征信息是纹理方向信息。Further: the video decoder also includes a texture adaptive interpolation module, and the texture adaptive interpolation module selects an appropriate interpolation method according to the texture feature information extracted by the texture analyzer at the decoding end to construct sub-pixel points, wherein, The texture feature information is texture direction information.
进一步地:纹理自适应插值模块和纹理自适应扫描模块使用同种的或不同种的解码单元纹理特征信息进行控制。Further: the texture adaptive interpolation module and the texture adaptive scanning module use the same or different texture feature information of the decoding unit for control.
本发明的有益效果是,本发明纹理自适应视频编解码系统把视频图像的纹理特征纳入到视频编解码系统中,以提高视频编码的压缩效率和主观质量。The beneficial effect of the present invention is that the texture adaptive video codec system of the present invention incorporates the texture features of video images into the video codec system, so as to improve the compression efficiency and subjective quality of video coding.
附图说明Description of drawings
图1是纹理自适应视频编码系统示意图;Fig. 1 is a schematic diagram of a texture adaptive video coding system;
图2是纹理自适应视频解码系统示意图;Fig. 2 is a schematic diagram of a texture adaptive video decoding system;
图3是纹理自适应视频编解码系统示意图;Fig. 3 is a schematic diagram of a texture adaptive video codec system;
图4是n×m编码块原始数据示意图;Fig. 4 is a schematic diagram of the original data of an n × m coding block;
图5是n×m参考图像块的数据示意图;FIG. 5 is a schematic diagram of data of n×m reference image blocks;
图6是帧内预测模式作为编码端纹理分析器信号输入示意图;Fig. 6 is a schematic diagram of an intra prediction mode as a signal input to a texture analyzer at the encoding end;
图7是Sobel算子示意图;Figure 7 is a schematic diagram of the Sobel operator;
图8纹理自适应插值模块的示意图;Figure 8 is a schematic diagram of a texture adaptive interpolation module;
图9是整像素点和亚像素点示意图;Fig. 9 is a schematic diagram of integer pixels and sub-pixels;
图10是纹理自适应扫描模块的示意图;Fig. 10 is a schematic diagram of a texture adaptive scanning module;
图11是纵向优先扫描顺序示意图;FIG. 11 is a schematic diagram of a vertical priority scanning sequence;
图12是横向优先扫描顺序示意图;FIG. 12 is a schematic diagram of a horizontal priority scanning sequence;
图13是实施实例1示意图:纹理自适应视频编码系统;Fig. 13 is a schematic diagram of implementation example 1: texture adaptive video coding system;
图14是实施实例2示意图:纹理自适应视频解码系统;Fig. 14 is a schematic diagram of implementation example 2: texture adaptive video decoding system;
图15是实施实例4示意图:纹理自适应视频编码系统;Fig. 15 is a schematic diagram of implementation example 4: texture adaptive video coding system;
图16是实施实例5示意图:纹理自适应视频解码系统;Fig. 16 is a schematic diagram of implementation example 5: texture adaptive video decoding system;
图17是实施实例7示意图:纹理自适应视频编码系统;Fig. 17 is a schematic diagram of implementation example 7: texture adaptive video coding system;
图18是实施实例8示意图:纹理自适应视频解码系统;Fig. 18 is a schematic diagram of implementation example 8: texture adaptive video decoding system;
图19是实施实例10示意图:纹理自适应视频编码系统;Fig. 19 is a schematic diagram of implementation example 10: texture adaptive video coding system;
图20是实施实例11示意图:纹理自适应视频解码系统。Fig. 20 is a schematic diagram of implementation example 11: texture adaptive video decoding system.
具体实施方式Detailed ways
本发明涉及一种纹理自适应视频编码系统(图1所示)、一种纹理自适应视频解码系统(图2所示)和一种纹理自适应视频编解码系统(图3所示),注意图3中所示的“传输信道”不包含在纹理自适应视频编解码系统中。The present invention relates to a texture adaptive video coding system (shown in Figure 1), a texture adaptive video decoding system (shown in Figure 2) and a texture adaptive video coding system (shown in Figure 3), note The "transport channel" shown in Figure 3 is not included in the texture adaptive video codec system.
纹理自适应视频编解码系统涉及范围很广,下面先对本发明中涉及的名词进行举例说明。The texture adaptive video coding and decoding system involves a wide range, and the terms involved in the present invention will be described with examples below.
A、编码单元的实例A. Examples of coding units
编码单元是纹理自适应的单元,它由视频像素点组成的集合。编码单元的形式很多,在早期的差分脉冲调制编码系统中,编码单元为一个一个单独的像素点;在当前许多视频编码标准中编码单元是矩形像素块,包括方块;而最新有文献中提到的编码单元是三角形、梯形等不同的形式;编码单元也可以是一个条带(slice)、一个帧、一个场等形式;此外,编码单元还可以由不相邻的像素点组成。A coding unit is a texture-adaptive unit, which is a collection of video pixels. There are many forms of coding units. In the early differential pulse modulation coding system, the coding unit is a single pixel; in many current video coding standards, the coding unit is a rectangular pixel block, including a square; and the latest literature mentioned The coding unit is in different forms such as triangle and trapezoid; the coding unit can also be in the form of a slice, a frame, a field, etc.; in addition, the coding unit can also be composed of non-adjacent pixels.
编码块是编码单元的一种实例,它是由像素点组成的矩形块,矩形块大小为n×m,代表该编码块高度为n个像素点,宽度为m个像素点。比如16×16的编码块,16×8的编码块,8×16的编码块,8×8的编码块,8×4的编码块,4×8的编码块,4×4的编码块。以下将以编码块为例来给出具体实施实例,在没有特别说明时,将使用编码块代替编码单元。但实施实例中列举的方法同样可用于其它形式的编码单元。A coding block is an example of a coding unit, which is a rectangular block composed of pixels. The size of the rectangular block is n×m, which means that the coding block has a height of n pixels and a width of m pixels. For example, a 16×16 coding block, a 16×8 coding block, an 8×16 coding block, an 8×8 coding block, an 8×4 coding block, a 4×8 coding block, and a 4×4 coding block. Hereinafter, a specific implementation example will be given by taking a coding block as an example. Unless otherwise specified, a coding block will be used instead of a coding unit. However, the methods listed in the implementation examples can also be used for other types of coding units.
B、解码单元的实例B. Examples of decoding units
解码单元和编码单元是同一事物在系统不同位置的不同说法。编码单元是纹理自适应视频编码系统中的概念,与此对应的,在纹理自适应视频解码系统中,它就被称为解码单元。所以A中提到编码单元的举例和说明也适合对解码单元的举例和说明。The decoding unit and the encoding unit are different expressions of the same thing in different positions of the system. A coding unit is a concept in a texture-adaptive video coding system, and correspondingly, it is called a decoding unit in a texture-adaptive video decoding system. Therefore, the example and description of the encoding unit mentioned in A is also suitable for the example and description of the decoding unit.
C、视频编码功能模块的实例C. Examples of video encoding functional modules
视频编码器中编码功能模块包括以下模块中的一个或多个:预测模块、插值模块、变换模块、反变换模块、量化模块、反量化模块、扫描模块、反扫描模块、去块滤波模块、熵编码模块等。这些编码功能模块可以一个细分为多个或多个合并成一个编码功能模块,比如插值模块可以分成二分之一像素插值模块和四分之一像素插值模块;比如变换模块和量化模块合并为变换量化模块。视频编码器也可以有其它的功能划分方法,形成一套新的编码功能模块。The encoding function module in the video encoder includes one or more of the following modules: prediction module, interpolation module, transformation module, inverse transformation module, quantization module, inverse quantization module, scanning module, inverse scanning module, deblocking filtering module, entropy Encoding modules, etc. These coding functional modules can be subdivided into multiple or combined into one coding functional module. For example, the interpolation module can be divided into a half-pixel interpolation module and a quarter-pixel interpolation module; for example, the transformation module and the quantization module are combined into Transform quantization module. The video encoder may also have other functional division methods to form a new set of encoding functional modules.
视频编码器中的编码功能模块通过一定的方式相联,完成编码压缩的功能。The encoding function modules in the video encoder are connected in a certain way to complete the function of encoding and compression.
对于一个编码功能模块,其工作模式可以多样,比如插值模块,采用不同的滤波器抽头个数和滤波器系数就是不同的工作模式。For an encoding function module, its working mode can be various, for example, the interpolation module adopts different numbers of filter taps and filter coefficients to be different working modes.
D、视频解码功能模块的实例D. Examples of video decoding function modules
视频解码器中的解码功能模块包括以下模块中的一个或多个:预测模块、插值模块、反变换模块、反量化模块、反扫描模块、去块滤波模块、熵解码模块等。这些解码功能模块可以一个细分为多个或多个合并成一个解码功能模块,比如插值模块可以分成二分之一像素插值模块和四分之一像素插值模块;比如反变换模块和反量化模块合并为反变换量化模块。视频解码器也可以有其它的功能划分方法,形成一套新的解码功能模块。The decoding function modules in the video decoder include one or more of the following modules: prediction module, interpolation module, inverse transformation module, inverse quantization module, inverse scanning module, deblocking filtering module, entropy decoding module and the like. These decoding function modules can be subdivided into multiple or combined into one decoding function module. For example, the interpolation module can be divided into a half-pixel interpolation module and a quarter-pixel interpolation module; such as an inverse transformation module and an inverse quantization module. Combined into an inverse transform quantization module. The video decoder may also have other functional division methods to form a new set of decoding functional modules.
视频解码器中的解码功能模块通过一定的方式相联,完成解码重建的功能。The decoding function modules in the video decoder are connected in a certain way to complete the function of decoding and reconstruction.
对于一个解码功能模块,其工作模式可以多样,比如插值模块,采用不同的滤波器抽头个数和滤波器系数就是不同的工作模式。For a decoding function module, its working mode can be varied, for example, the interpolation module adopts different numbers of filter taps and filter coefficients to be different working modes.
E、纹理特征信息的实例E. Examples of texture feature information
纹理特征信息的表示方式可以有纹理方向信息、纹理强度信息、纹理方向强度信息,也可以有其它的表示方式,例如纹理结构等。The representation of texture feature information may include texture direction information, texture strength information, texture direction strength information, or other representations, such as texture structure.
E-1、纹理方向信息E-1. Texture direction information
纹理方向信息主观表现为图像中纹理的朝向,一般用纹理倾斜角度表示。倾斜角度是连续量,在使用时,可以量化为离散量。量化的时候可以选择不同的精度,把纹理分成不同种类的方向。量化时,将角度属于同一量化区域的纹理倾斜角度归为同一类纹理方向。比如,量化精度为4档次时,纹理方向信息可以分为横向纹理、纵向纹理、左对角纹理和右对角纹理。当然,有些编码单元,它没有明显的纹理方向,也可以说各个纹理方向对应的纹理强度相当,称之为平坦区域,平坦区域是一种特殊的纹理方向信息。Texture direction information is subjectively expressed as the orientation of the texture in the image, which is generally represented by the texture tilt angle. The tilt angle is a continuous quantity, and it can be quantized into a discrete quantity when used. When quantizing, you can choose different precisions and divide the texture into different types of directions. During quantization, texture tilt angles whose angles belong to the same quantization area are classified as the same type of texture direction. For example, when the quantization precision is 4 grades, the texture direction information can be divided into horizontal texture, vertical texture, left diagonal texture and right diagonal texture. Of course, some coding units have no obvious texture direction, and it can also be said that the texture intensity corresponding to each texture direction is equivalent, which is called a flat area, and the flat area is a special texture direction information.
图像中的边沿的方向是一种纹理方向信息的实例。The direction of an edge in an image is an example of texture direction information.
E-2、纹理强度信息E-2. Texture strength information
纹理强度信息主观表现为图像中纹理的明显程度,可以用梯度强度表示、也可以用能量强度表示,还可以用其它方法表示。Texture intensity information is subjectively expressed as the degree of texture in the image, which can be expressed by gradient intensity, energy intensity, or other methods.
E-3、纹理方向强度信息E-3. Texture direction strength information
纹理方向强度信息是指将纹理方向按E-1分成不同种类,而每一种类的纹理方向都有与之对应的强度信息。纹理方向强度信息就是对应于各纹理方向的纹理强度信息。The texture direction intensity information means that the texture directions are divided into different types according to E-1, and each type of texture direction has corresponding intensity information. The texture direction intensity information is the texture intensity information corresponding to each texture direction.
F、编码端纹理分析器的输入信号的实例F. An example of the input signal of the texture analyzer at the encoding end
F-1、原始图像数据F-1. Raw image data
原始图像数据是指由原始图像原始像素值组成或构建的数据。构建方式多种多样,例如插值方式、滤波方式、像素重复方式等。Raw image data refers to data that consists of or is constructed from the raw pixel values of a raw image. There are various construction methods, such as interpolation methods, filtering methods, pixel repetition methods, etc.
F-2、参考图像数据F-2. Reference image data
参考图像数据是指由已解码重建图像的像素值组成或构建的数据。构建方式多种多样,例如插值方式、滤波方式、像素重复方式等。Reference image data refers to data consisting of or constructed from pixel values of a decoded reconstructed image. There are various construction methods, such as interpolation methods, filtering methods, pixel repetition methods, etc.
F-3、编码功能模块输出数据F-3. Output data of encoding function module
编码功能模块输出的与当前编码单元对应的数据。The data corresponding to the current encoding unit output by the encoding function module.
例如,功能模块为帧内预测模块,输出数据为当前编码单元的帧内预测模式。图6是此例的示意图。For example, the functional module is an intra-frame prediction module, and the output data is the intra-frame prediction mode of the current coding unit. Figure 6 is a schematic diagram of this example.
编码功能模块输出的与(一个或多个)已编码的编码单元对应的数据。The data corresponding to the encoded encoding unit(s) output by the encoding function module.
例如,功能模块为帧内预测模块,输出数据为当前编码单元上方和左方的编码单元的帧内预测模式。为了供编码端纹理分析器分析当前编码单元的纹理特征,这些信息应该经过一定时间的缓存后输入编码端纹理分析器。For example, the functional module is an intra-frame prediction module, and the output data is the intra-frame prediction modes of the coding units above and to the left of the current coding unit. In order for the texture analyzer at the encoding end to analyze the texture features of the current coding unit, the information should be cached for a certain period of time and then input to the texture analyzer at the encoding end.
编码功能模块输出数据不限于帧内预测模块输出的帧内预测模式,它可以是变换模块输出的变换系数;扫描模块输出的扫描顺序;帧间预测模块输出的帧间预测模式等。The output data of the encoding function module is not limited to the intra prediction mode output by the intra prediction module, it can be the transformation coefficient output by the transformation module; the scanning sequence output by the scanning module; the inter prediction mode output by the inter prediction module, etc.
注意,编码功能模块输出数据是指编码功能模块的部分或者全部输出数据,例如这里帧内预测模式只是帧内预测模块的部分输出数据。Note that the output data of the encoding function module refers to part or all of the output data of the encoding function module, for example, the intra prediction mode here is only part of the output data of the intra prediction module.
F-4、输入信号为多种数据F-4, the input signal is a variety of data
输入信号包括以下数据中的多种数据:原始图像数据、参考图像数据和编码功能模块输出数据。The input signal includes multiple kinds of data in the following data: original image data, reference image data and output data of the encoding function module.
例如,编码单元的原始图像数据和编码单元帧间匹配单元作为输入信号都送入编码端纹理分析器。For example, the original image data of the coding unit and the inter-frame matching unit of the coding unit are both sent to the texture analyzer at the coding end as input signals.
其中,图4是编码单元的原始图像数据的一个例子,编码单元为编码块,它是一个n×m的块P,Pji代表第(j,i)位置的像素值,是该像素点的原始像素值。Among them, Figure 4 is an example of the original image data of the coding unit. The coding unit is a coding block, which is an n×m block P, and P ji represents the pixel value at the (j,i)th position, which is the raw pixel value.
匹配单元是与编码单元最相近的参考图像数据。构成匹配单元的像素点与当前编码单元不在同一帧图像中,称为帧间匹配单元;构成匹配单元的像素点与当前编码单元在同一帧图像中,称为帧内匹配单元。图5是帧间匹配单元的一个例子,帧间匹配单元R,是大小为n×m的块。Rji代表第(j,i)位置像素点的值。A matching unit is the closest reference image data to a coding unit. The pixels constituting the matching unit and the current coding unit are not in the same frame image, which is called an inter-frame matching unit; the pixels constituting the matching unit and the current coding unit are in the same frame image, which is called an intra-frame matching unit. FIG. 5 is an example of an inter-frame matching unit. The inter-frame matching unit R is a block of size n×m. R ji represents the value of the pixel at the (j,i)th position.
G、解码端纹理分析器的输入信号的实例G. An example of the input signal of the texture analyzer at the decoding end
G-1、参考图像数据G-1. Reference image data
解码端纹理分析器的输入信号为参考图像数据时,它与F-2相一致。When the input signal of the texture analyzer at the decoding end is the reference image data, it is consistent with F-2.
G-2、解码功能模块输出数据G-2. Decoding function module output data
解码功能模块输出的与当前解码单元对应的数据。The data corresponding to the current decoding unit output by the decoding function module.
例如,一、熵解码模块进行码流解析后,获得当前解码单元纹理特征信息,将此纹理特征信息输出到解码端纹理分析器;二、功能模块为帧内预测模块,输出数据为当前解码单元的帧内预测模式。For example, first, the entropy decoding module obtains the texture feature information of the current decoding unit after parsing the code stream, and outputs the texture feature information to the texture analyzer at the decoding end; second, the functional module is an intra prediction module, and the output data is the current decoding unit The intra prediction mode for .
解码功能模块输出的与(一个或多个)已解码的解码单元对应的数据。Data corresponding to the decoded decoding unit(s) output by the decoding function module.
例如,功能模块为帧内预测模块,输出数据为当前解码单元上方和左方的解码单元的帧内预测模式。为了供解码端纹理分析器分析当前解码单元的纹理特征,这些信息应该经过一定时间的缓存后输入解码端纹理分析器。For example, the functional module is an intra-frame prediction module, and the output data is the intra-frame prediction modes of the decoding units above and to the left of the current decoding unit. In order for the texture analyzer at the decoding end to analyze the texture features of the current decoding unit, the information should be buffered for a certain period of time before being input to the texture analyzer at the decoding end.
解码功能模块输出数据不限于这些例子,它还可以是变换模块输出的变换系数;扫描模块输出的扫描顺序;帧间预测模块输出的帧间预测模式等。The output data of the decoding function module is not limited to these examples, and it can also be the transform coefficient output by the transform module; the scan order output by the scan module; the inter prediction mode output by the inter prediction module, etc.
注意,解码功能模块输出数据是指解码功能模块的部分或者全部输出数据,例如这里帧内预测模式只是熵解码模块的部分输出数据。Note that the output data of the decoding function module refers to part or all of the output data of the decoding function module, for example, the intra prediction mode here is only part of the output data of the entropy decoding module.
G-3、输入信号为多种数据G-3, the input signal is a variety of data
输入信号包括以下数据中的多种数据:参考图像数据和解码功能模块输出数据。The input signal includes multiple kinds of data in the following data: reference image data and decoding function module output data.
例如,解码单元的帧间匹配单元与解码单元的上方和左方解码单元的帧间预测模式作为输入信号都送入解码端纹理分析器。For example, the inter-frame matching unit of the decoding unit and the inter-frame prediction modes of the upper and left decoding units of the decoding unit are sent as input signals to the texture analyzer at the decoding end.
H、编码端纹理分析器提取编码单元纹理特征信息的实例H. An example of extracting coding unit texture feature information by the texture analyzer at the coding end
H-1、输入信号为原始图像数据H-1. The input signal is the original image data
编码端纹理分析器的输入信号如F-1所述。The input signal of the texture analyzer at the encoding end is as described in F-1.
以编码单元的原始图像数据为输入信号的情况为例子,此处编码单元为编码块,图4是一个n×m的编码块P,Pji代表第(j,i)位置像素点的值,是该像素点的原始数据。纹理分析器以Sobel算子来提取得到编码块P的纹理特征信息,即纹理方向信息和纹理强度信息。图7是Sobel算子x方向和y方向梯度的求取方法。根据Sobel算子可以求得Pji的x方向和y方向的梯度。Take the case where the original image data of the coding unit is the input signal as an example, where the coding unit is the coding block, and Figure 4 is an n×m coding block P, where P ji represents the value of the pixel at the (j,i)th position, is the original data of the pixel. The texture analyzer uses the Sobel operator to extract the texture feature information of the coding block P, that is, texture direction information and texture intensity information. Fig. 7 is the calculation method of Sobel operator x-direction and y-direction gradient. According to the Sobel operator, the gradients in the x-direction and y-direction of P ji can be obtained.
其x方向的梯度为:The gradient in the x direction is:
hx(Pji)=Pj-1,i-1+2×Pj-1,i+Pj-1,i+1-Pj+1,i-1-2×Pj+1,i-Pj+1,i+1 hx(P ji )=P j-1,i-1 +2×P j-1,i +P j-1,i+1 -P j+1,i-1 -2×P j+1,i -P j+1,i+1
其y方向的梯度为:The gradient in the y direction is:
hy(Pji)=Pj-1,i+1+2×Pj,i+1+Pj+1,i+1-Pj-1,i-1-2×Pj-1,i-Pj-1,i+1 hy(P ji )=P j-1,i+1 +2×P j,i+1 +P j+1,i+1 -P j-1,i-1 -2×P j-1,i -P j-1,i+1
则Pji的梯度方向为Then the gradient direction of P ji is
Dir(Pji)=arctan(hy(Pji)/hx(Pji)),arctan为反正切函数;Dir(P ji )=arctan(hy(P ji )/hx(P ji )), arctan is arctangent function;
Pji的梯度强度为The gradient strength of P ji is
Mag(Pji)=sqrt(hx((Pji)^2+hy(Pji)^2),sqrt为求根函数,^2指平方。Mag(P ji )=sqrt(hx((P ji )^2+hy(P ji )^2), sqrt is the root function, and ^2 refers to the square.
将Dir(Pji)量化成相应精度的档次,每个档次对应一种纹理方向信息,。比如四个量化档次:横向、纵向、左对角和右对角。Quantize Dir(P ji ) into grades of corresponding precision, and each grade corresponds to a kind of texture direction information. For example, four quantitative grades: horizontal, vertical, left diagonal and right diagonal.
为了确定P的纹理方向信息和纹理强度信息,依次求出P11到Pn-1,m-1这(n-2)×(m-2)个像素点的Dir值和Mag值,根据Dir量化档次,将这些点分类,求出每类中像素点Mag值的和,Mag值的和是P的纹理方向强度信息。占主导地位的纹理方向强度信息就是P的纹理强度信息,它所对应的纹理方向信息定为P的纹理方向信息;如果分类中没有占主导地位的纹理强度信息,可以认为P是平坦区域。In order to determine the texture direction information and texture intensity information of P, the Dir value and Mag value of the (n-2)×(m-2) pixel points from P 11 to P n-1, m-1 are calculated in turn, according to Dir Quantify the grade, classify these points, and calculate the sum of the Mag value of the pixel points in each class, and the sum of the Mag value is the texture direction intensity information of P. The dominant texture direction intensity information is the texture intensity information of P, and its corresponding texture direction information is defined as the texture direction information of P; if there is no dominant texture intensity information in the classification, P can be considered as a flat area.
纹理分析器还可以利用原始图像数据采用其它的算子或者方法提取与此实例中表达方式相同或不同的纹理特征信息。The texture analyzer can also use other operators or methods to extract texture feature information that is the same as or different from the expression in this example by using the original image data.
H-2、输入信号为参考图像数据H-2. The input signal is the reference image data
编码端纹理分析器的输入信号如F-2所述。The input signal of the texture analyzer at the encoding end is as described in F-2.
参考图像数据以帧间匹配单元为例子,编码单元为编码块,帧间匹配单元为帧间匹配块,例如图5。纹理特征信息提取方法可以采用H-1中例举的方法。The reference image data takes an inter-frame matching unit as an example, the coding unit is a coding block, and the inter-frame matching unit is an inter-frame matching block, as shown in FIG. 5 . The method for extracting texture feature information can adopt the method exemplified in H-1.
H-3、输入信号为编码功能模块输出数据H-3. The input signal is the output data of the encoding function module
编码端纹理分析器的输入信号如F-3所述。The input signal of the texture analyzer at the encoding end is as described in F-3.
例一:输入信号为F-3中当前编码单元的帧内预测模式,帧内预测模式是有方向性的,比如横向预测模式,纵向预测模式,左对角预测模式,右对角预测模式和DC预测模式。编码端纹理分析器根据预测模式决定该编码块的纹理特征信息,这里只是纹理方向信息。如果预测模式为横向预测模式,纹理方向为横向纹理;如果预测模式为纵向预测模式,纹理方向为纵向纹理;如果预测模式为左对角预测模式,纹理方向为左对角纹理;如果预测模式为右对角预测模式,纹理方向为右对角纹理;如果预测模式为DC预测模式,纹理特征信息为平坦区域,无明显纹理。Example 1: The input signal is the intra prediction mode of the current coding unit in F-3. The intra prediction mode is directional, such as horizontal prediction mode, vertical prediction mode, left diagonal prediction mode, right diagonal prediction mode and DC prediction mode. The texture analyzer at the coding end determines the texture feature information of the coding block according to the prediction mode, here it is only the texture direction information. If the prediction mode is horizontal prediction mode, the texture direction is horizontal texture; if the prediction mode is vertical prediction mode, the texture direction is vertical texture; if the prediction mode is left diagonal prediction mode, the texture direction is left diagonal texture; if the prediction mode is Right diagonal prediction mode, the texture direction is right diagonal texture; if the prediction mode is DC prediction mode, the texture feature information is a flat area without obvious texture.
例二:输入信号为帧间预测模块输出的当前编码单元上方和左方的编码单元的帧间预测模式信息。帧间预测模式是指帧间预测时的块大小。比如,当编码块上方和左方的编码块的帧间预测模式为16×8时,确定该编码块为横向纹理方向;当编码块上方和左方的编码块的帧间预测模式为8×16时,确定该编码块为纵向纹理方向;其它情况,确定该编码块为平坦区域。Example 2: The input signal is the inter prediction mode information of the coding unit above and to the left of the current coding unit output by the inter prediction module. The inter prediction mode refers to the block size at the time of inter prediction. For example, when the inter-frame prediction mode of the coding block above and to the left of the coding block is 16×8, it is determined that the coding block is in the horizontal texture direction; when the inter-frame prediction mode of the coding block above and to the left of the coding block is 8×8 At 16, it is determined that the coding block is in the longitudinal texture direction; in other cases, it is determined that the coding block is a flat area.
H-4、输入信号为组合数据H-4. The input signal is combined data
编码端纹理分析器的输入信号如F-4所述。The input signal of the texture analyzer at the encoding end is as described in F-4.
此处,原始图像数据以编码单元的原始数据为例子,参考图像数据以帧间匹配单元为例子。这两种信号作为编码端纹理分析器的输入信号,编码单元为编码块,编码端纹理分析器先求得它们间的差分信号,差分信号是指编码块和帧间匹配块的差值;对差分信号采用H-1中例举的方法处理,以获得编码单元纹理特征信息。Here, the original image data is exemplified by the original data of the coding unit, and the reference image data is exemplified by the inter-frame matching unit. These two signals are used as the input signals of the texture analyzer at the encoding end, and the encoding unit is the encoding block, and the texture analyzer at the encoding end first obtains the differential signal between them, and the differential signal refers to the difference between the encoding block and the matching block between frames; The differential signal is processed by the method exemplified in H-1 to obtain the texture feature information of the coding unit.
I、解码端纹理分析器提取纹理特征信息实例I. An example of extracting texture feature information by the texture analyzer at the decoding end
I-1、输入信号为参考图像数据I-1. The input signal is the reference image data
解码端纹理分析器的输入信号为参考图像数据时,如G-1所述,可以按H-2中例举的方法提取解码单元纹理特征信息。When the input signal of the texture analyzer at the decoding end is reference image data, as described in G-1, the texture feature information of the decoding unit can be extracted according to the method exemplified in H-2.
I-2、输入信号为解码功能模块输出数据I-2. The input signal is the output data of the decoding function module
例一:码流中包含解码单元纹理特征信息,解码端纹理分析器的输入信号是熵解码模块通过码流解析获得的解码块的纹理特征信息或者解码块的纹理特征信息的一种编码形式,将此信息输入到解码端纹理分析器,解码端纹理分析器直接利用或者解码这些信息,得到解码块的纹理特征信息。Example 1: The code stream contains the texture feature information of the decoding unit, and the input signal of the texture analyzer at the decoding end is the texture feature information of the decoding block obtained by the entropy decoding module through code stream analysis or a coded form of the texture feature information of the decoding block. This information is input to the texture analyzer at the decoding end, and the texture analyzer at the decoding end directly uses or decodes the information to obtain texture feature information of the decoding block.
例二:解码端纹理分析器的输入信号为帧内预测模块输出的当前解码块的帧内预测模式。解码端纹理分析器按照H-3中例举的方法,确定解码块的纹理特征信息。Example 2: The input signal of the texture analyzer at the decoding end is the intra prediction mode of the current decoding block output by the intra prediction module. The texture analyzer at the decoding end determines the texture feature information of the decoding block according to the method exemplified in H-3.
例三:输入信号为帧间预测模块输出的当前解码单元上方和左方的解码单元的帧间预测模式信息。解码端纹理分析器按照H-3中例举的方法,确定解码块的纹理特征信息。Example 3: The input signal is the inter prediction mode information of the decoding unit above and to the left of the current decoding unit output by the inter prediction module. The texture analyzer at the decoding end determines the texture feature information of the decoding block according to the method exemplified in H-3.
J、编码功能模块、解码功能模块的工作模式与纹理特征相适应的实例J. An example of adapting the working mode of the encoding function module and the decoding function module to the texture feature
J-1、纹理自适应插值模块J-1, texture adaptive interpolation module
图8是纹理自适应插值模块示意图。编码端纹理分析器提取编码块的纹理特征信息,由纹理特征信息控制纹理自适应插值模块,使其选择一种相适应的插值方法,也就是工作模式,来构建亚像素点。图8中纹理自适应插值模块有N类纹理方向插值,每一类对应一种工作模式。编码端纹理分析器提取的纹理特征信息有纹理方向信息。纹理自适应插值模块在获得编码块的纹理方向信息后,根据纹理方向的不同选择不同的工作模式,来进行亚像素的插值构建。图9是需要插值的亚像素点示意图,图中大写字母A-P代表整像素的位置;小写字母a-o代表整像素点A对应的亚像素点的位置。亚像素点可以分成二分之一像素点和四分之一像素点。其中,b、h、j代表二分之一像素点,其它小写字母代表四分之一像素点。下面表格列出的是编码块不同纹理方向所对应的插值工作模式。当然,插值滤波器的设计不仅仅局限于这种方法,还可以有其它设计方法。Fig. 8 is a schematic diagram of a texture adaptive interpolation module. The texture analyzer at the coding end extracts the texture feature information of the coding block, and the texture adaptive interpolation module is controlled by the texture feature information to make it choose an appropriate interpolation method, that is, the working mode, to construct sub-pixel points. The texture adaptive interpolation module in Figure 8 has N types of texture direction interpolation, and each type corresponds to a working mode. The texture feature information extracted by the texture analyzer at the encoding end includes texture direction information. After obtaining the texture direction information of the coding block, the texture adaptive interpolation module selects different working modes according to the texture direction to perform sub-pixel interpolation construction. FIG. 9 is a schematic diagram of sub-pixel points that need to be interpolated. In the figure, the uppercase letters A-P represent the position of the integer pixel; the lowercase letters a-o represent the position of the sub-pixel point corresponding to the integer pixel point A. Sub-pixels can be divided into half-pixels and quarter-pixels. Among them, b, h, and j represent half pixels, and other lowercase letters represent quarter pixels. The following table lists the interpolation working modes corresponding to different texture directions of the coding block. Of course, the design of the interpolation filter is not limited to this method, and other design methods are also possible.
纹理自适应插值模块位于纹理自适应视频编码系统称为纹理自适应插值编码功能模块,位于纹理自适应视频解码系统称为纹理自适应插值解码功能模块。The texture adaptive interpolation module located in the texture adaptive video coding system is called the texture adaptive interpolation encoding function module, and the texture adaptive interpolation module located in the texture adaptive video decoding system is called the texture adaptive interpolation decoding function module.
J-2、纹理自适应扫描模块实例J-2. Example of Texture Adaptive Scanning Module
图10是纹理自适应扫描方法示意图。编码端纹理分析器提取编码块的纹理特征信息,纹理自适应扫描模块根据编码端纹理分析器提取的纹理特征信息选择一种相适应的扫描顺序,每种扫描顺序对应一种工作模式。扫描顺序有纵向优先的扫描顺序;有横向优先的扫描顺序;有无方向优先的扫描顺序等。如图11所示,是8×8块两种纵向优先的扫描顺序的例子,图中右边的扫描顺序比左边的扫描顺序纵向优先程度大;如图12所示,是8×8块两种横向优先的扫描顺序的例子,图中右边的扫描顺序比左边的扫描顺序横向优先程度大;无方向优先的扫描顺序,比如zigzag扫描顺序。纹理分析器输出的纹理特征信息是纹理方向信息和纹理强度信息,纹理方向信息分成横向纹理、纵向纹理和其它纹理三种;纹理强度信息分成强、弱两种。下表举例说明了使用纹理方向信息和纹理强度信息控制纹理自适应扫描模块工作模式的情况,比如对于强的横向纹理,自适应扫描模块采用图11右图所示的扫描顺序作为工作模式;对于弱的其它纹理,自适应扫描模块采用zigzag扫描顺序作为工作模式等。Fig. 10 is a schematic diagram of a texture adaptive scanning method. The texture analyzer at the encoding end extracts texture feature information of the encoding block, and the texture adaptive scanning module selects an appropriate scanning sequence according to the texture feature information extracted by the texture analyzer at the encoding end, and each scanning sequence corresponds to a working mode. The scanning order has a vertical priority scanning sequence; a horizontal priority scanning sequence; a direction priority scanning sequence, etc. As shown in Figure 11, it is an example of two vertical priority scanning orders for 8×8 blocks. The scanning order on the right in the figure has a higher vertical priority than the scanning order on the left; as shown in Figure 12, there are two types of 8×8 blocks. An example of a horizontal priority scanning order, the scanning order on the right in the figure has a greater horizontal priority than the scanning order on the left; the scanning order without direction priority, such as the zigzag scanning order. The texture feature information output by the texture analyzer is texture direction information and texture intensity information. The texture direction information is divided into three types: horizontal texture, vertical texture and other textures; the texture intensity information is divided into two types: strong and weak. The following table exemplifies the use of texture direction information and texture intensity information to control the working mode of the texture adaptive scanning module. For example, for strong horizontal textures, the adaptive scanning module uses the scanning sequence shown in the right figure of Figure 11 as the working mode; for Weak other textures, the adaptive scanning module uses the zigzag scanning order as the working mode, etc.
下表举例说明了仅使用纹理方向信息控制纹理自适应扫描模块工作模式的情况,比如对于横向纹理,自适应扫描模块采用图11右图所示的扫描顺序作为工作模式;对于纵向纹理,自适应扫描模块采用图12右图所示的扫描顺序作为工作模式;对于其它纹理,自适应扫描模块采用zigzag扫描顺序作为工作模式。The following table exemplifies the situation where only texture direction information is used to control the working mode of the texture adaptive scanning module. For example, for horizontal textures, the adaptive scanning module uses the scanning sequence shown in the right figure of Figure 11 as the working mode; for vertical textures, the adaptive scanning module The scanning module adopts the scanning order shown in the right figure of Figure 12 as the working mode; for other textures, the adaptive scanning module adopts the zigzag scanning order as the working mode.
纹理自适应扫描模块位于纹理自适应视频编码系统称为纹理自适应扫描编码功能模块,位于纹理自适应视频解码系统称为纹理自适应扫描解码功能模块。The texture adaptive scanning module located in the texture adaptive video coding system is called the texture adaptive scanning coding function module, and the texture adaptive video decoding system is called the texture adaptive scanning decoding function module.
当然,还有其它编码功能模块、解码功能模块与纹理特征相适应的例子,比如变换模块、量化模块等等,不再一一例举。Of course, there are other examples where the encoding functional modules and decoding functional modules adapt to texture features, such as transformation modules, quantization modules, etc., and will not be listed one by one.
实施实例1Implementation example 1
图13是实施实例1的示意图,它是一个纹理自适应视频编码系统。编码端纹理分析器的输入信号为编码单元的参考图像数据,如F-2所述,输出信息是编码单元的纹理方向信息,纹理方向信息的提取方法如H-2所例举的方法。输出的纹理方向信息控制视频编码器中纹理自适应插值模块的工作模式,使得纹理自适应插值模块根据纹理方向信息以J-1中所例举的方法选取相适应的插值工作模式。视频编码器中的其它编码功能模块不根据纹理特征信息进行自适应。Fig. 13 is a schematic diagram of implementation example 1, which is a texture adaptive video coding system. The input signal of the texture analyzer at the coding end is the reference image data of the coding unit, as described in F-2, and the output information is the texture direction information of the coding unit, and the method of extracting the texture direction information is as exemplified in H-2. The output texture direction information controls the working mode of the texture adaptive interpolation module in the video encoder, so that the texture adaptive interpolation module selects an appropriate interpolation working mode according to the texture direction information by the method exemplified in J-1. Other encoding functional modules in the video encoder do not perform self-adaptation according to texture feature information.
实施实例2Implementation example 2
图14是实施实例2的示意图,它是一个纹理自适应视频解码系统。解码端纹理分析器的输入信号为解码单元的参考图像数据,如G-1所述,输出信息是解码单元的纹理方向信息,纹理方向信息的提取方法如I-1所例举的方法。输出的纹理方向信息控制视频解码器中纹理自适应插值模块的工作模式,使得纹理自适应插值模块根据纹理方向信息以J-1中所例举的方法选取相适应的插值工作模式。视频解码器中的其它解码功能模块不根据纹理特征信息进行自适应。Fig. 14 is a schematic diagram of implementation example 2, which is a texture adaptive video decoding system. The input signal of the texture analyzer at the decoding end is the reference image data of the decoding unit, as described in G-1, and the output information is the texture direction information of the decoding unit, and the extraction method of the texture direction information is as exemplified in I-1. The output texture direction information controls the working mode of the texture adaptive interpolation module in the video decoder, so that the texture adaptive interpolation module selects an appropriate interpolation working mode according to the texture direction information by the method exemplified in J-1. Other decoding functional modules in the video decoder do not perform self-adaptation according to texture feature information.
实施实例3Implementation example 3
实施实例3的纹理自适应视频编解码系统包含实施实例1的纹理自适应视频编码系统和实施实例2的纹理自适应视频解码系统。The texture adaptive video coding system implementing Example 3 includes the texture adaptive video coding system implementing Example 1 and the texture adaptive video decoding system implementing Example 2.
实施实例4Implementation Example 4
图15是实施实例4的示意图,它是一个纹理自适应视频编码系统。编码端纹理分析器的输入信号为编码单元的参考图像数据,如F-2所述,输出信息是编码单元的纹理方向信息和纹理强度信息,提取方法采用H-2所例举的方法。纹理方向信息控制视频编码器中纹理自适应插值模块,使得纹理自适应插值模块根据纹理方向信息以J-1中所例举的方法选取相适应的插值工作模式;纹理方向信息和纹理强度信息控制视频编码器中纹理自适应扫描模块,使得纹理自适应扫描模块按照J-2中所例举的方法选取相适应的扫描工作模式。视频编码器中的其它编码功能模块不根据纹理特征信息进行自适应。Fig. 15 is a schematic diagram of implementation example 4, which is a texture adaptive video coding system. The input signal of the texture analyzer at the coding end is the reference image data of the coding unit, as described in F-2, and the output information is the texture direction information and texture intensity information of the coding unit, and the extraction method adopts the method exemplified in H-2. The texture direction information controls the texture self-adaptive interpolation module in the video encoder, so that the texture self-adaptive interpolation module selects the appropriate interpolation mode according to the texture direction information in the method exemplified in J-1; the texture direction information and the texture strength information control The texture adaptive scanning module in the video encoder enables the texture adaptive scanning module to select an appropriate scanning working mode according to the method exemplified in J-2. Other encoding functional modules in the video encoder do not perform self-adaptation according to texture feature information.
实施实例5Implementation Example 5
图16是实施实例5的示意图,它是一个纹理自适应视频解码系统。解码端纹理分析器的输入信号为需要解码的解码单元的参考图像数据,如G-1所述,输出信息是需要解码单元的纹理方向信息和纹理强度信息,提取方法采用I-1所例举的方法。纹理方向信息控制视频解码器中纹理自适应插值模块,使得纹理自适应插值模块根据纹理方向信息以J-1中所例举的方法选取相适应的插值工作模式;纹理方向信息和纹理强度信息控制视频解码器中纹理自适应扫描模块,使得纹理自适应扫描模块按照J-2中所例举的方法选取相适应的反扫描工作模式。视频解码器中的其它解码功能模块不根据纹理特征信息进行自适应。Fig. 16 is a schematic diagram of implementation example 5, which is a texture adaptive video decoding system. The input signal of the texture analyzer at the decoding end is the reference image data of the decoding unit that needs to be decoded. As described in G-1, the output information is the texture direction information and texture intensity information of the decoding unit. The extraction method adopts the example in I-1 Methods. The texture direction information controls the texture self-adaptive interpolation module in the video decoder, so that the texture self-adaptive interpolation module selects the appropriate interpolation mode according to the texture direction information in the method exemplified in J-1; the texture direction information and the texture strength information control The texture adaptive scanning module in the video decoder enables the texture adaptive scanning module to select an appropriate inverse scanning working mode according to the method exemplified in J-2. Other decoding functional modules in the video decoder do not perform self-adaptation according to texture feature information.
实施实例6Implementation Example 6
实施实例6的纹理自适应视频编解码系统包含实施实例4的纹理自适应视频编码系统和实施实例5的纹理自适应视频解码系统。The texture adaptive video coding system implementing Example 6 includes the texture adaptive video coding system implementing Example 4 and the texture adaptive video decoding system implementing Example 5.
实施实例7Implementation example 7
图17是实施实例7的示意图,它是一个纹理自适应视频编码系统。编码端纹理分析器的输入信号为需要编码的编码单元的原始数据,如F-1所述,输出信息是需要编码单元的纹理方向信息和纹理强度信息,提取方法采用H-1所例举的方法。纹理方向信息控制视频编码器中纹理自适应插值模块,使得纹理自适应插值模块根据纹理方向信息按J-1中所例举的方法选取相适应的插值工作模式;纹理方向信息和纹理强度信息控制视频编码器中纹理自适应扫描模块,使得纹理自适应扫描模块按照J-2中所例举方法选取相适应的扫描工作模式。视频编码器中的其它编码功能模块不根据纹理特征信息进行自适应。Fig. 17 is a schematic diagram of
实施实例8Implementation Example 8
图18是实施实例8的示意图,它是一个纹理自适应视频解码系统。解码端纹理分析器的输入信号为熵解码模块的输出信号,如G-2所述,输出信息是需要解码的解码单元的纹理方向信息和纹理强度信息,纹理方向信息和纹理强度信息的提取方法采用I-2中所例举的方法。纹理方向信息控制视频解码器中纹理自适应插值模块,使得纹理自适应插值模块根据纹理方向特征信息按J-1中所例举的方法选取相适应的插值工作模式;纹理方向信息和纹理强度信息控制视频解码器中纹理自适应扫描模块,使得纹理自适应扫描模块按照J-2中所例举的方法选取相适应的反扫描工作模式。视频解码器中的其它解码功能模块不根据纹理特征信息进行自适应。Fig. 18 is a schematic diagram of implementation example 8, which is a texture adaptive video decoding system. The input signal of the texture analyzer at the decoding end is the output signal of the entropy decoding module. As described in G-2, the output information is the texture direction information and texture intensity information of the decoding unit that needs to be decoded, and the extraction method of the texture direction information and texture intensity information Use the method exemplified in I-2. The texture direction information controls the texture self-adaptive interpolation module in the video decoder, so that the texture self-adaptive interpolation module selects an appropriate interpolation work mode according to the texture direction characteristic information according to the method exemplified in J-1; texture direction information and texture intensity information Control the texture adaptive scanning module in the video decoder, so that the texture adaptive scanning module selects an appropriate inverse scanning working mode according to the method exemplified in J-2. Other decoding functional modules in the video decoder do not perform self-adaptation according to texture feature information.
实施实例9Implementation example 9
实施实例9的纹理自适应视频编解码系统包含实施实例7的纹理自适应视频编码系统和实施实例8的纹理自适应视频解码系统。The texture adaptive video coding system implementing Example 9 includes the texture adaptive video coding system implementing Example 7 and the texture adaptive video decoding system implementing Example 8.
实施实例10Implementation example 10
图19是实施实例10的示意图,它是一个纹理自适应视频编码系统。编码端纹理分析器的输入信号为帧内预测模块输出的帧内预测模式,如F-3所述,输出信息是编码单元的纹理方向信息,纹理方向信息的提取方法采用H-3中所例举的方法。纹理方向信息控制视频编码器中纹理自适应插值模块和纹理自适应扫描模块,使得纹理自适应插值模块根据纹理方向信息按J-1中所例举的方法选取相适应的插值工作模式;使得纹理自适应扫描模块根据纹理方向信息,按照J-2中方法选取相适应的扫描工作模式。视频编码器中的其它编码功能模块不根据纹理特征信息进行自适应。Fig. 19 is a schematic diagram of
实施实例11Implementation Example 11
图20是实施实例11的示意图,它是一个纹理自适应视频解码系统。解码端纹理分析器的输入信号为帧内预测模块输出的帧内预测模式,如G-2所述,输出信息是需要解码单元的纹理方向信息,纹理方向信息的提取方法采用I-2中所例举的方法。纹理方向信息控制视频解码器中纹理自适应插值模块和纹理自适应扫描模块,使得纹理自适应插值模块根据纹理方向信息按J-1中例举的方法选取相适应的插值工作模式;使得纹理自适应扫描模块根据纹理方向信息,按照J-2中所例举的方法选取相适应的反扫描工作模式。视频解码器中的其它解码功能模块不根据纹理特征信息进行自适应。Fig. 20 is a schematic diagram of
实施实例12Implementation example 12
实施实例12的纹理自适应视频编解码系统包含实施实例10的纹理自适应视频编码系统和实施实例11的纹理自适应视频解码系统。The texture adaptive video coding system implementing Example 12 includes the texture adaptive video coding system implementing Example 10 and the texture adaptive video decoding system implementing Example 11.
上述实施例用来解释说明本发明,而不是对本发明进行限制,在本发明的精神和权利要求的保护范围内,对本发明作出的任何修改和改变,都落入本发明的保护范围。The above-mentioned embodiments are used to illustrate the present invention, rather than to limit the present invention. Within the spirit of the present invention and the protection scope of the claims, any modification and change made to the present invention will fall into the protection scope of the present invention.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110388181.6A CN102413330B (en) | 2007-06-12 | 2007-06-12 | Texture-adaptive video coding/decoding system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110388181.6A CN102413330B (en) | 2007-06-12 | 2007-06-12 | Texture-adaptive video coding/decoding system |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 200710069093 Division CN101325707B (en) | 2007-06-12 | 2007-06-12 | System for encoding and decoding texture self-adaption video |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102413330A CN102413330A (en) | 2012-04-11 |
CN102413330B true CN102413330B (en) | 2014-05-14 |
Family
ID=45915139
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201110388181.6A Expired - Fee Related CN102413330B (en) | 2007-06-12 | 2007-06-12 | Texture-adaptive video coding/decoding system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102413330B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104661023B (en) * | 2015-02-04 | 2018-03-09 | 天津大学 | Image or method for video coding based on predistortion and training wave filter |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6339658B1 (en) * | 1999-03-09 | 2002-01-15 | Rockwell Science Center, Llc | Error resilient still image packetization method and packet structure |
CN1431828A (en) * | 2002-01-07 | 2003-07-23 | 三星电子株式会社 | Optimum scanning method for change coefficient in coding/decoding image and video |
CN1662066A (en) * | 2004-02-26 | 2005-08-31 | 中国科学院计算技术研究所 | A method for selecting an intra-frame prediction mode |
CN1757240A (en) * | 2003-03-03 | 2006-04-05 | 皇家飞利浦电子股份有限公司 | Video encoding |
CN1879418A (en) * | 2003-11-13 | 2006-12-13 | 高通股份有限公司 | Selective and/or scalable complexity control for video codecs |
-
2007
- 2007-06-12 CN CN201110388181.6A patent/CN102413330B/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6339658B1 (en) * | 1999-03-09 | 2002-01-15 | Rockwell Science Center, Llc | Error resilient still image packetization method and packet structure |
CN1431828A (en) * | 2002-01-07 | 2003-07-23 | 三星电子株式会社 | Optimum scanning method for change coefficient in coding/decoding image and video |
CN1757240A (en) * | 2003-03-03 | 2006-04-05 | 皇家飞利浦电子股份有限公司 | Video encoding |
CN1879418A (en) * | 2003-11-13 | 2006-12-13 | 高通股份有限公司 | Selective and/or scalable complexity control for video codecs |
CN1662066A (en) * | 2004-02-26 | 2005-08-31 | 中国科学院计算技术研究所 | A method for selecting an intra-frame prediction mode |
Also Published As
Publication number | Publication date |
---|---|
CN102413330A (en) | 2012-04-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101325707B (en) | System for encoding and decoding texture self-adaption video | |
JP6851429B2 (en) | Image decoding device, image decoding method, image coding device and image coding method | |
JP5413191B2 (en) | Moving picture encoding method and apparatus, and moving picture decoding apparatus | |
KR101947657B1 (en) | Method and apparatus for encoding intra prediction information | |
CN102598669B (en) | Encoding / decoding method and device for high-resolution moving images | |
KR101211665B1 (en) | Method and apparatus for intra prediction encoding and decoding of image | |
CN100586184C (en) | Infra-frame prediction method | |
EP3119091B1 (en) | Image filtering apparatus and method, and video encoding/decoding apparatus and method using the same | |
KR100667808B1 (en) | Method and apparatus for intra prediction encoding and decoding of images | |
CN101394560B (en) | Mixed production line apparatus used for video encoding | |
CN103327325B (en) | The quick self-adapted system of selection of intra prediction mode based on HEVC standard | |
KR20130098122A (en) | Device and method for encoding/decoding | |
KR20060119736A (en) | Video signal encoding method | |
KR100677118B1 (en) | Motion estimation method and device | |
CN101841713A (en) | Video coding method for reducing coding code rate and system | |
KR101433169B1 (en) | Mode prediction and quantization matrix and scanning application method and apparatus according to intra directionality | |
CN1194544C (en) | Video Coding Method Based on Time-Space Domain Correlation Motion Vector Prediction | |
CN102065293B (en) | Image compression method based on space domain predictive coding | |
CN101540900A (en) | Method for reducing blocking artifacts in video streams | |
CN105791868A (en) | Video coding method and equipment | |
CN102413330B (en) | Texture-adaptive video coding/decoding system | |
JP5800077B2 (en) | Video decoding method | |
Dai et al. | Adaptive block-based image coding with pre-/post-filtering | |
KR20100044333A (en) | Video encoding apparatus, and apparatus and method of 2 dimensional ordering transform for image signal, and recording medium therefor | |
KR101247024B1 (en) | Method of motion estimation and compensation using in-loop preprocessing filtering |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140514 |
|
CF01 | Termination of patent right due to non-payment of annual fee |