WO2017045344A1 - 一种视频帧的下采样方法和上采样方法以及传输处理方法 - Google Patents
一种视频帧的下采样方法和上采样方法以及传输处理方法 Download PDFInfo
- Publication number
- WO2017045344A1 WO2017045344A1 PCT/CN2016/073415 CN2016073415W WO2017045344A1 WO 2017045344 A1 WO2017045344 A1 WO 2017045344A1 CN 2016073415 W CN2016073415 W CN 2016073415W WO 2017045344 A1 WO2017045344 A1 WO 2017045344A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frequency component
- video frame
- low
- cosine transform
- matrix
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
Definitions
- the invention belongs to the field of image processing, and in particular relates to a downsampling method, an upsampling method and a transmission processing method for a video frame.
- DCT Discrete Cosine Transform
- Mitra of the University of California, Santa Barbara proposed a sub-band DCT approximation scheme
- Park of Seoul National University proposed a modular representation scheme for non-reverse DCT
- the State University of New York Chen proposed the DCT-Wiener scheme, in which the Wiener filter It is used to estimate high-frequency components
- Siu of the Hong Kong Polytechnic University proposes a self-learning k-NN MMSE estimation scheme to improve PSNR
- Mitra applies the correlation between DCT coefficients to improve visual quality.
- these schemes are based on the assumption that most of the energy is concentrated on the DCT low-frequency coefficients, but when this assumption is not met, the performance of these schemes will deteriorate.
- the premise of using compressed sensing theory is that the compressed sampled signal must be sparse, because Therefore, the image sampling processing using the compressed sensing must first perform the sparse transform on the image, but often after the image is sparsely transformed, the amount of data of the image will become larger, resulting in a large bandwidth occupied by the image transmission.
- the applicant found that the spatial components corresponding to the high-frequency DCT coefficients are sparse, that is, when the low-frequency coefficients of the video frames are removed and the high-frequency components are preserved, only the video frames are in the video frames. Fast changing areas and edge areas.
- the present invention provides a downsampling method and an upsampling method for video frames, which perform downsampling and corresponding video frame data by utilizing the sparsity of the spatial domain corresponding to the high frequency coefficients of the DCT. Upsampling achieves low data throughput and high reconstruction accuracy.
- the method for downsampling a video frame in the present invention includes the following steps:
- the present invention also discloses an upsampling method for a video frame, including the following steps:
- the present invention further provides a method for processing a video frame, including the following steps:
- the transmitting end down-samples the low-frequency component and the high-frequency component in the video frame according to the above-mentioned downsampling method to obtain the down sampling result T lf of the low-frequency component and the down-sampling result T hf of the high-frequency component;
- the receiving end receives the down sampling result T lf of the low frequency component and the down sampling result T hf of the high frequency component, and performs the restoration of the video frame by using the above up sampling method.
- the present invention most of the energy of the video frame is concentrated on the low frequency coefficient of the discrete cosine transform and the air space corresponding to the high frequency coefficient is sparse. These two properties are first used in the downsampling of the video frame.
- the components and the high-frequency components are separated, and the DCT low-frequency coefficients of the video frame are truncated and retained.
- the spatial components corresponding to the high-frequency DCT coefficients are compressed and sampled by the compressed sensing theory, and the low-frequency components and the high-frequency components are respectively downsampled and retained.
- the edge information corresponding to the high frequency coefficient and the information of the rapidly changing region correspondingly, in the process of upsampling the video frame, for the low frequency component and the high frequency component of the down sampling,
- the inverse discrete cosine transform and the l 1 norm reconstruction algorithm are used to reconstruct the low frequency component and the high frequency component respectively, and the reserved DCT low frequency coefficients are zeroed to the same data size as the original image, and reconstructed by the compressed sensing reconstruction algorithm.
- the spatial component corresponding to the high-frequency DCT coefficient is added, which is the high-similar reconstructed image of the original image, which is retained in the downsampling.
- upsampling the reconstructed image can be reduced to achieve high accuracy.
- the method of the present invention has a higher similarity to the original video frame at the same downsampling ratio as the conventional scheme.
- FIG. 1 is a flow chart of a video frame up and down sampling scheme of the present invention
- FIG. 2 is an exploded view of the high frequency component and the low frequency component of the video frame DCT;
- FIG. 2(a) is the original video frame;
- FIG. 2(b) is a spatial domain diagram of the original video frame retaining the discrete cosine transform low frequency coefficient and the high frequency coefficient zero padding
- Figure 2 (c) is a spatial map corresponding to the high frequency coefficients of the discrete cosine transform of the original video frame;
- FIG. 3 is a flow chart of a video frame downsampling method of the present invention.
- FIG. 4 is a flow chart of a method for sampling a video frame of the present invention.
- FIG. 6 is a simulation comparison diagram of SSIM data of the present invention and the existing hybrid interpolation method under different downsampling ratio conditions.
- the up-and-down sampling process is divided into the high-frequency part and the low-frequency part of the video frame, respectively, using the discrete cosine transform energy concentrated in the low-frequency coefficient and the high-frequency coefficient corresponding to the spatial domain is sparse Two properties.
- Figure 2(b) is more ambiguous than Figure 2(a), but basically the video frame can be determined.
- the content is sufficient to see that the airspace map corresponding to the low-frequency coefficients concentrates most of the energy in the original video frame.
- the grayscale value of the video frame in Figure 2(c) is mostly zero or close to zero (corresponding to the black part in the figure). ), only the sparse value exists in the edge region, it can be seen that the spatial domain corresponding to the high frequency component of the video frame is sparse.
- the video uses video frames as a processing unit, and the video frames are represented in the form of a matrix.
- the elements of the matrix are pixels, and the video frames are used as images.
- the length and width are also in pixels.
- the mentioned video frame and video frame data are the same concept, and both represent a matrix of a certain video frame.
- the video frame transmitting end uses the discrete cosine transform and the compressed sensing to downsample the low frequency component and the high frequency component of the video frame, respectively, including the following steps:
- C 1Dm and C 1Dn are both one-dimensional discrete cosine transform matrices.
- M D is the number of image columns after truncation Truncated matrix is a column
- R represents the set of real numbers
- the image width N D are truncated columns
- E is the identity matrix.
- ⁇ is the measurement matrix [DLDonoho, "Compressed sensing," IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006], and the measurement matrix is a definition in the theory of compressed sensing.
- Quantity is a kind of random matrix, and satisfies the finite equidistant property (RIP) as the measurement matrix.
- RIP finite equidistant property
- the sampled data is sent to the video receiving end.
- the receiving end separately samples the low frequency component and the high frequency component of the video frame by using discrete cosine transform and compressed sensing, and then performs The restored video frame data is integrated, as shown in FIG. 4, the method includes the following steps:
- U c and U r are the upsampled line zero complement matrix and the downsampled column complement zero matrix, respectively, and E is an identity matrix.
- ⁇ ' is the reconstructed ⁇
- mat() is the inverse of vec(), which converts the vector into a matrix. It represents the 0 norm of the vector ⁇ , which is the number of non-zero elements in the vector ⁇ , argmin( ⁇ ) is the minimum value, st is the constraint condition, ⁇ is the measurement matrix, and T hf is the high-frequency component downsampled. results, I hf 'is the reconstructed discrete cosine transform high-frequency component corresponding to the spatial data.
- the original video frame matrix I' can be restored by adding the reconstructed spatial domain high-frequency component to the low-frequency spatial domain component obtained by inverse discrete cosine transform:
- the simulation of the SSIM data using the peak signal-to-noise ratio PSNR and the structural similarity is performed at different sampling rates of the lower upsampling method of the present invention and the existing downsampling method.
- PSNR and SSIM are both quantitative evaluations of the similarity between the two images. The higher the value, the higher the similarity.
- the two pictures compared with the method of the present invention are: the original video frame and Upsampled restored image; the two images compared by the existing downsampling method are: the original video frame and its corresponding upsampled restored image.
- the peak signal-to-noise ratio (PSNR) of the method of the present invention is 1.239 dB higher than the discrete cosine transform-Wiener interpolation method at different downsampling rates. It can be seen that as the sampling rate increases, The peak signal-to-noise ratio (PSNR) of the inventive method has a higher rate of improvement than the discrete cosine transform-Wiener interpolation method, and the hybrid interpolation method has a small increase in the peak signal-to-noise ratio PSNR as the downsampling rate increases.
- the present invention In terms of the method, the performance of the low-frequency component down-sampling data fixed (fixed DCT cut-off rate) is higher than that of the high-frequency component down-sampled data (fixed compression sampling rate).
- the structural similarity measurement SSIM of the method of the present invention is higher than the discrete cosine transform-Wiener interpolation method by 0.0067 or more at different downsampling rates.
- the same down sampling rate is used to compare different images by using the method of the present invention and DCT Wiener hybrid interpolation and bicubic interpolation.
- the method has a downsampling rate of 0.5 and is processed for different images (12 images are listed in the table).
- the method of the present invention is at least 2.5 dB higher than the DCT Wiener hybrid interpolation method and the bicubic interpolation;
- the method of the present invention is at least 0.05 higher than the DCT Wiener hybrid interpolation method and the bicubic interpolation average, which is sufficient to see the method of the present invention. Stability and general applicability.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
本发明公开了一种视频帧的下采样方法和上采样方法以及传输处理方法，利用视频帧经过离散余弦变换后大部分能量都集中在低频系数中和高频系数所对应的空间域是稀疏的这两个性质，将视频帧经过离散余弦变换后，下采样截取低频系数，将高频系数所对应的空间域利用压缩感知理论进行下采样；在上采样端，将下采样得到的低频系数通过高频系数补零的方式拓展到与原视频帧同样的大小，然后利用逆离散余弦变换回到空间域；同时，利用压缩感知重构算法恢复高频系数所对应的空间域。最后低频系数补零的空间域成分加上高频压缩感知重构算法得到的空间域成分就得到与原视频帧高度一致的重构视频帧。本发明方法在与传统方案相同的下采样比下，上采样得到的视频帧对原视频帧具有更高的相似度。
Description
本发明属于图像处理领域，尤其涉及一种视频帧的下采样方法和上采样方法以及传输处理方法。
随着显示器件的发展，高清视频在消费市场上非常流行。然而，有限的带宽资源限制了高清图像的传输，比如，4k高清电视是人们非常喜爱的，但其每一个视频帧包含了大量的像素数据，导致了在传输时占用了大量的带宽资源。为了突破带宽的限制，学者提出了视频帧下采样和上采样概念。上、下采样属于一种伸缩编码，在下采样过程中，减少高清图像传输的数据量；而在上采样过程中，恢复出高清图像原有的数据量。
近年来，国内外学者提出了很多上、下采样方案，如插值，预测和估计等。然而，这些方法只优化了上采样过程。而在下采样过程中，原视频帧数据只是在空间域或者变换域中被截断。
一些学者论证了在小波域或者DCT(离散余弦变换)域的下采样和上采样可以获得很好的性能。DCT被广泛地应用在视频编码方案中，如AVC/H.264。因为视频帧或者图像中大部分的能量都集中在低频成分中，很多下采样方案是在DCT域中高频成分被简单地截断。例如：加州大学圣芭芭拉分校的Mitra提出了子带DCT近似方案；首尔大学的Park提出了无逆DCT的模块表示方案；纽约州立大学Chen提出了DCT-维纳方案，其中维纳滤波器被用来估计高频成分；香港理工大学的Siu提出了基于自学习的k-NN MMSE估计方案来提高PSNR；Mitra应用了DCT系数之间的相关性来提高视觉质量。然而，这些方案都基于大部分能量都集中于DCT低频系数上这个假设，但这个假设不满足时，这些方案的性能将变差。因此，当视频帧中存在较多快速变换的区域或者边缘区域，即能量在高频DCT系数部分较为集中，上述关于DCT的下采样方案的PSNR(峰值信噪比)和SSIM(结构相似度测量)将会下降。
另一方面，2006年T.Tao等人提出了压缩感知(Compressive Sensing；CS)理论[D.L.Donoho,“Compressed sensing,”IEEE Transactions on Information Theory,vol.52,no.4,pp.1289–1306,2006]引起了国内外学者的广泛关注。利用压缩感知理论能够在远低于奈奎斯特采样速率下进行采样，使压缩和采样同时进行，从而减少了大量的采样数据，且能保证对信号的准确重构。但利用压缩感知理论的前提是压缩采样的信号必须是稀疏的，因
此，利用压缩感知进行图像采样处理，必须先对图像进行稀疏变换，但往往图像经过稀疏变换以后，图像的数据量将变大，导致图像传输占用大的带宽。
申请人通过对视频帧的DCT域进行大量理论研究和仿真实验，发现高频DCT系数所对应的空域成分是稀疏的，即将视频帧的低频系数去除而保留高频成分时，在视频帧中只有快速变化区域和边缘区域。
发明内容
发明目的：为了克服现有技术中存在的不足，本发明提供一种视频帧的下采样方法和上采样方法，通过利用DCT高频系数所对应空域的稀疏性对视频帧数据进行下采样和相应的上采样，达到低数据量传输和高重构精度的效果。
技术方案：为实现上述目的，本发明中视频帧的下采样方法包括以下步骤：
(1)将视频帧Ioriginal进行离散余弦变换得到视频帧Ioriginal在余弦变换域下的系数IDCT；
(2)将所述余弦变换域下的系数IDCT中的高频系数利用截断矩阵进行截断，得到低频成分的下采样结果Tlf；
(3)将所述余弦变换域下的系数IDCT中的低频系数置零得到高频成分，并将高频成分通过逆离散余弦变换回到空间域Ihf；
(4)将所述空间域Ihf排列成向量，然后利用测量矩阵进行压缩采样，得到高频成分的下采样结果Thf。
相应的，本发明还公开了一种视频帧的上采样方法，包括以下步骤：
(1)将低频成分的下采样结果Tlf中的高频系数补零，然后使用逆离散余弦变换回到空间域，得到低频空间域成分Ilf'；
(2)将高频成分的下采样结果Thf利用压缩感知重构算法重构，并排列成一个矩阵，得到空间域高频成分矩阵I'hf；
(3)将所述低频空间域成分Ilf'和空间域高频成分I'hf进行叠加恢复出原视频帧I'。
进一步地，本发明还提供一种视频帧的传输处理方法，包括以下步骤：
(1)发送端对输入的视频帧按照上述下采样方法对视频帧中的低频成分和高频成分分别进行下采样得到低频成分的下采样结果Tlf和高频成分的下采样结果Thf；
(2)将所述低频成分的下采样结果Tlf和高频成分的下采样结果Thf进行发送；
(3)接收端接收到所述低频成分的下采样结果Tlf和高频成分的下采样结果Thf，并利用上述上采样方法进行视频帧的还原。
有益效果：本发明中利用视频帧的大部分能量集中在离散余弦变换的低频系数且高频系数所对应的空域是稀疏的这两个性质，在对视频帧进行下采样的过程中先对低频成分和高频成分进行分离，将视频帧的DCT低频系数截断并保留，高频DCT系数所对应的空域成分采用压缩感知理论进行压缩采样，对低频系成分和高频成分分别进行下采样，保留了视频帧的大部分信息，特别是高频系数所对应的边缘信息和快速变化区域的信息；相应的，在对视频帧进行上采样的过程中，针对下采样的低频成分和高频成分，利用逆离散余弦变换和l1范数重构算法分别重构出低频成分和高频成分，将保留的DCT低频系数补零到与原图像相同的数据大小，同时利用压缩感知重构算法重构出高频DCT系数所对应的空域成分，两者相加即为原图像的高相似度重构图像，在下采样保留了视频帧的大部分信息的基础上，上采样重构还原的图像便可达到很高的精度。本发明方法在与传统方案相同的下采样比下，上采样得到的视频帧对原视频帧具有更高的相似度。
图1是本发明的视频帧上、下采样方案的流程图；
图2是视频帧DCT高频成分和低频成分的分解图；图2(a)是原视频帧；图2(b)是原视频帧保留离散余弦变换低频系数并且高频系数补零的空域图；图2(c)是原视频帧离散余弦变换高频系数对应的空域图；
图3是本发明的视频帧下采样方法的流程图；
图4是本发明的视频帧上采样方法的流程图；
图5是本发明与现有的混合插值法在不同采样比条件下PSNR数据的仿真对比图；
图6是本发明与现有的混合插值法在不同下采样比条件下SSIM数据的仿真对比图。
下面结合实施例对本发明作更进一步的说明。
图1中可以看出，上下采样过程都分别分为对视频帧高频部分和低频部分进行的，利用的是离散余弦变换能量集中在低频系数和高频系数所对应的空间域是稀疏的这两个性质。
图2中可以看出，图2(b)中相对于图2(a)较为模糊，但基本上可以确定视频帧
的内容，足以见得，低频系数对应的空域图集中了原视频帧中的大部分能量，图2(c)中视频帧的灰度值大部分是零或者接近于零(对应于图中黑色部分)，只有边缘区域存在稀疏值，可以看出视频帧的高频成分所对应的空间域是稀疏的。
在图像处理过程中，视频以视频帧为一个处理单元，视频帧以矩阵的形式进行表示，矩阵的元素为像素点，视频帧作为图像，其长度和宽度也都以像素为单位，全文中所提到的视频帧和视频帧数据为同一概念，均表示指代某一视频帧的矩阵。
图3中视频帧发送端利用离散余弦变换和压缩感知分别对视频帧的低频成分和高频成分进行下采样，包括以下步骤：
(1)输入原始的视频帧Ioriginal数据；
(2)将视频帧Ioriginal进行离散余弦变换C2D得到原视频帧在余弦变换域下的系数IDCT∈RM×N：
其中，M是图像的长度，N是图像的宽度，flf表示低频系数，fhf表示高频系数
C1Dm、C1Dn均是一维离散余弦变换矩阵，C1Dm的长度为M，它的元素是
i,j是位置坐标，当j＝1时，
当j＝2,…,M时， 
C1Dn长度为N，它的元素是 
当j＝1时， 
当j＝2,…,N时，
表示Kronecker积；C1Dn'为C1Dn的转置。
(3)将视频帧在余弦变换域下的系数IDCT的高频系数进行行截断Dc和列截断Dr，得到低频系数对应的空间域视频帧图像，即低频成分的下采样结果Tlf：
其中， 
是行截断矩阵，MD是截断后的图像列数 
是列截断矩阵，R表示实数集，ND是列截断后的图像宽度，E是单
位矩阵。
(4)将视频帧在余弦变换域下的系数IDCT的低频系数置零，并将高频系数通过逆离散余弦变换回到空间域Ihf，该空间域中仅存在高频成分：
其中
是逆离散余弦变换矩阵。
(5)将空间域Ihf排列成一个向量，然后利用压缩感知理论进行压缩采样，得到高频成分下采样结果Thf：
Thf＝Φvec(Ihf)＝Φλ
其中Φ是测量矩阵(measurement matrix)[D.L.Donoho,“Compressed sensing,”IEEE Transactions on Information Theory,vol.52,no.4,pp.1289–1306,2006]，测量矩阵是压缩感知理论里的一个定义量，是一种随机矩阵，并且满足有限等距性质(RIP)即可为测量矩阵，所谓有限等距性，简单解释就是矩阵的每一列近似正交，vec()是将矩阵排列成一个向量的操作，一般按照矩阵的从左到右，从上到下排列，λ是中间变量，λ＝vec(Ihf)。
在对视频帧进行下采样之后，将采样后的数据发送给视频接收端，相应的，接收端利用离散余弦变换和压缩感知分别对视频帧的低频成分和高频成分进行上采样处理，然后进行整合得到复原后的视频帧数据，如图4所示，该方法包括以下步骤：
(1)将低频成分下采样结果Tlf的高频系数补零，然后使用逆离散余弦变换回到空间域Ilf'：
其中，Uc和Ur分别是上采样行补零矩阵和下采样列补零矩阵，E是单位矩阵。
(2)将高频成分下采样结果Thf利用压缩感知重构算法重构，并排列成一个矩阵，得到空间域高频成分矩阵I'hf：
I'hf＝mat(λ')
其中，λ'是重构的λ，mat()是vec()的逆操作，即将向量转化成矩阵，
表示求向量λ的0范数，即为向量λ中非零元素的个数，argmin(·)是求最小值，s.t.表示约束条件，Φ是测量矩阵，Thf是高频成分下采样后的结果，Ihf'是重构的离散余弦变换高频成分对应的空域数据。
(3)将重构的空间域高频成分加上由逆离散余弦变换得到的低频空间域成分，即可恢复出原视频帧矩阵I'：
I'＝I'lf+I'hf
为了验证本发明方法比现有技术的优势，在本发明的下上采样方法与现有的下上采样方法在不同的采样率下，利用峰值信噪比PSNR和结构相似度测量SSIM数据的仿真对比图进行比较两种方法的性能，PSNR和SSIM都是对两幅图相似性的一种量化评价，值越高，相似度越高，本发明方法对比的两幅图为：原视频帧和上采样还原的图像；现有下采样方法对比的两幅图为：原视频帧和其相应上采样还原的图像。
如图5所示，在不同的下采样率下，本发明方法的峰值信噪比PSNR比离散余弦变换-维纳插值法高1.239dB以上，可以看出，随着采样率的升高，本发明方法的峰值信噪比PSNR的提升率高于离散余弦变换-维纳插值法，而混合插值法随着下采样率的升高，峰值信噪比PSNR增长的很少；另外，对于本发明方法而言，低频成分下采样数据固定(固定DCT截断率)的性能比高频成分下采样数据固定(固定压缩采样率)的性能较高。
如图6所示，在不同的下采样率下，本发明方法的结构相似度测量SSIM比离散余弦变换-维纳插值法高0.0067以上。
为了验证本发明方法对不同图像均有较好的普遍适用性，采用相同的下采样率利用本发明方法和DCT维纳混合插值法和双三次插值对不同图像进行处理对比，表1中三种方法的下采样率为0.5，针对不同图像进行处理(表中罗列了12个图像)，就PSNR而言，本发明方法比DCT维纳混合插值法和双三次插值平均高至少2.5dB；就SSIM而言，本发明方法比DCT维纳混合插值法和双三次插值平均高至少0.05，足以见得本发明方法的
稳定性和普遍适用性。
表1本发明方法与DCT维纳混合插值法和双三次插值的性能对比表
以上详细描述了本发明的优选实施方式，但是，本发明并不限于上述实施方式中的具体细节，在本发明的技术构思范围内，可以对本发明的技术方案进行多种等同变换，这些等同变换均属于本发明的保护范围。
Claims (4)
- 一种视频帧的下采样方法，其特征在于，包括以下步骤：(1)将视频帧Ioriginal进行离散余弦变换得到视频帧Ioriginal在余弦变换域下的系数IDCT；(2)将所述余弦变换域下的系数IDCT中的高频系数利用截断矩阵进行截断，得到低频成分的下采样结果Tlf；(3)将所述余弦变换域下的系数IDCT中的低频系数置零得到高频成分，并将高频成分通过逆离散余弦变换回到空间域Ihf；(4)将所述空间域Ihf排列成向量，然后利用测量矩阵进行压缩采样，得到高频成分的下采样结果Thf。
- 根据权利要求1所述的视频帧的下采样方法，其特征在于，所述截断矩阵包括行截断矩阵Dr和列截断矩阵Dc，表达式如下：
式中，MD是截断后的图像列数，R表示实数集，ND是列截断后的图像宽度，E是单位矩阵，M是图像的长度，N是图像的宽度。 - 一种视频帧的上采样方法，其特征在于，包括以下步骤：(1)将低频成分的下采样结果Tlf中的高频系数补零，然后使用逆离散余弦变换回到空间域，得到低频空间域成分Ilf'；(2)将高频成分的下采样结果Thf利用压缩感知重构算法重构，并排列成一个矩阵，得到空间域高频成分矩阵I'hf；(3)将所述低频空间域成分Ilf'和空间域高频成分I'hf进行叠加恢复出原视频帧I'。
- 一种视频帧的传输处理方法，其特征在于，包括以下步骤：(1)发送端对输入的视频帧按照权利要求1所述的下采样方法对视频帧中的低频成分和高频成分分别进行下采样得到低频成分的下采样结果Tlf和高频成分的下采样结果Thf；(2)将所述低频成分的下采样结果Tlf和高频成分的下采样结果Thf进行发送；(3)接收端接收到所述低频成分的下采样结果Tlf和高频成分的下采样结果Thf，并利用权利要求2所述的上采样方法进行视频帧的还原。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510593694.9A CN105263027B (zh) | 2015-09-17 | 2015-09-17 | 一种视频帧的下采样方法和上采样方法以及传输处理方法 |
| CN201510593694.9 | 2015-09-17 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017045344A1 true WO2017045344A1 (zh) | 2017-03-23 |
Family
ID=55102508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2016/073415 WO2017045344A1 (zh) | 2015-09-17 | 2016-02-04 | 一种视频帧的下采样方法和上采样方法以及传输处理方法 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105263027B (zh) |
| WO (1) | WO2017045344A1 (zh) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112106304A (zh) * | 2018-05-11 | 2020-12-18 | 三菱电机株式会社 | 通信系统、通信方法及存储介质 |
| CN114025168A (zh) * | 2021-09-30 | 2022-02-08 | 浙江大华技术股份有限公司 | 视频图像的处理方法、处理设备、电子设备以及存储介质 |
| CN114155874A (zh) * | 2021-12-09 | 2022-03-08 | 云知声智能科技股份有限公司 | 一种特征提取方法、装置、电子设备和存储介质 |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105263027B (zh) * | 2015-09-17 | 2019-01-29 | 东南大学 | 一种视频帧的下采样方法和上采样方法以及传输处理方法 |
| CN109040116B (zh) * | 2018-09-06 | 2020-03-27 | 广州宏途教育网络科技有限公司 | 一种基于云端服务器的视频会议系统 |
| CN109413152B (zh) * | 2018-09-25 | 2021-02-26 | 上海瑾盛通信科技有限公司 | 图像处理方法、装置、存储介质及电子设备 |
| CN114372995A (zh) * | 2020-10-14 | 2022-04-19 | 晶晨半导体(上海)股份有限公司 | 显示驱动方法、模块及芯片、电子设备和存储介质 |
| WO2022109910A1 (zh) * | 2020-11-26 | 2022-06-02 | 深圳市大疆创新科技有限公司 | 视频编码方法、装置、编码器和存储介质 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102378011A (zh) * | 2010-08-12 | 2012-03-14 | 华为技术有限公司 | 一种图像上采样的方法、装置和系统 |
| CN103098464A (zh) * | 2010-08-04 | 2013-05-08 | Nxp股份有限公司 | 在频域中具有下采样的视频解码器 |
| WO2014173970A1 (en) * | 2013-04-26 | 2014-10-30 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Upsampling and signal enhancement |
| CN104469389A (zh) * | 2014-12-09 | 2015-03-25 | 上海交通大学 | 基于变换下采样的低码率视频编码方法及系统 |
| CN105263027A (zh) * | 2015-09-17 | 2016-01-20 | 东南大学 | 一种视频帧的下采样方法和上采样方法以及传输处理方法 |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20030053108A (ko) * | 2001-12-22 | 2003-06-28 | 한국과학기술원 | 블록 이산코사인변환 영역에서의 영상 크기 변환 방법 |
| CN102281443A (zh) * | 2011-08-22 | 2011-12-14 | 天津师范大学 | 基于优化的层式离散余弦变换的压缩感知图像处理方法 |
| CN104883579B (zh) * | 2015-06-08 | 2017-11-14 | 电子科技大学 | 一种基于空‑频域的联合视频图像的上采样方法 |
-
2015
- 2015-09-17 CN CN201510593694.9A patent/CN105263027B/zh active Active
-
2016
- 2016-02-04 WO PCT/CN2016/073415 patent/WO2017045344A1/zh active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103098464A (zh) * | 2010-08-04 | 2013-05-08 | Nxp股份有限公司 | 在频域中具有下采样的视频解码器 |
| CN102378011A (zh) * | 2010-08-12 | 2012-03-14 | 华为技术有限公司 | 一种图像上采样的方法、装置和系统 |
| WO2014173970A1 (en) * | 2013-04-26 | 2014-10-30 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Upsampling and signal enhancement |
| CN104469389A (zh) * | 2014-12-09 | 2015-03-25 | 上海交通大学 | 基于变换下采样的低码率视频编码方法及系统 |
| CN105263027A (zh) * | 2015-09-17 | 2016-01-20 | 东南大学 | 一种视频帧的下采样方法和上采样方法以及传输处理方法 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112106304A (zh) * | 2018-05-11 | 2020-12-18 | 三菱电机株式会社 | 通信系统、通信方法及存储介质 |
| CN112106304B (zh) * | 2018-05-11 | 2022-03-25 | 三菱电机株式会社 | 通信系统、通信方法及存储介质 |
| CN114025168A (zh) * | 2021-09-30 | 2022-02-08 | 浙江大华技术股份有限公司 | 视频图像的处理方法、处理设备、电子设备以及存储介质 |
| CN114155874A (zh) * | 2021-12-09 | 2022-03-08 | 云知声智能科技股份有限公司 | 一种特征提取方法、装置、电子设备和存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105263027A (zh) | 2016-01-20 |
| CN105263027B (zh) | 2019-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017045344A1 (zh) | 一种视频帧的下采样方法和上采样方法以及传输处理方法 | |
| CN106097251B (zh) | 非均匀稀疏采样视频超分辨率方法 | |
| Sandić-Stanković et al. | DIBR-synthesized image quality assessment based on morphological multi-scale approach | |
| US9830682B2 (en) | Upsampling and signal enhancement | |
| CN104883579B (zh) | 一种基于空‑频域的联合视频图像的上采样方法 | |
| CN113450280A (zh) | 一种由粗到细地融合时空信息的压缩视频质量增强方法 | |
| Bagawade Ramdas et al. | Wavelet transform techniques for image resolution enhancement: a study | |
| Harikrishna et al. | Satellite image resolution enhancement using dwt technique | |
| CN102842123B (zh) | 图像上采样中边缘清晰度提升的稀疏域残差补偿修正方法 | |
| Park et al. | Hybrid image upsampling method in the discrete cosine transform domain | |
| Pandey et al. | Block wise image compression & Reduced Blocks Artifacts Using Discrete Cosine Transform | |
| CN105392014A (zh) | 一种优化的小波变换图像压缩方法 | |
| Tai et al. | A fast algorithm for single image super resolution in both wavelet and spatial domain | |
| CN104683818A (zh) | 基于双正交不变集多小波的图像压缩方法 | |
| Yang et al. | A new image super-resolution method in the wavelet domain | |
| Kapinaiah et al. | Block DCT to wavelet transcoding in transform domain | |
| EP2645712A1 (en) | Image downsampling | |
| Nahar | A compression original image based on the DDWT technique and enhancement SNR | |
| Viswanath et al. | Wavelet transcoding in the block discrete cosine transform space | |
| Pokle et al. | Lossy Image Compression using Discrete Cosine Transform | |
| KR101650897B1 (ko) | 저해상도 컨텐츠를 위한 윈도우 사이즈 주밍 방법 및 그 장치 | |
| Zhizhong et al. | Translation invariance-based super resolution method for mixed resolution multiview video | |
| Nahar et al. | A Novel Improve and Compression for the Medical Image Technique Based on the Double Density Wavelet | |
| Xiong et al. | Super-resolution for low quality thumbnail images | |
| Dhara et al. | A novel hybrid technique in spatio-frequecncy domain for better quality image and video up-sampling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16845462 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16845462 Country of ref document: EP Kind code of ref document: A1 |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16845462 Country of ref document: EP Kind code of ref document: A1 |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14.12.2018) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16845462 Country of ref document: EP Kind code of ref document: A1 |