CN101674483A - Selection method of synthetic virtual reference image based on dynamic texture - Google Patents
Selection method of synthetic virtual reference image based on dynamic texture Download PDFInfo
- Publication number
- CN101674483A CN101674483A CN 200910272285 CN200910272285A CN101674483A CN 101674483 A CN101674483 A CN 101674483A CN 200910272285 CN200910272285 CN 200910272285 CN 200910272285 A CN200910272285 A CN 200910272285A CN 101674483 A CN101674483 A CN 101674483A
- Authority
- CN
- China
- Prior art keywords
- matrix
- xhat
- row
- frame
- ahat
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to the field of video coding technology, in particular to a selection method of a synthetic virtual reference image based on dynamic texture. The invention comprises the following steps: inputting n frames of reference images in a reference image list at a coding end, wherein, n is larger than or equal to 2; utilizing a dynamic texture model to generate one frame of virtual image; taking the virtual image as one frame of reference image for being processed by a coder; carrying out matching and selecting by a coder from a real reference image and the virtual image; and calculating and obtaining an optimum prediction model and a reference frame. The invention synthesizes one frame of virtue image by calculating the dynamic texture model, guides the frame virtue image tothe virtue reference image selection algorithm of interframe prediction, thereby improving the efficiency of the interframe prediction.
Description
Technical field
The present invention relates to technical field of video coding, relate in particular to a kind of based on the synthetic virtual reference image system of selection of dynamic texture.
Background technology
Inter prediction technology in the existing video coding technique is with the reconstructed image of the front multiframe reference picture as present frame, it is carried out time domain prediction reduce the residual error data amount.Because the delay of life period between reference picture and the present image is therefore lower for sequence prediction efficient such as nonlinear motion, background illumination variations.Though there is the scholar to propose based on the synthetic virtual reference image technology of dynamic texture (referring to document: A.Stojanovic, M.Wien, J.R.Ohm.Dynamic texturesynthesis for is inter coding:Proceedings of the 15th IEEE International Conference onImage Processing H.264/AVC, San Diego, California, USA, 2008:1608-1611), it is in the efficient of to a certain degree having improved inter prediction, but because there is defective in this model solution method, though this method gains to some extent for partial sequence, loss is to a certain degree arranged then on the whole.
Summary of the invention
The purpose of this invention is to provide a kind of virtual reference image system of selection of synthesizing,, be introduced in the virtual reference image system of selection of inter prediction, promote the efficient of inter prediction by finding the solution the synthetic frame virtual image of dynamic texture model based on dynamic texture.
For achieving the above object, the present invention adopts following technical scheme:
A kind of based on the synthetic virtual reference image system of selection of dynamic texture, may further comprise the steps:
N frame reference picture in the input coding end reference picture list, described n 〉=2;
Utilize the dynamic texture model to generate a frame virtual image;
Above-mentioned virtual image is supplied coder processes as a frame reference picture;
Encoder carries out match search from true reference picture and above-mentioned virtual image;
Calculate and obtain optimum prediction model and reference frame.
The described step of utilizing the dynamic texture model to generate a frame virtual image further comprises following substep:
(1) note Y (y
1, y
2... y
n) be n frame reference brightness and the chromatic value matrix in the coding side reference picture list, deposited the brightness value and the chromatic value of all pixels in the frame in every row successively;
(2) svd is a singular value decomposition method, Y is carried out singular value decomposition obtain U, and S, V be totally 3 matrixes:
[U，S，V]＝svd(Y，0)
(3) take out all elements that 1～n is listed as in the matrix U, form Matrix C hat:
Chat＝U(:，1:n)
(4) elements 1～n in the matrix S is capable, 1～n row form matrix S (1:n, 1:n), with all elements of 1～n row in the matrix V form matrix V (:, 1:n) and to its ask transposition obtain matrix (V (:, 1:n)) ', then S (1:n, 1:n) and (V (:, 1:n)) ' multiplying each other obtains matrix Xhat:
Xhat＝S(1:n，1:n)*(V(:，1:n))′
(5) first column element that takes out among the matrix Xhat forms matrix x0:
x0＝Xhat(:，1)
The all elements formation matrix Xhat that (6) 2～n is listed as among the taking-up matrix Xhat (:, 2:n), the all elements formation matrix of 1～(n-1) row among the taking-up matrix Xhat (Xhat (:, 1:(n-1)), and it is asked generalized inverse matrix pinv (Xhat (:, 1:(n-1))), then Xhat (:, 2:n) and pinv (Xhat (:, 1:(n-1))) multiplying each other obtains matrix A hat:
Ahat＝Xhat(:，2:n)*pinv(Xhat(:，1:(n-1)))
The all elements formation matrix Xhat that (7) 2～n is listed as among the taking-up matrix Xhat (:, 2:n), the all elements formation matrix of 1～(n-1) row among the taking-up matrix Xhat (Xhat (:, 1:(n-1)), the Ahat that step (6) is obtained and (Xhat (:, 1:(n-1)) multiply each other, then Xhat (:, 2:n) and Ahat*Xhat (:, 1:(n-1)) subtract each other and obtain Vhat:
Vhat＝Xhat(:，2:n)-Ahat*Xhat(:，1:(n-1))
(8) Vhat is carried out singular value decomposition and obtain U
v, S
v, V
vTotally 3 matrixes:
(U
v，S
v，V
v)＝svd(Vhat，0)
(9) take out matrix U
vIn all elements of 1 (n-2) row, i.e. U
v(1:(n-2)), take out matrix S
vIn 1～(n-2) row, the element of 1～(n-2) row, i.e. S
v(1:(n-2), 1:(n-2)), to the n-1 evolution, i.e. sqrt (n-1) is U
v(1:(n-2)) and S
v(1:(n-2), 1:(n-2)) multiply each other and obtain Bhat divided by sqrt (n-1) then:
Bhat＝U
v(1:(n-2))*S
v(1:(n-2)，1:(n-2))/sqrt(n-1)
(10) obtain the columns of Bhat, as the value of len;
(11) first column element of x0 as matrix x:
X(:，1)＝x0
(12) establish variable t, t carried out the circulation assignment n time: with the element of the t of matrix X row form matrix X (:, t), with X (:, t) and matrix A hat multiply each other, obtain Ahat*X (:, t), the pseudo random sequence randn (len, 1) of capable 1 row of structure len, Bhat and randn (len, 1) multiplied each other obtain Bhat*randn (len, 1), obtain then with Ahat*X (:, t) and Bhat*randn (len, 1) addition, the value x that the result is listed as matrix X t+1 (:, t+1); Again the element of matrix X t+1 row form matrix X (:, t+1), with X (:, t+1) and Matrix C hat multiply each other, the result as matrix I t+1 row I (:, t+1):
for?t＝1:n
X(:，t+1)＝Ahat*X(:，t)+Bhat*randn(len，1)
I(:，t+1)＝Chat*X(:，t+1)
End
(13) with the value of the n+1 among the matrix I that obtains row brightness and chromatic value matrix as virtual image:
I＝I(:，t+1)。
Described encoder carries out in the step of match search from true reference picture and above-mentioned virtual image, and the method for match search is the match search method that the encoder acquiescence adopts.
The present invention has the following advantages and good effect:
1) by finding the solution the synthetic frame virtual image of dynamic texture model, is introduced in the virtual reference image selection algorithm of inter prediction, promotes the efficient of inter prediction.
Description of drawings
Fig. 1 is the flow chart based on the synthetic virtual reference image system of selection of dynamic texture provided by the invention.
Wherein,
N frame reference picture in the S1-input coding end reference picture list, S2-utilizes the dynamic texture model to generate a frame virtual image, S3-supplies coder processes as a frame reference picture as a frame reference picture with above-mentioned virtual image, the S4-encoder carries out match search from true reference picture and above-mentioned virtual image, S5-calculates and obtains optimum prediction model and reference frame.
Embodiment
The invention will be further described in conjunction with the accompanying drawings with specific embodiment below:
Following technical scheme is specifically adopted in virtual reference image system of selection of synthesizing based on dynamic texture provided by the invention, referring to Fig. 1, may further comprise the steps:
S1: the n frame reference picture in the input coding end reference picture list, described n 〉=2;
S2: utilize the dynamic texture model to generate a frame virtual image;
S3: above-mentioned virtual image is supplied coder processes as a frame reference picture;
S4: encoder carries out match search from true reference picture and above-mentioned virtual image;
S5: calculate and obtain optimum prediction model and reference frame.
Further describe the detailed process of implementing under the environment concrete below, and the technique effect of obtaining:
Employing reference software JM12.4 H.264 is as encoder, type of coding is IPPPPP, the reference frame number is 5, open the RDO option, " container " sequence of choosing QCIF resolution sizes (176 * 144) is as cycle tests, with the 11st two field picture in the coding container sequence is example, and concrete implementation step is as follows:
1,5 frame reference pictures in the input coding end reference picture list;
2, utilize the dynamic texture model to generate a frame virtual image, further comprise following substep:
(1) note Y (y1, y2, y5) be the brightness and the chromatic value matrix of the 5 frame reference pictures (being the reconstructed image of the 6th～10 frame in the contaier sequence) in the coding side reference picture list, deposited the brightness value and the chromatic value of all pixels in the frame in every row successively.
(2) svd is a singular value decomposition method, Y is carried out singular value decomposition obtain U, and S, V be totally 3 matrixes:
[U，S，V]＝svd(Y，0)
(3) take out 1～5 all elements that is listed as in the matrix U, form Matrix C hat:
Chat＝U(:，1:5)
(4) element with the row of 1～5 in the matrix S, 1～5 row forms matrix S (1:5,1:5), with all elements of 1～5 row in the matrix V form matrix V (:, 1:5) and to its ask transposition obtain matrix (V (:, 1:5)) ', then S (1:5,1:5) and (V (:, 1:5)) ' multiplying each other obtains matrix Xhat:
Xhat＝S(1:5，1:5)*(V(:，1:5))′
(5) first column element that takes out among the matrix Xhat forms matrix x0:
x0＝Xhat(:，1)
(6) all elements that take out 2～5 row among the matrix Xhat form matrix Xhat (:, 2:5), the all elements formation matrix of 1～4 row among the taking-up matrix Xhat (Xhat (:, 1:4), and it is asked generalized inverse matrix pinv (Xhat (:, 1:4)), then Xhat (:, 2:5) and pinv (Xhat (:, 1:4)) multiplying each other obtains matrix A hat:
Ahat＝Xhat(:，2:5)*pinv(Xhat(:，1:4))
(7) all elements that take out 2～5 row among the matrix Xhat form matrix Xhat (:, 2:5), the all elements formation matrix of 1～4 row among the taking-up matrix Xhat (Xhat (:, 1:4), the Ahat that step (6) is obtained with (Xhat (:, 1:4) multiply each other, then Xhat (:, 2:5) and Ahat*Xhat (:, 1:4) subtract each other and obtain Vhat:
Vhat＝Xhat(:，2:5)-Ahat*Xhat(:，1:4)
(8) Vhat is carried out singular value decomposition and obtain U
v, S
v, V
vTotally 3 matrixes:
(U
v，S
v，V
v)＝svd(Vhat，0)
(9) take out matrix U
vIn all elements of 13 row, i.e. U
v(1:3), take out matrix S
vIn 1～3 the row, 1～3 row be S
v(1:3,1:3), to 4 evolutions, i.e. sqrt (4) is U
v(1:3) and S
v(1:3,1:3) multiply each other and obtain Bhat divided by sqrt (4) then:
Bhat＝U
v(1:3)*S
v(1:3，1:3)/sqrt(4)
(10) obtain the columns of Bhat, as the value of len, this moment len=3
(11) first column element of x0 as matrix x:
X(:，1)＝x0
(12) establish variable t, t carried out the circulation assignment 5 times: with the element of the t of matrix X row form matrix X (:, t), with X (:, t) and matrix A hat multiply each other, obtain Ahat*X (:, t).Construct the pseudo random sequence randn (3,1) of 3 row 1 row, Bhat and randn (3,1) are multiplied each other obtains Bhat*randn (3,1), obtain then with Ahat*X (:, t) and Bhat*randn (3,1) addition, the value x that the result is listed as matrix X t+1 (:, t+1); Again the element of matrix X t+1 row form matrix X (:, t+1), with X (:, t+1) and Matrix C hat multiply each other, the result as matrix I t+1 row I (:, t+1):
for?t＝1:5
X(:，t+1)＝Ahat*X(:，t)+Bhat*randn(3，1)
I(:，t+1)＝Chat*X(:，t+1)
End
(13) with the value of the 6th among the matrix I that obtains row brightness and chromatic value matrix as virtual image:
I＝I(:，t+1)
3, above-mentioned virtual image is supplied coder processes as a frame reference picture;
4, encoder carries out match search from true reference picture and above-mentioned virtual image;
The method of above-mentioned match search is the match search method that the encoder acquiescence adopts.
5, calculate acquisition optimum prediction model and reference frame.
The obtained technique effect of the present invention is as follows:
Present embodiment is tested the container sequence of QCIF form.The coding frame number is 300 frames, and order is IPPPPPP.With method proposed by the invention and H.264 canonical algorithm compare, the peak value signal to noise ratio PSNR of coded image gain and code check saving result are as shown in table 1, therefrom the present invention has better compression efficiency as can be seen.
Table 1:container sequential test result:
| Sequence | The PSNR gain | Code check is saved (%) |
| ??container | ??0.08 | ??1.37 |
Claims (3)
1. a virtual reference image system of selection of synthesizing based on dynamic texture is characterized in that, may further comprise the steps:
N frame reference picture in the input coding end reference picture list, described n 〉=2;
Utilize the dynamic texture model to generate a frame virtual image;
Above-mentioned virtual image is supplied coder processes as a frame reference picture;
Encoder carries out match search from true reference picture and above-mentioned virtual image;
Calculate and obtain optimum prediction model and reference frame.
2. according to claim 1 based on the synthetic virtual reference image system of selection of dynamic texture, it is characterized in that:
The described step of utilizing the dynamic texture model to generate a frame virtual image further comprises following substep:
(1) note Y (y
1, y
2... y
n) be n frame reference brightness and the chromatic value matrix in the coding side reference picture list, deposited the brightness value and the chromatic value of all pixels in the frame in every row successively;
(2) svd is a singular value decomposition method, Y is carried out singular value decomposition obtain U, and S, V be totally 3 matrixes:
[U，S，V]＝svd(Y，0)
(3) take out all elements that 1～n is listed as in the matrix U, form Matrix C hat:
Chat＝U(：，1：n)
(4) elements 1～n in the matrix S is capable, 1～n row form matrix S (1:n, 1:n), with all elements of 1～n row in the matrix V form matrix V (:, 1:n) and to its ask transposition obtain matrix (V (:, 1:n)) ', then S (1:n, 1:n) and (V (:, 1:n)) ' multiplying each other obtains matrix Xhat:
Xhat＝S(1：n，1：n)*(V(：，1：n))′
(5) first column element that takes out among the matrix Xhat forms matrix x0:
x0＝Xhat(：，1)
The all elements formation matrix Xhat that (6) 2～n is listed as among the taking-up matrix Xhat (:, 2:n), the all elements formation matrix of 1～(n-1) row among the taking-up matrix Xhat (Xhat (:, 1:(n-1)), and it is asked generalized inverse matrix pinv (Xhat (:, 1:(n-1))), then Xhat (:, 2:n) and pinv (Xhat (:, 1:(n-1))) multiplying each other obtains matrix A hat:
Ahat＝Xhat(：，2：n)*pinv(Xhat(：，1：(n-1)))
The all elements formation matrix Xhat that (7) 2～n is listed as among the taking-up matrix Xhat (:, 2:n), the all elements formation matrix of 1～(n-1) row among the taking-up matrix Xhat (Xhat (:, 1:(n-1)), the Ahat that step (6) is obtained and (Xhat (:, 1:(n-1)) multiply each other, then Xhat (:, 2:n) and Ahat*Xhat (:, 1:(n-1)) subtract each other and obtain Vhat:
Vhat＝Xhat(：，2：n)-Ahat*Xhat(：，1：(n-1))
(8) Vhat is carried out singular value decomposition and obtain U
v, S
v, V
vTotally 3 matrixes:
(U
v，S
v，V
v)＝svd(Vhat，0)
(9) take out matrix U
vIn all elements of 1 (n-2) row, i.e. U
v(1:(n-2)), take out matrix S
vIn 1～(n-2) row, the element of 1～(n-2) row, i.e. S
v(1:(n-2), 1:(n-2)), to the n-1 evolution, i.e. sqrt (n-1) is U
v(1:(n-2)) and S
v(1:(n-2), 1:(n-2)) multiply each other and obtain Bhat divided by sqrt (n-1) then:
Bhat＝U
v(1：(n-2))*S
v(1：(n-2)，1：(n-2))/sqrt(n-1)
(10) obtain the columns of Bhat, as the value of len;
(11) first column element of x0 as matrix x:
X(：，1)＝x0
(12) establish variable t, t carried out the circulation assignment n time: with the element of the t of matrix X row form matrix X (:, t), with X (:, t) and matrix A hat multiply each other, obtain Ahat*X (:, t), the pseudo random sequence randn (len, 1) of capable 1 row of structure len, Bhat and randn (len, 1) multiplied each other obtain Bhat*randn (len, 1), obtain then with Ahat*X (:, t) and Bhat*randn (len, 1) addition, the value x that the result is listed as matrix X t+1 (:, t+1); Again the element of matrix X t+1 row form matrix X (:, t+1), with X (:, t+1) and Matrix C hat multiply each other, the result as matrix I t+1 row I (:, t+1):
for?t＝1：n
X(：，t+1)＝Ahat*X(：，t)+Bhat*randn(len，1)
I(：，t+1)＝Chat*X(：，t+1)
End
(13) with the value of the n+1 among the matrix I that obtains row brightness and chromatic value matrix as virtual image:
I＝I(：，t+1)。
3. according to claim 1 and 2 based on the synthetic virtual reference image system of selection of dynamic texture, it is characterized in that:
Described encoder carries out in the step of match search from true reference picture and above-mentioned virtual image, and the method for match search is the match search method that the encoder acquiescence adopts.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910272285 CN101674483B (en) | 2009-09-28 | 2009-09-28 | Selection method of synthetic virtual reference image based on dynamic texture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910272285 CN101674483B (en) | 2009-09-28 | 2009-09-28 | Selection method of synthetic virtual reference image based on dynamic texture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101674483A true CN101674483A (en) | 2010-03-17 |
| CN101674483B CN101674483B (en) | 2011-05-18 |
Family
ID=42021433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910272285 Expired - Fee Related CN101674483B (en) | 2009-09-28 | 2009-09-28 | Selection method of synthetic virtual reference image based on dynamic texture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101674483B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970542A (en) * | 2012-11-30 | 2013-03-13 | 上海晨思电子科技有限公司 | Video data conversion method and device and intelligent television |
| CN106791829A (en) * | 2016-11-18 | 2017-05-31 | 华为技术有限公司 | The method for building up and equipment of virtual reference frame |
-
2009
- 2009-09-28 CN CN 200910272285 patent/CN101674483B/en not_active Expired - Fee Related
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970542A (en) * | 2012-11-30 | 2013-03-13 | 上海晨思电子科技有限公司 | Video data conversion method and device and intelligent television |
| CN102970542B (en) * | 2012-11-30 | 2015-06-10 | 上海晨思电子科技有限公司 | Video data conversion method and device and intelligent television |
| CN106791829A (en) * | 2016-11-18 | 2017-05-31 | 华为技术有限公司 | The method for building up and equipment of virtual reference frame |
| WO2018090600A1 (en) * | 2016-11-18 | 2018-05-24 | 华为技术有限公司 | Method for establishing virtual reference frame, and device |
| CN106791829B (en) * | 2016-11-18 | 2020-01-21 | 华为技术有限公司 | Method and equipment for establishing virtual reference frame |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101674483B (en) | 2011-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Maximum frame rate video acquisition using adaptive compressed sensing | |
| Chen et al. | Dictionary learning-based distributed compressive video sensing | |
| US9641729B2 (en) | System and method for encoder-integrated media denoising | |
| CN102427527A (en) | Method for reconstructing non key frame on basis of distributed video compression sensing system | |
| CN1719735A (en) | Method or device for coding a sequence of source pictures | |
| CN105306944A (en) | Chrominance component prediction method in hybrid video coding standard | |
| CN106713929A (en) | Video interframe prediction enhancement method based on deep neural network | |
| CN101335892A (en) | Hybrid distributed video encoding method based on intra-frame intra-frame mode decision | |
| CN107637078A (en) | Real transform and transformation by reciprocal direction in video decoding system and method | |
| CN105408935A (en) | Upsampling and signal enhancement | |
| US20130101029A1 (en) | Multimedia data encoding | |
| CN101674483B (en) | Selection method of synthetic virtual reference image based on dynamic texture | |
| CN1325220A (en) | Motion-vector coding method | |
| CN1703094A (en) | Image interpolation apparatus and methods that apply quarter pel interpolation to selected half pel interpolation results | |
| CN104683818B (en) | Method for compressing image based on biorthogonal invariant set m ultiwavelet | |
| CN110100437A (en) | For damaging the hybrid domain cooperation loop filter of Video coding | |
| CN1139354A (en) | Method and apparatus for limiting band of moving-picture signal | |
| US20140119435A1 (en) | System and method for video and graphic compression using mulitple different compression techniques and compression error feedback | |
| US8705609B2 (en) | Moving image coding device and method | |
| JP2021047404A5 (en) | ||
| CN1741612A (en) | Quantization method during video image coding | |
| CN107707917A (en) | A kind of video adaptive sample rate establishing method based on comentropy | |
| Takeuchi et al. | A gamut-extension method considering color information restoration using convolutional neural networks | |
| CN105227959A (en) | For odd encoder point shuffling flowing water method and the device thereof of Video coding | |
| CN107689797B (en) | Compressed sensing signal reconstruction method based on block Hadamard measurement matrix with arbitrary dimension |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110518 Termination date: 20160928 |