CN108600745B - Video quality evaluation method based on time-space domain slice multi-map configuration - Google Patents
Video quality evaluation method based on time-space domain slice multi-map configuration Download PDFInfo
- Publication number
- CN108600745B CN108600745B CN201810882119.4A CN201810882119A CN108600745B CN 108600745 B CN108600745 B CN 108600745B CN 201810882119 A CN201810882119 A CN 201810882119A CN 108600745 B CN108600745 B CN 108600745B
- Authority
- CN
- China
- Prior art keywords
- time
- video
- map
- slice
- space domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention discloses a video quality evaluation method based on time-space domain slice multi-map configuration, and belongs to the technical field of image video analysis. The method uses a time-space domain slicing thought to convert an original video sequence and a distorted video sequence into a time-space domain slicing representation form, extracts a distortion friendly edge image and a frame difference image on a space domain slice, then extracts a gradient amplitude value and gradient direction composition change image and a Laplace correction static image on all slice sequences, and forms an image map with the original image so as to complete image map configuration. And then, introducing the time-space domain stability of the video to be evaluated into the slice field to perform map generation calculation, introducing a 2D image quality evaluation method, and calculating a difference value of a generated map reference-distortion pair. And finally, automatically determining the weight of the contribution degree of each atlas to the video distortion in a learning mode by applying a neural network method. Compared with the prior art, the method has the characteristics of high subjective consistency, strong compatibility, high algorithm stability and the like.
Description
Technical Field
The invention relates to a video quality evaluation method, in particular to a video quality evaluation method based on time-space domain slice multi-map configuration, and belongs to the technical field of image video analysis.
Background
With the development of scientific technology, the cost of image and video generation and transmission becomes lower and lower, which makes the image and video become more and more common and indispensable in daily life as an excellent information transmission medium. However, images and video have the potential to introduce distortion at various stages of production and transmission. The distortion will affect the experience of people watching the multimedia data and seriously affect the physical and mental health of people.
In recent years, people have made great progress in the field of image quality evaluation research, but progress in the video field is relatively slow. How to suppress the propagation of low-quality videos and ensure the visual experience of people still is a problem to be solved urgently. Therefore, the ability of automatically evaluating the quality of the video is provided for the media for video generation and transmission, so that the quality of the video at the output end of the media is improved, and the method has important significance for solving the problem.
Disclosure of Invention
The invention aims to solve the problems that the existing video quality evaluation method is low in prediction accuracy and poor in information representation capability, and a 2D image quality evaluation method is not successfully applied to the field of video quality evaluation, and provides a video quality evaluation method based on time-space domain slice multi-map configuration.
The method of the invention refers to the time-space domain slicing idea proposed by Ngo et al. The time-space domain slicing idea adopts a time-space domain combined mode to represent original video information again, and effectively solves the contradiction between video time-space information extraction and high computation complexity. The idea is to regard the video as a cuboid in a three-dimensional coordinate system, and three coordinate axes respectively represent the height (H), the width (W) and the time (T) of the video. By slicing the video along different axes, different information representations of the video are obtained. This idea can be formulated as:
ISTS(i,d)＝{Vd|d∈[T,W,H],i∈[1,N]} (1)
where V is the input video sequence, the superscript d represents the different dimensions of the video, the ranges are height (H), width (W) and time (T) mentioned above, I represents the index value of the slice sequence, and I represents the index value of the slice sequenceSTSAnd (i, d) is the generated time-space domain slice sequence.
The method is realized by the following technical scheme. A video quality evaluation method based on time-space domain slice multi-map configuration comprises the following steps:
step one, converting the original and distorted video sequences into a time-space domain slice representation form as a basic unit of subsequent processing.
And secondly, extracting distortion-friendly edge images and frame difference images on the spatial domain slices, extracting variation images and static images on all slice sequences, and forming an atlas with the original image, thereby completing atlas configuration.
And step three, introducing the time-space domain stability of the video to be evaluated into the slicing field, and performing map generation calculation.
And step four, introducing a 2D image quality evaluation method, and calculating difference values of generated map reference-distortion pairs.
And step five, automatically determining the weight of the contribution degree of each map to the video distortion in a learning mode by applying a neural network method.
Advantageous effects
Compared with the prior art, the method has the characteristics of high subjective consistency, strong compatibility, high algorithm stability and the like. The method can convert the common 2D image quality evaluation method into a high-performance video quality evaluation method, can be used in cooperation with a video processing related application system, can be embedded into an actual application system (such as a video showing system, a network transmission system and the like) and monitors the quality of the video in real time; the method can be used for evaluating the advantages and disadvantages of various video processing algorithms and tools (such as compression coding of stereo images, video acquisition tools and the like); the method can be used for quality audit of video works, and prevents the inferior video products from harming physical and mental health of audiences.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The method of the present invention is further described in detail below with reference to the drawings and examples.
A video quality evaluation method based on time-space domain slice multi-map configuration is shown in figure 1 and comprises the following steps:
step one, converting the original and distorted video sequences into a time-space domain slice representation form as a basic unit of subsequent processing.
And secondly, extracting distortion-friendly edge images and frame difference images on the spatial domain slices, extracting variation images and static images on all slice sequences, and forming an atlas with the original image, thereby completing atlas configuration.
The spatial domain slice can best reflect the structure of the image, so the edge map and the frame difference map of the spatial domain slice are extracted for optimizing the time-space domain information. The preferred method is as follows:
IDIFF(i,T)＝{ISTS(i,T)-ISTS(i-1,T)|i∈[2,N]} (3)
wherein, IEDGE(I, T) is the edge map generated, IDIFFAnd (i, T) is the generated frame difference map. I isSTS(i, T) represents a time-space domain slice, i represents an index value of a slice sequence, T is a time dimension of the slice, N is a maximum index value of the slice, fh、fvA corresponding distortion-friendly edge filtering kernel.
In this embodiment, the specific values of the edge filtering kernel are:
SI13＝[-0.0052625,-0.0173466,-0.0427401,-0.0768961,
-0.957739,-0.0696751,0,0.6696751,0.0957739,
0.0768961,0.0427401,0.0173466,0.0052625].(4)
the vector is respectively copied horizontally and vertically to obtain the two edge filtering kernels.
In order to optimize the basic information representation capability of the spatio-temporal-spatial domain slices, a gradient amplitude and direction composition change graph and a Laplace corrected Gaussian stationary graph are respectively extracted to complete optimization. The preferred extraction method is as follows:
ILAP(i,d)＝ISTS(i,d)-IUP(i,d) (8)
wherein, IGM(I, d) is the gradient magnitude map generated, IGO(I, d) is the gradient pattern generated, IGAU(I, d) is the generated Gaussian filter map, ILAP(i, d) are the generated Laplace maps. I denotes the index value of the slice sequence, d denotes the different dimensions of the video, ISTSAnd (i, d) is a time-space domain slice sequence. Gx、GyIs a Gaussian gradient filter kernel in the horizontal and vertical directions, fgIs a Gaussian blur Filter kernel, IUP(i, d) are Gaussian filtered upsampled maps.
And step three, introducing the time-space domain stability of the video to be evaluated into the slicing field, and performing map generation calculation.
Preferably, the generation calculation of the atlas is performed on only half of the slices, and the calculation amount is further reduced.
And step four, calculating and generating a difference value of the map reference-distortion pair in an average aggregation mode by adopting a 2D image quality evaluation method.
Difference value PmThe specific calculation method of (i', d) is as follows:
wherein, I represents a certain image in the map sequence, the superscripts ref and dis represent a reference video sequence map and a distorted video sequence map respectively, m represents the map class, I' represents the index value of the map sequence, and d represents different dimensionalities of the video. The IQA represents the adopted 2D full-reference image quality evaluation method, the calculated result is that each group of image pairs in the atlas sequence generate a difference value containing distortion information, for each type of atlas, the atlas difference fraction of the atlas is obtained in an average aggregation mode, and then the vector S consisting of the difference fractions of the atlases of all types can be obtained.
And step five, automatically determining the weight of the contribution degree of each map to the video distortion in a learning mode by applying a neural network method.
The learning characterization formula is:
Q＝θRS (10)
wherein, θ is the weight parameter vector to be learned, S is the map difference score vector, Q is the final video quality score representation, and R represents the vector transposition.
Examples
The method of the invention is implemented on three video quality evaluation databases, including LIVE, IVP and CSIQ. The basic information of these databases is shown in table 1. Two full-reference video quality evaluation methods with excellent performance are selected for comparison with the method.
TABLE 1 database basic information
In addition, because the method is a framework for converting the 2D image quality evaluation method into the video evaluation method, three full-reference image quality evaluation methods (PSNR, SSIM and VIF) are selected to be combined with the method to complete the experiment, and meanwhile, the results of the three full-reference methods are also added with comparison for testing the improvement of the performance of the method by the framework of the method. 20% of data are selected for testing in each experiment, SRCC, KRCC, PLCC and RMSE are taken as indexes, the median value is obtained after 1000 times of repetition, and the experimental result is shown in Table 2.
TABLE 2 comparison of algorithmic Performance across three databases
Table 3 shows the performance of each algorithm on each distortion type, and it can be seen from table 2 that the performance of the 2D method is significantly improved in the test of three databases by using the method of the present invention, and meanwhile, the improved PSNR is better than STRRED and ViS3 in the results of 3 databases, which indicates that the 2D method can achieve very competitive performance through the improvement of the framework.
TABLE 3 comparison of Algorithm Performance on each distortion category
Claims (5)
1. A video quality evaluation method based on time-space domain slice multi-map configuration is characterized by comprising the following steps:
converting an original video sequence and a distorted video sequence into a time-space domain slice representation form as a basic unit of subsequent processing;
secondly, extracting distortion-friendly edge images and frame difference images on the spatial domain slices, and extracting variation images and Gaussian stills on all slice sequences; forming an atlas by the extracted edge graph, frame difference graph, change graph and Gaussian stationary graph together with the original graph so as to complete atlas configuration, wherein the change graph comprises a gradient amplitude graph and a gradient directional graph, and the Gaussian stationary graph comprises a Gaussian filter graph and a Laplace graph;
step three, introducing the time-space domain stability of the video to be evaluated into the slicing field, and performing map generation calculation on half of slices according to the mode of the step two;
step four, introducing a 2D image quality evaluation method, and calculating difference values of generated map reference-distortion pairs;
and step five, automatically determining the weight of the contribution degree of each map to the video distortion in a learning mode by applying a neural network method.
2. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the edge map is as follows:
wherein, IEDGE(I, T) represents the generated edge map, ISTS(i, T) represents a time-space domain slice, i represents an index value of a slice sequence, T represents a time dimension of a slice, fh、fvA corresponding distortion-friendly edge filtering kernel.
3. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the frame difference map comprises the following steps:
IDIFF(i,T)＝{ISTS(i,T)-ISTS(i-1,T)|i∈[2,N]} (2)
wherein, IDIFF(I, T) represents the generated frame difference map, ISTS(i, T) represents a time-space domain slice, i represents an index value of a slice sequence, T is a time dimension of the slice, and N is a maximum index value of the slice.
4. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the variation map is as follows:
wherein, IGM(I, d) represents the generated gradient magnitude map, IGO(i, d) is the generated gradient directional diagram, i represents the index value of the slice sequence; d represents different dimensionalities of the video, the video is regarded as a cuboid in a three-dimensional coordinate system, three coordinate axes respectively represent the height, the width and the time of the video, and the value range of d is the height, the width and the time; i isSTS(i, d) is a time-space domain slice sequence, Gx、GyIs a Gaussian gradient filter kernel in the horizontal and vertical directions.
5. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the Gaussian stationary map comprises the following steps:
ILAP(i,d)＝ISTS(i,d)-IUP(i,d) (6)
wherein, IGAU(i, d) is the generated Gauss filter map, ILAP (i, d) is the generated Laplace map, iAn index value representing a slice sequence; d represents different dimensionalities of the video, the video is regarded as a cuboid in a three-dimensional coordinate system, three coordinate axes respectively represent the height, the width and the time of the video, and the value range of d is the height, the width and the time; i isSTS(i, d) is a time-space domain slice sequence, fgIs a Gaussian blur Filter kernel, IUP(i, d) are Gaussian filtered upsampled maps.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810882119.4A CN108600745B (en) | 2018-08-06 | 2018-08-06 | Video quality evaluation method based on time-space domain slice multi-map configuration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810882119.4A CN108600745B (en) | 2018-08-06 | 2018-08-06 | Video quality evaluation method based on time-space domain slice multi-map configuration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108600745A CN108600745A (en) | 2018-09-28 |
| CN108600745B true CN108600745B (en) | 2020-02-18 |
Family
ID=63623050
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810882119.4A Active CN108600745B (en) | 2018-08-06 | 2018-08-06 | Video quality evaluation method based on time-space domain slice multi-map configuration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108600745B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6363116B1 (en) * | 1997-04-04 | 2002-03-26 | Tektronix, Inc. | Picture quality assessment using spatial location with or without subsampling |
| CN101742355A (en) * | 2009-12-24 | 2010-06-16 | 厦门大学 | Method for partial reference evaluation of wireless videos based on space-time domain feature extraction |
| CN103731664A (en) * | 2013-12-25 | 2014-04-16 | 华为技术有限公司 | Method for full reference type video quality assessment, apparatus for full reference type video quality assessment and video quality testing device for full reference type video quality assessment |
| CN104023230A (en) * | 2014-06-23 | 2014-09-03 | 北京理工大学 | Non-reference image quality evaluation method based on gradient relevance |
| CN106028026A (en) * | 2016-05-27 | 2016-10-12 | 宁波大学 | Effective objective video quality evaluation method based on temporal-spatial structure |
| CN107220974A (en) * | 2017-07-21 | 2017-09-29 | 北京印刷学院 | A kind of full reference image quality appraisement method and device |
| CN106341677B (en) * | 2015-07-07 | 2018-04-20 | 中国科学院深圳先进技术研究院 | Virtual view method for evaluating video quality |
-
2018
- 2018-08-06 CN CN201810882119.4A patent/CN108600745B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6363116B1 (en) * | 1997-04-04 | 2002-03-26 | Tektronix, Inc. | Picture quality assessment using spatial location with or without subsampling |
| CN101742355A (en) * | 2009-12-24 | 2010-06-16 | 厦门大学 | Method for partial reference evaluation of wireless videos based on space-time domain feature extraction |
| CN103731664A (en) * | 2013-12-25 | 2014-04-16 | 华为技术有限公司 | Method for full reference type video quality assessment, apparatus for full reference type video quality assessment and video quality testing device for full reference type video quality assessment |
| CN104023230A (en) * | 2014-06-23 | 2014-09-03 | 北京理工大学 | Non-reference image quality evaluation method based on gradient relevance |
| CN106341677B (en) * | 2015-07-07 | 2018-04-20 | 中国科学院深圳先进技术研究院 | Virtual view method for evaluating video quality |
| CN106028026A (en) * | 2016-05-27 | 2016-10-12 | 宁波大学 | Effective objective video quality evaluation method based on temporal-spatial structure |
| CN107220974A (en) * | 2017-07-21 | 2017-09-29 | 北京印刷学院 | A kind of full reference image quality appraisement method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108600745A (en) | 2018-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | No-reference and robust image sharpness evaluation based on multiscale spatial and spectral features | |
| CN102722876B (en) | Residual-based ultra-resolution image reconstruction method | |
| CN102902961B (en) | Face super-resolution processing method based on K neighbor sparse coding average value constraint | |
| CN102547368B (en) | Objective evaluation method for quality of stereo images | |
| CN110889895B (en) | Face video super-resolution reconstruction method fusing single-frame reconstruction network | |
| CN109523513B (en) | Stereoscopic image quality evaluation method based on sparse reconstruction color fusion image | |
| CN107635136B (en) | View-based access control model perception and binocular competition are without reference stereo image quality evaluation method | |
| CN108134937B (en) | Compressed domain significance detection method based on HEVC | |
| CN109255358B (en) | 3D image quality evaluation method based on visual saliency and depth map | |
| CN104869421B (en) | Saliency detection method based on overall motion estimation | |
| CN103354617B (en) | Boundary strength compressing image quality objective evaluation method based on DCT domain | |
| CN109242834A (en) | It is a kind of based on convolutional neural networks without reference stereo image quality evaluation method | |
| CN104994375A (en) | Three-dimensional image quality objective evaluation method based on three-dimensional visual saliency | |
| Ma et al. | Reduced-reference stereoscopic image quality assessment using natural scene statistics and structural degradation | |
| CN106815839A (en) | A kind of image quality blind evaluation method | |
| CN108259893B (en) | Virtual reality video quality evaluation method based on double-current convolutional neural network | |
| CN109257592B (en) | Stereoscopic video quality objective evaluation method based on deep learning | |
| CN105160667A (en) | Blind image quality evaluation method based on combining gradient signal and Laplacian of Gaussian (LOG) signal | |
| CN108470336B (en) | Stereo image quality evaluation method based on stack type automatic encoder | |
| CN110717892A (en) | Tone mapping image quality evaluation method | |
| CN108830829B (en) | Non-reference quality evaluation algorithm combining multiple edge detection operators | |
| CN109447903A (en) | A kind of method for building up of half reference type super-resolution reconstruction image quality evaluation model | |
| CN106157251B (en) | A kind of face super-resolution method based on Cauchy's regularization | |
| CN113096015B (en) | Image super-resolution reconstruction method based on progressive perception and ultra-lightweight network | |
| CN103903239B (en) | A kind of video super-resolution method for reconstructing and its system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |