CN108600745B - A video quality evaluation method based on spatiotemporal slice multi-atlas configuration - Google Patents

Abstract

The invention discloses a video quality evaluation method based on multi-atlas configuration of temporal and spatial slices, and belongs to the technical field of image and video analysis. This method uses the idea of spatiotemporal slices to convert original and distorted video sequences into spatiotemporal slice representations, extracts distortion-friendly edge maps and frame difference maps on the spatial slices, and then extracts gradient magnitudes and gradients over all slice sequences The change map composed of the orientation and the still map of Laplace correction, and the original map form a map to complete the map configuration. After that, the spatiotemporal stability of the video to be evaluated is introduced into the slice field to generate the atlas, and the 2D image quality evaluation method is introduced to calculate the difference value of the generated atlas reference-distortion pair. Finally, a neural network method is applied to automatically determine the weight of the contribution of each atlas to video distortion in a learning manner. Compared with the prior art, the method of the invention has the characteristics of high subjective consistency, strong compatibility and high algorithm stability.

Description

A video quality evaluation method based on spatiotemporal slice multi-atlas configuration

technical field

The invention relates to a video quality evaluation method, in particular to a video quality evaluation method based on spatiotemporal slice multi-atlas configuration, and belongs to the technical field of image and video analysis.

Background technique

With the development of science and technology, the cost of generating and disseminating images and videos has become lower and lower, which makes images and videos, as an excellent medium for information dissemination, more and more common in daily life, and more and more The more indispensable. However, images and videos have the potential to introduce distortion at various stages of production and dissemination. The resulting distortion will affect people's experience when viewing these multimedia materials, and seriously affect people's physical and mental health.

In recent years, people have made great progress in the field of image quality evaluation, but the development in the field of video is relatively slow. How to curb the spread of low-quality videos and ensure people's visual experience is still an urgent problem to be solved. Therefore, it is of great significance to solve this problem that the media that generates and spreads the video have the ability to automatically evaluate the quality of the video, so as to improve the quality of the video at the output end of the media.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problems of low prediction accuracy, poor information representation ability, and unsuccessful application of 2D image quality evaluation methods in the field of video quality evaluation of existing video quality evaluation methods, and proposes a multi-atlas configuration based on spatiotemporal slices video quality assessment method.

The method of the present invention refers to the spatiotemporal slice idea proposed by Ngo et al. The spatiotemporal slicing idea uses a spatiotemporal joint method to re-represent the original video information, which effectively resolves the contradiction between video spatiotemporal information extraction and high computational complexity. This idea regards the video as a cuboid located in a three-dimensional coordinate system, and the three coordinate axes represent the height (H), width (W) and time (T) of the video respectively. Slicing the video along different axes results in different information representations of the video. This idea can be formulated as:

I _STS (i,d)={V ^d |d∈[T,W,H],i∈[1,N]} (1)

Among them, V is the input video sequence, the superscript d represents the different dimensions of the video, the value range is the height (H), width (W) and time (T) mentioned above, i represents the index value of the slice sequence, I _STS (i,d) is the generated spatiotemporal slice sequence.

The method of the present invention is realized by the following technical solutions. A video quality evaluation method based on spatiotemporal slice multi-atlas configuration, comprising the following steps:

Step 1. Convert the original and distorted video sequences to the spatiotemporal slice representation as the basic unit for subsequent processing.

Step 2: Extract the distortion-friendly edge map and frame difference map on the air domain slice, extract the change map and still image from all slice sequences, and form a map with the original map, thereby completing the map configuration.

Step 3: Introduce the spatiotemporal stability of the video to be evaluated into the slice field, and perform atlas generation and calculation.

Step 4: A 2D image quality evaluation method is introduced, and the difference value of the generated atlas reference-distortion pair is calculated.

Step 5: Apply the neural network method to automatically determine the weight of the contribution of each atlas to the video distortion in a learning manner.

beneficial effect

Compared with the prior art, the method of the invention has the characteristics of high subjective consistency, strong compatibility and high algorithm stability. This method can convert ordinary 2D image quality evaluation methods into high-performance video quality evaluation methods, can be used in conjunction with video processing related application systems, and can be embedded in practical application systems (such as video projection systems, network transmission systems, etc.) , to monitor the quality of video in real time; it can be used to evaluate the pros and cons of various video processing algorithms and tools (such as compression coding of stereoscopic images, video capture tools, etc.); it can be used to review the quality of video works to prevent the harm of inferior video products The physical and mental health of the audience.

Description of drawings

Figure 1 is a flow chart of the method of the present invention.

Detailed ways

The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

A video quality evaluation method based on spatiotemporal slice multi-atlas configuration, as shown in Figure 1, includes the following steps:

Step 1. Convert the original and distorted video sequences to the spatiotemporal slice representation as the basic unit for subsequent processing.

Step 2: Extract the distortion-friendly edge map and frame difference map on the air domain slice, extract the change map and still image from all slice sequences, and form a map with the original map, thereby completing the map configuration.

The spatial slice can best reflect the structure of the image. Therefore, the edge map and frame difference map of the spatial slice are extracted to optimize the spatiotemporal information. The preferred method is as follows:

I _DIFF (i,T)={I _STS (i, T)-I _STS (i-1,T)|i∈[2,N]} (3)

Among them, I _EDGE (i, T) is the generated edge map, and I _DIFF (i, T) is the generated frame difference map. I _STS (i, T) represents the spatiotemporal slice, i represents the index value of the slice sequence, T is the temporal dimension of the slice, N is the maximum index value of the slice, and f _h and f _v are the corresponding distortion-friendly edge filter kernels.

In this embodiment, the specific value of the edge filter kernel is:

SI13=[-0.0052625,-0.0173466,-0.0427401,-0.0768961,

-0.957739,-0.0696751,0,0.6696751,0.0957739,

0.0768961,0.0427401,0.0173466,0.0052625].(4)

The above two edge filter kernels can be obtained by duplicating this vector horizontally and vertically.

In order to optimize the basic information representation ability of spatiotemporal slices, the change map composed of gradient magnitude and direction, and the Laplace-corrected Gaussian still map are extracted to complete the optimization. The preferred extraction method is as follows:

I _LAP (i,d)=I _STS (i,d)-I _UP (i,d) (8)

Among them, I _GM (i, d) is the generated gradient magnitude map, I _GO (i, d) is the generated gradient direction map, I _GAU (i, d) is the generated Gaussian filter map, I _LAP (i, d) ) is the generated Laplace graph. i represents the index value of the slice sequence, d represents the different dimensions of the video, and _ISTS (i, d) is the spatiotemporal slice sequence. G _x and G _y are the Gaussian gradient filter kernels in the horizontal and vertical directions, f _g is the Gaussian blur filter kernel, and I _UP (i,d) is the up-sampled image after Gaussian filtering.

Step 3: Introduce the spatiotemporal stability of the video to be evaluated into the slice field, and perform atlas generation and calculation.

Preferably, only half of the slices can be calculated to generate the map to further reduce the amount of calculation.

Step 4: Using the 2D image quality evaluation method, the difference value of the generated atlas reference-distortion pair is calculated by means of average aggregation.

The specific calculation method of the difference value P _m (i′,d) is:

Among them, I represents an image in the atlas sequence, the superscripts ref and dis represent the reference video sequence atlas and the distorted video sequence atlas respectively, m denotes the atlas category, i' denotes the index value of the atlas sequence, and d denotes the different dimensions of the video . IQA represents the 2D full reference image quality evaluation method adopted. The calculated result is that each group of image pairs in this type of atlas sequence generates a difference value containing distortion information. For each type of atlas, the average aggregation method is used to obtain this type of atlas. The map difference score of the map, and then a vector S composed of the difference scores of all categories of maps can be obtained.

Step 5: Apply the neural network method to automatically determine the weight of the contribution of each atlas to the video distortion in a learning manner.

The learning representation formula is:

Q=θ ^R S (10)

Among them, θ is the weight parameter vector to be learned, S is the map difference score vector, Q is the final video quality score representation, and R represents the vector transpose.

Example

The method of the present invention is implemented on three video quality evaluation databases, including LIVE, IVP and CSIQ. The basic information of these databases is shown in Table 1. The method of the present invention is compared by selecting two full-reference video quality evaluation methods with superior performance.

Table 1 Basic information of the database

In addition, because this method is a framework for transforming the 2D image quality evaluation method into a video evaluation method, three full reference image quality evaluation methods (PSNR, SSIM and VIF) are selected to complete the experiment in combination with the method of the present invention. The results of the reference method are also added for comparison to test the improvement of the performance of these methods by the method framework of the present invention. In each experiment, 20% of the data was selected for testing, with SRCC, KRCC, PLCC, and RMSE as indicators, and repeated 1000 times to take the median value. The experimental results are shown in Table 2.

Table 2 Comparison of algorithm performance on three databases

Table 3 shows the performance of each algorithm on each type of distortion. Combining with Table 2, it can be seen that the performance of the 2D method has been significantly improved in the tests of the three databases using the method of the present invention. At the same time, after the improvement The PSNR results of the 3 libraries are better than STRRED and ViS3, indicating that the 2D method can achieve very competitive performance after being improved by this framework.

Table 3 Comparison of algorithm performance on each distortion category

Claims (5)

1. A video quality evaluation method based on time-space domain slice multi-map configuration is characterized by comprising the following steps:

converting an original video sequence and a distorted video sequence into a time-space domain slice representation form as a basic unit of subsequent processing;

secondly, extracting distortion-friendly edge images and frame difference images on the spatial domain slices, and extracting variation images and Gaussian stills on all slice sequences; forming an atlas by the extracted edge graph, frame difference graph, change graph and Gaussian stationary graph together with the original graph so as to complete atlas configuration, wherein the change graph comprises a gradient amplitude graph and a gradient directional graph, and the Gaussian stationary graph comprises a Gaussian filter graph and a Laplace graph;

step three, introducing the time-space domain stability of the video to be evaluated into the slicing field, and performing map generation calculation on half of slices according to the mode of the step two;

step four, introducing a 2D image quality evaluation method, and calculating difference values of generated map reference-distortion pairs;

and step five, automatically determining the weight of the contribution degree of each map to the video distortion in a learning mode by applying a neural network method.

2. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the edge map is as follows:

wherein, I_EDGE(I, T) represents the generated edge map, I_STS(i, T) represents a time-space domain slice, i represents an index value of a slice sequence, T represents a time dimension of a slice, f_h、f_vA corresponding distortion-friendly edge filtering kernel.

3. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the frame difference map comprises the following steps:

I_DIFF(i,T)＝{I_STS(i,T)-I_STS(i-1,T)|i∈[2,N]} (2)

wherein, I_DIFF(I, T) represents the generated frame difference map, I_STS(i, T) represents a time-space domain slice, i represents an index value of a slice sequence, T is a time dimension of the slice, and N is a maximum index value of the slice.

4. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the variation map is as follows:

wherein, I_GM(I, d) represents the generated gradient magnitude map, I_GO(i, d) is the generated gradient directional diagram, i represents the index value of the slice sequence; d represents different dimensionalities of the video, the video is regarded as a cuboid in a three-dimensional coordinate system, three coordinate axes respectively represent the height, the width and the time of the video, and the value range of d is the height, the width and the time; i is_STS(i, d) is a time-space domain slice sequence, G_x、G_yIs a Gaussian gradient filter kernel in the horizontal and vertical directions.

5. The video quality evaluation method based on the time-space domain slice multi-map configuration as claimed in claim 1, wherein in the second step, the method for extracting the Gaussian stationary map comprises the following steps:

I_LAP(i,d)＝I_STS(i,d)-I_UP(i,d) (6)

wherein, I_GAU(i, d) is the generated Gauss filter map, ILAP (i, d) is the generated Laplace map, iAn index value representing a slice sequence; d represents different dimensionalities of the video, the video is regarded as a cuboid in a three-dimensional coordinate system, three coordinate axes respectively represent the height, the width and the time of the video, and the value range of d is the height, the width and the time; i is_STS(i, d) is a time-space domain slice sequence, f_gIs a Gaussian blur Filter kernel, I_UP(i, d) are Gaussian filtered upsampled maps.