CN112954313A - Method for calculating perception quality of panoramic image - Google Patents
Method for calculating perception quality of panoramic image Download PDFInfo
- Publication number
- CN112954313A CN112954313A CN202110178463.7A CN202110178463A CN112954313A CN 112954313 A CN112954313 A CN 112954313A CN 202110178463 A CN202110178463 A CN 202110178463A CN 112954313 A CN112954313 A CN 112954313A
- Authority
- CN
- China
- Prior art keywords
- quality
- calculating
- panoramic image
- viewing
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000008447 perception Effects 0.000 title claims abstract description 14
- 238000005070 sampling Methods 0.000 claims abstract description 48
- 238000013441 quality evaluation Methods 0.000 claims abstract description 26
- 230000000007 visual effect Effects 0.000 claims abstract description 13
- 230000002123 temporal effect Effects 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims description 15
- 230000015654 memory Effects 0.000 claims description 15
- 230000000694 effects Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000001303 quality assessment method Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000006403 short-term memory Effects 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000006399 behavior Effects 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 6
- 241000282414 Homo sapiens Species 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000021824 exploration behavior Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000004886 head movement Effects 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 206010021403 Illusion Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention provides a method for calculating the perception quality of a panoramic image, which comprises the steps of acquiring the watching track of the panoramic image, extracting a watching view port by utilizing linear projection in combination with the sampling rate and the watching view field range, and acquiring a group of video sequences sampled along a watching path; taking the video sequence as two-dimensional video data shot by a virtual camera, and calculating the quality of each frame of image in the two-dimensional video data by using a two-dimensional image quality evaluation method; and fusing the quality scores of all the frame images by a temporal weighting method to obtain a final overall perception quality score. The invention can apply the traditional video quality evaluation method to the immersive scene, and can effectively and objectively predict the visual quality of the panoramic image.
Description
Technical Field
The invention relates to the technical field of image data processing, in particular to a method for calculating the perception quality of a panoramic image.
Background
Virtual Reality (VR) imaging is a technique that can capture and create a complete natural scene, which is typically taken to form a single omnidirectional image, also commonly referred to as a static 360 ° panorama. Because people can freely explore in an immersive virtual environment, panoramic images provide a viewing experience that is very different from traditional multimedia data. Therefore, in view of their importance for panoramic image acquisition, compression, storage, transmission and reproduction, understanding how humans perceive panoramic visual distortion has become a new direction of research.
Since a panoramic image is usually projected onto a 2D plane and then stored, there is an attempt to apply a 2D image quality evaluation model to quantify the perceptual distortion in the projection. However, different geometric projections have different distortion problems, for example, the equidistant cylindrical projection can generate severe distortion in the two polar region, and the cubic projection has a sampling rate of more than 190% compared with the spherical surface. And there is no significant link between the distortion of the 2D plane measurement and the perceived distortion seen by the observer at the sphere. In order to eliminate the mismatch between the plane and the spherical surface, some panoramic image quality evaluation models firstly perform local quality measurement on the plane, and then use the spherical surface area as a weight to weight the local quality to obtain a final quality score. Meanwhile, the method which works with the method has the same work as the method for calculating the quality of the whole image on the spherical surface.
In 2D image quality assessment, the behavior of the experimenter is well controlled in a laboratory environment and is generally considered similar without explicit modeling. However, this assumption is not applicable to panoramic image quality evaluation. With Head Mounted Displays (HMDs) worn, people can explore interest in scenes by head and eye movement as much as possible. Therefore, different people have different attention areas to the same panoramic image, but no quality evaluation model can provide a good solution for the watching behavior when the perception quality of the panoramic image is predicted at present. In order to form an effective and reliable panoramic image quality evaluation model, modeling of visual exploration behaviors is very important in the quality evaluation process.
However, the existing method does not evaluate the perceptual quality of the panoramic image, so that an effective and reliable panoramic image quality evaluation model cannot be formed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method for calculating the perception quality of a panoramic image, which can effectively evaluate the perception quality of the panoramic image.
In order to achieve the purpose, the invention is realized by the following technical scheme: a method of computing perceived quality of a panoramic image, comprising: acquiring a watching track of a panoramic image, extracting a watching viewport by utilizing linear projection in combination with a sampling rate and a watching field range, and acquiring a group of video sequences sampled along a watching path; taking the video sequence as two-dimensional video data shot by a virtual camera, and calculating the quality of each frame of image in the two-dimensional video data by using a two-dimensional image quality evaluation method; and fusing the quality scores of all the frame images by a temporal weighting method to obtain a final overall perception quality score.
Preferably, the acquiring of the viewing trajectory of the panoramic image includes: setting a viewing starting point P0(φ0，θ0) Wherein phi0And theta0Respectively representing the longitude and latitude of the starting point; setting viewing time T and viewing path p (T):
for any time T e [0, T ∈]And obtaining corresponding spherical coordinates (phi, theta), wherein P (0) ═ 0, then the coordinates of the sampling point of the path at the time t are P (t) + P (0).
Preferably, the viewing path is from a starting point (φ)00) start viewing, gradually rotate to (phi) along the equatorial region0-pi/2, 0); then, clockwise rotates from (phi)0-pi/2, 0) to (phi)0+ pi/2, 0), and finally returning to the start point position.
Preferably, extracting the viewing viewport by using a straight-line projection, and obtaining a set of video sequences sampled along the viewing path includes: obtaining a sample point (phi) in a viewing pathC，θC) Rotating the spherical surface to a state that the Z axis points to the sampling point through the rotation matrix Rot; determining values of all pixel points in the viewport with the sampling point as the center and the view field as the size, calculating the corresponding sampling point coordinate (m, n) of the pixel for the pixel coordinate (m, n) of the viewportThe label (u, v); according to the coordinates (u, v) of the sampling point, calculating the corresponding spherical Cartesian coordinates (x, y, z) of the sampling point; calculating coordinates of sampling points after the spherical surface rotates through a rotation matrix; obtaining 2D plane coordinates (x) of sampling points through an equidistant column projection formula2D，y2D) And obtaining the corresponding pixel value.
Preferably, when calculating the coordinates (u, v) of the corresponding sampling point of the pixel, the following formula is adopted for calculation:
wherein F is the angle of the visual field, WvAnd HvRespectively, the length and width of the view port.
Preferably, after the viewing trajectory of the panoramic image is acquired, a sampling rate and a viewing field of view range are further set, wherein the setting of the sampling rate and the viewing field of view range includes: the sampling rate R at which the panoramic image is converted into a video sequence is set,
wherein R isetIs the sampling frequency, s, set by the head-mounted display1To adjust the parameters; setting a viewing field range at the time of sampling: the current sampling point P (t) + P (0) is set to [ - π/6, π/6 in the longitudinal and latitudinal directions as the center]The viewing field of view F of size.
Preferably, the calculating the quality of each frame of image in the two-dimensional video data by using the two-dimensional image quality evaluation method with the video sequence as the two-dimensional video data shot by the virtual camera comprises: evaluating the quality of the video frame by adopting a two-dimensional image quality evaluation method; calculating the quality of a single frame of video, using the following formula: qij＝D(Xij，Yij) Wherein Y isijRepresenting a jth frame of viewport image, X, obtained from an ith user's view path sampleijRepresenting a lossless quality version of the image, and D represents the two-dimensional image quality assessment model employed.
Preferably, the fusing the quality scores of all the frame images by a temporal weighting method to obtain a final global perceptual quality score includes: defining a quality memory unit: defining a quality memory unit in each video frame, wherein the quality memory unit is calculated by adopting the following formula:
wherein k is 2 xr is a simulated short-term memory related parameter;
adjusting the quality of the video frame affected by the near cause effect, and calculating by adopting the following formula:
wherein sort () represents a pair array { Q }j，...，Qmin{j+K，N}Carry out ascending sorting to become a new array
w is a normalized weighting vector specified by a gaussian function descent portion;
adjusted time-transformed quality fraction QjThe memory and current factors are obtained through linear combination calculation, and the adjustment calculation is carried out by adopting the following formula:
wherein α is a preset weighting parameter; averaging the quality scores of all video frames to serve as the quality score of the video and serve as the representation of the quality score of the panoramic image, and calculating by adopting the following formula:
calculate the average of the quality scores of all videos:
compared with the prior art, the invention has the beneficial effects that: therefore, the method and the device combine the real watching behavior of the user, extract the watching view port of the user by using the linear projection through the structured watching path and combining the sampling rate and the watching view field range, and obtain a group of video sequences sampled along the watching path. The invention provides different motion shooting videos for representing panoramic images watched by different viewers, and the novel characterization method can apply the existing traditional video quality evaluation method to an immersive scene and can effectively and objectively predict the visual quality of the panoramic images.
Drawings
Fig. 1 is a frame diagram of an embodiment of a method for calculating perceived quality of a panoramic image according to the present invention.
FIG. 2 is a flow chart of an embodiment of the method for calculating perceived quality of a panoramic image according to the present invention;
fig. 3 is a diagram of the effect of significance check of the embodiment of the method for calculating the perceived quality of the panoramic image.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention relates to a method for calculating the perceived quality of a panoramic image, which can be implemented by a computer device, for example, the computer device comprises a processor and a memory, the memory stores a computer program, and the computer program can implement the method for calculating the perceived quality of the panoramic image.
Because the visual experience provided by the panoramic image is greatly different from the traditional multimedia data, the watching behavior of the human exploration immersive environment plays an important role in the perception quality, the panoramic image quality evaluation method is designed by modeling the visual exploration behavior, a brand-new panoramic image quality evaluation method is designed, the evaluation effect of the panoramic image can be effectively improved, and the using method of the panoramic image quality evaluation method is characterized by comprising the user watching behavior, image projection conversion and video quality calculation.
Referring to fig. 1, in the present embodiment, a panorama distortion map 11 is obtained first, where the panorama distortion map 11 is a 3D drawing, and the present embodiment first applies the panorama distortion map 11 to obtain a group of video sequences 12 sampled along a viewing path, that is, a group of images of 2D videos is formed. Then, a 2D image quality evaluation algorithm 13 is used to calculate the video sequence 12 to obtain the frame-level quality 14 of each frame of image of the video sequence, and then the image is subjected to quality pooling 15 to finally obtain the image quality prediction result 16.
Referring to fig. 2, the present embodiment first performs step S1 to structure the viewing path, thereby acquiring the viewing trajectory of the panoramic image. In step S1, it is necessary to extract the user viewing behavior, specifically, it is necessary to set a starting point of the viewing path, the viewing time T, and calculate the viewing path. Preferably, the panoramic image is a three-dimensional image, and the points of the image have latitude and longitude to mark the coordinates of each point. Where the starting point provides the longitude and latitude at the center of the initial viewing window, the viewing time records the time it takes the experimenter to explore the entire panoramic image, and the viewing path describes the two-dimensional trajectory of the human eye on the viewing sphere while viewing the field of view.
Specifically, the starting point is set to P0(φ0，θ0) Wherein phi0，θ0Representing the longitude and latitude, respectively, of the starting point, representing the center point of the viewport at which the viewer initially begins exploring the virtual scene. The inventor conducts a large number of experiments, and the results of the experiments show that different starting points have different influences on the visual significance, so that the perception quality of people on the same scene is influenced. Moreover, at different starting points, the watching path and the watching time can also be different from person to person, and personalized experience of looking and feeling is generated.
The viewing time T records the time taken by the viewer to explore the panoramic image, and assuming a reasonable head movement speed, a short viewing time means that the viewer can only see little image content, highlighting the importance of the initial viewport in the quality evaluation process. Conversely, at longer viewing times, the viewer may be more susceptible to the last view of the viewport content.
Viewing path p (t) set in the present embodiment:
a 2D trajectory is described as the eyes of a viewer navigate through a visual area. Usually a time point T ∈ [0, T ] is required]As an input, 2D spherical coordinates (Φ, θ) are generated, where P (0) ═ 0, 0 can be described as P (t) + P (0) at a particular point in time.
When the above user behavior information (i.e., viewing starting point, viewing time, and viewing path) is known, it will be directly input as an image perception model for actual user viewing viewport extraction. However, since such behavior information is sometimes not available, the present embodiment designs a default viewing behavior based on a priori knowledge. Specifically, the start points are set at four points evenly distributed along the equator, and the viewing time T is set to 15 s. Meanwhile, in order to reduce the computational complexity, the embodiment designs a simple viewing path, and takes into account the situation that people tend to view the equatorial region more. First from a starting point (phi)00) start viewing, gradually rotate to (phi) along the equatorial region0-pi/2, 0); then, the viewer rotates clockwise from (φ)0-pi/2, 0) to (phi)0+ pi/2, 0), and finally the viewer returns to the starting point position to end the browsing.
Then, step S2 is executed to extract the viewing viewport by using linear projection in combination with the sampling rate and the viewing field of view, and obtain a set of video sequences sampled along the viewing path. In particular, given the viewing behavior, the panoramic image may be converted into a set of video sequences containing global motion by means of the viewing behavior data, just as a static scene is captured by a moving camera. Such a video sequence is derived from a rectilinear projection viewport sample along a scan viewing path. Specifically, given the current sample point P (t) + P (0) as the viewport center, the viewing horizon F is set to a size of [ -pi/6, pi/6 ] set in the longitude and latitude directions, inspired by visual psychology theory. After a viewing path is given, the extraction of the viewport needs to set a corresponding sampling rate, specifically, the sampling rate can be calculated by using the following formula:
wherein R isetIs a set maximum sampling rate that takes into account the maximum sampling rate set by the limits of the human eye, s1Is a stride parameter, specifically a number greater than or equal to 1. According to equation 1, it can be obtained that the finally converted video sequence contains N ═ R × T frame images.
For a certain sample point (phi) in the viewing pathC，θC) The corresponding view port is extracted using a line projection. Firstly, determining values of all pixel points in a viewport which takes a sampling point as a center and a view field as a size. Given a certain pixel coordinate (m, n) of the viewport, the corresponding sampling point coordinate (u, v) is calculated by the following formula:
wherein, F is the angle of the view field, and Wv and Hv are the length and width of the view port, respectively. Calculating the corresponding spherical Cartesian coordinates (x, y, z) according to the coordinates (u, v) of the sampling point, wherein the calculation formula is as follows:
after obtaining the cartesian coordinates (x, y, Z) of the spherical surface of the sampling point, rotating the spherical surface to a state that the Z axis points to the sampling point by rotating the matrix Rot, and calculating the formula of the spherical rotation matrix Rot as follows:
calculating the coordinates of the sampling points after the spherical surface rotates through a rotation matrix, wherein the calculation formula is as follows:
[ x, y, z ] ═ Rot × [ x, y, z ] (formula 5)
Finally, obtaining 2D plane coordinates (x) of the sampling points through an equidistant columnar projection formula2D，y2D) And obtaining a corresponding pixel value, wherein the calculation formula is as follows:
where W and H are the width and height, respectively, of the 2D planar figure.
Next, step S3 is executed to calculate the quality of each frame of image in the two-dimensional video data by using the two-dimensional image quality evaluation method using the video sequence as the two-dimensional video data captured by the virtual camera. In general, any existing VQA (visual question and answer) model may be used in step S3 to assess the perceptual quality of the panoramic image. In this embodiment, several common two-dimensional image quality evaluation methods are used to evaluate the quality of a video frame, including peak signal-to-noise ratio (PSNR), Structural Similarity (SSIM), visual information fidelity metric (VIF), Normalized Laplacian Pyramid Distance (NLPD), depth image result, and texture similarity metric (DISTS). For video frame-level quality, the present embodiment calculates the following formula:
Qij＝D(Xij，Yij) (formula 7)
Wherein Y isijRepresenting the j-th frame of video image, X, obtained from the viewing path sample of the i-th userijRepresenting a lossless quality version of the image, and D represents the two-dimensional image quality assessment model employed. Q obtained by calculationijIs the quality of a frame of image.
Finally, step S4 is executed to fuse the quality scores of all the frame images by a temporal weighting method to obtain a final global perceptual quality score. For simplicity of description, the symbol i will be omitted in the following equations 8 to 13, i.e., the viewing quality calculation process of a single user will be described.
Step S4 specifically executes the following steps: firstly, defining a quality memory unit, and simulating a situation that a human being is difficult to tolerate poor quality and a hysteresis response facing an improvement situation, wherein the embodiment sets a quality memory unit in each video frame, and the set memory unit is expressed by the following formula:
where K is 2 xr, the analog short-term memory related parameter, Q1 is the quality score of the first frame image, K is similar to a window size, and the preset value of K is 2 seconds times the FPS of the "video", i.e. how many frames are seen in 2 seconds. J-K is the first value of this window. For example, when K is 5 and j is 10, it is necessary to calculate j-K is 5 to j-1 is 9, that is, the frame with the lowest quality score in the 5 th frame (5 th to 9 th frames) and the following 5 th frames (5 th to 9 th frames). When j is less than 5, the window size is insufficient, so equation 8 is represented by a two-step equation.
According to the near-cause effect, this embodiment simulates a situation where human beings are greatly influenced by recent memory content, so that the quality of a video frame needs to be adjusted to a certain extent, specifically, the following formula is adopted for adjustment:
wherein sort () represents the pair { Q }j，...，Qmin{j+K，N}Is arranged in ascending order to become
w is a normalized weight vector specified by a decreasing part of the Gaussian function, and the adjusted time-transformed quality fraction QjThe memory factor and the current factor are obtained through linear combination calculation, and the calculation formula is as follows:
where α is a preset weighting parameter, i.e., a parameter that balances two components.
When a group of video sequences is given, the quality scores of all video images are averaged to be used as the quality score of the virtual video and used as the representation of the quality score of the panoramic image, and specifically, the following formula is adopted for calculation:
where N is the number of video frames.
And finally, obtaining the experience quality scores of all users by calculating the average value of the quality scores of all videos, and calculating by adopting the following formula:
where M is the number of users.
To verify the feasibility of the method of this embodiment, the present invention uses three VR databases for subjective testing: the JXUFE, OIQA and LIVE 3D databases were subjected to comparative testing. The JXUFE database comprises 36 distorted images with resolution of 7680 × 3840 in an equidistant columnar projection format, wherein 24 distorted images are local splicing distortion, and 12 distorted images are H.265 compressed distortion; the OIQA database contains 320 distorted images in the equidistant cylindrical projection format with resolutions ranging from 11332 × 5666 to 11320 × 6660, which are obtained from 16 original pictures through four distortion types, i.e., JPEG2k (JP2K), Gaussian Noise (GN) and Gaussian Blur (GB), and combining five distortion levels. The LIVE 3D database contains 15 stereoscopic reference panoramic images in the equidistant cylindrical projection format, the resolution of the stereoscopic reference panoramic images is 4096 × 2048, the database has 6 distortion types-GN, GB, Downsampling (DS), stitching distortion (ST), VP9 compression, h.264 compression, h.265 compression, and sets corresponding 5 distortion levels of distorted images.
The present invention uses two evaluation criteria: the Pearson Linear Correlation Coefficient (PLCC) and the spearman grade correlation coefficient (SRCC) are used for quantifying the quality prediction performance, and a good quality prediction model has higher PLCC and SRCC values.
The following table 1 is a table for comparing the performance of the panoramic IQA algorithm on the JXUFE and OIQA databases, and the following table 2 is a table for comparing the performance of the panoramic IQA algorithm on the LIVE 3D database.
TABLE 1 comparison of panoramic IQA Algorithm Performance on JXUFE, OIQA databases
TABLE 2 comparison of Performance of panoramic IQA Algorithm on LIVE 3D databases
Fig. 3 is a diagram showing the effect of significance check of an embodiment of the method for calculating the perceived quality of a panoramic image according to the present invention, wherein fig. 3(a) shows the effect of the juxe database, fig. 3(b) shows the effect of the OIQA database, and fig. 3(c) shows the effect of the LIVE 3D database. In fig. 3, black blocks indicate that the performance of the row model is significantly better than that of the column model, white blocks indicate that the performance of the column model is significantly better than that of the row model, and gray blocks indicate that there is no significant difference in the performance of the row model and the column model.
As can be seen from table 1, table 2 and fig. 3, on the basis of the original 2D image quality evaluation algorithm, the method provided by the present invention can effectively improve the correlation between the quality evaluation method and the subjective evaluation, and therefore, the image quality evaluation is more accurate and objective.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. A method for calculating the perceived quality of a panoramic image, comprising:
acquiring a watching track of a panoramic image, extracting a watching view port by utilizing linear projection in combination with a sampling rate and a watching view field range, and acquiring a group of video sequences sampled along the watching path;
taking the video sequence as two-dimensional video data shot by a virtual camera, and calculating the quality of each frame of image in the two-dimensional video data by using a two-dimensional image quality evaluation method;
and fusing the quality scores of all the frame images by a temporal weighting method to obtain a final overall perception quality score.
2. The method for calculating the perceived quality of the panoramic image according to claim 1, wherein:
acquiring a viewing trajectory of a panoramic image includes:
setting a viewing starting point P0(φ0，θ0) Wherein phi0And theta0Respectively representing the longitude and latitude of the starting point;
setting viewing time T and viewing path p (T):
for any time T e [0, T ∈]And obtaining corresponding spherical coordinates (phi, theta), wherein P (0) ═ 0, then the coordinates of the sampling point of the path at the time t are P (t) + P (0).
3. The method for calculating the perceived quality of the panoramic image according to claim 2, wherein:
the viewing path is from a starting point (phi)00) start viewing, gradually rotate to (phi) along the equatorial region0-pi/2, 0); then, clockwise rotates from (phi)0-pi/2, 0) to (phi)0+ pi/2, 0), and finally returning to the start point position.
4. A method for calculating perceptual quality of a panoramic image, as defined in claim 3, wherein:
extracting a viewing viewport using a line projection, and obtaining a set of video sequences sampled along the viewing path comprises:
obtaining a sampling point (phi) in the viewing pathC，θC) Rotating the spherical surface to a state that the Z axis points to the sampling point through the rotation matrix Rot;
determining values of all pixel points in the viewport with the sampling point as the center and the view field as the size, and calculating the corresponding sampling point coordinate (u, v) of the pixel for the pixel coordinate (m, n) of the viewport;
according to the coordinates (u, v) of the sampling points, calculating the corresponding spherical Cartesian coordinates (x, y, z) of the sampling points;
calculating coordinates of sampling points after the spherical surface rotates through the rotation matrix;
obtaining the 2D plane coordinates (x) of the sampling points by an equidistant column projection formula2D，y2D) And obtaining the corresponding pixel value.
5. The method for calculating the perceived quality of the panoramic image according to claim 4, wherein:
when the coordinates (u, v) of the corresponding sampling point of the pixel are calculated, the following formula is adopted for calculation:
wherein F is the angle of the visual field, WvAnd HvRespectively, the length and width of the view port.
6. A method for calculating perceptual quality of a panoramic image, as defined in any one of claims 1 to 5, wherein:
after the track of watching of acquireing panoramic picture, still set up the sampling rate and watch the field of vision scope, wherein, set up the sampling rate and watch the field of vision scope and include:
the sampling rate R at which the panoramic image is converted into a video sequence is set,
wherein R isetIs the sampling frequency, s, set by the head-mounted display1To adjust the parameters;
setting a viewing field range at the time of sampling: the current sampling point P (t) + P (0) is centered, and a viewing field F of [ -pi/6, pi/6 ] size is set in the longitudinal and latitudinal directions.
7. A method for calculating perceptual quality of a panoramic image, as defined in any one of claims 1 to 5, wherein:
using the view port sequence as two-dimensional video data shot by a virtual camera, and calculating the quality of each frame of image in the two-dimensional video data by using a two-dimensional image quality evaluation method comprises the following steps:
evaluating the quality of the video frame by adopting a two-dimensional image quality evaluation method;
calculating the quality of a single frame of video, using the following formula:
Qij＝D(Xij，Yij) Wherein Y isijRepresenting a jth frame of viewport image, X, obtained from an ith user's view path sampleijRepresenting a lossless quality version of the image, and D represents the two-dimensional image quality assessment model employed.
8. A method for calculating perceptual quality of a panoramic image, as defined in any one of claims 1 to 5, wherein:
fusing the quality scores of all the frame images by a temporal weighting method to obtain a final global perception quality score, wherein the step of fusing the quality scores of all the frame images by the temporal weighting method comprises the following steps:
defining a quality memory unit: defining a quality memory unit in each video frame, wherein the quality memory unit is calculated by adopting the following formula:
wherein k is 2 xr is a simulated short-term memory related parameter;
adjusting the quality of the video frame affected by the near cause effect, and calculating by adopting the following formula:
wherein sort () represents a pair array { Q }j，...，Qmin{j+K，N}Carry out ascending sorting to become a new array
w is a normalized weighting vector specified by a gaussian function descent portion;
adjusted time-transformed quality fraction QjThe memory and current factors are obtained through linear combination calculation, and the adjustment calculation is carried out by adopting the following formula:
averaging the quality scores of all video frames to serve as the quality score of the video and serve as the representation of the quality score of the panoramic image, and calculating by adopting the following formula:
calculate the average of the quality scores of all videos:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110178463.7A CN112954313A (en) | 2021-02-09 | 2021-02-09 | Method for calculating perception quality of panoramic image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110178463.7A CN112954313A (en) | 2021-02-09 | 2021-02-09 | Method for calculating perception quality of panoramic image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112954313A true CN112954313A (en) | 2021-06-11 |
Family
ID=76244787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110178463.7A Pending CN112954313A (en) | 2021-02-09 | 2021-02-09 | Method for calculating perception quality of panoramic image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112954313A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116843684A (en) * | 2023-08-30 | 2023-10-03 | 江西财经大学 | End-to-end panoramic image quality evaluation method based on dynamic visual content |
| CN117176983A (en) * | 2023-08-10 | 2023-12-05 | 苏州瀚易特信息技术股份有限公司 | Video generation evaluation system based on panoramic image synthesis |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013105720A1 (en) * | 2012-01-10 | 2013-07-18 | 에스케이플래닛 주식회사 | Device and method for analyzing quality of three-dimensional stereoscopic image |
| US20160212432A1 (en) * | 2013-09-06 | 2016-07-21 | Zhou Wang | Method and system for objective perceptual video quality assessment |
| CN110691236A (en) * | 2019-09-18 | 2020-01-14 | 宁波大学 | Panoramic video quality evaluation method |
| CN110930370A (en) * | 2019-11-04 | 2020-03-27 | 北京工业大学 | Panoramic image quality evaluation method |
-
2021
- 2021-02-09 CN CN202110178463.7A patent/CN112954313A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013105720A1 (en) * | 2012-01-10 | 2013-07-18 | 에스케이플래닛 주식회사 | Device and method for analyzing quality of three-dimensional stereoscopic image |
| US20160212432A1 (en) * | 2013-09-06 | 2016-07-21 | Zhou Wang | Method and system for objective perceptual video quality assessment |
| CN110691236A (en) * | 2019-09-18 | 2020-01-14 | 宁波大学 | Panoramic video quality evaluation method |
| CN110930370A (en) * | 2019-11-04 | 2020-03-27 | 北京工业大学 | Panoramic image quality evaluation method |
Non-Patent Citations (1)
| Title |
|---|
| SUI X J等: "Omnidirectional Images as Moving Camera Videos", ARXIV:2005.10547V1 [EESS.IV], 21 May 2020 (2020-05-21), pages 1 - 5 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117176983A (en) * | 2023-08-10 | 2023-12-05 | 苏州瀚易特信息技术股份有限公司 | Video generation evaluation system based on panoramic image synthesis |
| CN117176983B (en) * | 2023-08-10 | 2024-01-26 | 苏州瀚易特信息技术股份有限公司 | Video generation evaluation system based on panoramic image synthesis |
| CN116843684A (en) * | 2023-08-30 | 2023-10-03 | 江西财经大学 | End-to-end panoramic image quality evaluation method based on dynamic visual content |
| CN116843684B (en) * | 2023-08-30 | 2023-11-14 | 江西财经大学 | End-to-end panoramic image quality evaluation method based on dynamic visual content |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xu et al. | Assessing visual quality of omnidirectional videos | |
| Chen et al. | Study of 3D virtual reality picture quality | |
| CN109615703B (en) | Augmented reality image display method, device and equipment | |
| Yang et al. | 3D panoramic virtual reality video quality assessment based on 3D convolutional neural networks | |
| Fang et al. | Light filed image quality assessment by local and global features of epipolar plane image | |
| Sui et al. | Perceptual quality assessment of omnidirectional images as moving camera videos | |
| Tian et al. | A multi-order derivative feature-based quality assessment model for light field image | |
| Tian et al. | Light field image quality assessment via the light field coherence | |
| CN110544301A (en) | Three-dimensional human body action reconstruction system, method and action training system | |
| US20110273369A1 (en) | Adjustment of imaging property in view-dependent rendering | |
| CN107105333A (en) | A kind of VR net casts exchange method and device based on Eye Tracking Technique | |
| CN107452031B (en) | Virtual ray tracking method and light field dynamic refocusing display system | |
| KR100560464B1 (en) | Multi-view display system with viewpoint adaptation | |
| CN113518996A (en) | Damage detection from multiview visual data | |
| CN103780895B (en) | A kind of three-dimensional video quality evaluation method | |
| Liu et al. | Blind omnidirectional image quality assessment based on structure and natural features | |
| CN112954313A (en) | Method for calculating perception quality of panoramic image | |
| CN111897433A (en) | Method for realizing dynamic gesture recognition and control in integrated imaging display system | |
| Xu et al. | Quality assessment of stereoscopic 360-degree images from multi-viewports | |
| Azevedo et al. | A viewport-driven multi-metric fusion approach for 360-degree video quality assessment | |
| CN111813689B (en) | Game testing method, apparatus and medium | |
| Jabar et al. | Perceptual analysis of perspective projection for viewport rendering in 360° images | |
| CN114648482A (en) | Quality evaluation method and system for three-dimensional panoramic image | |
| Zhang et al. | Full-reference stability assessment of digital video stabilization based on riemannian metric | |
| CN114979689B (en) | Multi-machine-position live broadcast guide method, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |