CN113395602A - Modeling method for user experience quality QoE in adaptive point cloud video streaming media - Google Patents

Modeling method for user experience quality QoE in adaptive point cloud video streaming media Download PDF

Info

Publication number
CN113395602A
CN113395602A CN202110709824.6A CN202110709824A CN113395602A CN 113395602 A CN113395602 A CN 113395602A CN 202110709824 A CN202110709824 A CN 202110709824A CN 113395602 A CN113395602 A CN 113395602A
Authority
CN
China
Prior art keywords
point cloud
quality
gof
frame group
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110709824.6A
Other languages
Chinese (zh)
Other versions
CN113395602B (en
Inventor
黎洁
王枭
李奇越
张聪
韩玲
王慧宇
陈勇
彭涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei University of Technology
Original Assignee
Hefei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei University of Technology filed Critical Hefei University of Technology
Priority to CN202110709824.6A priority Critical patent/CN113395602B/en
Publication of CN113395602A publication Critical patent/CN113395602A/en
Application granted granted Critical
Publication of CN113395602B publication Critical patent/CN113395602B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a modeling method for user experience quality QoE in a self-adaptive point cloud video streaming media, which comprises the following steps: 1. acquiring an original point cloud video, segmenting the original point cloud video into a temporal frame group, segmenting the original point cloud video into spatial blocks, and finally compressing each block into different quality levels; 2. performing quality selection according to the user view prediction and the network bandwidth, and calculating the quality level of the selection of the cut block in the user view; 3. re-fusing the cut blocks and the frame groups to obtain the processed point cloud video, the pause time generated when the processed point cloud video is transmitted by streaming media, and switching the quality; 4. evaluating the objective quality of the processed point cloud video; 5. and grading the experience quality of the processed point cloud video, and performing QoE modeling according to the grading, the objective quality, the pause time and the quality switching. The method and the device can more comprehensively carry out QoE modeling on the point cloud video, and accurately predict the subjective experience quality of users in the point cloud video streaming media under the condition of network bandwidth change.

Description

Modeling method for user experience quality QoE in adaptive point cloud video streaming media
Technical Field
The invention relates to the field of multimedia video transmission, in particular to a modeling method for user experience quality QoE in self-adaptive point cloud video streaming media.
Background
Chen-Ying et al at Xiamen university invented a deep learning-based laser scanning SLAM indoor three-dimensional point cloud quality evaluation method (publication No. CN110246112A), which includes: s1, acquiring high-quality point cloud through a laser scanning SLAM device; s2, degrading the high-quality point cloud to obtain a simulation point cloud; s3, carrying out track measurement analysis on the simulation point cloud; s4, extracting a plane from the high-quality point cloud and the simulation point cloud, and performing local consistency noise analysis and geometric rule analysis on the plane to quantize the quality of the point cloud; s5, segmenting the high-quality point cloud and the simulation point cloud to obtain point cloud blocks; s6, normalizing the point cloud blocks and inputting the point cloud blocks into a PointNet + + neural network for model training to obtain a network model; s7, carrying out point cloud quality analysis on the point cloud to be evaluated through the step S4 to obtain a point cloud quality level value; and S8, predicting the point cloud to be evaluated through the neural network model obtained in the step S6, and judging that the point cloud belongs to high-quality point cloud or degraded point cloud. Although the method provides a method for quantifying the point cloud quality and establishes a classification standard and a frame for evaluating an indoor three-dimensional point cloud model under a SLAM system, the method belongs to an objective quality evaluation method for static point clouds and cannot accurately reflect the user experience quality when point cloud video streaming media.
The Zhao Tie Song et al of the Fuzhou university invent a video QoE evaluation system and method based on long-term memory (publication number: CN112101788A), including: the system comprises a video sequence generation module, an acquisition module, a database based on long-term memory and a QoE evaluation model; the video sequence generation module comprises an acceptability test video sequence generation module and a long-term memory influence test video sequence generation module; the acquisition module is used for acquiring the acceptability of experimenters to the video and subjective opinion scores of the experimenters with different long-term memories to the video; the database based on the long-term memory is divided into at least three types of databases according to the interval of the overall acceptability, and the acceptability and subjective opinion score information of experimenters to videos are stored. Although the method considers the influence of long-term memory on the QoE evaluation model, other influence factors considered by the method on the QoE evaluation model are too few, and the method cannot comprehensively and accurately reflect the experience quality of the user.
The invention discloses a subjective QoE assessment method (publication number: CN109831705A) aiming at HTTP video stream, which comprehensively considers the end-to-end transmission and playing process of the HTTP video stream and determines objective perception parameters influencing the QoE and user subjective scoring parameters; and obtaining a QoE IFs data set X; numerically preprocessing the variables of the characterization types in the QoE IFs data set X, and performing dimensionality reduction and feature extraction by using principal component analysis to obtain principal components; calculating a principal component comprehensive score and weighting each index to determine importance ranking of each QoE IFs; and selecting the influence factors with greater importance, establishing a nonlinear relationship mapping model library between the influence factors and the mean opinion score, and taking the model with the minimum mean absolute error as the optimal model for QoE estimation. Although the index weighting method based on PCA is added in the QoE evaluation, the method considers too many objective perception parameters influencing the QoE, has no pertinence to factors influencing the QoE of the user, and cannot reflect the experience quality of the user in the transmission process specifically.
The invention discloses an encrypted video QoE evaluation method (publication number: CN108696403A) based on network flow characteristic structure learning, which is characterized in that a network data flow characteristic independent of data content is extracted from a QoS parameter through HAS video service data flow characteristic analysis, on the basis, a mapping model of 'network data flow characteristic → video KQI → user MOS' is trained and established by utilizing a machine learning method, and the evaluation of the encrypted video QoE is directly realized in a data acquisition platform. Although the method acquires the acquired video service QoS parameters based on the network data flow in the modeling process, the method has less correlation with the video content and low accuracy.
Chen Daqing et al of university at Zhejiang invents a non-interference mobile video user experience quality index modeling method (publication number: CN107888579A), comprising the following steps: clicking videos of an HPD mode and an HLS mode on a mobile phone, capturing flow information of the videos, simultaneously recording the pause condition of each video by using a pause analysis tool, and marking the real value of the video transmission mode category; obtaining a true value of the video blockage degree by analyzing the video blockage condition; obtaining a real value of video definition and network characteristics of the flow by analyzing the video flow; for videos in two transmission modes, machine learning is carried out by using network characteristics of video flow and real classes of the transmission modes, and a model which can be used for transmission mode classification is trained; and for the video of each transmission mode, respectively using the network characteristics of the video flow and the true value of the user experience quality index to carry out machine learning, and training a model which can be used for evaluating the user experience quality index. Although the method considers the influence of video snapping on the user experience quality, the method is difficult to implement for the point cloud video and cannot effectively reflect the user experience quality.
Disclosure of Invention
The invention aims to avoid the defects of the prior art, provides a simple and feasible modeling method aiming at the user experience quality QoE in the self-adaptive point cloud video streaming media, so as to more comprehensively model the point cloud video QoE and accurately predict the user experience quality in the point cloud video streaming media under the condition of network bandwidth change, thereby solving the problem that the QoE modeling can not comprehensively and accurately reflect the user experience quality.
The invention adopts the following technical scheme for solving the technical problems:
the invention relates to a modeling method for user experience quality QoE in a self-adaptive point cloud video streaming media, which is characterized by comprising the following steps of:
step one, a server acquires A original point cloud videos and divides each original point cloud video into K frame groups with the same frame number in time, wherein the K frame groups divided by any one original point cloud video are marked as { gof1,gof2,...,gofk,...gofK}，gofkRepresenting a kth frame group obtained by segmenting any original point cloud video, wherein K is more than or equal to 1 and less than or equal to K;
the kth frame group gofkSpatially divided into C slices of equal size, the kth frame group gofkThe c-th cut block obtained after cutting is marked as chunkk,c，1≤c≤C；
The c th cut chunk was cutk,cCompressed into L kinds of blocks with different quality levels, wherein the kth frame group gofkMiddle c cut chunkk,cThe first quality grade cut obtained after compression is marked as chunkk,c,l(ii) a Thus obtaining compressed blocks of all frame groups and storing the compressed blocks in a server; l is more than or equal to 1 and less than or equal to L;
step two, the client collects the visual angle information of the original point cloud video watched by the user to obtain the cut block set watched by the user in each frame group, wherein the gof of the kth frame group watched by the userkIs noted as fovk(ii) a If the user is in the kth frame group gofkIn which the c-th cut chunk is viewedk,cThen the c th cut chunk is cutk,cAdding a cut block set fovk；
The client end watches the kth frame group gofkOf the collection of dices fovkAs predicted the user views the kth frame group gofkIs cut into blocksSynprefovk(ii) a Or the client side enables the user to watch the k-1 th frame group gofk-1Of the collection of dices fovk-1As predicted the user views the kth frame group gofkOf a slice set prefovkThereby completing the prediction of the view angle information;
thirdly, the client selects the quality of the cut blocks in the user view angle according to the prediction result of the view angle information and the t-th network bandwidth, calculates the quality grade selected by the cut blocks in the user view angle under the t-th network bandwidth, and calculates the pause time and quality switching generated when the point cloud video in the user view angle plays streaming media; t ═ 1, 2,. T; t represents the total number of network bandwidth classes;
step four, according to the quality grade selected by the user in the view angle under the t-type network bandwidth, the client downloads the blocks of the corresponding quality grade from the server and re-fuses the blocks to obtain a processed point cloud video predicted by the view angle information; thereby obtaining G processed point cloud videos and pause time generated when the G processed point cloud videos are played in a streaming media, and switching the quality; g ═ T × 2 × a;
evaluating the G processed point cloud videos through objective quality indexes by the client side, so as to obtain objective quality of the G processed point cloud videos;
step six, the client renders and plays the G processed point cloud videos, and the E-th user scores the experience quality of the played G processed point cloud videos, so that G scores of all E users are obtained; e1, 2,. E; e represents the total number of users;
step seven, carrying out mean processing on the scores of the g-th processed point cloud video by all E users to obtain the mean opinion score MOS of the g-th processed point cloud videog(ii) a Thus obtaining G mean opinion scores MOS of the processed point cloud video, G being 1, 2.. G;
step eight, taking the mean opinion score MOS as a dependent variable, taking the objective quality, the pause time and the quality switching as independent variables, and respectively making scatter diagrams of the mean opinion score MOS and the objective quality, the pause time and the quality switching; and performing function fitting on the dependent variable and the independent variable to obtain a user QoE model of the point cloud video streaming media.
The modeling method of the invention is also characterized in that the third step is carried out according to the following process:
3.1, selecting the cut blocks with the same quality grade in the user visual angle by the client;
step 3.2, obtaining the kth frame group gof in the user visual angle by using the formula (1)kQuality grade of dicing Lk：
Figure BDA0003133108500000041
s.t.
xk,l∈[0,1] (2)
Figure BDA0003133108500000042
In the formulae (1) to (3), xk,lIs zero-one variable and represents the kth frame group gofkWhether or not to select a slice with quality class l, if x k,l1, denotes the kth frame group gofkSelecting a slice with quality class l, if x k,l0 denotes the kth frame group gofkSelecting no cut block with quality grade of l;
step 3.3, the kth frame group gof is expressed by the formula (4) to the formula (8)kPause time Stall before streaming media playk：
Figure BDA0003133108500000043
Figure BDA0003133108500000044
Figure BDA0003133108500000045
Figure BDA0003133108500000051
Figure BDA0003133108500000052
In the formula (4), Sk,lIndicating the kth frame group gof within the user's viewkThe data volume of the cut blocks under different quality levels;
in the formula (5), BkIndicating the kth frame group gofkA corresponding bandwidth;
in formula (6), Tek,lIndicating the kth frame group gof within the user's viewkDecoding time at different quality levels;
in the formula (7), f represents the kth frame group gofkFps is the number of point cloud frames played by the player per second;
step 3.4, obtaining the gof of the kth frame group by using the formula (9)kQuality switch occurring at streaming media playbackk：
Figure BDA0003133108500000053
In the formula (9), Lk,cIndicating the kth frame group gofkMiddle c cut chunkk,cQuality class of Lk-1,cIndicating the k-1 th frame group gofk-1Middle c cut chunkk-1,cQuality class of Dk,cDenotes the c th cut chunkk,cAnd the impact of viewpoint distance on quality switching, and there are:
Figure BDA0003133108500000054
in the formula (10), Disk,cDenotes the c th cut chunkk,cTo the viewpoint distance of the user; boxsizek,cDenotes the c th cut chunkk,cThe kth frame group gofkThe length of the diagonal line of the boundary frame where the corresponding point cloud space is located;
step 3.5, when the client selects the quality, the zero-one variable x is corrected by using a zero-one integer programming methodk,lSolving to obtain the kth frame group gof of the point cloud video in the user view angle under the t-th network bandwidthkHighest quality grade of medium cut block
Figure BDA0003133108500000055
And minimum pause time
Figure BDA0003133108500000056
Further, the k frame group gof is obtainedkQuality switching occurring during streaming media playback
Figure BDA0003133108500000057
The fifth step is carried out according to the following processes:
step 5.1, finding the nearest plane corresponding to each point in the processed point cloud video in the original point cloud video, and evaluating the distance information of the G processed point cloud videos by using a formula (11) to obtain the objective quality PSNR (Peak Signal to noise ratio) containing the distance informationp：
Figure BDA0003133108500000061
In the formula (11), p2planeRMSDRepresenting the average value of the distance from each point in the processed point cloud video to the corresponding nearest plane in the original point cloud video; dMAXRepresenting the maximum value in the diagonal length of a boundary box where a point cloud space corresponding to each frame of an original point cloud video is located;
step 5.2, evaluating the color information of the G processed point cloud videos by using the formula (12) to obtain objective quality PSNR (Peak Signal to noise ratio) containing the color informationC：
Figure BDA0003133108500000062
PSNR in the formula (12)Y,PSNRU,PSNRVRespectively representing peak signal-to-noise ratios of a Y component, a U component and a V component in a YUV color space;
and 5.3, obtaining the total objective quality PCPSNR of the G processed point cloud videos by using the formula (13):
PCPSNR＝PSNRP+PSNRC (13)。
compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, the objective quality index of the point cloud sequence is considered in point cloud modeling, and meanwhile, pause and quality switching generated in the transmission of the self-adaptive point cloud video streaming media can be added, so that the defect that only the objective quality index is considered in the past research of point cloud quality evaluation is overcome, and more comprehensive modeling can be performed. Under the condition of network bandwidth change, the subjective experience quality of people in the point cloud video streaming media can be more accurately predicted through the modeling.
2. The invention analyzes the influence factors of the QoE of the point cloud video streaming media, comprising the following steps: objective quality of the point cloud video, pause and quality switching generated by streaming media transmission are achieved, then detailed subjective quality experiments are conducted to obtain a QoE model through user experience, and meanwhile the relation between the block cutting distance and the quality switching in the QoE model is analyzed, so that the user experience quality is measured more accurately.
3. The invention provides a novel objective quality measuring method for measuring the objective quality of a point cloud video, and the method considers the distance information and the color information of points simultaneously, can more comprehensively reflect the objective quality of the point cloud video, and thus improves the accuracy of a QoE model.
4. The QoE model under the point cloud QoE video streaming media is researched for the first time, and the problem that few researches are conducted on the QoE of the point cloud QoE video streaming media in the prior art is solved; meanwhile, data analysis is carried out by using function fitting, the relation between the user QoE and the influence factors such as objective quality, pause time and quality switching is effectively analyzed, and the problem that the influence factors are less considered when the user QoE of the point cloud video streaming media is researched in the prior art is solved, so that the QoE model can reflect the user QoE more comprehensively and accurately.
Drawings
FIG. 1 is a block diagram of an adaptive point cloud video streaming system according to the present invention;
fig. 2 is a scatter plot of dependent variables MOS versus their respective variables in the present invention.
Detailed Description
In this embodiment, a modeling method for user quality of experience QoE in adaptive point cloud video streaming media is applied to an adaptive point cloud video streaming media system; as shown in fig. 1, the adaptive point cloud streaming media system is composed of a server and a client, wherein the server firstly segments an original point cloud video into temporal frame groups, then segments the original point cloud video into spatial segments, and finally compresses each segment into different quality levels and uploads the quality levels to the server; the client side integrates the view angle information, the bandwidth information and the current buffer state to calculate the quality grade of the blocks in the corresponding user view angle under the current network bandwidth, and then uploads the information to the server; the server transmits the cut blocks with corresponding quality grades to the client for decoding according to the cut block information watched by the user and the cut block quality grades obtained through calculation, then the cut blocks are fused into the processed point cloud video again, and the point cloud video is rendered for the user to watch; the modeling method comprises the following steps:
step 1, a server acquires 4 original point cloud videos, and divides each original point cloud video into 30 frame groups with the same frame number in time, wherein the 30 frame groups divided by any one original point cloud video are marked as { gof [ ]1,gof2,...,gofk,...gof30}，gofkRepresenting a kth frame group obtained by segmenting any original point cloud video, wherein k is more than or equal to 1 and less than or equal to 30;
the kth frame group gofkSpatially divided into 12 slices of the same size, the kth frame group gofkThe c-th cut block obtained after cutting is marked as chunkk,c，1≤c≤12；
The c th cut chunk was cutk,cCompressed into 5 different quality level slices, where the kth frame group gofkMiddle c cut chunkk,cThe first quality grade cut obtained after compression is marked as chunkk,c,l(ii) a Thus obtaining compressed blocks of all frame groups and storing the compressed blocks in a server; l is more than or equal to 1 and less than or equal to 5;
step 2, the client collects the visual angle information of the original point cloud video watched by the user to obtain the cut block set watched by the user in each frame group, wherein the gof of the kth frame group watched by the userkIs noted as fovk(ii) a If the user is in the kth frame group gofkIn which the c-th cut chunk is viewedk,cThen the c th cut chunk is cutk,cAdding a cut block set fovk；
The client end watches the kth frame group gofkOf the collection of dices fovkAs predicted the user views the kth frame group gofkOf a slice set prefovk(ii) a Or the client side enables the user to watch the k-1 th frame group gofk-1Of the collection of dices fovk-1As predicted the user views the kth frame group gofkOf a slice set prefovkThereby completing the prediction of the view angle information;
step 3, the client selects the quality of the cut blocks in the user view angle according to the prediction result of the view angle information and the t-th network bandwidth, and the cut blocks in the view angle select the same quality grade during quality selection; t is 1, 2, 3, 4; the total number T of the bandwidth configuration types is 4;
step 3.1, obtaining the kth frame group gof in the user visual angle by using the formula (1)kQuality grade of dicing Lk：
Figure BDA0003133108500000081
s.t.
xk,l∈[0,1] (2)
Figure BDA0003133108500000082
In the formulae (1) to (3), xk,lIs zero-one variable and represents the kth frame group gofkWhether or not to select a slice with quality class l, if x k,l1, denotes the kth frame group gofkSelecting a slice with quality class l, if x k,l0 denotes the kth frame group gofkSelecting no cut block with quality grade of l; equation (3) indicates that there is one and only one quality class for the kth frame group;
step 3.2, the kth frame group gof is expressed by the formula (4) to the formula (8)kPause time Stall before streaming media playk：
Figure BDA0003133108500000083
Figure BDA0003133108500000084
Figure BDA0003133108500000085
Figure BDA0003133108500000086
Figure BDA0003133108500000087
S in formula (4)kIndicating the kth frame group gof within the user's viewkThe data size corresponding to the quality grade selected by the block; in the formula (4), Sk,lIndicating the kth frame group gof within the user's viewkThe data volume of the cut blocks under different quality levels;
formula (5) TwkIndicating the kth frame group gof within the user's viewkThe download time of the tile; in the formula (5), BkIndicating the kth frame group gofkA corresponding bandwidth;
formula (6) TekIndicating the kth frame group gof within the user's viewkDecoding time corresponding to the quality level selected by the slice; in formula (6), Tek,lIndicating the kth frame group gof within the user's viewkDecoding time at different quality levels;
formula (7) TpkRepresents a playback time of the k-th frame group; in the formula (7), f represents the kth frame group gofkFps is the number of point cloud frames played by the player per second;
equation (8) shows that if the total time required for downloading and decoding the kth intra view block is longer than the play time of the previous frame group, the difference between the two is the pause time; in the equation (8), since the number of frames of each frame group is the same, the playback time of a frame group previous to the kth frame group is represented by the playback time of the kth frame group; equation (8) represents that the buffering duration of the kth frame group is the playing time of the previous frame group;
step 3.3, obtaining the gof of the kth frame group by using the formula (9)kQuality switch occurring at streaming media playbackk：
Figure BDA0003133108500000091
Equation (9) indicates that the quality switch occurring when the kth frame group is played back in streaming media is the sum of the quality switches generated by each slice within the view angle; in the formula (9), the quality switching of the c-th slice in the k-th frame group compared with the c-th slice in the k-1 th frame group is represented by an absolute value of two level differences; in the formula (9), Lk,cIndicating the kth frame group gofkMiddle c cut chunkk,cQuality class of Lk-1,cIndicating the k-1 th frame group gofk-1Middle c cut chunkk-1,cQuality grade of (2); dk,cDenotes the c th cut chunkk,cAnd the impact of viewpoint distance on quality switching, and there are:
Figure BDA0003133108500000092
in the formula (10), Dk,cRepresenting the impact of the slice and viewpoint distance on the quality switch; dis (disease)k,cDenotes the c th cut chunkk,cTo the viewpoint distance of the user; boxsizek,cDenotes the c th cut chunkk,cThe kth frame group gofkThe length of the diagonal line of the boundary frame where the corresponding point cloud space is located;
step 3.4, when the client selects the quality, the zero-one variable x is corrected by using a zero-one integer programming methodk,lSolving to obtain the kth frame group gof of the point cloud video in the user view angle under the t-th network bandwidthkHighest quality grade of medium cut block
Figure BDA0003133108500000093
And minimum pause time
Figure BDA0003133108500000101
Further, the k frame group gof is obtainedkQuality switching occurring during streaming media playback
Figure BDA0003133108500000102
Step 4, according to the quality grade selected by the user within the view angle under the t-type network bandwidth, the client downloads the blocks of the corresponding quality grade from the server and re-fuses the blocks to obtain a processed point cloud video predicted by view angle information; predicting according to the quality grade selected by the block cutting in the user view angle and two view angles under 4 network bandwidths, thereby obtaining G-32 processed point cloud videos and the pause time generated when the G-32 processed point cloud videos are played in a streaming media, and switching the quality; g-32-4 × 2 × 4;
the specific configuration of the processed point cloud video is shown in table 1:
TABLE 1 Point cloud video configuration
Point cloud video Bandwidth configuration (Mb/s) View angle prediction
Longdress
70，120，230，340 Perfect prediction, nearest viewpoint
Loot
70，120，230，340 Perfect prediction, nearest viewpoint
Soldier 130，230，350，480 Perfect prediction, nearest viewpoint
Basketballplayer 130，230，350，480 Perfect prediction, nearest viewpoint
Step 5, the client evaluates the G processed point cloud videos through objective quality indexes;
step 5.1, finding the nearest plane corresponding to each point in the processed point cloud video in the original point cloud video, and evaluating the distance information of the G processed point cloud videos by using a formula (11) to obtain the objective quality PSNR (Peak Signal to noise ratio) containing the distance informationp：
Figure BDA0003133108500000103
In the formula (11), p2planeRMSDRepresenting the average value of the distance from each point in the processed point cloud video to the corresponding nearest plane in the original point cloud video; dMAXRepresenting the maximum value in the diagonal length of a boundary box where a point cloud space corresponding to each frame of an original point cloud video is located;
step 5.2, evaluating the color information of the G processed point cloud videos by using the formula (12) to obtain objective quality PSNR (Peak Signal to noise ratio) containing the color informationC：
Figure BDA0003133108500000104
PSNR in the formula (12)Y,PSNRU,PSNRVRespectively representing peak signal-to-noise ratios of a Y component, a U component and a V component in a YUV color space;
step 5.3, obtaining the total objective quality PCPSNR of the 32 processed point cloud videos by using the formula (13):
PCPSNR＝PSNRP+PSNRC (13)
step 6, the client renders and plays the processed point cloud videos with G being 32, and the E-th user scores the experience quality of the played 32 processed point cloud videos, so that 32 scores of all users with E being 34 are obtained; 1, 2, 34; e-34 represents the total number of users;
step 7, carrying out average processing on the scores of the g-th processed point cloud video by all the users with E-34 points to obtain the mean opinion score MOS of the g-th processed point cloud videog(ii) a Obtaining the average opinion scores MOS of the 32 processed point cloud videos, wherein G is 1, 2.
Step 8, taking the mean opinion score MOS as a dependent variable, taking the objective quality, the pause time and the quality switching as independent variables, and respectively making scatter diagrams of the mean opinion score MOS and the objective quality, the pause time and the quality switching; the scatter diagram is shown in fig. 2, and the relationship between the dependent variable MOS and each variable is observed, so that it can be seen that the three influencing factors and the MOS all have a relatively good linear relationship, and then a linear mathematical model conforming to the scatter diagram is selected to perform linear regression fitting on the dependent variable and the independent variable, so as to obtain a user QoE model of the point cloud video streaming media, as shown in formula (14):
QoE＝-0.471+0.037×PSNRC-0.313×Stall-0.007×Quality_switch (14)
PSNR in formula (14)CThe objective Quality of the point cloud video is represented, Stall represents pause generated when the point cloud video is transmitted in the streaming media, and Quality _ switch represents Quality switching generated when the point cloud video is transmitted in the streaming media.

Claims (3)

1. A modeling method for user experience quality QoE in adaptive point cloud video streaming media is characterized by comprising the following steps:
step one, a server acquires A original point cloud videos and divides each original point cloud video into K frame groups with the same frame number in time, wherein the K frame groups divided by any one original point cloud video are marked as { gof1,gof2,...,gofk,...gofK}，gofkRepresenting a kth frame group obtained by segmenting any original point cloud video, wherein K is more than or equal to 1 and less than or equal to K;
the kth frame group gofkSpatially divided into C slices of equal size, the kth frame group gofkThe c-th cut block obtained after cutting is marked as chunkk,c，1≤c≤C；
The c th cut chunk was cutk,cCompressed into L kinds of blocks with different quality levels, wherein the kth frame group gofkMiddle c cut chunkk,cThe first quality grade cut obtained after compression is marked as chunkk,c,l(ii) a Thus obtaining compressed blocks of all frame groups and storing the compressed blocks in a server; l is more than or equal to 1 and less than or equal to L;
step two, the client collects the visual angle information of the original point cloud video watched by the user to obtain the cut block set watched by the user in each frame group, wherein the gof of the kth frame group watched by the userkIs noted as fovk(ii) a If the user is in the k frameGroup gofkIn which the c-th cut chunk is viewedk,cThen the c th cut chunk is cutk,cAdding a cut block set fovk；
The client end watches the kth frame group gofkOf the collection of dices fovkAs predicted the user views the kth frame group gofkOf a slice set prefovk(ii) a Or the client side enables the user to watch the k-1 th frame group gofk-1Of the collection of dices fovk-1As predicted the user views the kth frame group gofkOf a slice set prefovkThereby completing the prediction of the view angle information;
thirdly, the client selects the quality of the cut blocks in the user view angle according to the prediction result of the view angle information and the t-th network bandwidth, calculates the quality grade selected by the cut blocks in the user view angle under the t-th network bandwidth, and calculates the pause time and quality switching generated when the point cloud video in the user view angle plays streaming media; t ═ 1, 2,. T; t represents the total number of network bandwidth classes;
step four, according to the quality grade selected by the user in the view angle under the t-type network bandwidth, the client downloads the blocks of the corresponding quality grade from the server and re-fuses the blocks to obtain a processed point cloud video predicted by the view angle information; thereby obtaining G processed point cloud videos and pause time generated when the G processed point cloud videos are played in a streaming media, and switching the quality; g ═ T × 2 × a;
evaluating the G processed point cloud videos through objective quality indexes by the client side, so as to obtain objective quality of the G processed point cloud videos;
step six, the client renders and plays the G processed point cloud videos, and the E-th user scores the experience quality of the played G processed point cloud videos, so that G scores of all E users are obtained; e1, 2,. E; e represents the total number of users;
step seven, carrying out mean processing on the scores of the g-th processed point cloud video by all E users to obtain the mean opinion score MOS of the g-th processed point cloud videog(ii) a Thereby obtainingG, mean opinion scores MOS of the processed point cloud videos, G being 1, 2.. G;
step eight, taking the mean opinion score MOS as a dependent variable, taking the objective quality, the pause time and the quality switching as independent variables, and respectively making scatter diagrams of the mean opinion score MOS and the objective quality, the pause time and the quality switching; and performing function fitting on the dependent variable and the independent variable to obtain a user QoE model of the point cloud video streaming media.
2. The modeling method of claim 1, wherein the third step is performed as follows:
3.1, selecting the cut blocks with the same quality grade in the user visual angle by the client;
step 3.2, obtaining the kth frame group gof in the user visual angle by using the formula (1)kQuality grade of dicing Lk：
Figure FDA0003133108490000021
s.t.
xk,l∈[0,1] (2)
Figure FDA0003133108490000022
In the formulae (1) to (3), xk,lIs zero-one variable and represents the kth frame group gofkWhether or not to select a slice with quality class l, if xk,l1, denotes the kth frame group gofkSelecting a slice with quality class l, if xk,l0 denotes the kth frame group gofkSelecting no cut block with quality grade of l;
step 3.3, the kth frame group gof is expressed by the formula (4) to the formula (8)kPause time Stall before streaming media playk：
Figure FDA0003133108490000031
Figure FDA0003133108490000032
Figure FDA0003133108490000033
Figure FDA0003133108490000034
Figure FDA0003133108490000035
In the formula (4), Sk,lIndicating the kth frame group gof within the user's viewkThe data volume of the cut blocks under different quality levels;
in the formula (5), BkIndicating the kth frame group gofkA corresponding bandwidth;
in formula (6), Tek,lIndicating the kth frame group gof within the user's viewkDecoding time at different quality levels;
in the formula (7), f represents the kth frame group gofkFps is the number of point cloud frames played by the player per second;
step 3.4, obtaining the gof of the kth frame group by using the formula (9)kQuality switch occurring at streaming media playbackk：
Figure FDA0003133108490000036
In the formula (9), Lk,cIndicating the kth frame group gofkMiddle c cut chunkk,cQuality class of Lk-1,cIndicating the k-1 th frame group gofk-1Middle c cut chunkk-1,cQuality class of Dk,cDenotes the c th cut chunkk,cAnd the impact of viewpoint distance on quality switching, and there are:
Figure FDA0003133108490000037
in the formula (10), Disk,cDenotes the c th cut chunkk,cTo the viewpoint distance of the user; boxsizek,cDenotes the c th cut chunkk,cThe kth frame group gofkThe length of the diagonal line of the boundary frame where the corresponding point cloud space is located;
step 3.5, when the client selects the quality, the zero-one variable x is corrected by using a zero-one integer programming methodk,lSolving to obtain the kth frame group gof of the point cloud video in the user view angle under the t-th network bandwidthkHighest quality grade of medium cut block
Figure FDA0003133108490000038
And minimum pause time
Figure FDA0003133108490000041
Further, the k frame group gof is obtainedkQuality switching occurring during streaming media playback
Figure FDA0003133108490000042
3. The modeling method of claim 1, wherein the fifth step is performed as follows:
step 5.1, finding the nearest plane corresponding to each point in the processed point cloud video in the original point cloud video, and evaluating the distance information of the G processed point cloud videos by using a formula (11) to obtain the objective quality PSNR (Peak Signal to noise ratio) containing the distance informationp：
Figure FDA0003133108490000043
In the formula (11), p2planeRMSDRepresenting the average value of the distance from each point in the processed point cloud video to the corresponding nearest plane in the original point cloud video; dMAXRepresenting the maximum value in the diagonal length of a boundary box where a point cloud space corresponding to each frame of an original point cloud video is located;
step 5.2, evaluating the color information of the G processed point cloud videos by using the formula (12) to obtain objective quality PSNR (Peak Signal to noise ratio) containing the color informationC：
Figure FDA0003133108490000044
PSNR in the formula (12)Y,PSNRU,PSNRVRespectively representing peak signal-to-noise ratios of a Y component, a U component and a V component in a YUV color space;
and 5.3, obtaining the total objective quality PCPSNR of the G processed point cloud videos by using the formula (13):
PCPSNR＝PSNRP+PSNRC (13)。
CN202110709824.6A 2021-06-25 2021-06-25 Modeling method for user experience quality QoE in adaptive point cloud video streaming media Active CN113395602B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110709824.6A CN113395602B (en) 2021-06-25 2021-06-25 Modeling method for user experience quality QoE in adaptive point cloud video streaming media

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110709824.6A CN113395602B (en) 2021-06-25 2021-06-25 Modeling method for user experience quality QoE in adaptive point cloud video streaming media

Publications (2)

Publication Number Publication Date
CN113395602A true CN113395602A (en) 2021-09-14
CN113395602B CN113395602B (en) 2022-03-22

Family

ID=77623861

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110709824.6A Active CN113395602B (en) 2021-06-25 2021-06-25 Modeling method for user experience quality QoE in adaptive point cloud video streaming media

Country Status (1)

Country Link
CN (1) CN113395602B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100316012A1 (en) * 2009-06-12 2010-12-16 Cygnus Broadband Systems and Methods for Intelligent Discard in a Communication Network
CN106454424A (en) * 2016-11-10 2017-02-22 中广热点云科技有限公司 Hot cloud management platform and method thereof
CN106604026A (en) * 2016-12-16 2017-04-26 浙江工业大学 Quality-of-experience (QoE) evaluation method of mobile streaming media user
CN107888579A (en) * 2017-11-06 2018-04-06 浙江大学 A kind of mobile video user experience quality index modeling method of non-interfering type
CN108696403A (en) * 2018-03-23 2018-10-23 中国科学技术大学 A kind of encrypted video QoE evaluating methods based on the study of network flow latent structure
CN109302623A (en) * 2018-11-01 2019-02-01 南京大学 A kind of transmission method based on QoE model dynamic self-adapting video
CN109451303A (en) * 2018-12-24 2019-03-08 合肥工业大学 A kind of modeling method for user experience quality QoE in VR video
CN109831705A (en) * 2019-02-22 2019-05-31 西安交通大学 A kind of subjective QoE appraisal procedure for HTTP video stream traffic
CN110246112A (en) * 2019-01-21 2019-09-17 厦门大学 Three-dimensional point cloud quality evaluating method in the room laser scanning SLAM based on deep learning
CN110266714A (en) * 2019-06-28 2019-09-20 合肥工业大学 VR video under a kind of QoE driving adaptively acquires and transmission method
CN111435991A (en) * 2019-01-11 2020-07-21 上海交通大学 Point cloud code stream packaging method and system based on grouping
CN112101788A (en) * 2020-09-15 2020-12-18 福州大学 Video QoE evaluation system and method based on long-term memory

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100316012A1 (en) * 2009-06-12 2010-12-16 Cygnus Broadband Systems and Methods for Intelligent Discard in a Communication Network
CN106454424A (en) * 2016-11-10 2017-02-22 中广热点云科技有限公司 Hot cloud management platform and method thereof
CN106604026A (en) * 2016-12-16 2017-04-26 浙江工业大学 Quality-of-experience (QoE) evaluation method of mobile streaming media user
CN107888579A (en) * 2017-11-06 2018-04-06 浙江大学 A kind of mobile video user experience quality index modeling method of non-interfering type
CN108696403A (en) * 2018-03-23 2018-10-23 中国科学技术大学 A kind of encrypted video QoE evaluating methods based on the study of network flow latent structure
CN109302623A (en) * 2018-11-01 2019-02-01 南京大学 A kind of transmission method based on QoE model dynamic self-adapting video
CN109451303A (en) * 2018-12-24 2019-03-08 合肥工业大学 A kind of modeling method for user experience quality QoE in VR video
CN111435991A (en) * 2019-01-11 2020-07-21 上海交通大学 Point cloud code stream packaging method and system based on grouping
CN110246112A (en) * 2019-01-21 2019-09-17 厦门大学 Three-dimensional point cloud quality evaluating method in the room laser scanning SLAM based on deep learning
CN109831705A (en) * 2019-02-22 2019-05-31 西安交通大学 A kind of subjective QoE appraisal procedure for HTTP video stream traffic
CN110266714A (en) * 2019-06-28 2019-09-20 合肥工业大学 VR video under a kind of QoE driving adaptively acquires and transmission method
CN112101788A (en) * 2020-09-15 2020-12-18 福州大学 Video QoE evaluation system and method based on long-term memory

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ZHI LIU等: "Fuzzy Logic-Based Adaptive Point Cloud Video Streaming", 《IEEE OPEN JOURNAL OF THE COMPUTER SOCIETY》 *
孙伟等: "基于服务质量的无线传感器网络MAC协议模型研究", 《电子测量与仪器学报》 *

Also Published As

Publication number Publication date
CN113395602B (en) 2022-03-22

Similar Documents

Publication Publication Date Title
Bampis et al. Towards perceptually optimized end-to-end adaptive video streaming
Duanmu et al. A quality-of-experience index for streaming video
Bampis et al. Towards perceptually optimized adaptive video streaming-a realistic quality of experience database
CN107454446A (en) Video frame management method and its device based on Quality of experience analysis
Venkataraman et al. Evaluating quality of experience for streaming video in real time
Aguiar et al. Video quality estimator for wireless mesh networks
Issa et al. Quality-of-experience perception for video streaming services: Preliminary subjective and objective results
Yim et al. Subjective quality assessment for youtube ugc dataset
CN112055263B (en) 360-degree video streaming transmission system based on significance detection
Li et al. UGC-VIDEO: perceptual quality assessment of user-generated videos
Usman et al. A novel no-reference metric for estimating the impact of frame freezing artifacts on perceptual quality of streamed videos
CN114598864A (en) Full-reference ultrahigh-definition video quality objective evaluation method based on deep learning
CN113395602B (en) Modeling method for user experience quality QoE in adaptive point cloud video streaming media
Li et al. A real-time blind quality-of-experience assessment metric for http adaptive streaming
CN112767385B (en) No-reference image quality evaluation method based on significance strategy and feature fusion
CN107465914B (en) Method for evaluating video quality based on Local textural feature and global brightness
Srinivasan et al. On the robustness of action recognition methods in compressed and pixel domain
Martinez et al. A no-reference autoencoder video quality metric
Vranješ et al. ECVQ and EVVQ video quality databases
CN111277899B (en) Video quality evaluation method based on short-term memory and user expectation
Ebenezer et al. Subjective and objective quality assessment of high-motion sports videos at low-bitrates
Søgaard et al. Quality assessment of adaptive bitrate videos using image metrics and machine learning
Al-Juboori et al. Content characterization for live video compression optimization
Mizdos et al. Subjective and objective evaluation of spatial and temporal information of video content
Lamichhane et al. A no reference deep learning based model for quality assessment of UGC videos

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant