CN103873854A - Method for determining number of stereoscopic image subjective assessment testees and experiment data - Google Patents

Abstract

The invention discloses a method for determining the number of stereoscopic image subjective assessment testees and experiment data, and relates to the field of stereoscopic image quality subjective assessment. The method comprises the following steps of through analyzing preprocessing data, determining the number of the subjective assessment testees by perverting a confidence interval formula to shorten an experiment cycle and reduce the cost; performing a subjective assessment experiment by utilizing the determined reasonable number of qualified testees, including the processes of establishing a stereoscopic image database, selecting reasonable testing environmental conditions, selecting the testees according to standards and training and testing the testees; lastly, according to the image quality subjective assessment standard regulation and the Grubbs' test method, eliminating abnormal testees and reasonably screening abnormal data in experiment data to avoid the influence of the testees, the experiment environment and other uncertain factors on the accuracy of an experiment result. The method is based on stereoscopic image quality subjective assessment, and the accuracy of the experiment data result is improved from the angle of statistics.

Description

A kind of definite method of stereo-picture subjective assessment subject's quantity and experimental data

Technical field

The present invention relates to stereo-picture field, relate in particular to a kind of definite method of stereo-picture subjective assessment subject's quantity and experimental data.

Background technology

At present, stereoscopic imaging technology is by industry-by-industry extensive use.This technology comprises collection, compression, transmission, processing and the demonstration of stereo-picture, therefore, how to utilize stereo image quality evaluation method to weigh the image lesion degree that three-dimensional imaging produces in these processes, become one of the study hotspot in stereoscopic imaging technology field.Stereo image quality evaluation is divided into subjective assessment and objective evaluation two classes, and with respect to method for objectively evaluating, subjective evaluation method is directly to utilize people to pass judgment on stereo image quality, can reflect more intuitively, more accurately the real quality of stereo-picture.Subjective quality assessment must pass through all too many levels, mainly comprises screening and the processing etc. of selection, subject's selected, the training of foundation, the test environment conditions in stereoscopic image data storehouse and testing process, experimental data.But the factors such as the uncertainty of experimental situation and subject's state, all can directly affect accuracy and the accuracy of subjective assessment data, meanwhile, subject's quantity can directly affect experiment test time and subjective experiment cost.Therefore, the method for the definite method of a set of practicable subject's quantity and experimental data screening proposed most important.

In recent years, existing Chinese scholars starts stereoscopic image quality subjective evaluation scheme and studies.The ITU of International Telecommunication Union has proposed three-dimensional television image subjective assessment standard I TU-R BT.1438 on the basis of plane picture quality subjective evaluation standard I TU-R BT.500-11 ^[1], experimental situation, test source, subject and testing process have been carried out to simple and general introduction; Document [2], [3], [4] are from watching condition, watching the aspect such as personnel, method of testing definition subjective assessment scheme; In the subjective evaluation method that document [5] proposes, set up and comprised compressed and add the stereo-picture storehouse of making an uproar and degrading; Document [6] has been analyzed the various factors that affects three-dimensional perceived quality from subjective aspect, and has discussed the feature in existing each stereoscopic image data storehouse; Document [7] from setting up stereoscopic image data storehouse, subject's selection, experiment condition, a more perfect stereo image quality subjective assessment scheme has been set up in subject's the aspect such as training and test.Different from stereo image quality subjective assessment scheme, document [8] has been realized the mutual three-dimensional video-frequency quality subjective testing system of a personal-machine by software programming; In the three-dimensional video-frequency quality subjective evaluation method that document [9] proposes, adopt SPSS instrument to carry out data processing and comparison.In addition, stereo image quality subjective evaluation method is often used for each factor and characteristic that research is relevant to three-dimensional imaging, and document [10] has been inquired into right view quality and the impact of disparity map quality on depth perception by subjective experiment; Document [11] is inquired into the relation of brightness and three-dimensional perception by subjective experiment.

Inventor finds realizing in process of the present invention, and the major defect of prior art is as follows:

At present, subjective assessment scheme for stereo-picture there is no unified standard, in the stereo image quality subjective assessment scheme of setting up or use, all the screening technique of the definite method of subject's quantity and experimental data is not made to concrete regulation simultaneously, and the accuracy of subject's quantity and experimental data has directly determined the accuracy of complexity, experimental cost and the experimental result of subjective experiment, it is also the key factor that affects subjective assessment score value.

Summary of the invention

A kind of definite method that the invention provides stereo-picture subjective assessment subject's quantity and experimental data, this method has improved the correctness with subjective assessment result, described below:

A definite method for stereo-picture subjective assessment subject's quantity and experimental data, said method comprising the steps of:

(1) choose n name subject and carry out stereo image quality and evaluate in advance, obtain subjective assessment mark, calculating mean value and standard deviation;

(2), according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains subject's quantity N of subjective experiment actual needs;

(3) choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

(4) whether meet normal distribution according to every width test pattern evaluation score, adopt the different abnormal threshold value p of judge mark _iand q _i, calculate the index of passing judgment on abnormal data

with by relatively, eliminate abnormal subject;

(5) adopt Grubbs' test method to filter out the abnormal data in every width test pattern evaluation score;

(6) utilize and do not have abnormal subject and data after screening, the whole normally mean values of experimental datas that calculate every width test pattern obtain final subjective assessment result.

Described subject's quantity is definite to be specially:

N＝(t ²S ²)/(r ²mean ²) （1）

Wherein, r represents acceptable relative percentage deviation, mean, S represent that respectively a small amount of subject carries out mean value and the standard deviation of the pre-subjective assessment mark of evaluating to single width stereo image quality, t gets initial value 1.96, substitution formula (1) calculates the initial value of subject's quantity N, then uses t value (looking into t distribution table can obtain) the substitution formula (1) corresponding to N initial value to calculate N value, constantly iteration, until the N value calculating no longer changes, i.e. subject's quantity.

The abnormal data that described employing Grubbs' test method filters out in every width test pattern evaluation score is specially:

First, the evaluation score of every width test pattern is arranged in ascending order mode; According to formula (2) and (3) calculating parameter α _iand γ _i,

α _i＝(x _imax-u _i)/S _i （2）

γ _i＝(u _i-x _imin)/S _i （3）

Wherein, x _imaxand x _iminrepresent respectively maximum and minimum value in whole evaluation score of i width test pattern; u _iand S _irepresent respectively mean value and the standard deviation of i width test pattern evaluation score; Finally, obtain by looking into tables of critical values the critical value G that corresponding subject's quantity and confidence level are 95%, if α _i> G, eliminates abnormal data x _imaxif, γ _i> G, eliminates abnormal data x _imin; Finally, repeat above-mentioned checking procedure until no longer there is abnormal data in experimental data.

The beneficial effect of technical scheme provided by the invention is: stereo image quality subjective assessment scheme that this method is perfect, compared with prior art, this method has been studied emphatically the method that subject's quantity is determined and experimental data is screened.Experimental result shows, this method is practical, has both ensured the reliability of experimental result, has saved again experimental resources, has improved the correctness with subjective assessment result.

Brief description of the drawings

Fig. 1 is the left and right viewpoint of girl reference picture;

Fig. 2 is the left and right viewpoint of family reference picture;

Fig. 3 is the left and right viewpoint of boy reference picture;

Fig. 4 is the left and right viewpoint of flower reference picture;

Fig. 5 is the left and right viewpoint of tree reference picture;

Fig. 6 is the left and right viewpoint of river reference picture;

Fig. 7 is stereo image quality subjective assessment scheme flow chart;

Fig. 8 is 3D WINDOWS-19A0 type Computerized 3 D imaging device.

Embodiment

For making the object, technical solutions and advantages of the present invention clearer, below embodiment of the present invention is described further in detail.

In order to improve and the correctness of subjective assessment result, the embodiment of the present invention provides a kind of definite method of stereo-picture subjective assessment subject's quantity and experimental data, with reference to the left and right viewpoint of stereo-picture, as shown in Fig. 1～6, subjective assessment scheme flow process is referring to Fig. 7, described below:

101: choose n name subject and carry out stereo image quality and evaluate in advance, obtain subjective assessment mark, calculating mean value and standard deviation;

First, utilize comparatively perfect stereo image quality subjective assessment system to carry out pretest to a small amount of subject of n name who chooses, comprise set up stereoscopic image data storehouse, determine watch condition, according to the selected subject of standard, train and test ^[7], finally obtain n name subject single width stereo image quality carried out to the pre-subjective assessment mark { x evaluating ₁, x ₂... x _n, calculate respectively their average and standard deviation according to formula (1) and formula (2),

$\mathrm{mean}=\frac{1}{n}{\mathrm{\Σ}}_{i=1}^n{x}_i---\left(1\right)$

$S=\sqrt{\frac{1}{n-1}{\mathrm{\Σ}}_{i=1}^n{(x_i-\mathrm{mean})}^2}---\left(2\right)$

102: according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains subject's quantity N of subjective experiment actual needs;

Calculate subject's quantity N according to formula (3),

N＝(t ²S ²)/(r ²mean ²) （3）

Wherein, r represents acceptable relative percentage deviation, and mean, S represent that respectively a small amount of subject carries out mean value and the standard deviation of the pre-subjective assessment mark of evaluating to single width stereo image quality; First t gets initial value 1.96, and substitution formula (3) calculates the initial value of subject's quantity N, then, use again t value (looking into t distribution table can obtain) the substitution formula (3) corresponding to N initial value to calculate N value, constantly iteration, last, until the N value calculating no longer changes, i.e. subject's quantity.

103: choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

First, suppose common use I width test pattern, set the subject N name that participates in subjective experiment according to above-mentioned calculating, again test x by comparatively perfect stereo image quality subjective assessment system _i,jrepresent the evaluation score of j name subject to i width test pattern, calculate mean value and the standard deviation of every width test pattern evaluation score, as formula (4) and formula (5),

$u_i=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{x}_{i,j}---\left(4\right)$

$S_i=\sqrt{\frac{1}{N-1}{\mathrm{\Σ}}_{j=1}^N{(u_i-x_{i,j})}^2}---\left(5\right)$

Wherein, u _iand S _irepresent respectively mean value and the standard deviation of i width test pattern evaluation score.

104: whether meet normal distribution according to every width test pattern evaluation score, adopt the different abnormal threshold value p of judge mark _iand q _i, calculate the index of passing judgment on abnormal data

with by relatively, eliminate abnormal subject;

First,, respectively according to second moment and the Fourth-order moment of formula (6) and (7) calculating i width test pattern evaluation score, draw the coefficient of kurtosis of i width test pattern evaluation score, as formula (8), according to the requirement of ITU-R BT.500-11, if 2≤β _2i≤ 4, the evaluation result of i width test pattern is normal distribution; Otherwise, be Non-Gaussian Distribution.

$m_{2i}=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{(x_{i,j}-u_i)}^2---\left(6\right)$

$m_{4i}=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{(x_{i,j}-u_i)}^4---\left(7\right)$

${\mathrm{\β}}_{2i}=m_{4i}/m_{2i}^2---\left(8\right)$

For every width test pattern, calculate subject and pass judgment on the whether abnormal threshold value p of mark _iand q _iif evaluation score meets normal distribution, utilize formula (9) and (10) to calculate; Otherwise, utilize formula (11) and (12) to calculate,

p _i＝u _i+2S _i （9）

q _i＝u _i-2S _i （10）

$p_i=u_i+\sqrt{20}{S}_i---\left(11\right)$

$q_i=u_i-\sqrt{20}{S}_i---\left(12\right)$

Then, j name subject is classified to the judge mark of all test patterns, obtain passing judgment on the measuring parameter P of two class situations _jand Q _j.Establish initial value and be 0, by whole evaluation score x of j name subject _i,j(i=1,2 ..., I) respectively with the p of corresponding test pattern _i, q _icompare; If the evaluation score of i width test pattern meets normal distribution, according to formula (9) and (10) judgement, if x _i,j>=p _i, P _j=P _j+ 1, if x _i,j≤ q _i, Q _j=Q _j+ 1; Otherwise, according to formula (11) and (12) judgement.

Finally, calculate according to formula (13) and formula (14) index of passing judgment on abnormal data

with

if and

eliminate all evaluating datas of j name subject, otherwise, evaluating data retained.

$R_j^1=\frac{1}{I}(P_j+Q_j)---\left(13\right)$

$R_j^2=\left|\frac{P_j-Q_j}{P_j+Q_j}\right|---\left(14\right)$

105: adopt Grubbs' test method to filter out the abnormal data in every width test pattern evaluation score;

First, the evaluation score of every width test pattern is arranged in ascending order mode, show that maximum and the minimum value in whole evaluation score of i width test pattern is x _imaxand x _imin, according to formula (15) and (16) calculating parameter α _iand γ _i, wherein, u _iand S _irepresent respectively mean value and the standard deviation of i width test pattern evaluation score.

α _i＝(x _imax-u _i)/S _i （15）

γ _i＝(u _i-x _imin)/S _i （16）

Finally, obtain by looking into tables of critical values the critical value G that corresponding subject's quantity and confidence level are 95%, if α _i> G, eliminates abnormal data x _imaxif, γ _i> G, eliminates abnormal data x _imin.Afterwards, repeat above-mentioned checking procedure until no longer there is abnormal data in experimental data.

106: utilize not have abnormal subject and data after screening, the whole normally mean values (being MOS value) of experimental datas that calculate every width test pattern obtain final subjective assessment result.

The mean value (being MOS value) that calculates whole normal experimental datas of every width test pattern according to formula (17) obtains final subjective assessment result.

${\mathrm{MOS}}_i=\frac{1}{K}{\mathrm{\Σ}}_{j=1}^K{x}_{i,j}---\left(17\right)$

Wherein, MOS _irepresent the subjective assessment value of i width image, K represents final normal subject's quantity, x _i,jrepresent the evaluation score of j name subject to i width image.

In order to verify that subject's quantity that this method proposes determines algorithm and experimental data filtering algorithm, this method has been carried out subjective assessment to the image of JPEG compression artefacts and random noise, Gaussian noise distortion, simply introduces experimentation below:

This method has been chosen 48 width test patterns for determining the pretest of subject's quantity, comprising girl, family, boy, flower, tree and six scenes of river, they are respectively through 85%, 65%, 45%, 25%, 15%, 5% JPEG compression and add random noise, Gaussian noise; Six width are with reference to stereo-picture left and right visual point image as shown in Fig. 1～6, and experimental data is provided by University Of Tianjin's broadband wireless communications and three-dimensional imaging research institute.According to ITU-R BT.1438[1] standard regulation, experiment adopts DSIS (Double-Stimulus Impairment Scale, two stimulation damage scaling laws), recruit 15 visual performances and stereoscopic vision all normal subject participate in pretest, evaluation equipment is 3DWINDOWS-19A0(19 inch, 1280 × 1024), as Fig. 8, laboratory brightness is for a little less than extremely, according to DSIS standards of grading, the quality of distorted image is given a mark, the total time of each subject's training and testing can not more than 30 minutes.

Result verification:

The present embodiment determines that by subject's quantity of utilizing this patent introduction algorithm calculates subject's quantity (concrete outcome is in table 1 and table 2) that different test patterns need, wherein, setting confidence level is 95%, because the evaluation score fluctuation of second-rate image is larger, therefore the percentage deviation the accepted r that average ratings mark is greater than to 3 pretest image is made as 5%, and other image is made as 8%.

Table 1 and table 2 shown respectively through JPEG compression and added the some experimental data of the test scene of noise, known through observing, and subject's quantity and scene content, image processing method formula are relevant with image lesion degree.Determine the calculating of algorithm by subject's quantity, the stereoscopic image data storehouse of setting up for this patent, select 30 subjects to participate in subjective assessment, both met the requirement of most test patterns to subject's quantity, ensure accuracy and the reliability of experimental result, again can be because of the excessive waste that causes experimental resources of subject's quantity.For the correctness of further checking this method and 30 subjects can guarantee the accuracy of this experimental result, this method increases subject's quantity to 35 partial test image is carried out to subjective assessment, find that experimental result and 30 subject's measured results are very approaching, specifically in table 1 and table 2.

Table 1 is about the proof of algorithm result of compressed image

Table 2 is about the proof of algorithm result that adds the image of making an uproar

Eliminate abnormal, suspicious subject and experimental data by this method, thereby make experimental result more accurately, reliably.In order to verify correctness and the validity of this experimental data screening technique, the present embodiment compares the final appraisal results MOS value of subjective experiment and the subject's that is eliminated part evaluation score, specifically in table 3.Observe known, subject's the evaluation score that is eliminated and the final MOS value of subjective experiment all differ more than 1.5 points, depart from most of subjects' evaluation score scope, will affect the accuracy of subjective assessment result, thereby verified the correctness of this patent experimental data screening technique.Therefore, this method can correctly be determined subject's quantity and screening experiment data, has ensured the reliability of experimental result, has saved experimental resources, eliminates abnormal, suspicious experiment value, thereby ensures the consistency of experimental result.

Table 3 is about the result of data screening algorithm

List of references:

[1]International Telecommunication Union(ITU),Recommendation ITU-R BT.1438,Subjective assessment of stereoscopic television pictures[S],2000.

[2]W.J.Tam,G.Alain,L.Zhang,et al.Smoothing depth maps for improved stereoscopic image quality[C].Three-Dimensional TV,Video and DisplayⅢ,Philadelphia,PA,USA,2004.

[3]Xu Wang,Mei Yu,You Yang,et al.Research on subjective stereoscopic image quality assessment[C].Multimedia Content Access:Algorithms and SystemsⅢ,San Jose,CA,United States,2009.

[4]CHENG Yu-qing and JIANG Xiu-hua.Latest research result of subjective and objective evaluation methodology for the video quality of3DTV[J].Journal of communication university of China(science and technology).2012,19(1):31-36.

[5]Junming Zhou,Gangyi Jiang and Xiangying Mao.Subjective quality analyses of stereoscopic images in3DTV system[C],IEEE Visual Communications and Image Processing(VCIP),Tainan:2011,1～4

[6] Zhou Wujie, Yu Mei, Zhou Junming, etc., stereo image quality evaluation method research [C], China's Telecommunication's international conference in 2010, Nanning: 2010

[7] Zhang Yingjing, Li Sumei, Wei Jinjin, etc., the subjective assessment scheme [J] of stereo image quality, photon journal, 2012,41(5): 602～607

[8]NukhetOzbek,GizemErtan and OktayKarakus.Interactive quality assessment for asymmetric coding of3D video[C],3DTV Conference:The True Vision-Capture,Transmission and Display of3D Video(3DTV-CVON),Antalya:2011,1～4

[9]A.S.Umar,R.M.Swash and A.H.Sadka,Subjective quality assessment of3D videos[C],2011AFRICON,Livingstone,Zambia:2011:1～6

[10]ChaohuiLv,Jingwei Huang and YinghuaShen,Subjective assessment of noised stereo images[C],International Conference on Multimedia Technology(ICMT),Hangzhou,China:2011,783～785

[11]Mahsa T.Pourazad,Zicong Mai,PanosNasiopoulos,et al,Effect of brightness on the quality of visual3D perception[C],201118th IEEE International Conference on Image Processing(ICIP),Brussels:2011,989～992

It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, the invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.

The foregoing is only preferred embodiment of the present invention, in order to limit the present invention, within the spirit and principles in the present invention not all, any amendment of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (3)

1. a definite method for stereo-picture subjective assessment subject quantity and experimental data, is characterized in that, said method comprising the steps of:

(1) choose n name subject and carry out stereo image quality and evaluate in advance, obtain subjective assessment mark, calculating mean value and standard deviation;

(2), according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains subject's quantity N of subjective experiment actual needs;

(3) choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

(4) whether meet normal distribution according to every width test pattern evaluation score, adopt the different abnormal threshold value p of judge mark _iand q _i, calculate the index of passing judgment on abnormal data

with

by relatively, eliminate abnormal subject;

(5) adopt Grubbs' test method to filter out the abnormal data in every width test pattern evaluation score;

(6) utilize and do not have abnormal subject and data after screening, the whole normally mean values of experimental datas that calculate every width test pattern obtain final subjective assessment result.

2. definite method of a kind of stereo-picture subjective assessment subject's quantity according to claim 1 and experimental data, is characterized in that, described subject's quantity is definite to be specially:

N＝(t ²S ²)/(r ²mean ²) （1）

Wherein, r represents acceptable relative percentage deviation, mean, S represent that respectively a small amount of subject carries out mean value and the standard deviation of the pre-subjective assessment mark of evaluating to single width stereo image quality, t gets initial value 1.96, substitution formula (1) calculates the initial value of subject's quantity N, then uses corresponding to the t value substitution formula (1) of N initial value and calculate N value, constantly iteration, until the N value calculating no longer changes, i.e. subject's quantity.

3. definite method of a kind of stereo-picture subjective assessment subject's quantity according to claim 1 and experimental data, is characterized in that, the abnormal data that described employing Grubbs' test method filters out in every width test pattern evaluation score is specially:

First, the evaluation score of every width test pattern is arranged in ascending order mode; According to formula (2) and (3) calculating parameter α _iand γ _i,

α _i＝(x _imax-u _i)/S _i （2）

γ _i＝(u _i-x _imin)/S _i （3）

Wherein, x _imaxand x _iminrepresent respectively maximum and minimum value in whole evaluation score of i width test pattern; u _iand S _irepresent respectively mean value and the standard deviation of i width test pattern evaluation score; Finally, obtain by looking into tables of critical values the critical value G that corresponding subject's quantity and confidence level are 95%, if α _i> G, eliminates abnormal data x _imaxif, γ _i> G, eliminates abnormal data x _imin; Finally, repeat above-mentioned checking procedure until no longer there is abnormal data in experimental data.

Images