CN103873854B - The defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data - Google Patents

Abstract

The invention discloses the defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data, relate to stereo image quality subjective assessment field, the method comprises: by analyzing preprocessed data, inverse confidential interval formula determination subjective assessment subject quantity, thus shorten experimental period, cost-saving; The qualified subject of the above-mentioned fair amount determined is utilized to carry out subjective assessment, comprise the flow process set up stereoscopic image data storehouse, select reasonable test environment conditions, select subject according to standard, implement testee's training and test, finally, according to picture quality subjective assessment standard regulation and Grubbs' test method, eliminate abnormal subject and abnormal data rationally in screening experiment data, avoid the impact of the uncertain factor such as subject and experimental situation on experimental result accuracy.This method, on the basis of stereo image quality subjective assessment, from the angle of probability statistics, improves the accuracy of experimental data.

Description

The defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data

Technical field

The present invention relates to stereo-picture field, particularly relate to the defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data.

Background technology

At present, stereoscopic imaging technology is by industry-by-industry extensive use.This technology comprises the collection of stereo-picture, compression, transmission, process and display, therefore, the image lesion degree how utilizing stereo image quality evaluation method measurement three-dimensional imaging to produce in these processes, has become one of the study hotspot in stereoscopic imaging technology field.Stereo image quality evaluation is divided into subjective assessment and objective evaluation two class, and relative to method for objectively evaluating, subjective evaluation method directly utilizes people to pass judgment on stereo image quality, can reflect the real quality of stereo-picture more intuitively, more accurately.Subjective quality assessment has to pass through all too many levels, mainly comprises the foundation in stereoscopic image data storehouse, the selection of test environment conditions, selected, the training of subject and testing process, the screening of experimental data and process etc.But the factors such as the uncertainty of experimental situation and subject's state, all directly can affect accuracy and the accuracy of subjective assessment data, meanwhile, the quantity of subject can directly affect experiment test time and subjective experiment cost.Therefore, the method proposing method that a set of practicable subject's quantity determines and experimental data screening is most important.

In recent years, existing Chinese scholars starts stereoscopic image quality subjective evaluation scheme and studies.International Telecommunication Union ITU proposes three-dimensional television image subjective assessment standard I TU-R BT.1438 on the basis of plane picture quality subjective evaluation standard I TU-R BT.500-11 ^[1], simple and general introduction has been carried out to experimental situation, test source, subject and testing process; Document [2], [3], [4] are from viewing condition, the viewing aspect such as personnel, method of testing definition subjective assessment scheme; Establish in the subjective evaluation method that document [5] proposes and comprise compressed and add the stereo-picture storehouse of making an uproar and degrading; Document [6] analyzes the various factors affecting three-dimensional perceived quality from subjective aspect, and discusses the feature in each stereoscopic image data storehouse existing; Document [7] from set up stereoscopic image data storehouse, the selection of subject, experiment condition, subject the aspect such as training and test establish a more perfect stereo image quality subjective assessment scheme.With stereo image quality subjective assessment scheme unlike, document [8] achieves the mutual stereoscopic video quality subjective testing system of a personal-machine by software programming; SPSS instrument is adopted to carry out data processing and compare in the stereoscopic video quality subjective evaluation method that document [9] proposes.In addition, stereo image quality subjective evaluation method is often used for studying each factor and the characteristic relevant to three-dimensional imaging, and document [10] has inquired into right view quality and disparity map quality to the impact of depth perception by subjective experiment; Document [11] inquires into the relation of brightness and three-dimensional perception by subjective experiment.

Inventor is realizing finding in process of the present invention, and the major defect of prior art is as follows:

At present, subjective assessment scheme for stereo-picture there is no unified standard, the method simultaneously all do not determined subject's quantity in the stereo image quality subjective assessment scheme set up or use and the screening technique of experimental data make concrete regulation, and the accuracy of subject's quantity and experimental data directly determines the accuracy of the complexity of subjective experiment, experimental cost and experimental result, is also the key factor affecting subjective assessment score value.

Summary of the invention

The invention provides the defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data, this method improves the correctness with subjective evaluation result, described below:

A defining method for stereo-picture subjective assessment subject's quantity and experimental data, said method comprising the steps of:

(1) choose n name subject and carry out stereo image quality pre assessment, obtain subjective assessment mark, calculating mean value and standard deviation;

(2) according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains the subject quantity N of subjective experiment actual needs;

(3) choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

(4) whether meet normal distribution according to every width test pattern evaluation score, adopt the threshold value p of different judge mark exceptions _iand q _i, calculate the index passing judgment on abnormal data with by comparing, eliminate abnormal subject;

(5) employing Grubbs' test method filters out the abnormal data in every width test pattern evaluation score;

(6) there are not abnormal subject and data after utilizing screening, the mean value calculating whole normally experimental datas of every width test pattern obtains final subjective evaluation result.

Described subject's quantity is determined to be specially:

N＝(t ²S ²)/(r ²mean ²) （1）

Wherein, r represents acceptable percentage deviation, mean, S represent that a small amount of subject carries out mean value and the standard deviation of the subjective assessment mark of pre assessment to single stereoscopic image quality respectively, t gets initial value 1.96, substitute into the initial value that formula (1) calculates subject quantity N, then substitute into formula (1) calculating N value by the t value (looking into t distribution table can obtain) corresponding to N initial value, continuous iteration, until the N value calculated no longer changes, i.e. subject's quantity.

The abnormal data that described employing Grubbs' test method filters out in every width test pattern evaluation score is specially:

First, the evaluation score of every width test pattern is arranged in ascending order mode; According to formula (2) and (3) calculating parameter α _iand γ _i,

α _i＝(x _imax-u _i)/S _i（2）

γ _i＝(u _i-x _imin)/S _i（3）

Wherein, x _imaxand x _iminrepresent the maximum in whole evaluation score of the i-th width test pattern and minimum value respectively; u _iand S _irepresent mean value and the standard deviation of the i-th width test pattern evaluation score respectively; Finally, by looking into tables of critical values and obtain corresponding subject's quantity and confidence level being the critical value G of 95%, if α _i> G, then eliminate abnormal data x _imaxif, γ _i> G, then eliminate abnormal data x _imin; Finally, above-mentioned checking procedure is repeated until no longer there is abnormal data in experimental data.

The beneficial effect of technical scheme provided by the invention is: stereo image quality subjective assessment scheme that this method is perfect, and compared with prior art, this method have studied emphatically subject's quantity and determines the method with experimental data screening.Experimental result shows, this method is practical, both ensure that the reliability of experimental result, has in turn saved experimental resources, improves the correctness with subjective evaluation result.

Accompanying drawing explanation

Fig. 1 is the left and right viewpoint of girl reference picture;

Fig. 2 is the left and right viewpoint of family reference picture;

Fig. 3 is the left and right viewpoint of boy reference picture;

Fig. 4 is the left and right viewpoint of flower reference picture;

Fig. 5 is the left and right viewpoint of tree reference picture;

Fig. 6 is the left and right viewpoint of river reference picture;

Fig. 7 is stereo image quality subjective assessment protocol procedures figure;

Fig. 8 is 3D WINDOWS-19A0 type Computerized 3 D imaging device.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly, below embodiment of the present invention is described further in detail.

In order to improve the correctness with subjective evaluation result, embodiments provide the defining method of a kind of stereo-picture subjective assessment subject's quantity and experimental data, as shown in figs. 1 to 6, subjective assessment protocol procedures is see Fig. 7, described below for the left and right viewpoint of reference stereo-picture:

101: choose n name subject and carry out stereo image quality pre assessment, obtain subjective assessment mark, calculating mean value and standard deviation;

First, utilize comparatively perfect stereo image quality subjective assessment system to carry out pretest to a small amount of subject of n name chosen, comprise and set up stereoscopic image data storehouse, determine viewing condition, select subject according to standard, carry out training and test ^[7], finally obtain n name subject carries out pre assessment subjective assessment mark { x to single stereoscopic image quality ₁, x ₂... x _n, their average and standard deviation is calculated respectively according to formula (1) and formula (2),

$\mathrm{mean}=\frac{1}{n}{\mathrm{\Σ}}_{i=1}^n{x}_i---\left(1\right)$

$S=\sqrt{\frac{1}{n-1}{\mathrm{\Σ}}_{i=1}^n{(x_i-\mathrm{mean})}^2}---\left(2\right)$

102: according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains the subject quantity N of subjective experiment actual needs;

Subject quantity N is calculated according to formula (3),

N＝(t ²S ²)/(r ²mean ²) （3）

Wherein, r represents acceptable percentage deviation, and mean, S represent that a small amount of subject carries out mean value and the standard deviation of the subjective assessment mark of pre assessment to single stereoscopic image quality respectively; First t gets initial value 1.96, substitutes into the initial value that formula (3) calculates subject quantity N, then, substitute into formula (3) by the t value (looking into t distribution table can obtain) corresponding to N initial value again and calculate N value, continuous iteration, finally, until the N value calculated no longer changes, i.e. subject's quantity.

103: choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

First, assuming that common use I width test pattern, participate in the subject N name of subjective experiment according to above-mentioned calculating setting, again tested by comparatively perfect stereo image quality subjective assessment system, x _i,jrepresent that jth name subject is to the evaluation score of the i-th width test pattern, calculates mean value and the standard deviation of every width test pattern evaluation score, as formula (4) and formula (5),

$u_i=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{x}_{i,j}---\left(4\right)$

$S_i=\sqrt{\frac{1}{N-1}{\mathrm{\Σ}}_{j=1}^N{(u_i-x_{i,j})}^2}---\left(5\right)$

Wherein, u _iand S _irepresent mean value and the standard deviation of the i-th width test pattern evaluation score respectively.

104: whether meet normal distribution according to every width test pattern evaluation score, adopt the threshold value p of different judge mark exceptions _iand q _i, calculate the index passing judgment on abnormal data with by comparing, eliminate abnormal subject;

First, calculate second moment and the Fourth-order moment of the i-th width test pattern evaluation score respectively according to formula (6) and (7), draw the coefficient of kurtosis of the i-th width test pattern evaluation score, as formula (8), according to the requirement of ITU-R BT.500-11, if 2≤β _2i≤ 4, then the evaluation result of the i-th width test pattern is normal distribution; Otherwise, be Non-Gaussian Distribution.

$m_{2i}=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{(x_{i,j}-u_i)}^2---\left(6\right)$

$m_{4i}=\frac{1}{N}{\mathrm{\Σ}}_{j=1}^N{(x_{i,j}-u_i)}^4---\left(7\right)$

${\mathrm{\β}}_{2i}=m_{4i}/m_{2i}^2---\left(8\right)$

For every width test pattern, calculate subject and pass judgment on the whether abnormal threshold value p of mark _iand q _iif evaluation score meets normal distribution, then formula (9) and (10) are utilized to calculate; Otherwise, utilize formula (11) and (12) to calculate,

p _i＝u _i+2S _i（9）

q _i＝u _i-2S _i（10）

$p_i=u_i+\sqrt{20}{S}_i---\left(11\right)$

$q_i=u_i-\sqrt{20}{S}_i---\left(12\right)$

Then, jth name subject is classified to the judge mark of all test patterns, obtain the measuring parameter P of judge two class situation _jand Q _j.Namely initial value is established to be 0, by whole evaluation score x of jth name subject _i,j(i=1,2 ..., I) respectively with the p of corresponding test pattern _i, q _icompare; If the evaluation score of the i-th width test pattern meets normal distribution, then judge, if x according to formula (9) and (10) _i,j>=p _i, then P _j=P _j+ 1, if x _i,j≤ q _i, then Q _j=Q _j+ 1; Otherwise, judge according to formula (11) and (12).

Finally, calculate according to formula (13) and formula (14) index passing judgment on abnormal data with if and then eliminate all evaluating datas of jth name subject, otherwise, retain evaluating data.

$R_j^1=\frac{1}{I}(P_j+Q_j)---\left(13\right)$

$R_j^2=\left|\frac{P_j-Q_j}{P_j+Q_j}\right|---\left(14\right)$

105: employing Grubbs' test method filters out the abnormal data in every width test pattern evaluation score;

First, the evaluation score of every width test pattern is arranged in ascending order mode, show that maximum in whole evaluation score of the i-th width test pattern and minimum value are x _imaxand x _imin, according to formula (15) and (16) calculating parameter α _iand γ _i, wherein, u _iand S _irepresent mean value and the standard deviation of the i-th width test pattern evaluation score respectively.

α _i＝(x _imax-u _i)/S _i（15）

γ _i＝(u _i-x _imin)/S _i（16）

Finally, by looking into tables of critical values and obtain corresponding subject's quantity and confidence level being the critical value G of 95%, if α _i> G, then eliminate abnormal data x _imaxif, γ _i> G, then eliminate abnormal data x _imin.Afterwards, above-mentioned checking procedure is repeated until no longer there is abnormal data in experimental data.

106: utilize and there are not abnormal subject and data after screening, the mean value (i.e. MOS value) calculating whole normally experimental datas of every width test pattern obtains final subjective evaluation result.

The mean value (i.e. MOS value) calculating the whole normal experimental data of every width test pattern according to formula (17) obtains final subjective evaluation result.

${\mathrm{MOS}}_i=\frac{1}{K}{\mathrm{\Σ}}_{j=1}^K{x}_{i,j}---\left(17\right)$

Wherein, MOS _irepresent the subjective assessment value of the i-th width image, K represents the quantity of final normal subject, x _i,jrepresent that jth name subject is to the evaluation score of the i-th width image.

In order to verify subject's quantity determination algorithm that this method proposes and experimental data filtering algorithm, the image of this method to JPEG compression artefacts and random noise, Gaussian noise distortion has carried out subjective assessment, simply introduces experimentation below:

This method have chosen 48 width test patterns for determining the pretest of subject's quantity, comprising girl, family, boy, flower, tree and river six scenes, they compress through the JPEG of 85%, 65%, 45%, 25%, 15%, 5% respectively and add random noise, Gaussian noise; With reference to stereo-picture left and right visual point image as shown in figs. 1 to 6, experimental data is provided by University Of Tianjin's broadband wireless communications and three-dimensional imaging research institute six width.Specify according to ITU-R BT.1438 [1] standard, experiment adopts DSIS (Double-Stimulus Impairment Scale, two stimulation damage scaling law), recruit all normal subject of 15 visual performances and stereoscopic vision and participate in pretest, evaluation equipment is 3DWINDOWS-19A0(19 inch, 1280 × 1024), as Fig. 8, laboratory brightness is extremely weak, give a mark according to the quality of DSIS standards of grading to distorted image, the total time of the training and testing of each subject can not more than 30 minutes.

Result verification:

Subject's quantity (concrete outcome is in table 1 and table 2) that the present embodiment will utilize subject's quantity determination algorithm of this patent introduction to calculate different test pattern needs, wherein, setting confidence level is 95%, because the evaluation score fluctuation of second-rate image is larger, therefore accepted percentage deviation r mean opinion score being greater than the pretest image of 3 is set to 5%, and other image is set to 8%.

Table 1 and table 2 respectively show through JPEG compression and the some experimental data of test scene adding noise, known through observing, and subject's quantity and scene content, image procossing mode are relevant with image lesion degree.By the calculating of subject's quantity determination algorithm, for the stereoscopic image data storehouse that this patent is set up, 30 subjects are selected to participate in subjective assessment, both the requirement of most test pattern to subject's quantity had been met, ensure that accuracy and the reliability of experimental result, the waste that can not cause experimental resources because subject's quantity is excessive again.For the correctness and 30 subjects of verifying this method further can guarantee the accuracy of this experimental result, this method increases subject's quantity to 35 and carries out subjective assessment to partial test image, find experimental result and 30 subject's measured results closely, specifically in table 1 and table 2.

Table 1 is about the proof of algorithm result of compressed image

Table 2 is about the proof of algorithm result adding image of making an uproar

Eliminate exception, suspicious subject and experimental data by this method, thus make experimental result more accurately, reliably.In order to verify correctness and the validity of this experimental data screening technique, the part evaluation score of the final appraisal results MOS value of subjective experiment with the subject that is eliminated compares, specifically in table 3 by the present embodiment.Observe known, the evaluation score of subject that is eliminated all differs more than 1.5 points with the final MOS value of subjective experiment, deviate from the evaluation score scope of most of subject, will the accuracy of subjective evaluation result be affected, thus demonstrate the correctness of this patent experimental data screening technique.Therefore, this method correctly can determine subject's quantity and screening experiment data, ensure that the reliability of experimental result, has saved experimental resources, eliminates abnormal, suspicious experiment value, thus ensures the consistency of experimental result.

Table 3 is about the result of data screening algorithm

List of references:

[1]International Telecommunication Union(ITU),Recommendation ITU-R BT.1438,Subjective assessment of stereoscopic television pictures[S],2000.

[2]W.J.Tam,G.Alain,L.Zhang,et al.Smoothing depth maps for improved stereoscopicimage quality[C].Three-Dimensional TV,Video and DisplayⅢ,Philadelphia,PA,USA,2004.

[3]Xu Wang,Mei Yu,You Yang,et al.Research on subjective stereoscopic image qualityassessment[C].Multimedia Content Access:Algorithms and SystemsⅢ,San Jose,CA,United States,2009.

[4]CHENG Yu-qing and JIANG Xiu-hua.Latest research result of subjective and objectiveevaluation methodology for the video quality of3DTV[J].Journal of communicationuniversity of China(science and technology).2012,19(1):31-36.

[5]Junming Zhou,Gangyi Jiang and Xiangying Mao.Subjective quality analyses of stereoscopicimages in3DTV system[C],IEEE Visual Communications and Image Processing(VCIP),Tainan:2011,1～4

[6] Zhou Wujie, Yu Mei, Zhou Junming, etc., stereo image quality study on evaluation way [C], China's Telecommunication's international conference in 2010 years, Nanning: 2010

[7] Zhang Yingjing, Li Sumei, Wei Jinjin, etc., the subjective assessment scheme [J] of stereo image quality, photon journal, 2012,41(5): 602 ~ 607

[8]NukhetOzbek,GizemErtan and OktayKarakus.Interactive quality assessment for asymmetriccoding of3D video[C],3DTV Conference:The True Vision-Capture,Transmission andDisplay of3D Video(3DTV-CVON),Antalya:2011,1～4

[9]A.S.Umar,R.M.Swash and A.H.Sadka,Subjective quality assessment of3D videos[C],2011AFRICON,Livingstone,Zambia:2011:1～6

[10]ChaohuiLv,Jingwei Huang and YinghuaShen,Subjective assessment of noised stereoimages[C],International Conference on Multimedia Technology(ICMT),Hangzhou,China:2011,783～785

[11]Mahsa T.Pourazad,Zicong Mai,PanosNasiopoulos,et al,Effect of brightness on thequality of visual3D perception[C],201118th IEEE International Conference on ImageProcessing(ICIP),Brussels:2011,989～992

It will be appreciated by those skilled in the art that accompanying drawing is the schematic diagram of a preferred embodiment, the invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.

The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (1)

1. a defining method for stereo-picture subjective assessment subject quantity and experimental data, is characterized in that, said method comprising the steps of:

(1) choose n name subject and carry out stereo image quality pre assessment, obtain subjective assessment mark, calculating mean value and standard deviation;

(2) according to the requirement of required accuracy and confidence level, by inverse confidential interval formula, interative computation obtains the subject quantity N of subjective experiment actual needs;

(3) choose N name subject and carry out stereo image quality subjective assessment, calculate mean value and the standard deviation of every width test pattern evaluation score;

(4) whether meet normal distribution according to the mean value of every width test pattern evaluation score and standard deviation, adopt the threshold value p of different judge mark exceptions _iand q _i, calculate the index passing judgment on abnormal data with by comparing, eliminate abnormal subject;

(5) employing Grubbs' test method filters out the abnormal data in every width test pattern evaluation score;

(6) there are not abnormal subject and data after utilizing screening, the mean value calculating whole normally experimental datas of every width test pattern obtains final subjective evaluation result;

Wherein, described subject's quantity is determined to be specially:

N＝(t ²S ²)/(r ²mean ²) (1)

Wherein, r represents acceptable percentage deviation, mean, S represent that a small amount of subject carries out mean value and the standard deviation of the subjective assessment mark of pre assessment to single stereoscopic image quality respectively, t gets initial value 1.96, substitute into the initial value that formula (1) calculates subject quantity N, then substitute into formula (1) calculating N value by the t value corresponding to N initial value, continuous iteration, until the N value calculated no longer changes, i.e. subject's quantity;

Wherein, the abnormal data that described employing Grubbs' test method filters out in every width test pattern evaluation score is specially:

First, the evaluation score of every width test pattern is arranged in ascending order mode; According to formula (2) and (3) calculating parameter α _iand γ _i,

α _i＝(x _imax-u _i)/S _i(2)

γ _i＝(u _i-x _imin)/S _i(3)

Wherein, x _imaxand x _iminrepresent the maximum in whole evaluation score of the i-th width test pattern and minimum value respectively; u _iand S _irepresent mean value and the standard deviation of the i-th width test pattern evaluation score respectively; Finally, by looking into tables of critical values and obtain corresponding subject's quantity and confidence level being the critical value G of 95%, if α _i> G, then eliminate abnormal data x _imaxif, γ _i> G, then eliminate abnormal data x _imin; Finally, above-mentioned checking procedure is repeated until no longer there is abnormal data in experimental data;

Wherein, the threshold value p of different judge mark exceptions is adopted _iand q _i, calculate the index passing judgment on abnormal data with by comparing, the step of eliminating abnormal subject is specially:

For every width test pattern, calculate subject and pass judgment on the whether abnormal threshold value p of mark _iand q _iif evaluation score meets normal distribution, then formula (4) and (5) are utilized to calculate; Otherwise, utilize formula (6) and (7) to calculate,

p _i＝u _i+2S _i(4)

q _i＝u _i-2S _i(5)

$p_i=u_i+\sqrt{20}{S}_i---\left(6\right)$

$q_i=u_i-\sqrt{20}{S}_i---\left(7\right)$

The index passing judgment on abnormal data is calculated according to formula (8) and formula (9) with if and then eliminate all evaluating datas of jth name subject, otherwise, retain evaluating data;

$R_j^1=\frac{1}{I}(P_j+Q_j)---\left(8\right)$

$R_j^2=\left|\frac{P_j-Q_j}{P_j+Q_j}\right|---\left(9\right)$

Wherein, u _iand S _irepresent mean value and the standard deviation of the i-th width test pattern evaluation score respectively; Jth name subject is classified to the judge mark of all test patterns, obtains the measuring parameter P of judge two class situation _jand Q _j.