CN103218815A - Method for statistical calculation of image saliency map by means of natural scenes - Google Patents

Method for statistical calculation of image saliency map by means of natural scenes Download PDF

Info

Publication number
CN103218815A
CN103218815A CN2013101357628A CN201310135762A CN103218815A CN 103218815 A CN103218815 A CN 103218815A CN 2013101357628 A CN2013101357628 A CN 2013101357628A CN 201310135762 A CN201310135762 A CN 201310135762A CN 103218815 A CN103218815 A CN 103218815A
Authority
CN
China
Prior art keywords
image
conspicuousness
remarkable
wavelet coefficient
neighborhood
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2013101357628A
Other languages
Chinese (zh)
Other versions
CN103218815B (en
Inventor
黄虹
张建秋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jilian Network Technology Co ltd
Original Assignee
Fudan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fudan University filed Critical Fudan University
Priority to CN201310135762.8A priority Critical patent/CN103218815B/en
Publication of CN103218815A publication Critical patent/CN103218815A/en
Application granted granted Critical
Publication of CN103218815B publication Critical patent/CN103218815B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of image saliency map models, and particularly discloses a method for statistical calculation of an image saliency map by means of natural scenes. The image saliency map is calculated through multiplier random variables in Gaussian scale mixture statistical distribution of the natural scenes, and therefore an image saliency map model is built. Analysis shows that the image saliency map model put forward in the method has high consistency with a visual attention choice mechanism, namely, simulation is prevented from occurring repeatedly and visual simulation with high saliency is highlighted. Therefore, saliency distribution of the visual simulation from an image on human eyes is better described.

Description

Utilize the method for the remarkable figure of natural scene statistical computation image
Technical field
The invention belongs to the remarkable graph model technical field of image, be specifically related to utilize natural scene Gauss yardstick to mix the method that the multiplier stochastic variable in the statistical distribution is come the remarkable figure of computed image.
Background technology
(Visual Attention VA) is human visual system's (Human Visual System, important mechanisms HVS) to visual attention.Generally speaking, the visual stimulus that human eye can be subjected to coming from the outside in a large number a moment, but because the processing resource-constrained among the HVS can form one to handling the competitive relation of resource between the different visual stimulus.The stimulation of quantity of information maximum can be won competition in final all visual stimulus, and other thorns are goaded into action and are suppressed [1-2]HVS utilizes such VA choice mechanism just, makes full use of effective resource and handles a large amount of visual stimulus, thereby reduced the complexity that scene is analyzed [3]
In neurology and field of biology, the eye gaze point situation when some experiments are faced the different images scene by specific equipment records observer, other experiments then initiatively indicate their interesting areas in the different images scene by the observer.The purpose of these experiments all is to wish according to experimental result, the VA mechanism of research HVS.Result of study shows: on the one hand, the place that grey scale change is big in the image can cause the attention that HVS is bigger, comprises texture information, marginal information or the like.On the other hand, HVS has certain inhibiting effect to the redundant information that repeats, and is novel, can not cause HVS more concern with the information that repeats on every side.Therefore, we claim the few more place of occurrence number of visual stimulus stay in place form, and conspicuousness is big more [3]
Along with to the deepening continuously of VA Mechanism Study, some results of study have been applied among the problem of Flame Image Process, and have brought some suggestive results.But in the real-time processing of image, it is unpractical obtaining by experiment that human eye distributes to the blinkpunkt of different images, needs to set up the calculated VA model that can simulate HVS character, thereby realizes the prediction to the visual stimulus conspicuousness.In these models, the most basic is remarkable graph model, and it describes the attention level of HVS to diverse location in the scene with significantly scheming (Saliency map) [1]
The basic conclusion of visual attention is that the notice feature that Treisman﹠Gelade in 1980 proposes on experiment basis is integrated theoretical [5]1989, Wolfe etc. proposed Guided Search model [6], it realizes the search of target in the scene with the significance mapping graph.1985, Koch﹠Ullman set up the VA model framework according to the neurology theory [7]The classical remarkable drawing method of Itti﹠Koch [8]In the Koch﹠Ullman framework, adopt multiple dimensioned hyperchannel to come model VA mechanism, extract correlated characteristic, and obtain the remarkable figure higher with the HVS consistance by merging.STB (Saliency ToolBox) [9]Method is improved the Itti﹠Koch method in conjunction with the theory of visual cognition unanimity.In addition, document [16] has proposed a kind of remarkable graph model based on image Fourier transform phase spectrum (PFT-based Saliency map PFTSal), and has obtained widely approval [1] [4] [17]Recently, document [14] has proposed a kind of PCT based on cosine transform (Pulse Discrete cosine Tranform) model, iamge description that is called signature behind this model again by document [15], (signature-based Saliency map, signatureSal) consistance with HVS is higher than other existing remarkable graph model for the remarkable graph model that their experimental result confirmation is set up on this descriptor.Some other work comprises the Bruce method based on information theory [10], and based on the SUN model of bayes method [12]With the Surprise model [13]Deng.
The present invention proposes a kind of natural scene statistically significant graph model.The multiplier stochastic variable that it utilizes natural scene Gauss yardstick to mix in (GSM, Gaussian scale mixture) statistical distribution comes computed image significantly to scheme.Remarkable graph model of the present invention and VA mechanism have higher consistance.
Summary of the invention
The object of the present invention is to provide a kind of and the method remarkable figure of the corresponding to computed image of human visual system's notice vision choice mechanism, thereby set up the remarkable graph model of image.
The present invention utilize natural scene Gauss yardstick mix (Gaussian scale mixture, GSM) the multiplier stochastic variable in the statistical distribution comes computed image significantly to scheme, concrete steps are as follows:
(1) establishing image is gray level image
Figure 699257DEST_PATH_IMAGE001
,
Figure 213414DEST_PATH_IMAGE002
,
Figure 608624DEST_PATH_IMAGE003
Be respectively the row, column number of image, it is carried out the wavelet coefficient conversion, obtain a plurality of wavelet coefficient subbands.
(2) in each wavelet coefficient subband, to each coefficient
Figure 157417DEST_PATH_IMAGE004
Choose suitable wavelet coefficient neighborhood, the wavelet coefficient neighborhood is pulled into wavelet coefficient neighborhood vector
Figure 979879DEST_PATH_IMAGE005
, wherein
Figure 246913DEST_PATH_IMAGE006
Be the neighborhood size;
According to natural scene statistics (Natural Scene Statistics, the NSS) statistical property of model, the wavelet coefficient neighborhood vector of natural image
Figure 163922DEST_PATH_IMAGE005
The enough Gauss's yardstick mixing of energy (Gaussian scale mixture, GSM) describe, promptly by distribution , wherein
Figure 560585DEST_PATH_IMAGE008
Be the multiplier at random of sign neighborhood vector covariance variation,
Figure 49335DEST_PATH_IMAGE009
For zero-mean, covariance matrix are
Figure 786347DEST_PATH_IMAGE010
Gaussian random variable, therefore, the neighborhood vector Probability density function Be expressed as:
Figure 715623DEST_PATH_IMAGE013
(1)
Wherein,
Figure 459588DEST_PATH_IMAGE014
Be multiplier at random
Figure 103059DEST_PATH_IMAGE008
Probability density function profiles;
Therefore, coefficient neighborhood vector
Figure 438225DEST_PATH_IMAGE011
About multiplier at random
Figure 901568DEST_PATH_IMAGE008
Obedience zero-mean, covariance matrix are
Figure 980382DEST_PATH_IMAGE015
Gaussian distribution, the conditional probability distribution function representation is:
Figure 462048DEST_PATH_IMAGE016
(6)。
(3) estimated value of calculating Gauss yardstick mixed distribution multiplier variable
When the wavelet coefficient neighborhood of choosing enough little, promptly
Figure 233695DEST_PATH_IMAGE006
Enough hour, suppose multiplier In this neighborhood, remain unchanged, therefore can be temporarily with
Figure 535680DEST_PATH_IMAGE008
Regard that one is determined amount or constant as, at this moment, neighborhood Corresponding multiplier
Figure 565133DEST_PATH_IMAGE008
Can pass through the conditional probability distribution function
Figure 3068DEST_PATH_IMAGE017
Maximal possibility estimation obtain, promptly [18]:
Figure 158106DEST_PATH_IMAGE018
(7)
Note
Figure 630675DEST_PATH_IMAGE010
Characteristic value decomposition be
Figure 744125DEST_PATH_IMAGE019
, wherein For
Figure 628084DEST_PATH_IMAGE010
Eigenvector The matrix that constitutes,
Figure 442773DEST_PATH_IMAGE022
For
Figure 855300DEST_PATH_IMAGE010
Eigenwert The matrix that constitutes.
Therefore,
Figure 48570DEST_PATH_IMAGE008
The maximal possibility estimation result be:
Figure 972664DEST_PATH_IMAGE024
Figure 173018DEST_PATH_IMAGE026
(8)
Do not lose universality ground hypothesis
Figure 474686DEST_PATH_IMAGE027
, have
Figure 835261DEST_PATH_IMAGE011
Covariance matrix
Figure 691221DEST_PATH_IMAGE028
Therefore, can use
Figure 61023DEST_PATH_IMAGE011
Covariance matrix
Figure 686039DEST_PATH_IMAGE029
Replace
Figure 483094DEST_PATH_IMAGE010
, obtain
Figure 826350DEST_PATH_IMAGE030
(9)
The band subscript
Figure 999843DEST_PATH_IMAGE031
Figure 731563DEST_PATH_IMAGE011
Expression
Figure 433940DEST_PATH_IMAGE011
Transposition, suppose little wave vector Be a feature samples, so
Figure 976096DEST_PATH_IMAGE011
Set constituted the feature space of image.Because wavelet coefficient average is zero, therefore,
Figure 310126DEST_PATH_IMAGE032
For
Figure 448983DEST_PATH_IMAGE011
To this feature space center The Mahalanobis distance [22]
Figure 16548DEST_PATH_IMAGE034
Character according to the Mahalanobis distance: Mahalanobis distance has been taken all factors into consideration the relation between each dimension of eigenvector, sample
Figure 1821DEST_PATH_IMAGE011
Mahalanobis distance to the feature space center
Figure 46001DEST_PATH_IMAGE035
It is big more,
Figure 116725DEST_PATH_IMAGE011
The probability that belongs to this feature space is low more, that is to say sample " conspicuousness " high more, vice versa.
According to the description of formula (9), can know
Figure 744332DEST_PATH_IMAGE036
With
Figure 224992DEST_PATH_IMAGE035
There is proportional relation, promptly
(10
Figure 92640DEST_PATH_IMAGE038
)
Therefore,
Figure 786926DEST_PATH_IMAGE036
Be the effective description of sample in the feature space conspicuousness.The feature samples conspicuousness is high more, corresponding
Figure 172908DEST_PATH_IMAGE036
Be worth big more; On the contrary, the feature samples conspicuousness is low more, corresponding
Figure 218225DEST_PATH_IMAGE036
Be worth more little.
According to the character that image wavelet decomposes, wavelet coefficient can extract the discontinuous information of image, compares with frequency field simultaneously, and it can describe the notice intensity distributions of HVS to scene.Adjacent whole coefficients on its space, yardstick and direction constitute a neighborhood if we choose each wavelet coefficient, and the whole coefficient tables in the neighborhood are shown as a neighborhood vector
Figure 347855DEST_PATH_IMAGE011
So The eigenvector of visual stimulus in this neighborhood has just been described.Therefore, by
Figure 453531DEST_PATH_IMAGE011
The feature space that constitutes of whole set, the space distribution situation of visual stimulus has not only been described, described simultaneously in the relevant information between the adjacent visual stimulus on space, yardstick and the direction, described space adjacent visual signature correlativity, described the direction adjacent equidirectional adjacent coefficient of visual signature correlativity different scale with yardstick different directions adjacent coefficient and then described adjacent visual signature correlativity of yardstick etc. as adjacent coefficient in the same subband.Just can show by formula (6) and top analysis so: by
Figure 720564DEST_PATH_IMAGE036
Constitute Matrix can be under the prerequisite of taking all factors into consideration correlativity between visual stimulus space distribution situation and the adjacent visual stimulus, comprehensive description visual scene conspicuousness distributes, and such comprehensive description is quite consistent with VA mechanism described in neurology and the field of biology.
(4) the pairing conspicuousness of all wavelet coefficient subbands is merged, can access complete natural scene statistically significant figure NSSSal (Natural Scene Statistical Saliency map) model:
( 2)
In the formula,
Figure 804878DEST_PATH_IMAGE041
The expression yardstick The superposition that the conspicuousness of last different directions is described;
Figure 65275DEST_PATH_IMAGE042
Add (the across-scale addition) that represents different scale [9], the description of the conspicuousness on all yardsticks is interpolated into
Figure 478808DEST_PATH_IMAGE043
Back addition, wherein
Figure 454854DEST_PATH_IMAGE044
Be the Gaussian Blur kernel function, it is used for remarkable figure is carried out certain smoothing effect [15]
(5) formula (11) is adjusted to the gray scale dynamic range
Figure 995557DEST_PATH_IMAGE045
, near 1 position, big more with regard to corresponding region conspicuousness in the presentation video, the place littler than other values is more prone to be attracted to the attention of human eye to value more.
(6) if image is the coloured image of RGB modulation
Figure 107869DEST_PATH_IMAGE046
, red channel is arranged
Figure 443036DEST_PATH_IMAGE047
, green channel
Figure 906378DEST_PATH_IMAGE048
With blue channel
Figure 719613DEST_PATH_IMAGE049
Calculate corresponding gray scale passage
Figure 217591DEST_PATH_IMAGE050
, red green antagonism is right Right with the champac antagonism
Figure 408718DEST_PATH_IMAGE052
The gray scale passage
Figure 291223DEST_PATH_IMAGE050
For:
Figure 378128DEST_PATH_IMAGE053
(12)
According to the treatment mechanism of human eye to chromatic information, the redness (R) of four broad sense modulation, green (G), blue (B) are respectively with yellow (Y) passage:
(13)
Figure 742299DEST_PATH_IMAGE055
(14)
Figure 162916DEST_PATH_IMAGE056
(15)
Figure 635486DEST_PATH_IMAGE057
(16)
It is right to obtain red green antagonism
Figure 217777DEST_PATH_IMAGE051
Right with the champac antagonism
Figure 143007DEST_PATH_IMAGE052
For:
Figure 836157DEST_PATH_IMAGE058
(17)
Figure 163233DEST_PATH_IMAGE059
(18)
(7) to the gray scale passage
Figure 447584DEST_PATH_IMAGE050
, red green antagonism is to passage
Figure 594531DEST_PATH_IMAGE051
Right with the champac antagonism
Figure 356951DEST_PATH_IMAGE052
Calculate according to step (1)-(5), be designated as respectively
Figure 272954DEST_PATH_IMAGE060
,
Figure 728207DEST_PATH_IMAGE061
With
Figure 628029DEST_PATH_IMAGE062
Remarkable figure, then with this three's weighted mean remarkable figure as this coloured image
Figure 928561DEST_PATH_IMAGE063
, that is:
Figure 964650DEST_PATH_IMAGE064
(19)
Wherein,
Figure 305983DEST_PATH_IMAGE065
,
Figure 693102DEST_PATH_IMAGE066
With
Figure 797324DEST_PATH_IMAGE067
Be respectively the weight of 3 passages, and have
Figure 687920DEST_PATH_IMAGE068
According to the present invention, among the remarkable figure that image calculation is obtained, the position correspondence image conspicuousness that pixel value is high more is high more; The position correspondence image conspicuousness that pixel value is low more is low more.
Remarkable graph model and visual attention choice mechanism that the present invention proposes have higher consistance, promptly can be simultaneously in the stimulation that suppresses to repeat, outstanding conspicuousness higher visual stimulates, thereby described image better the conspicuousness that human eye vision stimulates is distributed.
Description of drawings
Fig. 1: can handle pyramid decomposition and the wavelet coefficient neighborhood is chosen.
Embodiment
Following the present invention is by embodiment, and the contrast remarkable graph model NSSSal/cNSSSal of the present invention and other remarkable graph model extract the performance of remarkable figure to natural image.Simultaneously, also will be by disclosed Bruce database [10]With the ImgSal database [11], their AUC (Area Under the Curve, ROC area under a curve) are carried out qualitative assessment.
Used the complete pyramid handled that image is carried out wavelet decomposition in the experiment, to keep the directivity of image.Adopt the decomposition scale number
Figure 484974DEST_PATH_IMAGE069
, direction number , be respectively
Figure 1723DEST_PATH_IMAGE071
,
Figure 481246DEST_PATH_IMAGE072
,
Figure 449202DEST_PATH_IMAGE073
With
Figure 545334DEST_PATH_IMAGE074
, as shown in Figure 1.Simultaneously, choose each wavelet coefficient
Figure 725780DEST_PATH_IMAGE004
(indicating with the black blockage among the figure) constitutes its neighborhood vector jointly with the interior adjacent coefficient of its same sub-band, with the coefficient (all indicating with the grey blockage among the figure) of same position on the female yardstick of same locational coefficient and equidirectional in the yardstick different directions subband
Figure 856547DEST_PATH_IMAGE005
, the neighborhood size
Figure 729825DEST_PATH_IMAGE075
Choose the gaussian kernel variance and be remarkable figure width 0.045 times.
For any width of cloth natural image, the remarkable figure that note is obtained by certain remarkable graph model is
Figure 313253DEST_PATH_IMAGE076
Selected threshold
Figure 562969DEST_PATH_IMAGE077
, according to threshold value
Figure 282663DEST_PATH_IMAGE078
Right Carry out binaryzation, after the note binaryzation
Figure 318938DEST_PATH_IMAGE076
For According to the subjective notice distribution plan that provides in the database
Figure 946546DEST_PATH_IMAGE080
(Fixation Density Map), its positive class rate (TPR, True Positive Rate) is:
Figure 161626DEST_PATH_IMAGE081
(20)
Wherein,
Figure 906597DEST_PATH_IMAGE082
Multiplication between the symbolic representation pixel,
Figure 763695DEST_PATH_IMAGE083
Represent 1 norm value, i.e. matrix
Figure 130085DEST_PATH_IMAGE084
In be the number of 1 element.
Similarly, its false alarm rate (FPR, False Positive Rate) is:
Figure 781646DEST_PATH_IMAGE085
(21)
For given threshold value , will obtain all images in the database
Figure 205860DEST_PATH_IMAGE086
Mean value as this remarkable graph model in threshold value is
Figure 754653DEST_PATH_IMAGE078
The time positive class rate
Figure 577116DEST_PATH_IMAGE087
, same, will Mean value as being in threshold value
Figure 980732DEST_PATH_IMAGE078
The time false alarm rate
Figure 649611DEST_PATH_IMAGE089
By choosing different threshold values , with
Figure 131725DEST_PATH_IMAGE087
Be the longitudinal axis, with Be transverse axis, draw the ROC curve.The ROC area under a curve Provide the subjective notice Uniformity of Distribution of remarkable graph model and human eye to measure, Value approaches 1 more, illustrates that the consistance of remarkable graph model and VA mechanism is high more.
(1) the remarkable graph model Performance Evaluation of gray level image
Table 1 is behind the image gray processing in Bruce, the ImgSal database, to be obtained by NSSSal, Itti﹠Koch, PFTSal, signatureSal Model Calculation Value,
Figure 794175DEST_PATH_IMAGE091
Value approaches 1 more, represents that the consistance of remarkable graph model and VA mechanism is high more.As can be seen, on these two databases, Itti﹠Koch's
Figure 172067DEST_PATH_IMAGE091
The result is minimum, and PFTSal is close with signatureSal model result on statistical significance, and the NSSSal model is compared to the two, has more to approach 1
Figure 772812DEST_PATH_IMAGE091
Value is the highest with VA mechanism consistance.
Table 1: gray level image is significantly schemed the contrast of AUC value
Figure 970576DEST_PATH_IMAGE093
。
(2) the remarkable graph model Performance Evaluation of coloured image
Further, to colored natural image, we have contrasted cNSSSal and Itti﹠Koch [8], PQFTSal [16], signatureSal [15]ROC (Receiver Operating Characteristive) curve, and ROC area under a curve AUC (Area Under the Curve).Get in the experiment
Figure 314969DEST_PATH_IMAGE094
, promptly 3 passages are got equal weight.
Table 2 is to be obtained by cNSSSal, Itti﹠Koch, PQFTSal, signatureSal Model Calculation on Bruce, the ImgSal database Value.Different significantly graph models after having considered chromatic information,
Figure 281788DEST_PATH_IMAGE095
Value all can correspondingly increase, and the value of cNSSSal is still the highest than other model.
Table 2: gray level image is significantly schemed the contrast of AUC value
Figure 522277DEST_PATH_IMAGE097
。
Practical application effect of the present invention
Significantly graph model all is widely used in the Flame Image Process problems such as image zoom of adapting to image compression, video frequency abstract, coding and image progressive transmission, image segmentation, image and video quality assessment, Target Recognition, perception of content.Below with the validity and the superiority that are applied as example explanation the present invention remarkable graph model of the remarkable graph model of the present invention in the image quality measure problem.
Image quality measure is estimated the integrated mechanism of simulating human vision system, wish to obtain with human eye to the consistent result of the assessment of picture quality.Yet in a lot of existing image quality measure methods, notice vision choice mechanism important among the HVS but usually is left in the basket.When ignoring notice vision choice mechanism, image quality measure is estimated the hypothesis human eye can give same attention rate to comprising all targets such as natural scene and image fault.Yet, according to notice vision choice mechanism, the human vision model is not treated image in the mode of simple higher dimensional space signal, but the different attribute of image is had different susceptibilitys, as the shape of brightness, contrast, target and texture, direction, smoothness etc.Because the human visual system has different susceptibilitys to the heterogeneity of image, need consider so different susceptibility when therefore distorted image being carried out quality evaluation.Therefore, can improve the performance of image quality measure algorithm effectively in conjunction with notice vision choice mechanism.
In the experiment image that obtains is significantly schemed
Figure 738494DEST_PATH_IMAGE098
Be used as weights, readjust the shared weight proportion in net result of each several part quality assessment result in original image quality measure method, its objective is that expectation makes assessment result more consistent with human eye subjective evaluation result.Experiment is estimated (PSNR, MSSIM to three kinds of objective image quality evaluations [25], VIF [26]) be weighted with remarkable figure.
(1) based on the PSNR of remarkable figure
Because PSNR is based on pixel, so significantly figure can directly use as weighting matrix.Suppose
Figure DEST_PATH_IMAGE099
Be remarkable figure
Figure 355421DEST_PATH_IMAGE098
Last
Figure 707905DEST_PATH_IMAGE100
Gray-scale value on the individual pixel.Based on the PSNR of remarkable figure (Saliency-based PSNR SPSNR) is defined as:
Figure 119294DEST_PATH_IMAGE101
(22)
Wherein,
Figure 275338DEST_PATH_IMAGE102
Be the total number of pixels of image,
Figure 695955DEST_PATH_IMAGE103
,
Figure 902945DEST_PATH_IMAGE104
Be respectively image
Figure 750816DEST_PATH_IMAGE105
,
Figure 676046DEST_PATH_IMAGE106
Last
Figure 900354DEST_PATH_IMAGE100
The gray-scale value of each and every one pixel,
Figure 696272DEST_PATH_IMAGE107
It is the gradation of image dynamic range.Because the grey level of the remarkable figure that different natural images obtain is different, therefore use
Figure 980623DEST_PATH_IMAGE108
Weights are carried out normalized.
(2) based on the MSSIM of remarkable figure
Because MSSIM is based on image subblock, therefore at first the remarkable figure that obtains
Figure 861991DEST_PATH_IMAGE098
Be divided into and the same number of reference picture, onesize sub-piece.Suppose
Figure 889990DEST_PATH_IMAGE098
Be divided into
Figure 805993DEST_PATH_IMAGE109
The height piece, and with sub-piece
Figure 261246DEST_PATH_IMAGE110
,
Figure 161068DEST_PATH_IMAGE111
Corresponding sub-piece is
Figure 710867DEST_PATH_IMAGE112
, with sub-piece
Figure 746957DEST_PATH_IMAGE113
The Normalized Grey Level average
Figure 841951DEST_PATH_IMAGE114
Weight as the sub-piece of correspondence so just has
Figure 963491DEST_PATH_IMAGE115
(23)
Wherein,
Figure 333293DEST_PATH_IMAGE116
Expression
Figure 223888DEST_PATH_IMAGE113
In pixel count,
Figure 20943DEST_PATH_IMAGE117
Expression
Figure 567462DEST_PATH_IMAGE118
In
Figure 459064DEST_PATH_IMAGE119
Individual gray values of pixel points.Therefore (Saliency-based MSSIM SMSSIM) just may be defined as based on the MSSIM of remarkable figure
Figure 469745DEST_PATH_IMAGE120
(23)
Wherein,
Figure 437701DEST_PATH_IMAGE121
,
Figure 268254DEST_PATH_IMAGE122
Be respectively in reference picture and the distorted image
Figure 979858DEST_PATH_IMAGE123
The height piece,
Figure 579466DEST_PATH_IMAGE124
Sum for image subblock.
(3) based on the VIF of remarkable figure
Because spatial domain VIF is multiple dimensioned, therefore reference picture is adjusted under the different scale, calculate the remarkable figure under the different scale
Figure 718324DEST_PATH_IMAGE125
, wherein
Figure 770593DEST_PATH_IMAGE126
Be the yardstick index.Again according to the character of VIF, at yardstick based on sub-piece
Figure 285888DEST_PATH_IMAGE127
Following,
Figure 5583DEST_PATH_IMAGE125
Be divided into
Figure 315341DEST_PATH_IMAGE128
The height piece, note the
Figure 120486DEST_PATH_IMAGE123
The height piece is
Figure 642734DEST_PATH_IMAGE129
With sub-piece
Figure 943833DEST_PATH_IMAGE129
The Normalized Grey Level average
Figure 424493DEST_PATH_IMAGE130
Weight as the sub-piece of correspondence so just has
Figure 451355DEST_PATH_IMAGE132
(25)
Wherein, Expression
Figure 2739DEST_PATH_IMAGE129
In pixel count,
Figure 388721DEST_PATH_IMAGE134
Expression
Figure 168458DEST_PATH_IMAGE129
In
Figure 298088DEST_PATH_IMAGE119
Individual gray values of pixel points.Therefore, (Saliency-based VIF SVIF) just may be defined as based on the VIF of remarkable figure
Figure 830570DEST_PATH_IMAGE135
(26)
Wherein,
Figure 653032DEST_PATH_IMAGE136
The expression scale parameter,
Figure 654486DEST_PATH_IMAGE126
Be the yardstick index,
Figure 853386DEST_PATH_IMAGE128
Expression the
Figure 256686DEST_PATH_IMAGE126
Image subblock number under the individual yardstick,
Figure 250050DEST_PATH_IMAGE123
Be sub-piece index.
Figure 738800DEST_PATH_IMAGE137
Be reference picture
Figure 741391DEST_PATH_IMAGE126
Of individual yardstick
Figure 733618DEST_PATH_IMAGE123
The height piece,
Figure 632304DEST_PATH_IMAGE138
, Be respectively reference picture and the sub-piece of distorted image that human eye receives,
Figure 149053DEST_PATH_IMAGE140
Model parameter for correspondence.
Figure 245053DEST_PATH_IMAGE141
For
Figure 845799DEST_PATH_IMAGE137
With
Figure 43562DEST_PATH_IMAGE142
About
Figure 387956DEST_PATH_IMAGE140
Mutual information.
In the experiment to PSNR, MSSIM, adopt four kinds of different remarkable graph model (Itti﹠Koch with VIF [8], PQFTSal [16], signatureSal [15], cNSSSal) be weighted, estimate so derive five kinds of different implementation methods (IQA, Itti_IQA, PQFT_IQA, signature_IQA and cNSS_IQA) for every kind.
The matlab of MSSIM and spatial domain VIF realizes that program all comes from online original author's realization version.In the experiment, sub-block size gets 8 * 8 when calculating MSSIM and SMSSIM, and adjacent sub-blocks is 1 pixel distance at interval; And get 4 yardsticks during the VIF of computer memory territory, sub-block size is 3 * 3, sub-interblock non-overlapping copies.Simultaneously, for the fairness that guarantees to contrast, with three kinds of models extract remarkable figure and with its with image quality measure in conjunction with the time the parameter setting be consistent.Adopt the author of signatureSal to realize that the acquiescence fuzzy parameter (seeing document [15] for details) in the version blurs three kinds of remarkable figure.
At LIVE image quality measure database [28]Last checking performance of the present invention.The LIVE database comprises 982 width of cloth images, and wherein 779 width of cloth are distorted image.These images are obtained under different level of distortion by JPEG, JPEG2000, white noise, Gaussian Blur and channel this five kinds of distortion modes that decline fast by 29 width of cloth reference pictures.Give the subjective evaluation mark (DMOS) of every width of cloth image correspondence in the database, the scope of DMOS is [0,100], and the DMOS=0 representative image is undistorted, and along with the distortion level increase of image, the value of DMOS also can correspondingly increase.Compare by the assessment result that DMOS and image quality measure algorithm are obtained, but the performance of assess image quality assessment algorithm so just.
After picture quality estimated result and DMOS are carried out nonlinear regression and fitting, can come the quality evaluation of quantitative evaluation objective image to estimate consistance with the subjective quality assessment result by five objective evaluation indexs: 1) linearly dependent coefficient (Linear Correlation Coefficient, LCC), it has described prediction accuracy, and it is high more that its value approaches 1 expression forecasting accuracy more; 2) mean absolute error (Mean Absolute Error, MAE), the more little expression of its value prediction absolute error is more little; 3) root-mean-square error (Root Mean Squared Error, RMSE), its value more little expression predicted root mean square error more little; 4) peel off ratio (Outlier Ratio, OR), it has described the prediction consistance, the more little expression of its value prediction consistance is high more; 5) the Spearman rank correlation coefficient (Spearman ' s Rank Ordered Correlation Coefficient, SROCC), its describes the monotonicity of prediction, its value approaches 1 expression more and predicts that monotonicity is excellent more.The quantitative evaluation result is as follows:
Table 3: the remarkable graph model of coloured image is estimated performance boost relatively to image quality measure
Figure 558037DEST_PATH_IMAGE143
。
The value of overstriking is in each evaluation index in the table 3, the optimal result that several different implementation methods obtain.As can be seen, three kinds of different remarkable graph models (Itti﹠Koch, PQFTSal, signatureSal and cNSSSal) are all estimated performance to image quality measure and have been brought lifting.Simultaneously, the lifting that the remarkable figure of cNSSSal estimates performance to image quality measure is far superior to other remarkable graph model, has proved the superiority of the remarkable graph model of the present invention in the image quality measure problem.
List of references
[1] U Engelke, H Kaprykowsk, H-J Zepernick, and P Ndjiki-Nya. Visual attention in quality assessment [J]. IEEE Signal Processing Magazine, 2011, 28(6): 50-59.
[2] E Kowler. Eye movements: The past 25 years [J].Vision Res., 2011, 51(13): 1457-1583.
[3] M Carrasco. Visual Attention: The past 25 years [J]. Vision Res., 2011, 51(13): 1484-1525.
[4] A Toet. Computational versus psychophysical bottom-up image saliency: a comparative evaluation study [J]. IEEE Transactions, 2011, PAMI-33(11): 2131-2146.
[5] A M Treisman and G Gelade. A feature-integration theory of attention [J]. Cogn. Psychol., 1980, 12(1): 97–136.
[6] J M Wolfe, K R Cave, and S L Franzel. Guided search: An alternative to the feature integration model for visual search [J]. J. Exp. Psychol. Hum. Percept. Perform., 1989, 15(3): 419-433.
[7] C Koch and S Ullman. Shifts in selective visual attention: towards the underlying neural circuitry [J]. Human Neurobiol, 1985, 219-227.
[8] L Itti, C Koch and E Niebur. A model of saliency-based visual attention for rapid scene analysis [J]. IEEE Transactions, 1998, PAMI-20(11): 1254-1259.
[9] D Walther and C Koch. Modeling attention to salient proto-objects [J]. Neural Networks, 2006, 19(9): 1395-1407.
[10] N D B Bruce and J K Tsotsos. Saliency based on information maximization [A]. In Proc. Advances in Neural Information Processing System [C]. 2005. 155-162.
[11] J Li, M D Levine, X An, X Xu, and H He. Visual saliency based on scale-space analysis in the frequency domain [J]. IEEE Transactions, 2007, PAMI-PP(99): 1, 0.
[12] C Kanan, M H Tong, L Zhang, and G W Cottrel. SUN: Top-down saliency using natural statistics [J]. Visual Cognition, 2009, 17( 6/7): 979-1003.
[13] L Itti and P Baldi. Bayesian surprise attracts human attention [A]. In Proc. Advances in Neural Information Processing System [C]. 2009. 547-554.
[14] Y Ying, B Wang, L M Zhang. Pulse discrete cosine transform for saliency-based attention [A]. In Proc. IEEE ICDL’09 [C]. 2009. 1-6, 5-7.
[15] X Hou, J Harel, and C Koch. Image signature: highlighting sparse salient regions [J]. IEEE Transactions, 2012, PAMI-34(1): 194-201.
[16] M Qi and L M Zhang. Saliency-based image quality assessment criterion [A]. In Proc. Adavanced Intelligent Computing Theories and Applications [C]. 2008. 1124-1133.
[17] C Guo and L M Zhang. Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform [A]. In Proc. IEEE CVPR ’08[C]. 2008. 1-8, 23-28.
[18] M J Wainwright and E P Simoncelli. Scale mixtures of Gaussians and the statistics of natural images [J]. Adv. Neural Inf. Process. Syst., 2000, 12: 855-861.
[19] J Portilla, V Strela, M J Wainwright, and E P Simoncelli, Image denoising using scale mixture of Gaussians in the wavelet domain [J]. IEEE Transactions, 2003, Image Processing-12(11): 1338-1351.
[20] Z Wang and A C Bovik. Reduced- and no-reference image quality assessment [J]. IEEE, 2011, Signal Processing Magazine-28(6): 29-40.
[21] A K Moorthy and A C Bovik. Statistics of natural image distortions [A]. In Proc. IEEE ICASSP’10 [C]. 2010. 962-965, 14-19.
[22] P C Mahalanobis. On the generalized distance in statistics [J]. Proceedings of the National Institute of Sciences of India, 1936, 2(1): 49-55.
[23] E P Simoncelli, W T Freeman, E H Adelson, and D J Heeger. Shiftable multiscale transforms [J]. IEEE Transactions, 1992, Informaiton Theory-38(2): 587–607.
[24] Z Wang and A C Bovik. Mean squared error: love it or leave it A new look at signal fidelity measures [J]. IEEE Signal Processing Magazine, 2009, 26(1): 98-117.
[25] Z Wang, A C Bovik, H R Sheikh, and E P Simoncelli. Image quality assessment: From error measurement to structural similarity [J]. IEEE Transactions, 2004, Image Processing-13(4): 600-612.
[26] H R Sheikh and A C Bovik. Image information and visual quality [J]. IEEE Transactions, 2006, Image Processing-15(2): 430-444.
[27] H R. Sheikh, M F Sabir, and A C Bovik. A statistical evaluation of recent full reference image quality assessment algorithms [J]. IEEE Transactions, 2006, Image Processing-15(11): 3440-2451.
[28] H R Sheikh, Z Wang, A C Bovik, and L K Cormack. Image and Video Quality Assessment Research at LIVE [Online]. Available: http://live.ece.utexas.edu/research/quality.。

Claims (2)

1. the method for the remarkable figure of computed image is characterized in that utilizing the multiplier stochastic variable in the natural scene Gauss yardstick mixing statistical distribution to come computed image significantly to scheme, and concrete steps are as follows:
(1) establishing image is gray level image
Figure 2013101357628100001DEST_PATH_IMAGE001
,
Figure 806136DEST_PATH_IMAGE002
,
Figure 2013101357628100001DEST_PATH_IMAGE003
Be respectively the row, column number of image, it is carried out the wavelet coefficient conversion, obtain a plurality of wavelet coefficient subbands;
(2) in each wavelet coefficient subband, to each coefficient
Figure 313340DEST_PATH_IMAGE004
Choose suitable wavelet coefficient neighborhood, the wavelet coefficient neighborhood is pulled into wavelet coefficient neighborhood vector
Figure 2013101357628100001DEST_PATH_IMAGE005
, wherein
Figure 791726DEST_PATH_IMAGE006
Be the neighborhood size; According to the statistical property of natural scene statistical model, the wavelet coefficient neighborhood vector of natural image is described with Gauss's yardstick mixed distribution
Figure 435197DEST_PATH_IMAGE005
, promptly
Figure 2013101357628100001DEST_PATH_IMAGE007
, wherein,
Figure 366768DEST_PATH_IMAGE008
Be the multiplier at random of sign neighborhood vector covariance variation,
Figure 2013101357628100001DEST_PATH_IMAGE009
For zero-mean, covariance matrix are Gaussian random variable;
(3) the maximal possibility estimation result of calculating Gauss yardstick mixed distribution multiplier variable is:
Figure 2013101357628100001DEST_PATH_IMAGE011
(1)
Suppose
Figure 440084DEST_PATH_IMAGE012
, use
Figure 2013101357628100001DEST_PATH_IMAGE013
Covariance matrix
Figure 141324DEST_PATH_IMAGE014
Replace
Figure 178550DEST_PATH_IMAGE010
, obtain:
Figure 2013101357628100001DEST_PATH_IMAGE015
(2)
Suppose little wave vector
Figure 722663DEST_PATH_IMAGE013
Be a feature samples, so
Figure 605169DEST_PATH_IMAGE013
Set constituted the feature space of image;
Figure 957653DEST_PATH_IMAGE016
With
Figure 103463DEST_PATH_IMAGE013
The Mahalanobis distance
Figure 2013101357628100001DEST_PATH_IMAGE017
There is proportional relation, that is:
Figure 806977DEST_PATH_IMAGE018
(3)
Figure 227594DEST_PATH_IMAGE016
Be the effective description of sample in the feature space conspicuousness, the feature samples conspicuousness is high more, corresponding Be worth big more; On the contrary, the feature samples conspicuousness is low more, corresponding
Figure 407089DEST_PATH_IMAGE016
Be worth more little;
(4) the pairing conspicuousness of all wavelet coefficient subbands is merged, can access complete natural scene statistically significant figure NSSSal model:
Figure 2013101357628100001DEST_PATH_IMAGE019
(4)
In the formula,
Figure 535582DEST_PATH_IMAGE020
The expression yardstick
Figure 759890DEST_PATH_IMAGE008
The superposition that the conspicuousness of last different directions is described; Adding of expression different scale, the description of the conspicuousness on all yardsticks is interpolated into Back addition, wherein
Figure 2013101357628100001DEST_PATH_IMAGE023
Be the Gaussian Blur kernel function, it is used for remarkable figure is carried out certain smoothing effect;
(5) formula (4) is adjusted to the gray scale dynamic range
Figure 699213DEST_PATH_IMAGE024
, near 1 position, big more with regard to corresponding region conspicuousness in the presentation video, the place littler than other values is more prone to be attracted to the attention of human eye to value more;
(6) if image is the coloured image of RGB modulation
Figure 2013101357628100001DEST_PATH_IMAGE025
, to the gray scale passage of image
Figure 580581DEST_PATH_IMAGE026
, red green antagonism is to passage
Figure 2013101357628100001DEST_PATH_IMAGE027
Right with the champac antagonism
Figure 874159DEST_PATH_IMAGE028
Calculate respectively and be designated as according to step (1)-(5)
Figure 2013101357628100001DEST_PATH_IMAGE029
,
Figure 55742DEST_PATH_IMAGE030
With
Figure 2013101357628100001DEST_PATH_IMAGE031
Remarkable figure, then with this three's weighted mean remarkable figure, that is: as this coloured image
Figure 573311DEST_PATH_IMAGE032
(5)
Wherein,
Figure 2013101357628100001DEST_PATH_IMAGE033
,
Figure 941975DEST_PATH_IMAGE034
With
Figure 2013101357628100001DEST_PATH_IMAGE035
Be respectively the weight of 3 passages, and have
Figure 773665DEST_PATH_IMAGE036
2. method according to claim 1, among the remarkable figure that every width of cloth image calculation is obtained, the position correspondence image conspicuousness that pixel value is high more is high more; The position correspondence image conspicuousness that pixel value is low more is low more.
CN201310135762.8A 2013-04-19 2013-04-19 Utilize the method for natural scene statistical computation image saliency map Expired - Fee Related CN103218815B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310135762.8A CN103218815B (en) 2013-04-19 2013-04-19 Utilize the method for natural scene statistical computation image saliency map

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310135762.8A CN103218815B (en) 2013-04-19 2013-04-19 Utilize the method for natural scene statistical computation image saliency map

Publications (2)

Publication Number Publication Date
CN103218815A true CN103218815A (en) 2013-07-24
CN103218815B CN103218815B (en) 2016-03-30

Family

ID=48816558

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310135762.8A Expired - Fee Related CN103218815B (en) 2013-04-19 2013-04-19 Utilize the method for natural scene statistical computation image saliency map

Country Status (1)

Country Link
CN (1) CN103218815B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110298341A (en) * 2019-06-12 2019-10-01 上海大学 A kind of enhancing saliency prediction technique based on direction selection
CN110503162A (en) * 2019-08-29 2019-11-26 广东工业大学 A kind of media information prevalence degree prediction technique, device and equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101103378A (en) * 2005-01-10 2008-01-09 汤姆森许可贸易公司 Device and method for creating a saliency map of an image
CN102184557A (en) * 2011-06-17 2011-09-14 电子科技大学 Salient region detection method for complex scene
EP2461274A1 (en) * 2010-09-16 2012-06-06 Thomson Licensing Method and device of determining a saliency map for an image
CN102754126A (en) * 2010-02-12 2012-10-24 高等技术学校 Method and system for determining a quality measure for an image using multi-level decomposition of images

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101103378A (en) * 2005-01-10 2008-01-09 汤姆森许可贸易公司 Device and method for creating a saliency map of an image
CN102754126A (en) * 2010-02-12 2012-10-24 高等技术学校 Method and system for determining a quality measure for an image using multi-level decomposition of images
EP2461274A1 (en) * 2010-09-16 2012-06-06 Thomson Licensing Method and device of determining a saliency map for an image
CN102184557A (en) * 2011-06-17 2011-09-14 电子科技大学 Salient region detection method for complex scene

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
DIMITRIOS VERVERIDIS 等: "Gaussian Mixture Modeling by Exploiting the Mahalanobis Distance", 《IEEE TRANSACTIONS ON SIGNAL PROCESSING》 *
LORI MCCAY-PEET 等: "On Saliency, Affect and Focused Attention", 《PROCEEDING OF THE SIGCHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS》 *
ZHOU WANG 等: "Information Content Weighting for Perceptual Image Quality Assessment", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 *
吕宝成 等: "图像感兴趣区域检测技术研究", 《中国科技信息》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110298341A (en) * 2019-06-12 2019-10-01 上海大学 A kind of enhancing saliency prediction technique based on direction selection
CN110298341B (en) * 2019-06-12 2023-09-19 上海大学 Enhanced image significance prediction method based on direction selectivity
CN110503162A (en) * 2019-08-29 2019-11-26 广东工业大学 A kind of media information prevalence degree prediction technique, device and equipment

Also Published As

Publication number Publication date
CN103218815B (en) 2016-03-30

Similar Documents

Publication Publication Date Title
Gu et al. No-reference quality metric of contrast-distorted images based on information maximization
Gao et al. Biologically inspired image quality assessment
Jian et al. Visual-patch-attention-aware saliency detection
Sengur Wavelet transform and adaptive neuro-fuzzy inference system for color texture classification
CN101103378B (en) Device and method for creating a saliency map of an image
Zhang et al. Kurtosis-based no-reference quality assessment of JPEG2000 images
Wang et al. Illumination compensation for face recognition using adaptive singular value decomposition in the wavelet domain
Fei et al. Perceptual image quality assessment based on structural similarity and visual masking
CN104361593A (en) Color image quality evaluation method based on HVSs and quaternions
Bruce Features that draw visual attention: an information theoretic perspective
CN103607589B (en) JND threshold value computational methods based on hierarchy selection visual attention mechanism
Zhang et al. No-reference image quality assessment using structural activity
Li et al. Blind image quality assessment in the contourlet domain
Liu et al. An efficient no-reference metric for perceived blur
Kuo et al. Improved visual information fidelity based on sensitivity characteristics of digital images
CN103544488A (en) Face recognition method and device
CN110111347A (en) Logos extracting method, device and storage medium
CN103258326B (en) A kind of information fidelity method of image quality blind evaluation
Shi et al. A perceptual image quality index based on global and double-random window similarity
Ein-shoka et al. Quality enhancement of infrared images using dynamic fuzzy histogram equalization and high pass adaptation in DWT
Nikvand et al. Image distortion analysis based on normalized perceptual information distance
CN109509201A (en) A kind of SAR image quality evaluating method and device
CN103218815A (en) Method for statistical calculation of image saliency map by means of natural scenes
Bruni et al. Jensen–Shannon divergence for visual quality assessment
Thriveni Edge preserving Satellite image enhancement using DWT-PCA based fusion and morphological gradient

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20190704

Address after: Room 1103, Building 21, 39 Jibang Road, Zhongming Town, Shanghai 202163

Patentee after: SHANGHAI JILIAN NETWORK TECHNOLOGY Co.,Ltd.

Address before: 200433 No. 220, Handan Road, Shanghai, Yangpu District

Patentee before: Fudan University

TR01 Transfer of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20160330

CF01 Termination of patent right due to non-payment of annual fee