CN101493890A - Dynamic vision caution region extracting method based on characteristic - Google Patents

Abstract

The invention relates to a dynamic visual attention area extraction method based on features in the technical field of machine vision. The method comprises the following steps: first, an independent component analysis method is adopted for carrying out sparse decomposition to mass natural images so as to obtain a group of filtering base functions and a group of corresponding reconfigurable base functions and the input images are divided into small RGB blocks of m multiplied by m and projected on a group of the base to obtain the features of the images; second, effective encoding principle is utilized to measure an incremental encoding length index for each feature; third, according to the incremental encoding length index, the remarkable degree of each small block is processed through the energy reallocation of each feature and finally a remarkable map is obtained. The method can reduce a 'time slice', realize continuous sampling, therefore, data of different frames can direct the processing of remarkable degrees together, and the problem that the remarkable degrees of different frames require independent process is solved so as to realize the dynamic performance.

Description

Dynamic vision caution region extracting method based on feature

Technical field

What the present invention relates to is a kind of method of technical field of image processing, and specifically, what relate to is a kind of dynamic vision attention method for extracting region based on feature.

Background technology

Along with the constantly development of artificial intelligence technology ground, machine vision is more and more in real-life application, it mainly comes anthropomorphic dummy's visual performance with computing machine, but be not only to be the simple extension of human eye, the more important thing is part of functions with human brain---information extraction from the image of objective things, handle and understood, finally be used for actual detected, measurement and control.Since machine vision have speed fast, contain much information, function is many, it is in quality testing, authentication, object detection and identification, robot, the application of automatic vehicle etc. is very extensive.

Can be made at various aspects on the engineering at present and (comprise the visual angle, Snazzi degree, broad spectrum activity, dynamic perfromance) all surpasses the sensor of human eye ability, we can say that the exploration to " looking " has arrived to a certain degree, but Vision Builder for Automated Inspection not only needs " looking ", also needs " feel ".Because people's Selective Attention Mechanism has guaranteed that human eye obtains the high efficiency of information, has attracted people's attention and research, the extractive technique in various vision attentions zone is suggested and has obtained to use widely.For example, utilize extractive technique based on the vision attention zone of Selective Attention Mechanism to find area-of-interest in the image, preferentially in these zones, search for then, thereby improved the efficient of object detection and identification; The area-of-interest that utilization is found carries out picture compression efficiently (the interesting areas ratio of compression is low, other regional ratio of compression height), and image zooming (the scaling ratio of interesting areas is greater than other zones), or the like.The extractive technique in vision attention zone has huge advantage in the high efficiency of obtaining information, thereby in often occurring in the Vision Builder for Automated Inspection processing procedure.

Find that through the literature search to prior art the extractive technique in vision attention zone is the remarkable map (Saliency Map) that Koch and Ullman proposed in 1985, this technology was perfect through Itti and Koch afterwards, formed a whole set of system about remarkable map.Specifically can list of references: " LAURENT I; CHRISTOF K; ERNST N.A model of saliency-based visual attention for rapid sceneanlysis[J] .IEEE Transactions on PAMI; 1998; 20 (11): 1254-1259 ", (author: LAURENTI, CHRISTOF K, ERNST N, exercise question: a visual attention model that can be used for quick scene analysis, magazine: pattern analysis and machine intelligence IEEE proceedings, 1998 20 volumes based on significance (saliency), o. 11th, the 1254-1259 page or leaf).This method is based on the extractive technique in space, at first picture is divided into color, direction, brightness, several parallel passages such as texture, extract information respectively for each passage then, form one and kept the picture topological structure, but the feature topomap (feature maps) that simultaneously the response power of feature is had record, next, each feature topomap all passes through a series of yardsticks " sombrero (Difference of Gaussian) " function and carries out filtering, and it is that the Gaussian function of two different scales is asked the function that obtains after the difference.This function is very responsive for change detected, and a little less than the reflection very of the signal of Mass, has general biological meaning for one.Next, the victor of use competition network wins (Winner-Take-All) model entirely and compares for different attention zones, finally generates a map about each some significance of the overall situation, is called remarkable map.Though this method and the analytical technology based on the space afterwards have good performance in a lot of scenes, but nearly all inevasiblely being faced with following problem: 1) can only pay close attention to specific a part of visual cues, 2) distribution of notice is discontinuous in time.For example, when continuous image was observed, system just can't consider the situation of multiframe, and this just causes each constantly all to need to reanalyse remarkable map individually, made the continuity of system, and reliability all declines to a great extent.And, when the position of visual angle and object changes, because not to the tracking mechanism of feature, the prediction of new remarkable map is very possible to be offset with preceding frame.In addition, inhibition is for example returned in a series of vision attention behaviors, and the viewpoint transfer etc., all can't in analytical technology, well be realized based on the space.

Summary of the invention

The objective of the invention is to overcome the deficiencies in the prior art, a kind of dynamic vision caution region extracting method based on feature is provided, this method is based on feature itself to the definition of significance, but not the spatial distribution differences of feature, can eliminate " timeslice ", continuous sampling, thus the data of different frame (time) can instruct the processing of significance together, the significance that has solved different frame (time) needs the problem of independent processing, has realized dynamic.

The present invention is achieved by the following technical solutions, may further comprise the steps:

The first step, adopt the independent component analysis method that a large amount of natural images is carried out Sparse Decomposition, obtain one group of filtering basis function and one group of corresponding reconstruct basis function, the image of input is divided into the RGB fritter of m * m, and project on this group base, obtain the feature of this figure;

Second step, utilize the efficient coding principle, promptly when a system was efficient coding, the principle of its entropy maximum was for each feature is weighed the incremental encoding length index;

In the 3rd step,, by the energy of each feature is redistributed the significance of handling each fritter, thereby finally obtain remarkable map according to these incremental encoding length index.

The described first step, specific as follows:

1. will train picture to be divided into the colored fritter of RGB of several m * m pixel size, and with each fritter vectorization.Natural picture is sampled, obtains the colored fritter of RGB of a large amount of m * m, with it as training sample.The value of m can be 8,16 or 32.M is the length of side of the colored fritter of each RGB.

2. by independent component analysis (ICA) method of standard, train basis function (A, W).The number of basis function is m * m * 3=3m ², promptly $W=[{\mathrm{<figure-callout\; id="1"\; label="w"\; filenames="A20091004668800122.png"\; state="\{\{state\}\}">w</figure-callout>}}_1,w_2,\&CenterDot;\&CenterDot;\&CenterDot;w_{{3m}^2}],$ W wherein _iBe that (size of A is the same with W, 1≤i≤3m for i filtering basis function ²).A, W are the basis functions that the ICA method trains out, and its value can be got any range, by the input decision.

3. for any width of cloth picture X, be divided into the RGB fritter of n m * m, form sampling matrix X=[x ₁, x ₂... x _n], x wherein _kThe vectorization that is k image block represents that (1≤k≤n) is to x _kCarry out linear transformation $S_k={\mathrm{Wx}}_k=[s_{k,1},s_{k,2},...,s_{k,{3m}^2}],$ Wherein W is the filtering basis function that trains.S then _kBe the coefficient of correspondence of basis function, just picture fritter x _kFeature, s _{K, i}Be the coefficient of correspondence of i basis function, be the value of i feature.To all x _kAll do same processing, obtain the feature S=[S of X ₁, S ₂..., S _n].N is the number that input picture X is divided into the RGB fritter, and its value is determined by the value of the size of input picture X and m.

After first step processing finishes,, constructed 3m for input picture X ²Next individual feature S carried out for second step.

Described second step, specific as follows:

1. for each feature, computing activation rate p _i

$p_i=\frac{{\mathrm{\&Sigma;}}_k{s_{k,i}}^2}{{\mathrm{\&Sigma;}}_j{\mathrm{\&Sigma;}}_k{s_{k,j}}^2}---\left(2.1\right)$

This amount has been represented the average energy granting level of this feature.

2. consider the activity ratio p of entropy i feature _iOn variation, i.e. the incremental encoding length index of i feature, order $p=\{{\mathrm{<figure-callout\; id="1"\; label="p"\; filenames="A20091004668800122.png"\; state="\{\{state\}\}">p</figure-callout>}}_1,p_2,p_{{3m}^2}\},$ Probability distribution for stochastic variable.Suppose that the feature activation rate of certain particular moment is distributed as p, can be to p when i feature is activated _iBring a small disturbance ε, therefore new distribution

Become:

$\hat{p}=\left\{\begin{array}{cc}\frac{p_j+\mathrm{\&epsiv;}}{1+\mathrm{\&epsiv;}},\mathrm{if}& j=\mathrm{<figure-callout\; id="1"\; label="i\; p\; j"\; filenames="A20091004668800122.png"\; state="\{\{state\}\}">i</figure-callout>}& \\ \frac{p_j}{}\frac{{}_{}}{1+\mathrm{\&epsiv;}},\mathrm{if}& j\&NotEqual;i\end{array}\right.---\left(2.2\right)$

Therefore, the incremental encoding length of i feature is:

$\mathrm{ICL}\left(p_i\right)=\frac{\&PartialD;H\left(p\right)}{\&PartialD;p_i}=1-H\left(p\right)-p_i-\log p_i-p_i\log p_i---\left(2.3\right)$

The present invention is by the principle of predictive coding, with energy, feature and significance hook.Incremental encoding length (ICL) has been weighed the entropy rate of change to perception of each feature.This index is used for instructing energy distribution, thereby realizes that system in its entirety realizes predictive coding---and be that the system that makes that common information can be few as far as possible produces response, and rare information is understood the strong response of triggering system usually.

Described the 3rd step, specific as follows:

1. according to the incremental encoding length index of resultant each feature that goes out, divide notable feature S set F:

SF＝{i|ICL(p _i)＞0}(3.1)

Divide { SF, the feature that SF} is unique to have determined to cause the entropy of total system to increase.And this division has the explicit mathematical meaning, and for a feature, having only when other is rare on characteristic distribution, that is to say, when this feature is carried out new observation, can cause the entropy of global feature distribution p to increase.

2. according to the predictive coding principle, redistribute energy between each feature, for the feature i in the notable feature set, d assigns weight _i(i ∈ SF):

$d_i=\frac{\mathrm{ICL}\left(p_i\right)}{{\mathrm{\&Sigma;}}_{k\&Element;\mathrm{SF}}\mathrm{ICL}\left(P_k\right)}\mathrm{ifi}\&Element;\mathrm{SF}---\left(3.2\right)$

And, define it for non-notable feature $d_k=0(k\&NotElement;\mathrm{SF}).$

3. for picture fritter x _k, its significance is defined as m _k:

$m_k={\mathrm{\&Sigma;}}_{i\&Element;\mathrm{SF}}{d}_i{w}_i^T{x}_k---\left(3.3\right)$

4. had after the significance of each picture fritter,, generated the remarkable map M of picture in its entirety by reconstruct base A:

$M=\underset{k\&Element;\mathrm{SF}}{\mathrm{\&Sigma;}}{A}_k{m}_k---\left(3.4\right)$

A wherein _kK the column vector of expression reconstruct base A.

From formula (3.3), as can be seen, not constant for the significance of picture fritter, but can change along with the time.And because in the method for the invention, sampling is a kind of continuous process, and the weight of feature can change continuously along with the increase of sampling, so just can be successfully the sampling variation be interpreted as that context notes the influence of weight to feature.So-called " notable feature ", why remarkable, all be for the characteristic distribution of current context.

The invention has the beneficial effects as follows: (1) because the filtration base that adopts is that training in advance is good, therefore when handling a new input picture, does not need to train basis function again, makes processing speed very fast, and the efficient height can be accomplished real-time processing.(2) owing to adopted based on feature itself, but not the mode of the spatial distribution differences of feature comes significance is analyzed, and eliminated the structural restriction of picture space.On handling, because continuous sampling eliminated " timeslice ", thereby the data of different frame (time) can be instructed the processing of significance together, the significance that has solved different frame (time) needs the problem of independent processing, has realized dynamic.

Description of drawings

Fig. 1. the remarkable map of static images;

Wherein: (a), (d), (g) be the input picture, (b), the remarkable map that generates for the present invention of (e), (h), (c), (f), (i) eye movement data for marking.

Fig. 2. the remarkable map of screen (dynamic vision).

Embodiment

Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment is being to implement under the prerequisite with the technical solution of the present invention, provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

1. latent structure

(1) the colored block sizes of the RGB that adopts is 8 * 8, by a large amount of natural pictures is sampled, obtains 120000 8 * 8 the colored fritter of RGB, and the colored fritter of these RGB is the training data of basis function.

(2) (A, W), because the colored fritter of the RGB of employing 8 * 8 is used as training sample, i.e. the number of m=8, so basis function is 3 * 8 to utilize the ICA method to train basis function ²=192.

(3) for the input color picture, for example the size of picture is 800 * 640, is divided into 8000 8 * 8 the colored fritter of RGB, and promptly n=8000 forms sampling matrix X=[x ₁, x ₁..., x ₈₀₀₀], x wherein _kThe vectorization that is k image block is represented, it is carried out linear transformation S _k=Wx _k=[s _{K, 1}, s _{K, 2}..., s _{K, 192}], wherein W is the filtering basis function that trains.S then _kBe the coefficient of correspondence of basis function, just picture fritter x _kFeature.s _{K, i}Be i basis function coefficient of correspondence, be i feature.

2. weigh incremental encoding length (ICL) index

(1), calculates its activity ratio p according to formula (2.1) to each feature.

(2) according to the activity ratio of each feature, weigh its incremental encoding length index according to formula (2.3)

3. generate remarkable map

(1), utilize formula (3.1) to divide notable feature S set F according to the incremental encoding length index of each feature that obtains in 2.

(2) utilize formula (3.2), redistribute the energy of each feature in the notable feature set.

(3) for picture fritter x _k,, handle its significance m according to formula (3.3) _k

(4) significance of importing each fritter of picture has been arranged, utilized formula (3.3), obtained importing the remarkable map M of picture.

Example one: the remarkable map of still picture

The RGB fritter of employing 8 * 8 is trained basis function, and (A, W), their dimension is 192.

For size is 800 * 640 input picture, is divided into 8000 8 * 8 the colored fritter of RGB, and promptly n=8000 forms sampling matrix X=[x ₁, x ₁..., x ₈₀₀₀].And calculate the basis function coefficient of correspondence by formula S=WX, i.e. the feature of X.

Obtain each feature activation rate p by formula (2.1), and weigh the incremental encoding length index of each feature according to p and formula (2.3).

Incremental encoding length index and formula (3.1) according to each feature are divided notable feature S set F, and utilize formula (3.2), redistribute the energy of each feature in the notable feature set.So for picture fritter x _k,, handle its significance m according to formula (3.3) _k, and finally utilize formula (3.4), generate the remarkable map M of input picture.

When sequentially to each the image fritter sampling on the static images time, just can estimate the characteristic distribution characteristic of this picture, thereby construct out remarkable map, the remarkable map that generates can be further be analyzed with people's eye movement data, with the correctness of verification model.Among Fig. 1, (a), (d), (g) be the input picture, (b), the remarkable map that generates for the present invention of (e), (h), (c), (f), (i) eye movement data for marking.In an embodiment, adopted document " BRUCE N; TSOTSOS J.Saliency Basedon Information Maximization[J]; Advances in Neural InformationProcessing Systems; 2006; 18:155-162 " (authors: BRUCE N, TSOTSOS J. exercise question: based on the maximized significance of quantity of information, magazine: higher nerve information handling system, 2006 18 phases, the 155-162 page or leaf) the eye movement data that provided are as benchmark, the model that has compared and traditional model, and the result shows that the present invention has obtained maximum performance.

Example two: the remarkable map in the screen

Compare same in the past class methods, a big advantage of method of the present invention is that it is continuous.Incremental encoding length is a process of upgrading continuously.The variation of the distribution of feature activation rate can be based on the spatial domain, also can occupy time domain.If consider that it is a laplacian distribution that time domain changes, and supposes p ^tBe the t frame, can think p so ^tBe before characteristic response accumulation and:

$p^t=\frac{1}{Z}\underset{\mathrm{\&tau;}=0}{\overset{t-1}{\mathrm{\&Sigma;}}}\exp \left(\frac{\mathrm{\&tau;}-t}{\mathrm{\&lambda;}}\right){\hat{p}}^{\mathrm{\&tau;}}$

Wherein λ is the half life period, $Z=\&Integral;{\hat{p}}^t\left(x\right)\mathrm{dx}$ It is normalization function.

When screen being done the vision attention extraction, face the problem of target travel and the motion of observation visual angle usually.Yet, the attention model framework based on feature under, these problems all are readily solved because feature always along with object in the visual field the position and move.

The signal to noise ratio (snr) of analysis image, it is defined as follows:

$\mathrm{SNR}\left(t\right)=\frac{{\mathrm{\&Sigma;}}_{i\&Element;F}{m}_i^t}{{\mathrm{\&Sigma;}}_{j\&NotElement;F}{m}_j^t}$

Wherein F is a manual mark " prospect ".After respectively 250 frame pictures being carried out the craft mark, just each frame is handled its significance, its process is except feature activation rate p difference, and other processes are consistent with the remarkable map that generates still picture.Afterwards, the remarkable map that generates can be compared with manual mark, analyze snr value.Fig. 2 has reflected the result, the sectional drawing of the first behavior video among the figure, and second row has reacted signal to noise ratio (S/N ratio) of the present invention, last column then is the signal to noise ratio (S/N ratio) of Itti model, as can be seen from the figure, average signal-to-noise ratio of the present invention is 0.4803, be much better than existing main flow the Itti model 0.1680.

Claims (4)

1, a kind of dynamic vision caution region extracting method based on feature is characterized in that may further comprise the steps:

The first step, adopt the independent component analysis method that a large amount of natural images is carried out Sparse Decomposition, obtain one group of filtering basis function and one group of corresponding reconstruct basis function, the image of input is divided into the RGB fritter of m * m, and project on this group base, obtain the feature of this figure;

Second step is for each feature is weighed the incremental encoding length index;

In the 3rd step,, by the energy of each feature is redistributed the significance of handling each fritter, thereby finally obtain remarkable map according to these incremental encoding length index.

2, the dynamic vision caution region extracting method based on feature according to claim 1 is characterized in that, the described first step is specific as follows:

1. will train picture to be divided into the colored fritter of RGB of several m * m pixel size, and, natural picture be sampled each fritter vectorization, obtain the colored fritter of RGB of a large amount of m * m, as training sample, the value of m is 8 with it, 16 or 32, m is the length of side of the colored fritter of each RGB;

2. by the independent component analysis method of standard, (A, W), the number of basis function is m * m * 3=3m to train basis function ², promptly $W=[w_1,w_2,...w_{{3m}^2}],$ W wherein _iBe i filtering basis function, the size of A is the same with W, 1≤i≤3m ², A, W are the basis functions that the ICA method trains out;

3. for any width of cloth picture X, be divided into the RGB fritter of n m * m, form sampling matrix X=[x ₁, x ₂... x _n], x wherein _kThe vectorization that is k image block represents that 1≤k≤n is to x _kCarry out linear transformation $S_k={\mathrm{Wx}}_k=[s_{k,1},s_{k,2},...,s_{k,{3m}^2}],$ Wherein W is the filtering basis function that trains, then S _kBe the coefficient of correspondence of basis function, just picture fritter x _kFeature, s _{K, i}Be i basis function coefficient of correspondence, be the value of i feature, to all x _kAll do same processing, obtain the feature S=[S of X ₁, S ₂..., S _n], n is the number that input picture X is divided into the RGB fritter, its value is determined by the value of the size of input picture X and m.

3, the dynamic vision caution region extracting method based on feature according to claim 1 is characterized in that, and is described second step, specific as follows:

1. for each feature i, calculate its activity ratio p _i: $p_i=\frac{{\mathrm{\&Sigma;}}_k{s_{k,i}}^2}{{\mathrm{\&Sigma;}}_j{\mathrm{\&Sigma;}}_k{s_{k,j}}^2}$ , and order $p=\{p_1,p_2,p_{{3m}^2}\},$ Then p is the probability density distribution of a stochastic variable, and its entropy is H (p);

2. calculate the incremental encoding length ICL (p of i feature _i):

$\mathrm{ICL}\left(p_i\right)=\frac{\&PartialD;H\left(p\right)}{\&PartialD;p_i}=1-H\left(p\right)-p_i-\log p_i-p_i\log p_i.$

4, the dynamic vision caution region extracting method based on feature according to claim 1 is characterized in that, and is described the 3rd step, specific as follows:

1. according to the incremental encoding length index of resultant each feature that goes out, divide notable feature S set F:

SF＝{i|ICL(p _i)＞0}

2. according to the predictive coding principle, redistribute energy between each feature, for the feature i in the notable feature set, d assigns weight _i, i ∈ SF:

$d_i=\frac{\mathrm{ICL}\left(p_i\right)}{{\mathrm{\&Sigma;}}_{k\&Element;\mathrm{SF}}\mathrm{ICL}\left(P_k\right)},\mathrm{ifi}\&Element;\mathrm{SF}$

And, define its weight for non-notable feature $d_k=0(k\&NotElement;\mathrm{SF});$

3. so for picture fritter x _k, its significance is defined as m _k: $m_k={\mathrm{\&Sigma;}}_{i\&Element;\mathrm{SF}}{d}_i{w}_i^T{x}_k,$ D wherein _iWeight for i the feature of distributing in 2. going on foot;

4. had after the significance of each picture fritter,, generated the remarkable map M of picture in its entirety by reconstruct base A: $M=\underset{k\&Element;\mathrm{SF}}{\mathrm{\&Sigma;}}{A}_k{m}_k,$ A wherein _kK the column vector of expression reconstruct base A.

Images