CN101944178A - Significant region extraction method for intelligent monitoring - Google Patents

Abstract

The invention provides a significant region extraction method for intelligent monitoring, which comprises the following steps of: (1) multi-scale transformation; (2) local contrast characteristic extraction; (3) characteristic graph formation; (4) global normalization; and (5) linear combination. The significance of a significant graph is expressed by the gray scale of an image, the gray scale is larger, the spot is paid with more attention and the significance is greater; on the contrary, the significance is smaller. In the invention, the visual characteristic that human eyes are sensitive to strong contrast is taken into consideration, the image is extracted in a multi-scale local contrast way by fully using the three most basic and important low-grade characteristics, namely luminance, texture and color of a representational image, and the significant region of the image is extracted by a series of operation including iterative interpolation summation, normalization treatment and linear combination; and thus, the accuracy and integrality of the significant region extraction are obviously improved.

Description

The marking area extracting method that is used for intelligent monitoring

Technical field

The present invention relates to a kind of marking area extractive technique that can be applicable to intelligent monitoring, belong to image, multimedia signal processing technique field.

Background technology

Along with social development, the application of intelligent monitor system more and more generalizes.Such as, in some important public places such as customs, bank, station, shops rig camera can be housed all now, but the video data for the monitoring gained under most of situation all is with unified resolution monitoring scene to be gathered gained, usually often because resolution can not fully be applied in the important area, cause and can't differentiate effectively important area or object, this processing mode seems loaded down with trivial details and causes trouble.As: criminal investigation and case detection etc.Therefore, press for intelligent monitoring and can survey important area automatically.

Along with the development of bio-science, many achievements in research show that human eye is unequal for each regional degree of concern of visual pattern.For stimulating stronger image-region, human eye always drops into more vision attention; And for flat site, the vision attention that human eye drops into is less, this just human vision having the main cause place of high-efficiency information processing capacity and target acquistion function.If so attention mechanism of the more accurate emulation biological vision of intelligent monitor system energy, only to existing the marking area of interesting target to pay close attention in the image, then will not pay close attention to for flat site, redundant information in the image just can be given up in advance so, and possible important area, be the target area, then can obtain the concern of emphasis.Therefore, the research visual attention model that the can be applicable to intelligent monitoring marking area that extracts image is surveyed collection as the important area of monitoring.

Research to visual attention model extraction marking area has in recent years become the research focus, has proposed some visual attention models and has been applied to many aspects.Classical Itti attention model, low-level features such as the color by extracting input picture, brightness, direction are analyzed to merge and are significantly schemed.Its basic thought is the contrast of pixel in aspect such as color, brightness, direction and peripheral region is defined as the remarkable value of this point, contrasts by force more, and the remarkable value of this point is just big more.Though this model can find focus-of-attention, the marking area that obtains is quite inaccurate, and object bounds does not match in marking area and the image.For visual attention model is applied better in practice, just must on the Itti model based, improve.Because brightness, texture, color are the essential characteristics of token image, therefore, consideration is based on these three kinds of features, replace central peripheral difference in the past to calculate, adopt new part contrast characterization method to extract the marking area of image, make this marking area extracting method be more suitable for being applied in the intelligent monitoring.

Summary of the invention

The present invention is directed to deficiencies such as the extraction that has the existence of visual attention model extraction marking area algorithm at present is inaccurate, imperfect, a kind of marking area extracting method that is used for intelligent monitoring based on the local contrast of multiple dimensioned low-level features is proposed, the method is significantly improved on accuracy that marking area extracts and integrality, and meets the visual characteristic of human eye.

The marking area extracting method that is used for intelligent monitoring of the present invention may further comprise the steps:

(1) multi-scale transform: input picture is carried out Filtering Processing, obtain the image of input picture on m different scale, m ∈ 4,6,8};

(2) local contrast feature extraction: on different scale, calculate the part contrast figure of brightness, texture, three low-level features of color respectively, it is to adopt the pixel window of a slip to calculate the local control value of each position that local contrast figure calculates, adopt for brightness and to receive the method for (Weber-Fechner) rule based on weber-Fick and calculate its local contrast figure, calculate its local contrast figure for texture by gray variance, calculate its local contrast figure for color employing colour-difference method in based on the HSI color space of visually-perceptible;

(3) characteristic pattern forms: the part contrast figure on each yardstick carries out iteration interpolation summation operation to each feature, forms the characteristic pattern of each feature;

(4) global normalization: the remarkable value of each pixel in three width of cloth characteristic patterns is calculated in normalization, obtains three width of cloth and significantly schemes;

(5) linear combination: the remarkable figure of three width of cloth is carried out linear in conjunction with obtaining final remarkable figure.

Significantly the conspicuousness among the figure is to represent by the gray scale of image, and the place that gray-scale value is big more is subjected to degree of concern strong more, and conspicuousness is big more; Otherwise conspicuousness is more little.

Filtering in the described step (1) is to use the gaussian pyramid model that input picture is carried out multiple-stage filtering.

Calculating local contrast figure in the described step (2) adopts the local pixel window of a slip to calculate the local control value of each position; When calculating the control value of each position pixel, this position is corresponding with the center pixel of window, calculates this point and the contrast in the zone of window size on every side, and the value of gained is as the local control value of this location point; Brightness, texture, color are calculated respectively, obtained the local contrast figure of three width of cloth.

It is as follows to receive the local contrast of the brightness figure computing formula of (Weber-Fechner) rule based on weber-Fick:

$I_{\mathrm{CM}}(x,y)=\mathrm{clg}\frac{I_j^{\max }}{I_j^{\mathrm{avg}}}=\mathrm{clg}\frac{\max \{I_1,I_2,...I_n,...,I_N\}}{\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n}$

Wherein, I _CM(x, y) be pixel (x, the brightness control value of y) locating, c are constant,

With

Be respectively brightness maximal value and the mean value in j the window, the number of pixels in N=(2k+1) * (2k+1) the expression window.I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window.

Computing formula based on the local contrast of the texture of gray variance figure is as follows:

$T_{\mathrm{CM}}(x,y)={\left[\frac{1}{N-1}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{(I_n-\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n)}^2\right]}^{\frac{1}{2}}$

Wherein, I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window.

Part contrast figure computing method based on the HSI color space of visually-perceptible are as follows: at first to two value of color Y at the HSI color space ₁=(H ₁, S ₁, I ₁) ^TAnd Y ₂=(H ₂, S ₂, I ₂) ^T, the definition color difference is:

${\mathrm{\Δ}}_{\mathrm{HSI}}(Y_1,Y_2)=\sqrt{{\left({\mathrm{\Δ}}_I\right)}^2+{\left({\mathrm{\Δ}}_C\right)}^2}$

Δ wherein _I=| I ₁-I ₂| and ${\mathrm{\Δ}}_C=\sqrt{S_1^2+S_2^2-2S_1{S}_2\cos \mathrm{\θ}}$

$\mathrm{\θ}=\left\{\begin{array}{cc}|H_1-H_2|;& \mathrm{if}|H_1-H_2|\≤\mathrm{\π}\\ 2\mathrm{\π}-|H_1-H_2|;& \mathrm{if}|H_1-H_2|>\mathrm{\π}\end{array}\right.$

Therefore, the local contrast figure of color is calculated as follows formula:

$C_{\mathrm{CM}}(x,y)=\frac{1}{N-1}\left[\underset{n=1}{\overset{N-1}{\mathrm{\Σ}}}{\mathrm{\Δ}}_{\mathrm{HSI}}(Y(x,y),Y_n)\right]$

In step (2), calculate the part contrast figure of each feature on different scale respectively, i.e. brightness, texture, each correspondence of color obtain several local contrast figure.

The step that step (3) forms the characteristic pattern of each feature is:

A is inserted into the higher scale size of resolution that is adjacent from the minimum yardstick of resolution in the part contrast figure on the yardstick that resolution is lower;

Part contrast figure on the corresponding yardstick with it of part contrast figure of b after with interpolation carries out addition;

The above step a of c, b carry out repeated iterative operation, end up to adding up on the highest yardstick of resolution, obtain characteristic pattern.

The present invention considers that human eye is for the responsive visual characteristic of strong contrast, make full use of the most basic, most important brightness, texture and three low-level features of color of token image, image is carried out the extraction of multiple dimensioned local contrast, through the summation of iteration interpolation, normalized, the marking area of image is extracted in the operation of linear junction unification series; On accuracy that marking area extracts and integrality, be significantly improved.

Description of drawings

Fig. 1 is the frame diagram of the inventive method.

Fig. 2 is the remarkable figure that obtains by the inventive method emulation.

Fig. 3 is the contrast of the visual attention location location drawing that obtains by the eye movement experimental verification and the remarkable figure that obtains by the inventive method.

Embodiment

The present invention adopts the part based on multiple dimensioned low-level features to contrast and extracts marking area, the part contrast that is about to the pixel feature is extracted as the size of weighing significance, in conjunction with the multi-scale transform algorithm, under different scale, ask for contrast figure, and the algorithm that adopts the summation of iteration interpolation is combined into the characteristic pattern that a width of cloth and original image have identical size with the contrast figure of different scale.By normalization, linear then in conjunction with the remarkable figure that obtains image.

Fig. 1 has provided the general frame figure of the inventive method, and according to flow process shown in Figure 1, method of the present invention comprises following concrete steps:

1. multiple dimensioned local contrast figure forms

Input picture is carried out multiple-stage filtering with gaussian pyramid, and down-sampling obtains the image of original image on m different scale, and yardstick 1 is an input picture.Yardstick 2 to m is respectively 1/2 to 1/2 of input picture ^M-1, other increases along with sample stage, and the resolution of image reduces gradually.On each yardstick, calculate the part contrast figure of brightness, texture and color characteristic respectively.Corresponding each feature obtains the part contrast figure under the m width of cloth different scale respectively.Wherein: m ∈ 4,6,8}.

The specific algorithm of the part contrast figure of brightness, texture, three kinds of features of color is as follows:

Adopted a size to calculate the local control value of each position among the present invention for the local pixel window of (2k+1) * (2k+1) (k is a nonnegative integer).When calculating the control value of each position pixel, this position is corresponding with the center pixel of window, calculates this point and the contrast in the zone of window size on every side, and the value of gained is as the local control value of this location point.

The calculating of brightness contrast figure, receive (Weber-Fechner) rule as can be known according to weber-Fick: visually the logarithm of subjective luminance and light stimulus intensity is proportional.Therefore calculate of the brightness contrast of the subjective luminance of each pixel with following formula, that is: as this point

$I_{\mathrm{CM}}(x,y)=\mathrm{clg}\frac{I_j^{\max }}{I_j^{\mathrm{avg}}}=\mathrm{clg}\frac{\max \{I_1,I_2,...I_n,...,I_N\}}{\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n}$

Wherein, I _CM(x, y) be pixel (c is a constant for x, the brightness of y) locating contrast,

With

Be respectively brightness maximal value and the mean value in j the window, the number of pixels in N=(2k+1) * (2k+1) the expression window.I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window.

The calculating of texture contrast figure, we select for use the simplest gray variance to describe the coarse texture degree of image block.Computing formula is as follows:

$T_{\mathrm{CM}}(x,y)={\left[\frac{1}{N-1}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{(I_n-\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n)}^2\right]}^{\frac{1}{2}}$

Wherein, I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window.

The comparison based on the HSI color space colour-difference of visually-perceptible is considered in the calculating of color contrast figure.To two value of color Y at the HSI color space ₁=(H ₁, S ₁, I ₁) ^TAnd Y ₂=(H ₂, S ₂, I ₂) ^T, color difference is defined as:

${\mathrm{\Δ}}_{\mathrm{HSI}}(Y_1,Y_2)=\sqrt{{\left({\mathrm{\Δ}}_I\right)}^2+{\left({\mathrm{\Δ}}_C\right)}^2}$

Δ wherein _I=| I ₁-I ₂| and ${\mathrm{\Δ}}_C=\sqrt{S_1^2+S_2^2-2S_1{S}_2\cos \mathrm{\θ}}$

$\mathrm{\θ}=\left\{\begin{array}{cc}|H_1-H_2|;& \mathrm{if}|H_1-H_2|\≤\mathrm{\π}\\ 2\mathrm{\π}-|H_1-H_2|;& \mathrm{if}|H_1-H_2|>\mathrm{\π}\end{array}\right.$

The color contrast is calculated as follows formula:

$C_{\mathrm{CM}}(x,y)=\frac{1}{N-1}\left[\underset{n=1}{\overset{N-1}{\mathrm{\Σ}}}{\mathrm{\Δ}}_{\mathrm{HSI}}(Y(x,y),Y_n)\right]$

2. the formation of characteristic pattern

In the forming process of characteristic pattern, the present invention takes a kind of iteration interpolation summation algorithm.This algorithm begins to carry out successively upwards interpolation, summation from the minimum yardstick m of resolution.Finally on the highest yardstick 1 of resolution, obtain characteristic pattern.

The process of this algorithm is as follows:

for?σ＝1：m-1

% carries out the interpolation operation;

I _CM(m+1-σ)＝im_interpolation(I _CM(m+1-σ)，size(I _CM(m-σ)))

% sues for peace on same yardstick;

I _CM(m-σ)＝im_integrate(I _CM(m+1-σ)，I _CM(m-σ))

end

Wherein, σ is the yardstick of contrast figure.This algorithmic procedure is applied to brightness, texture and color contrast figure, obtains corresponding 3 width of cloth characteristic patterns.

3. global normalization is calculated

To by the brightness that obtains in the step 2, texture and color characteristic figure, the overall significance of each pixel is calculated in normalization, and method is as follows:

$S_{\mathrm{SM}}(x,y)=\frac{F(x,y)-\min (F_1,F_2,...,F_{M\×N})}{\max (F_1,F_2,...,F_{M\×N})-\min (F_1,F_2,...,F_{M\×N})}\×255$

In the formula, S _SM∈ { I _SM, T _SM, C _SMRepresent significantly to scheme F ∈ { I _FM, T _FM, C _FMRepresenting corresponding characteristic pattern, M and N represent S _SMWidth and the height.

4. linear combination

Through after the normalization, three width of cloth are significantly schemed I _SM, T _SM, C _SMSeparately through forming final remarkable figure after the weighting, associated methods as shown in the formula:

S＝w ₁×I _SM+w ₂×T _SM+w ₃×C _SM

W wherein _iThe expression weighted value, and satisfy $\underset{i=1}{\overset{3}{\mathrm{\Σ}}}{w}_i=1.$

For proving the performance of this vision mode, two experiments have been carried out.Emulation experiment 1 is selected great amount of images, uses the algorithm of the marking area extraction of the present invention's proposition, experimentizes.Experimental result as shown in Figure 2.Fig. 2 (a) is an input picture, the remarkable figure that obtains during multiple dimensioned factor m=6 among Fig. 2 (b).The part contrast vision mode algorithm based on multiple dimensioned low-level features of the present invention's proposition can extract accurate and complete marking area as seen from Figure 2, and the conspicuousness of remarkable position is more outstanding.The scope that the multiple dimensioned factor is not limited to fix can select different scale filtering to calculate according to the different images size, chooses the optimum filtering yardstick according to experimental result.In order to verify that whether the visual attention model that the present invention proposes meets human visual characteristic, has carried out the eye movement experiment under existing vision track hardware condition.Experimental result as shown in Figure 3.The marking area that extracts of the visual attention location model that proposes of the present invention meets human-eye visual characteristic as seen from Figure 3.

To sum up analyze, simulation result and eye movement experimental results show that the present invention is significantly improved and meets the visual characteristic of human eye on accuracy that marking area extracts and integrality.Therefore, can be used in the intelligent monitoring important area detection collection.

Claims (5)

1. marking area extracting method that is used for intelligent monitoring may further comprise the steps:

(1) multi-scale transform: input picture is carried out Filtering Processing, obtain the image of input picture on m different scale, m ∈ 4,6,8};

(2) local contrast feature extraction: on different scale, calculate the part contrast figure of brightness, texture, three low-level features of color respectively, it is to adopt the pixel window of a slip to calculate the local control value of each position that local contrast figure calculates, calculate its local contrast figure for the brightness employing based on weber-Fick nanofarad method then, calculate its local contrast figure for texture by gray variance, calculate its local contrast figure for color employing colour-difference method in based on the HSI color space of visually-perceptible;

(3) characteristic pattern forms: the part contrast figure on each yardstick carries out iteration interpolation summation operation to each feature, forms the characteristic pattern of each feature;

(4) global normalization: the remarkable value of each pixel in three width of cloth characteristic patterns is calculated in normalization, obtains three width of cloth and significantly schemes;

(5) linear combination: the remarkable figure of three width of cloth is carried out linear in conjunction with obtaining final remarkable figure.

2. the marking area extracting method that is used for intelligent monitoring according to claim 1 is characterized in that: the conspicuousness among the described remarkable figure is to represent by the gray scale of image, and the place that gray-scale value is big more is subjected to degree of concern strong more, and conspicuousness is big more; Otherwise conspicuousness is more little.

3. the marking area extracting method that is used for intelligent monitoring according to claim 1 is characterized in that: the filtering in the described step (1) is to use the gaussian pyramid model that input picture is carried out multiple-stage filtering.

4. the marking area extracting method that is used for intelligent monitoring according to claim 1, it is characterized in that: in the described step (2) when calculating the control value of each position pixel, this position is corresponding with the center pixel of window, calculate this point and the contrast in the zone of window size on every side, the value of gained is as the local control value of this location point; Brightness, texture, color are calculated respectively, obtained the local contrast figure of three width of cloth;

Computing formula based on the local contrast of weber-Fick nanofarad brightness then figure is as follows:

$I_{\mathrm{CM}}(x,y)=\mathrm{clg}\frac{I_j^{\max }}{I_j^{\mathrm{avg}}}=\mathrm{clg}\frac{\max \{I_1,I_2,...I_n,...,I_N\}}{\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n}$

Wherein, I _CM(x, y) be pixel (x, the brightness control value of y) locating, c are constant,

With

Be respectively brightness maximal value and the mean value in j the window, the number of pixels in N=(2k+1) * (2k+1) the expression window.I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window;

Computing formula based on the local contrast of the texture of gray variance figure is as follows:

$T_{\mathrm{CM}}(x,y)={\left[\frac{1}{N-1}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{(I_n-\frac{1}{N}\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{I}_n)}^2\right]}^{\frac{1}{2}}$

Wherein, I _n(n ∈ 1,2 ... N) brightness value of any pixel in the expression window;

Part contrast figure computing method based on the HSI color space of visually-perceptible are as follows: at first to two value of color Y at the HSI color space ₁=(H ₁, S ₁, I ₁) ^TAnd Y ₂=(H ₂, S ₂, I ₂) ^T, the definition color difference is:

${\mathrm{\Δ}}_{\mathrm{HSI}}(Y_1,Y_2)=\sqrt{{\left({\mathrm{\Δ}}_I\right)}^2+{\left({\mathrm{\Δ}}_C\right)}^2}$

Δ I=|I wherein ₁-I ₂| and ${\mathrm{\Δ}}_C=\sqrt{S_1^2+S_2^2-2S_1{S}_2\cos \mathrm{\θ}}$

$\mathrm{\θ}=\left\{\begin{array}{cc}|H_1-H_2|;& \mathrm{if}|H_1-H_2|\≤\mathrm{\π}\\ 2\mathrm{\π}-|H_1-H_2|;& \mathrm{if}|H_1-H_2|>\mathrm{\π}\end{array}\right.$

Therefore, the local contrast figure of color is calculated as follows formula:

$C_{\mathrm{CM}}(x,y)=\frac{1}{N-1}\left[\underset{n=1}{\overset{N-1}{\mathrm{\Σ}}}{\mathrm{\Δ}}_{\mathrm{HSI}}(Y(x,y),Y_n)\right].$

5. the marking area extracting method that is used for intelligent monitoring according to claim 1 is characterized in that: the step that forms the characteristic pattern of each feature in the described step (3) is:

A is inserted into the higher scale size of resolution that is adjacent from the minimum yardstick of resolution in the part contrast figure on the yardstick that resolution is lower;

Part contrast figure on the corresponding yardstick with it of part contrast figure of b after with interpolation carries out addition;

The above step a of c, b carry out repeated iterative operation, end up to adding up on the highest yardstick of resolution, obtain characteristic pattern.

Images