CN103632153A - Region-based image saliency map extracting method - Google Patents

Abstract

The invention discloses a region-based image saliency map extracting method. The method includes: firstly, calculating a global color histogram of an image to obtain an image saliency map based on the global color histogram; secondly, adopting superpixel segmentation technology to segment the image, calculating color contrast and space sparsity of each region, and weighting by the aid of similarity among the regions to obtain an image saliency map based on the region color contrast and an image saliency map based on the region space sparsity; finally, fusing the image saliency map based on the global color histogram, the image saliency map based on the region color contrast and the image saliency map based on the region space sparsity to obtain a final image saliency map. The method has the advantage that the obtained image saliency map can well reflect saliency changes of global and local regions, and conforms to image saliency semantic features.

Description

Region-based image saliency map extraction method

Technical Field

The present invention relates to a method for processing an image signal, and more particularly, to a method for extracting an image saliency map based on a region.

Background

In human visual reception and information processing, due to limited brain resources and difference in importance of external environment information, the human brain does not have the same sense as the external environment information but shows selective characteristics in the processing process. People are not evenly focused on every area of an image when watching the image or video clip, but are more focused on some salient areas. How to detect and extract the salient regions with high visual attention in the video is an important research content in the field of computer vision and content-based video retrieval.

The existing saliency map model is a selective attention model that models the visual attention mechanism of living beings, the method calculates the contrast of each pixel point with the surrounding background in the aspects of color, brightness and direction, and forms a saliency map by the saliency values of all the pixel points, however, the method can not well extract the saliency map information of the image, this is because pixel-based salient features do not reflect well the salient semantic features of the human eye when viewed, and the region-based salient features can effectively improve the stability and accuracy of extraction, and therefore, how to perform region segmentation on the image, how to extract the features of each region, how to describe the salient features of each region, how to measure the saliency of the region and the saliency between the regions is a problem to be researched and solved in the region-based saliency map extraction.

Disclosure of Invention

The invention aims to provide a region-based image saliency map extraction method which accords with salient semantic features and has higher extraction stability and accuracy.

The technical scheme adopted by the invention for solving the technical problems is as follows: a region-based image saliency map extraction method is characterized by comprising the following steps:

recording the source image to be processed as { I_i(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ H_i(x, y) }, H denotes { I_iHigh of (x, y) }, I_i(x, y) represents { I_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, wherein the 1 st component is an R component, the 2 nd component is a G component and the 3 rd component is a B component;

② first obtaining { I_i(x, y) } quantized image and global color histogram of quantized image, then according to { I }_i(x, y) } obtaining { I } from the quantized image_iThe color type of each pixel point in (x, y) } is determined according to { I }_iGlobal color histogram of quantized image of (x, y) } and { I_iThe color type of each pixel point in (x, y) } is obtained to obtain { I }_i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }_iThe coordinate position in the (x, y) } is the significant value of the pixel point of (x, y) based on the global color histogram;

(iii) using superpixel segmentation technique to divide { I_i(x, y) } into M non-overlapping regions, and then dividing { I }_i(x, y) } is re-represented as a set of M regions, denoted as { SP }_h}, recalculating { SP_hSimilarity between the respective regions in (will) { SP }_hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)_p,SP_q) Wherein M is more than or equal to 1, SP_hRepresents SP_hIn the h-th area, h is more than or equal to 1 and less than or equal to M, p is more than or equal to 1 and less than or equal to M, q is more than or equal to 1 and less than or equal to M, p is not equal to q, SP is equal to_pRepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th region in (1);

fourthly, according to the { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_i(x, y) } is an image saliency map based on regional color contrast, and is marked as { NGC (x, y) }, wherein the NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in the { NGC (x, y) };

fifthly, according to { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_i(x, y) } is an image saliency map based on region space sparsity and is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) };

sixthly, { I_i(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }_i(x, y) } region color contrast based image saliency maps { NGC (x, y) } and { I_i(x, y) } image saliency maps { NSS (x, y) } based on region space sparsity are fused to obtain { I_iThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).

The concrete process of the second step is as follows:

2- (1) pair of_iRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }_i(x, y) } quantized image, denoted as { P }_i(x, y) }, will { P_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as P_i(x,y)，

Wherein, the symbol

Is a rounded-down symbol;

2, calculating { P_i(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) represents_iThe number of all pixel points belonging to the kth color in (x, y) };

2-3 according to { P_i(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }_i(x, y) } the color type of the corresponding pixel point will be { I_iThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as k_xy，k_xy=P₃(x,y)×256+P₂(x,y)×16+P₁(x, y) wherein P₃(x, y) represents { P }_iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is₂(x, y) represents { P }_iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is₁(x, y) represents { P }_iColor of 1 st component of pixel point with coordinate position (x, y) in (x, y) }A value;

② 4, calculating { I_i(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { I_iThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> $D(k_{\mathrm{xy}},k)=\sqrt{{(p_{k_{\mathrm{xy}},1}-p_{k,1})}^2+{(p_{k_{\mathrm{xy}},2}-p_{k,2})}^2+{(p_{k_{\mathrm{xy}},3}-p_{k,3})}^2},$ wherein D (k)_xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}_xyThe euclidean distance between the seed color and the kth color, $p_{k_{\mathrm{xy}},2}=\mathrm{mod}(k_{\mathrm{xy}}/16),$

p_k,2=mod(k/16)，

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 1 st component corresponding to a seed color,denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 2 nd component corresponding to the seed color,

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyColor value of 3 rd component, p, corresponding to a color_k,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,3Representing the color value of the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder taking operation function;

② 5 according to { I_iThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }_i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.

In step (c) { SP_hSimilarity Sim (SP) between p-th and q-th regions in_p,SP_q) The acquisition process comprises the following steps:

③ 1, pair { SP_hQuantizing the color value of each component of each pixel point in each region to obtain { SP }_hQuantized region of each region in { SP } would be_hThe quantization region of the h-th region in (1) } is denoted as { P_h,i(x_h,y_h) Will { P }_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) Of the ith component of the pixelColor value P_h,i(x_h,y_h) Suppose { P_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) Has a pixel point of { I_iThe coordinate position in (x, y) } is (x, y), then

Wherein x is more than or equal to 1_h≤W_h,1≤y_h≤H_h，W_hRepresents SP_hWidth of the H-th area in (H) } H_hRepresents SP_hHeight of h-th area in (1), signIs a rounded-down symbol;

③ 2, calculate { SP_hColor histogram of quantized region of each region in { P }, will be { P_h,i(x_h,y_h) The color histogram of is noted asWherein,

represents { P_h,i(x_h,y_h) The number of all pixel points belonging to the kth color in the pixel;

③ 3, pair { SP_hNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histogramsThe normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,

represents SP_hH-th region of { P } quantization region of the h-th region_h,i(x_h,y_h) The probability of occurrence of a pixel belonging to the k-th color in the pixel,represents SP_hQuantization region of h' th region in { P }_h',i(x_h',y_h') X is more than or equal to 1 and the number of all pixel points belonging to the k color in the pixel_h'≤W_h',1≤y_h'≤H_h'，W_h'Represents SP_hWidth of H' th area in (H) }, H_h'Represents SP_hHeight of h' th area in (P) } h_h',i(x_h',y_h') Represents { P_h',i(x_h',y_h') The position of the middle coordinate is (x)_h',y_h') The color value of the ith component of the pixel point of (1);

③ 4, calculate { SP_hThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)_p,SP_q)，Sim(SP_p,SP_q)=Sim_c(SP_p,SP_q)×Sim_d(SP_p,SP_q)，Sim_c(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Sim_d(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe spatial similarity between the q-th regions in (j),

wherein, SP_pRepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th area in (1),represents SP_hQuantization region of the P-th region in { P } quantization region of the P-th region { P_p,i(x_p,y_p) The probability of occurrence of a pixel belonging to the k-th color in the pixel,

represents SP_hQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth region_q,i(x_q,y_q) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to x_p≤W_p,1≤y_p≤H_p，W_pRepresents SP_hWidth of p-th area in (H)_pRepresents SP_hHeight of P-th area in (P) }, P_p,i(x_p,y_p) Represents { P_p,i(x_p,y_p) The position of the middle coordinate is (x)_p,y_p) The color value of the ith component of the pixel point is more than or equal to 1 and less than or equal to x_q≤W_q,1≤y_q≤H_q，W_qRepresents SP_hWidth of the q-th area in (H) } m_qRepresents SP_hHeight of the q-th area in (P) } P_q,i(x_q,y_q) Represents { P_q,i(x_q,y_q) The position of the middle coordinate is (x)_q,y_q) Min () is a minimum function,

represents SP_hThe coordinate position of the center pixel point in the p-th region in (1),

represents SP_hThe coordinate position of the central pixel point in the qth region in (1)The symbol "ii |" is a euclidean distance symbol.

The specific process of the step IV is as follows:

fourthly-1, calculating { SP_hColor contrast of each region in { SP } will be { SP }_hColor contrast of the h-th area in (1) } is noted as

<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>

Wherein, SP_hRepresents SP_hH area in (SP)_qRepresents SP_hThe q-th area in (1),

represents SP_hTotal number of pixel points included in the h-th area in (Sim)_d(SP_h,SP_q) Represents SP_hH area in the with { SP }_hThe spatial similarity between the q-th regions in (j),

represents SP_hThe coordinate position of the center pixel point in the h-th area in (1),represents SP_hThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,

represents SP_hThe color mean vector of the h-th region in (j),represents SP_hThe color mean vector of the qth region in (j);

tetra-2, pair { SP_hNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }_hColor contrast of the h-th area in (1) } color contrastThe normalized color contrast obtained after normalization was recorded as

Wherein, NGC_minRepresents SP_hMinimum color contrast of M regions in (NGC) } NGC_maxRepresents SP_hMaximum color contrast in M regions in (j);

fourthly-3, calculating { SP_hColor contrast based saliency value for each region in the will SP_hColor-based contrast of the h-th area in (1) } inIs marked as

<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Represents SP_hSimilarity between the h region and the q region in (1);

fourthly-4, mixing the { SP_hThe significant value of each area based on the color contrast is taken as the significant value of all pixel points in the corresponding area, so as to obtain { I }_iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.

The concrete process of the fifth step is as follows:

fifthly-1, calculating { SP_hSpatial sparsity of each region in { SP } will be_hThe spatial sparsity of the h-th region in (1) } is noted as

<math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Represents SP_hThe similarity between the h-th and q-th regions in (1),

represents SP_hThe central pixel point in the h-th area in (I) } and (I)_i(x, y) } euclidean distance between center pixel points;

fifthly-2, for { SP_hNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsity_hSpatial sparsity of the h-th region in

The normalized space sparsity obtained after normalization is recorded as

Wherein NSS_minRepresents SP_hMinimum spatial sparsity, NSS, of M regions in_maxRepresents SP_hMaximum spatial sparsity in M regions in (j);

fifthly-3, calculating { SP_hSignificant value based on spatial sparsity for each region in the { SP will be_hSignificant value based on spatial sparsity of the h-th region in (1) } is noted

<math> <mrow> <msub> <msup> <mi>NSS</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow> </math>

Fifthly-4, mixing { SP_hThe significant value of each region based on space sparsity is used as the significant value of all pixel points in the corresponding region, so as to obtain { I }_iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.

Compared with the prior art, the invention has the advantages that:

1) according to the method, the image saliency map based on the global color histogram, the image saliency map based on the regional color contrast and the image saliency map based on the regional space sparsity are obtained through calculation respectively and are finally fused to obtain the image saliency map, the obtained image saliency map can better reflect the saliency change conditions of the global and local regions of the image, and the stability and the accuracy are high.

2) The method provided by the invention adopts a superpixel segmentation technology to segment the image, utilizes histogram features to respectively calculate the color contrast and space sparsity of each region, and finally utilizes the similarity between the regions to carry out weighting to obtain a final image saliency map based on the regions, so that the feature information conforming to the saliency semantics can be extracted.

Drawings

FIG. 1 is a block diagram of an overall implementation of the method of the present invention;

FIG. 2a is an original Image of "Image 1";

FIG. 2b is a real (Ground route) saliency map of an "Image 1" Image;

FIG. 2c is a global color histogram based Image saliency map of an "Image 1" Image;

FIG. 2d is a region color contrast based Image saliency map of an "Image 1" Image;

FIG. 2e is an Image saliency map based on region-space sparsity of an "Image 1" Image;

FIG. 2f is the final Image saliency map of the "Image 1" Image;

FIG. 3a is an original Image of "Image 2";

FIG. 3b is a real (Ground route) saliency map of an "Image 2" Image;

FIG. 3c is a global color histogram based Image saliency map of an "Image 2" Image;

FIG. 3d is a region color contrast based Image saliency map of an "Image 2" Image;

FIG. 3e is an Image saliency map based on region-space sparsity for an "Image 2" Image;

FIG. 3f is the final Image saliency map of the "Image 2" Image;

FIG. 4a is an original Image of "Image 3";

FIG. 4b is a real (Ground route) saliency map of an "Image 3" Image;

FIG. 4c is a global color histogram based Image saliency map for an "Image 3" Image;

FIG. 4d is a region color contrast based Image saliency map of an "Image 3" Image;

FIG. 4e is an Image saliency map based on region space sparsity for an "Image 3" Image;

FIG. 4f is the final Image saliency map of the "Image 3" Image;

FIG. 5a is an original Image of "Image 4";

FIG. 5b is a real (Ground route) saliency map of an "Image 4" Image;

FIG. 5c is a global color histogram based Image saliency map for an "Image 4" Image;

FIG. 5d is a region color contrast based Image saliency map of an "Image 4" Image;

FIG. 5e is an Image saliency map based on region-space sparsity for an "Image 4" Image;

FIG. 5f is the final Image saliency map of the "Image 4" Image;

FIG. 6a is an original Image of "Image 5";

FIG. 6b is a real (Ground route) saliency map of the "Image 5" Image;

FIG. 6c is a global color histogram based Image saliency map for an "Image 5" Image;

FIG. 6d is a region color contrast based Image saliency map of an "Image 5" Image;

FIG. 6e is an Image saliency map based on region space sparsity for an "Image 5" Image;

fig. 6f is a final Image saliency map of the "Image 5" Image.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The invention provides a region-based image saliency map extraction method, the overall implementation block diagram of which is shown in FIG. 1, and the method comprises the following steps:

recording the source image to be processed as { I_i(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ H_i(x, y) }, H denotes { I_iHigh of (x, y) }, I_i(x, y) represents { I_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, the 1 st component is an R component, the 2 nd component is a G component, and the 3 rd component is a B component.

Secondly, if only local saliency is considered, the saliency of the edge with violent change or the complicated background area in the image is higher, the saliency of the interior of the smooth target area is lower, and thus the global saliency needs to be considered, wherein the global saliency refers to the saliency of each pixel point relative to the global image, so that the method firstly obtains { I }_i(x, y) } quantized image and global color histogram of quantized image, then according to { I }_i(x, y) } obtaining { I } from the quantized image_iThe color type of each pixel point in (x, y) } is determined according to { I }_i(x,y)The global color histogram of the quantized image of { I } and_ithe color type of each pixel point in (x, y) } is obtained to obtain { I }_i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }_iAnd (x, y) the significant value of the pixel point with the coordinate position of (x, y) based on the global color histogram.

In this embodiment, the specific process of step two is:

2- (1) pair of_iRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }_i(x, y) } quantized image, denoted as { P }_i(x, y) }, will { P_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as P_i(x,y)，

Wherein, the symbol

To round the symbol down.

2, calculating { P_i(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) represents_iThe number of all pixel points belonging to the k-th color in (x, y) }.

2-3 according to { P_i(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }_i(x, y) } the color type of the corresponding pixel point will be { I_iThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as k_xy，k_xy=P₃(x,y)×256+P₂(x,y)×16+P₁(x, y) wherein P₃(x, y) represents { P }_iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is₂(x, y) represents { P }_iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is₁(x, y) represents { P }_i(x, y) } middle coordinateAnd (3) the color value of the 1 st component of the pixel point with the position (x, y).

② 4, calculating { I_i(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { I_iThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> $D(k_{\mathrm{xy}},k)=\sqrt{{(p_{k_{\mathrm{xy}},1}-p_{k,1})}^2+{(p_{k_{\mathrm{xy}},2}-p_{k,2})}^2+{(p_{k_{\mathrm{xy}},3}-p_{k,3})}^2},$ wherein D (k)_xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}_xyThe euclidean distance between the seed color and the kth color,

$p_{k_{\mathrm{xy}},2}=\mathrm{mod}(k_{\mathrm{xy}}/16),$

p_k,2=mod(k/16)，

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 1 st component corresponding to a seed color,denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 2 nd component corresponding to the seed color,

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyColor value of 3 rd component, p, corresponding to a color_k,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,3And (b) a color value representing the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder-taking operation function.

② 5 according to { I_iThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }_i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.

(iii) adopting super pixel (Superpixel) segmentation technique to divide { I }_i(x, y) } into M non-overlapping regions, and then dividing { I }_i(x, y) } is re-represented as a set of M regions, denoted as { SP }_hConsidering local saliency, similar areas in the image generally have lower saliency, so the invention calculates { SP_hSimilarity between the respective regions in (will) { SP }_hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)_p,SP_q) Wherein M is more than or equal to 1, SP_hRepresents SP_hIn the h-th area, h is more than or equal to 1 and less than or equal to M,1≤p≤M，1≤q≤M，p≠q，SP_prepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th region in (1). In the present embodiment, M =200 is taken.

In this embodiment, { SP ] in step (c)_hSimilarity Sim (SP) between p-th and q-th regions in_p,SP_q) The acquisition process comprises the following steps:

③ 1, pair { SP_hQuantizing the color value of each component of each pixel point in each region to obtain { SP }_hQuantized region of each region in { SP } would be_hThe quantization region of the h-th region in (1) } is denoted as { P_h,i(x_h,y_h) Will { P }_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) The color value of the ith component of the pixel point is recorded as P_h,i(x_h,y_h) Suppose { P_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) Has a pixel point of { I_iThe coordinate position in (x, y) } is (x, y), thenWherein x is more than or equal to 1_h≤W_h,1≤y_h≤H_h，W_hRepresents SP_hWidth of the H-th area in (H) } H_hRepresents SP_hHeight of h-th area in (1), sign

To round the symbol down.

③ 2, calculate { SP_hColor histogram of quantized region of each region in { P }, will be { P_h,i(x_h,y_h) The color histogram of is noted as

Wherein,

represents { P_h,i(x_h,y_h) The number of all pixel points belonging to the k-th color in the pixel.

③ 3, pair { SP_hNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histograms

The normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,represents SP_hH-th region of { P } quantization region of the h-th region_h,i(x_h,y_h) The probability of occurrence of a pixel belonging to the k-th color in the pixel,

represents SP_hQuantization region of h' th region in { P }_h',i(x_h',y_h') X is more than or equal to 1 and the number of all pixel points belonging to the k color in the pixel_h'≤W_h',1≤y_h'≤H_h'，W_h'Represents SP_hWidth of H' th area in (H) }, H_h'Represents SP_hHeight of h' th area in (P) } h_h',i(x_h',y_h') Represents { P_h',i(x_h',y_h') The position of the middle coordinate is (x)_h',y_h') The color value of the ith component of the pixel point of (1).

③ 4, calculate { SP_hThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)_p,SP_q)，Sim(SP_p,SP_q)=Sim_c(SP_p,SP_q)×Sim_d(SP_p,SP_q)，Sim_c(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Sim_d(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe spatial similarity between the q-th regions in (j),

wherein, SP_pRepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th area in (1),

represents SP_hQuantization region of the P-th region in { P } quantization region of the P-th region { P_p,i(x_p,y_p) The probability of occurrence of a pixel belonging to the k-th color in the pixel,

represents SP_hQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth region_q,i(x_q,y_q) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to x_p≤W_p,1≤y_p≤H_p，W_pRepresents SP_hWidth of p-th area in (H)_pRepresents SP_hHeight of P-th area in (P) }, P_p,i(x_p,y_p) Represents { P_p,i(x_p,y_p) The position of the middle coordinate is (x)_p,y_p) Pixel point of1 ≦ x for the color value of the ith component of_q≤W_q,1≤y_q≤H_q，W_qRepresents SP_hWidth of the q-th area in (H) } m_qRepresents SP_hHeight of the q-th area in (P) } P_q,i(x_q,y_q) Represents { P_q,i(x_q,y_q) The position of the middle coordinate is (x)_q,y_q) Min () is a minimum function,

represents SP_hThe coordinate position of the center pixel point in the p-th region in (1),

represents SP_hThe coordinate position of the central pixel point in the qth area in (1) is the symbol "iill" which is the euclidean distance symbol.

Fourthly, according to the { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.

In this embodiment, the specific process of step iv is:

fourthly-1, calculating { SP_hColor contrast of each region in { SP } will be { SP }_hColor contrast of the h-th area in (1) } is noted as

<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>

Wherein, SP_hRepresents SP_hH area in (SP)_qRepresents SP_hThe q-th area in (1),

represents SP_hTotal number of pixel points included in the h-th area in (Sim)_d(SP_h,SP_q) Represents SP_hH area in the with { SP }_hThe spatial similarity between the q-th regions in (j),

represents SP_hThe coordinate position of the center pixel point in the h-th area in (1),

represents SP_hThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,

represents SP_hColor mean vector of h-th region in (i.e. { SP) }_hAveraging the color vectors of all pixel points in the h-th area to obtain

Represents SP_hThe color mean vector of the q-th region in (j).

Tetra-2, pair { SP_hNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }_hColor contrast of the h-th area in (1) } color contrast

The normalized color contrast obtained after normalization was recorded as

Wherein, NGC_minRepresents SP_hMinimum color contrast of M regions in (NGC) } NGC_maxRepresents SP_hMaximum color contrast in M regions in (j).

Fourthly-3, calculating { SP_hColor contrast based saliency value for each region in the will SP_hThe h region in (1) has a significant value based on color contrast recorded as

<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Represents SP_hSimilarity between the h-th and q-th regions in (1).

Fourthly-4, mixing the { SP_hThe color contrast based saliency value for each region in the { SP } is taken as the saliency value for all pixel points in the corresponding region, i.e. for { SP }_hH region of { SP } will be_hThe significant value of the h-th area based on the color contrast is taken as the significant value of all the pixel points in the area, so as to obtain { I }_iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.

Fifthly, according to { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_i(x, y) } image saliency map based on region-space sparsity, denoted as { NSS (x, y) }, where NSS (x, y) represents the coordinate position in { NSS (x, y) }Is the pixel value of the pixel point of (x, y).

In this embodiment, the specific process of the fifth step is as follows:

fifthly-1, calculating { SP_hSpatial sparsity of each region in { SP } will be_hThe spatial sparsity of the h-th region in (1) } is noted as

<math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Watch (A)Show SP_hThe similarity between the h-th and q-th regions in (1),

represents SP_hThe central pixel point in the h-th area in (I) } and (I)_i(x, y) } euclidean distance between center pixel points.

Fifthly-2, for { SP_hNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsity_hSpatial sparsity of the h-th region in

The normalized space sparsity obtained after normalization is recorded as

Wherein NSS_minRepresents SP_hMinimum spatial sparsity, NSS, of M regions in_maxRepresents SP_hThe greatest spatial sparsity of the M regions in (j).

Fifthly-3, calculating { SP_hSignificant value based on spatial sparsity for each region in the { SP will be_hSignificant value based on spatial sparsity of the h-th region in (1) } is noted

<math> <mrow> <msub> <mrow> <mi>NS</mi> <msup> <mi>S</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> </mrow> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>.</mo> </mrow> </math>

Fifthly-4, mixing { SP_hThe spatial sparsity-based saliency value for each region in the { SP } is taken as the saliency value for all pixel points in the corresponding region, i.e., for { SP }_hH region of { SP } will be_hThe significant value of the h-th area based on space sparsity is used as the significant value of all pixel points in the area, so as to obtain { I }_iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.

Sixthly, { I_i(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }_iThe radical of (x, y) } or (y) or (c)Image saliency maps { NGC (x, y) } and { I } at regional color contrast_i(x, y) } image saliency maps { NSS (x, y) } based on region space sparsity are fused to obtain { I_iThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).

The method of the invention is used for extracting the saliency maps of five groups of images, namely Image1, Image2, Image3, Image4 and Image5 in a saliency object Image library MSRA provided by Microsoft Asian research institute. Fig. 2a shows an original Image of "Image 1", fig. 2b shows a real (Ground route) saliency map of an Image of "Image 1", fig. 2c shows an Image saliency map of an Image of "Image 1" based on a global color histogram, fig. 2d shows an Image saliency map of an Image of "Image 1" based on a regional color contrast, fig. 2e shows an Image saliency map of an Image of "Image 1" based on a regional spatial sparsity, and fig. 2f shows a final Image saliency map of an Image of "Image 1"; fig. 3a shows an original Image of "Image 2", fig. 3b shows a real (Ground route) saliency map of an Image of "Image 2", fig. 3c shows an Image saliency map of an Image of "Image 2" based on a global color histogram, fig. 3d shows an Image saliency map of an Image of "Image 2" based on a regional color contrast, fig. 3e shows an Image saliency map of an Image of "Image 2" based on a regional spatial sparsity, and fig. 3f shows a final Image saliency map of an Image of "Image 2"; FIG. 4a shows an original Image of "Image 3", FIG. 4b shows a real (Ground route) saliency map of an Image of "Image 3", FIG. 4c shows an Image saliency map of an Image of "Image 3" based on a global color histogram, FIG. 4d shows an Image saliency map of an Image of "Image 3" based on regional color contrast, FIG. 4e shows an Image saliency map of an Image of "Image 3" based on regional spatial sparsity, and FIG. 4f shows a final Image saliency map of an Image of "Image 3"; FIG. 5a shows an original Image of "Image 4", FIG. 5b shows a true (Ground route) saliency map of an Image of "Image 4", FIG. 5c shows an Image saliency map of an Image of "Image 4" based on a global color histogram, FIG. 5d shows an Image saliency map of an Image of "Image 4" based on regional color contrast, FIG. 5e shows an Image saliency map of an Image of "Image 4" based on regional spatial sparsity, and FIG. 5f shows a final Image saliency map of an Image of "Image 4"; fig. 6a shows an original Image of "Image 5", fig. 6b shows a real (Ground route) saliency map of an Image of "Image 5", fig. 6c shows an Image saliency map based on a global color histogram of an Image of "Image 5", fig. 6d shows an Image saliency map based on a regional color contrast of an Image of "Image 5", fig. 6e shows an Image saliency map based on a regional spatial sparsity of an Image of "Image 5", and fig. 6f shows a final Image saliency map of an Image of "Image 5". As can be seen from fig. 2a to fig. 6f, the image saliency map obtained by the method of the present invention can well conform to the features of the saliency semantics due to the consideration of the saliency change of the global and local regions.

Claims (5)

1. A region-based image saliency map extraction method is characterized by comprising the following steps:

recording the source image to be processed as { I_i(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ H_i(x, y) }, H denotes { I_iHigh of (x, y) }, I_i(x, y) represents { I_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, wherein the 1 st component is an R component, the 2 nd component is a G component and the 3 rd component is a B component;

② first obtaining { I_i(x, y) } quantized image and global color histogram of quantized image, then according to { I }_i(x, y) } obtaining { I } from the quantized image_iThe color type of each pixel point in (x, y) } is determined according to { I }_iGlobal color histogram of quantized image of (x, y) } and { I_iThe color type of each pixel point in (x, y) } is obtained to obtain { I }_i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }_iThe coordinate position in the (x, y) } is the significant value of the pixel point of (x, y) based on the global color histogram;

(iii) using superpixel segmentation technique to divide { I_i(x, y) } into M non-overlapping regions, and then dividing { I }_i(x, y) } is re-represented as a set of M regions, denoted as { SP }_h}, recalculating { SP_hSimilarity between the respective regions in (will) { SP }_hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)_p,SP_q) Wherein M is more than or equal to 1, SP_hRepresents SP_hIn the h-th area, h is more than or equal to 1 and less than or equal to M, p is more than or equal to 1 and less than or equal to M, q is more than or equal to 1 and less than or equal to M, p is not equal to q, SP is equal to_pRepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th region in (1);

fourthly, according to the { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_i(x, y) } is an image saliency map based on regional color contrast, and is marked as { NGC (x, y) }, wherein the NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in the { NGC (x, y) };

fifthly, according to { SP_hObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each other_i(x, y) } is an image saliency map based on region space sparsity and is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) };

sixthly, { I_i(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }_i(x, y) } region color contrast based image saliency maps { NGC (x, y) } and { I_i(x, y) } region-space sparsity-based image saliency map NSS (x,y) } to obtain { I }_iThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).

2. The method for extracting the image saliency map based on the region as claimed in claim 1, wherein the specific process of the step (II) is as follows:

2- (1) pair of_iRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }_i(x, y) } quantized image, denoted as { P }_i(x, y) }, will { P_iThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as P_i(x,y)，Wherein, the symbol

Is a rounded-down symbol;

2, calculating { P_i(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) represents_iThe number of all pixel points belonging to the kth color in (x, y) };

2-3 according to { P_i(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }_i(x, y) } the color type of the corresponding pixel point will be { I_iThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as k_xy，k_xy=P₃(x,y)×256+P₂(x,y)×16+P₁(x, y) wherein P₃(x, y) represents { P }_iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is₂(x, y) represents { P }_iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is₁(x, y) represents { P }_iThe color value of the 1 st component of the pixel point with the coordinate position (x, y) in (x, y) };

② 4, calculating{I_i(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { I_iThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> $D(k_{\mathrm{xy}},k)=\sqrt{{(p_{k_{\mathrm{xy}},1}-p_{k,1})}^2+{(p_{k_{\mathrm{xy}},2}-p_{k,2})}^2+{(p_{k_{\mathrm{xy}},3}-p_{k,3})}^2},$ wherein D (k)_xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}_xyThe euclidean distance between the seed color and the kth color,

$p_{k_{\mathrm{xy}},2}=\mathrm{mod}(k_{\mathrm{xy}}/16),$

p_k,2=mod(k/16)，

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 1 st component corresponding to a seed color,

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyThe color value of the 2 nd component corresponding to the seed color,

denotes the k-th in { H (k) |0 ≦ k ≦ 4095}_xyColor value of 3 rd component, p, corresponding to a color_k,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}_k,3Representing the color value of the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder taking operation function;

② 5 according to { I_iThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }_i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.

3. The method according to claim 1 or 2, wherein { SP (SP-plus-SP) } in step (c)_hSimilarity Sim (SP) between p-th and q-th regions in_p,SP_q) The acquisition process comprises the following steps:

③ 1, pair { SP_hQuantizing the color value of each component of each pixel point in each region to obtain { SP }_hQuantized region of each region in { SP } would be_hThe quantization region of the h-th region in (1) } is denoted as { P_h,i(x_h,y_h) Will { P }_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) The color value of the ith component of the pixel point is recorded as P_h,i(x_h,y_h) Suppose { P_h,i(x_h,y_h) The position of the middle coordinate is (x)_h,y_h) Has a pixel point of { I_iThe coordinate position in (x, y) } is (x, y), then

Wherein x is more than or equal to 1_h≤W_h,1≤y_h≤H_h，W_hRepresents SP_hWidth of the H-th area in (H) } H_hRepresents SP_hHeight of h-th area in (1), sign

Is a rounded-down symbol;

③ 2, calculate { SP_hColor histogram of quantized region of each region in { P }, will be { P_h,i(x_h,y_h) The color histogram of is noted as

Wherein,

represents { P_h,i(x_h,y_h) The number of all pixel points belonging to the kth color in the pixel;

③ 3, pair { SP_hNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histograms

The normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,

represents SP_hH-th region of { P } quantization region of the h-th region_h,i(x_h,y_h) The probability of occurrence of a pixel belonging to the k-th color in the pixel,

represents SP_hQuantization region of h' th region in { P }_h',i(x_h',y_h') All pixel points belonging to the k color in the pixel arrayNumber, 1 ≦ x_h'≤W_h',1≤y_h'≤H_h'，W_h'Represents SP_hWidth of H' th area in (H) }, H_h'Represents SP_hHeight of h' th area in (P) } h_h',i(x_h',y_h') Represents { P_h',i(x_h',y_h') The position of the middle coordinate is (x)_h',y_h') The color value of the ith component of the pixel point of (1);

③ 4, calculate { SP_hThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)_p,SP_q)，Sim(SP_p,SP_q)=Sim_c(SP_p,SP_q)×Sim_d(SP_p,SP_q)，Sim_c(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Sim_d(SP_p,SP_q) Represents SP_hThe p-th region in (f) and (SP)_hThe spatial similarity between the q-th regions in (j),

wherein, SP_pRepresents SP_hP-th area in (SP)_qRepresents SP_hThe q-th area in (1),represents SP_hQuantization region of the P-th region in { P } quantization region of the P-th region { P_p,i(x_p,y_p) The probability of occurrence of a pixel belonging to the k-th color in the pixel,

represents SP_hQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth region_q,i(x_q,y_q) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to x_p≤W_p,1≤y_p≤H_p，W_pRepresents SP_hWidth of p-th area in (H)_pRepresents SP_hHeight of P-th area in (P) }, P_p,i(x_p,y_p) Represents { P_p,i(x_p,y_p) The position of the middle coordinate is (x)_p,y_p) The color value of the ith component of the pixel point is more than or equal to 1 and less than or equal to x_q≤W_q,1≤y_q≤H_q，W_qRepresents SP_hWidth of the q-th area in (H) } m_qRepresents SP_hHeight of the q-th area in (P) } P_q,i(x_q,y_q) Represents { P_q,i(x_q,y_q) The position of the middle coordinate is (x)_q,y_q) Min () is a minimum function,

represents SP_hThe coordinate position of the center pixel point in the p-th region in (1),

represents SP_hThe coordinate position of the central pixel point in the qth area in (1) is the symbol "iill" which is the euclidean distance symbol.

4. The method for extracting the image saliency map based on the region of claim 3, characterized in that the specific process of the step (iv) is as follows:

fourthly-1, calculating { SP_hColor contrast of each region in { SP } will be { SP }_hColor contrast of the h-th area in (1) } is noted as

<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>

Wherein, SP_hRepresents SP_hH area in (SP)_qRepresents SP_hThe q-th area in (1),

represents SP_hTotal number of pixel points included in the h-th area in (Sim)_d(SP_h,SP_q) Represents SP_hH area in the with { SP }_hThe spatial similarity between the q-th regions in (j), represents SP_hThe coordinate position of the center pixel point in the h-th area in (1),

represents SP_hThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,

represents SP_hThe color mean vector of the h-th region in (j),

represents SP_hThe color mean vector of the qth region in (j);

tetra-2, pair { SP_hNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }_hColor contrast of the h-th area in (1) } color contrast

The normalized color contrast obtained after normalization was recorded as

<math> <mrow> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>NGC</mi> <mi>min</mi> </msub> </mrow> <mrow> <msub> <mi>NGC</mi> <mi>max</mi> </msub> <mo>-</mo> <msub> <mi>NGC</mi> <mi>min</mi> </msub> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, NGC_minRepresents SP_hMinimum color contrast of M regions in (NGC) } NGC_maxRepresents SP_hMaximum color contrast in M regions in (j);

fourthly-3, calculating { SP_hColor contrast based saliency value for each region in the will SP_hThe h region in (1) has a significant value based on color contrast recorded as

<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Represents SP_hSimilarity between the h region and the q region in (1);

fourthly-4, mixing the { SP_hThe significant value of each area based on the color contrast is taken as the significant value of all pixel points in the corresponding area, so as to obtain { I }_iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.

5. The method according to claim 4, wherein the specific process of step (c) is as follows:

fifthly-1, calculating { SP_hSpatial sparsity of each region in { SP } will be_hThe spatial sparsity of the h-th region in (1) } is noted as <math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)_h,SP_q) Represents SP_hThe similarity between the h-th and q-th regions in (1),represents SP_hThe central pixel point in the h-th area in (I) } and (I)_i(x, y) } euclidean distance between center pixel points;

fifthly-2, for { SP_hNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsity_hSpatial sparsity of the h-th region in

The normalized space sparsity obtained after normalization is recorded as

Wherein NSS_minRepresents SP_hMinimum spatial sparsity, NSS, of M regions in_maxRepresents SP_hMaximum spatial sparsity in M regions in (j);

fifthly-3, calculating { SP_hSignificant value based on spatial sparsity for each region in the { SP will be_hSignificant value based on spatial sparsity of the h-th region in (1) } is noted

<math> <mrow> <msub> <mrow> <mi>NS</mi> <msup> <mi>S</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> </mrow> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow> </math>

Fifthly-4, mixing { SP_hThe significant value of each region based on space sparsity is used as the significant value of all pixel points in the corresponding region, so as to obtain { I }_iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.

Images