CN103632153A - Region-based image saliency map extracting method - Google Patents

Region-based image saliency map extracting method Download PDF

Info

Publication number
CN103632153A
CN103632153A CN201310651864.5A CN201310651864A CN103632153A CN 103632153 A CN103632153 A CN 103632153A CN 201310651864 A CN201310651864 A CN 201310651864A CN 103632153 A CN103632153 A CN 103632153A
Authority
CN
China
Prior art keywords
msub
mrow
color
region
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201310651864.5A
Other languages
Chinese (zh)
Other versions
CN103632153B (en
Inventor
邵枫
姜求平
蒋刚毅
郁梅
李福翠
彭宗举
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Duyan Information Technology Co ltd
Original Assignee
Ningbo University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ningbo University filed Critical Ningbo University
Priority to CN201310651864.5A priority Critical patent/CN103632153B/en
Publication of CN103632153A publication Critical patent/CN103632153A/en
Application granted granted Critical
Publication of CN103632153B publication Critical patent/CN103632153B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The invention discloses a region-based image saliency map extracting method. The method includes: firstly, calculating a global color histogram of an image to obtain an image saliency map based on the global color histogram; secondly, adopting superpixel segmentation technology to segment the image, calculating color contrast and space sparsity of each region, and weighting by the aid of similarity among the regions to obtain an image saliency map based on the region color contrast and an image saliency map based on the region space sparsity; finally, fusing the image saliency map based on the global color histogram, the image saliency map based on the region color contrast and the image saliency map based on the region space sparsity to obtain a final image saliency map. The method has the advantage that the obtained image saliency map can well reflect saliency changes of global and local regions, and conforms to image saliency semantic features.

Description

Region-based image saliency map extraction method
Technical Field
The present invention relates to a method for processing an image signal, and more particularly, to a method for extracting an image saliency map based on a region.
Background
In human visual reception and information processing, due to limited brain resources and difference in importance of external environment information, the human brain does not have the same sense as the external environment information but shows selective characteristics in the processing process. People are not evenly focused on every area of an image when watching the image or video clip, but are more focused on some salient areas. How to detect and extract the salient regions with high visual attention in the video is an important research content in the field of computer vision and content-based video retrieval.
The existing saliency map model is a selective attention model that models the visual attention mechanism of living beings, the method calculates the contrast of each pixel point with the surrounding background in the aspects of color, brightness and direction, and forms a saliency map by the saliency values of all the pixel points, however, the method can not well extract the saliency map information of the image, this is because pixel-based salient features do not reflect well the salient semantic features of the human eye when viewed, and the region-based salient features can effectively improve the stability and accuracy of extraction, and therefore, how to perform region segmentation on the image, how to extract the features of each region, how to describe the salient features of each region, how to measure the saliency of the region and the saliency between the regions is a problem to be researched and solved in the region-based saliency map extraction.
Disclosure of Invention
The invention aims to provide a region-based image saliency map extraction method which accords with salient semantic features and has higher extraction stability and accuracy.
The technical scheme adopted by the invention for solving the technical problems is as follows: a region-based image saliency map extraction method is characterized by comprising the following steps:
recording the source image to be processed as { Ii(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ Hi(x, y) }, H denotes { IiHigh of (x, y) }, Ii(x, y) represents { IiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, wherein the 1 st component is an R component, the 2 nd component is a G component and the 3 rd component is a B component;
② first obtaining { Ii(x, y) } quantized image and global color histogram of quantized image, then according to { I }i(x, y) } obtaining { I } from the quantized imageiThe color type of each pixel point in (x, y) } is determined according to { I }iGlobal color histogram of quantized image of (x, y) } and { IiThe color type of each pixel point in (x, y) } is obtained to obtain { I }i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }iThe coordinate position in the (x, y) } is the significant value of the pixel point of (x, y) based on the global color histogram;
(iii) using superpixel segmentation technique to divide { Ii(x, y) } into M non-overlapping regions, and then dividing { I }i(x, y) } is re-represented as a set of M regions, denoted as { SP }h}, recalculating { SPhSimilarity between the respective regions in (will) { SP }hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)p,SPq) Wherein M is more than or equal to 1, SPhRepresents SPhIn the h-th area, h is more than or equal to 1 and less than or equal to M, p is more than or equal to 1 and less than or equal to M, q is more than or equal to 1 and less than or equal to M, p is not equal to q, SP is equal topRepresents SPhP-th area in (SP)qRepresents SPhThe q-th region in (1);
fourthly, according to the { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheri(x, y) } is an image saliency map based on regional color contrast, and is marked as { NGC (x, y) }, wherein the NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in the { NGC (x, y) };
fifthly, according to { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheri(x, y) } is an image saliency map based on region space sparsity and is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) };
sixthly, { Ii(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }i(x, y) } region color contrast based image saliency maps { NGC (x, y) } and { Ii(x, y) } image saliency maps { NSS (x, y) } based on region space sparsity are fused to obtain { IiThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).
The concrete process of the second step is as follows:
2- (1) pair ofiRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }i(x, y) } quantized image, denoted as { P }i(x, y) }, will { PiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as Pi(x,y),
Figure BDA00004304738400000311
Wherein, the symbol
Figure BDA00004304738400000312
Is a rounded-down symbol;
2, calculating { Pi(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) representsiThe number of all pixel points belonging to the kth color in (x, y) };
2-3 according to { Pi(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }i(x, y) } the color type of the corresponding pixel point will be { IiThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as kxy,kxy=P3(x,y)×256+P2(x,y)×16+P1(x, y) wherein P3(x, y) represents { P }iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is2(x, y) represents { P }iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is1(x, y) represents { P }iColor of 1 st component of pixel point with coordinate position (x, y) in (x, y) }A value;
② 4, calculating { Ii(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { IiThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> D ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , wherein D (k)xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}xyThe euclidean distance between the seed color and the kth color, p k xy , 2 = mod ( k xy / 16 ) ,
Figure BDA0000430473840000035
Figure BDA0000430473840000036
pk,2=mod(k/16),
Figure BDA0000430473840000038
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 1 st component corresponding to a seed color,denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 2 nd component corresponding to the seed color,
Figure BDA00004304738400000310
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyColor value of 3 rd component, p, corresponding to a colork,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,3Representing the color value of the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder taking operation function;
② 5 according to { IiThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.
In step (c) { SPhSimilarity Sim (SP) between p-th and q-th regions inp,SPq) The acquisition process comprises the following steps:
③ 1, pair { SPhQuantizing the color value of each component of each pixel point in each region to obtain { SP }hQuantized region of each region in { SP } would behThe quantization region of the h-th region in (1) } is denoted as { Ph,i(xh,yh) Will { P }h,i(xh,yh) The position of the middle coordinate is (x)h,yh) Of the ith component of the pixelColor value Ph,i(xh,yh) Suppose { Ph,i(xh,yh) The position of the middle coordinate is (x)h,yh) Has a pixel point of { IiThe coordinate position in (x, y) } is (x, y), then
Figure BDA0000430473840000046
Wherein x is more than or equal to 1h≤Wh,1≤yh≤Hh,WhRepresents SPhWidth of the H-th area in (H) } HhRepresents SPhHeight of h-th area in (1), signIs a rounded-down symbol;
③ 2, calculate { SPhColor histogram of quantized region of each region in { P }, will be { Ph,i(xh,yh) The color histogram of is noted asWherein,
Figure BDA0000430473840000042
represents { Ph,i(xh,yh) The number of all pixel points belonging to the kth color in the pixel;
③ 3, pair { SPhNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histogramsThe normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,
Figure BDA0000430473840000045
represents SPhH-th region of { P } quantization region of the h-th regionh,i(xh,yh) The probability of occurrence of a pixel belonging to the k-th color in the pixel,represents SPhQuantization region of h' th region in { P }h',i(xh',yh') X is more than or equal to 1 and the number of all pixel points belonging to the k color in the pixelh'≤Wh',1≤yh'≤Hh',Wh'Represents SPhWidth of H' th area in (H) }, Hh'Represents SPhHeight of h' th area in (P) } hh',i(xh',yh') Represents { Ph',i(xh',yh') The position of the middle coordinate is (x)h',yh') The color value of the ith component of the pixel point of (1);
③ 4, calculate { SPhThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)p,SPq),Sim(SPp,SPq)=Simc(SPp,SPq)×Simd(SPp,SPq),Simc(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Simd(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe spatial similarity between the q-th regions in (j),
Figure BDA0000430473840000052
wherein, SPpRepresents SPhP-th area in (SP)qRepresents SPhThe q-th area in (1),represents SPhQuantization region of the P-th region in { P } quantization region of the P-th region { Pp,i(xp,yp) The probability of occurrence of a pixel belonging to the k-th color in the pixel,
Figure BDA0000430473840000054
represents SPhQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth regionq,i(xq,yq) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to xp≤Wp,1≤yp≤Hp,WpRepresents SPhWidth of p-th area in (H)pRepresents SPhHeight of P-th area in (P) }, Pp,i(xp,yp) Represents { Pp,i(xp,yp) The position of the middle coordinate is (x)p,yp) The color value of the ith component of the pixel point is more than or equal to 1 and less than or equal to xq≤Wq,1≤yq≤Hq,WqRepresents SPhWidth of the q-th area in (H) } mqRepresents SPhHeight of the q-th area in (P) } Pq,i(xq,yq) Represents { Pq,i(xq,yq) The position of the middle coordinate is (x)q,yq) Min () is a minimum function,
Figure BDA0000430473840000055
represents SPhThe coordinate position of the center pixel point in the p-th region in (1),
Figure BDA0000430473840000056
represents SPhThe coordinate position of the central pixel point in the qth region in (1)The symbol "ii |" is a euclidean distance symbol.
The specific process of the step IV is as follows:
fourthly-1, calculating { SPhColor contrast of each region in { SP } will be { SP }hColor contrast of the h-th area in (1) } is noted as
Figure BDA0000430473840000061
<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>
Figure BDA00004304738400000613
Wherein, SPhRepresents SPhH area in (SP)qRepresents SPhThe q-th area in (1),
Figure BDA00004304738400000614
represents SPhTotal number of pixel points included in the h-th area in (Sim)d(SPh,SPq) Represents SPhH area in the with { SP }hThe spatial similarity between the q-th regions in (j),
Figure BDA0000430473840000063
Figure BDA0000430473840000064
represents SPhThe coordinate position of the center pixel point in the h-th area in (1),represents SPhThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,
Figure BDA0000430473840000066
represents SPhThe color mean vector of the h-th region in (j),represents SPhThe color mean vector of the qth region in (j);
tetra-2, pair { SPhNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }hColor contrast of the h-th area in (1) } color contrastThe normalized color contrast obtained after normalization was recorded as
Figure BDA0000430473840000069
Figure BDA00004304738400000610
Wherein, NGCminRepresents SPhMinimum color contrast of M regions in (NGC) } NGCmaxRepresents SPhMaximum color contrast in M regions in (j);
fourthly-3, calculating { SPhColor contrast based saliency value for each region in the will SPhColor-based contrast of the h-th area in (1) } inIs marked as
Figure BDA00004304738400000611
<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Represents SPhSimilarity between the h region and the q region in (1);
fourthly-4, mixing the { SPhThe significant value of each area based on the color contrast is taken as the significant value of all pixel points in the corresponding area, so as to obtain { I }iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.
The concrete process of the fifth step is as follows:
fifthly-1, calculating { SPhSpatial sparsity of each region in { SP } will behThe spatial sparsity of the h-th region in (1) } is noted as
Figure BDA0000430473840000071
<math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Represents SPhThe similarity between the h-th and q-th regions in (1),
Figure BDA0000430473840000073
represents SPhThe central pixel point in the h-th area in (I) } and (I)i(x, y) } euclidean distance between center pixel points;
fifthly-2, for { SPhNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsityhSpatial sparsity of the h-th region in
Figure BDA0000430473840000074
The normalized space sparsity obtained after normalization is recorded as
Figure BDA0000430473840000075
Wherein NSSminRepresents SPhMinimum spatial sparsity, NSS, of M regions inmaxRepresents SPhMaximum spatial sparsity in M regions in (j);
fifthly-3, calculating { SPhSignificant value based on spatial sparsity for each region in the { SP will behSignificant value based on spatial sparsity of the h-th region in (1) } is noted
Figure BDA0000430473840000077
<math> <mrow> <msub> <msup> <mi>NSS</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow> </math>
Fifthly-4, mixing { SPhThe significant value of each region based on space sparsity is used as the significant value of all pixel points in the corresponding region, so as to obtain { I }iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.
Compared with the prior art, the invention has the advantages that:
1) according to the method, the image saliency map based on the global color histogram, the image saliency map based on the regional color contrast and the image saliency map based on the regional space sparsity are obtained through calculation respectively and are finally fused to obtain the image saliency map, the obtained image saliency map can better reflect the saliency change conditions of the global and local regions of the image, and the stability and the accuracy are high.
2) The method provided by the invention adopts a superpixel segmentation technology to segment the image, utilizes histogram features to respectively calculate the color contrast and space sparsity of each region, and finally utilizes the similarity between the regions to carry out weighting to obtain a final image saliency map based on the regions, so that the feature information conforming to the saliency semantics can be extracted.
Drawings
FIG. 1 is a block diagram of an overall implementation of the method of the present invention;
FIG. 2a is an original Image of "Image 1";
FIG. 2b is a real (Ground route) saliency map of an "Image 1" Image;
FIG. 2c is a global color histogram based Image saliency map of an "Image 1" Image;
FIG. 2d is a region color contrast based Image saliency map of an "Image 1" Image;
FIG. 2e is an Image saliency map based on region-space sparsity of an "Image 1" Image;
FIG. 2f is the final Image saliency map of the "Image 1" Image;
FIG. 3a is an original Image of "Image 2";
FIG. 3b is a real (Ground route) saliency map of an "Image 2" Image;
FIG. 3c is a global color histogram based Image saliency map of an "Image 2" Image;
FIG. 3d is a region color contrast based Image saliency map of an "Image 2" Image;
FIG. 3e is an Image saliency map based on region-space sparsity for an "Image 2" Image;
FIG. 3f is the final Image saliency map of the "Image 2" Image;
FIG. 4a is an original Image of "Image 3";
FIG. 4b is a real (Ground route) saliency map of an "Image 3" Image;
FIG. 4c is a global color histogram based Image saliency map for an "Image 3" Image;
FIG. 4d is a region color contrast based Image saliency map of an "Image 3" Image;
FIG. 4e is an Image saliency map based on region space sparsity for an "Image 3" Image;
FIG. 4f is the final Image saliency map of the "Image 3" Image;
FIG. 5a is an original Image of "Image 4";
FIG. 5b is a real (Ground route) saliency map of an "Image 4" Image;
FIG. 5c is a global color histogram based Image saliency map for an "Image 4" Image;
FIG. 5d is a region color contrast based Image saliency map of an "Image 4" Image;
FIG. 5e is an Image saliency map based on region-space sparsity for an "Image 4" Image;
FIG. 5f is the final Image saliency map of the "Image 4" Image;
FIG. 6a is an original Image of "Image 5";
FIG. 6b is a real (Ground route) saliency map of the "Image 5" Image;
FIG. 6c is a global color histogram based Image saliency map for an "Image 5" Image;
FIG. 6d is a region color contrast based Image saliency map of an "Image 5" Image;
FIG. 6e is an Image saliency map based on region space sparsity for an "Image 5" Image;
fig. 6f is a final Image saliency map of the "Image 5" Image.
Detailed Description
The invention is described in further detail below with reference to the accompanying examples.
The invention provides a region-based image saliency map extraction method, the overall implementation block diagram of which is shown in FIG. 1, and the method comprises the following steps:
recording the source image to be processed as { Ii(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ Hi(x, y) }, H denotes { IiHigh of (x, y) }, Ii(x, y) represents { IiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, the 1 st component is an R component, the 2 nd component is a G component, and the 3 rd component is a B component.
Secondly, if only local saliency is considered, the saliency of the edge with violent change or the complicated background area in the image is higher, the saliency of the interior of the smooth target area is lower, and thus the global saliency needs to be considered, wherein the global saliency refers to the saliency of each pixel point relative to the global image, so that the method firstly obtains { I }i(x, y) } quantized image and global color histogram of quantized image, then according to { I }i(x, y) } obtaining { I } from the quantized imageiThe color type of each pixel point in (x, y) } is determined according to { I }i(x,y)The global color histogram of the quantized image of { I } andithe color type of each pixel point in (x, y) } is obtained to obtain { I }i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }iAnd (x, y) the significant value of the pixel point with the coordinate position of (x, y) based on the global color histogram.
In this embodiment, the specific process of step two is:
2- (1) pair ofiRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }i(x, y) } quantized image, denoted as { P }i(x, y) }, will { PiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as Pi(x,y),
Figure BDA0000430473840000091
Wherein, the symbol
Figure BDA0000430473840000092
To round the symbol down.
2, calculating { Pi(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) representsiThe number of all pixel points belonging to the k-th color in (x, y) }.
2-3 according to { Pi(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }i(x, y) } the color type of the corresponding pixel point will be { IiThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as kxy,kxy=P3(x,y)×256+P2(x,y)×16+P1(x, y) wherein P3(x, y) represents { P }iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is2(x, y) represents { P }iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is1(x, y) represents { P }i(x, y) } middle coordinateAnd (3) the color value of the 1 st component of the pixel point with the position (x, y).
② 4, calculating { Ii(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { IiThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> D ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , wherein D (k)xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}xyThe euclidean distance between the seed color and the kth color,
Figure BDA0000430473840000103
p k xy , 2 = mod ( k xy / 16 ) ,
Figure BDA0000430473840000105
pk,2=mod(k/16),
Figure BDA0000430473840000107
Figure BDA0000430473840000108
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 1 st component corresponding to a seed color,denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 2 nd component corresponding to the seed color,
Figure BDA00004304738400001010
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyColor value of 3 rd component, p, corresponding to a colork,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,3And (b) a color value representing the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder-taking operation function.
② 5 according to { IiThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.
(iii) adopting super pixel (Superpixel) segmentation technique to divide { I }i(x, y) } into M non-overlapping regions, and then dividing { I }i(x, y) } is re-represented as a set of M regions, denoted as { SP }hConsidering local saliency, similar areas in the image generally have lower saliency, so the invention calculates { SPhSimilarity between the respective regions in (will) { SP }hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)p,SPq) Wherein M is more than or equal to 1, SPhRepresents SPhIn the h-th area, h is more than or equal to 1 and less than or equal to M,1≤p≤M,1≤q≤M,p≠q,SPprepresents SPhP-th area in (SP)qRepresents SPhThe q-th region in (1). In the present embodiment, M =200 is taken.
In this embodiment, { SP ] in step (c)hSimilarity Sim (SP) between p-th and q-th regions inp,SPq) The acquisition process comprises the following steps:
③ 1, pair { SPhQuantizing the color value of each component of each pixel point in each region to obtain { SP }hQuantized region of each region in { SP } would behThe quantization region of the h-th region in (1) } is denoted as { Ph,i(xh,yh) Will { P }h,i(xh,yh) The position of the middle coordinate is (x)h,yh) The color value of the ith component of the pixel point is recorded as Ph,i(xh,yh) Suppose { Ph,i(xh,yh) The position of the middle coordinate is (x)h,yh) Has a pixel point of { IiThe coordinate position in (x, y) } is (x, y), thenWherein x is more than or equal to 1h≤Wh,1≤yh≤Hh,WhRepresents SPhWidth of the H-th area in (H) } HhRepresents SPhHeight of h-th area in (1), sign
Figure BDA0000430473840000117
To round the symbol down.
③ 2, calculate { SPhColor histogram of quantized region of each region in { P }, will be { Ph,i(xh,yh) The color histogram of is noted as
Figure BDA0000430473840000111
Wherein,
Figure BDA0000430473840000112
represents { Ph,i(xh,yh) The number of all pixel points belonging to the k-th color in the pixel.
③ 3, pair { SPhNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histograms
Figure BDA0000430473840000113
The normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,represents SPhH-th region of { P } quantization region of the h-th regionh,i(xh,yh) The probability of occurrence of a pixel belonging to the k-th color in the pixel,
Figure BDA0000430473840000127
represents SPhQuantization region of h' th region in { P }h',i(xh',yh') X is more than or equal to 1 and the number of all pixel points belonging to the k color in the pixelh'≤Wh',1≤yh'≤Hh',Wh'Represents SPhWidth of H' th area in (H) }, Hh'Represents SPhHeight of h' th area in (P) } hh',i(xh',yh') Represents { Ph',i(xh',yh') The position of the middle coordinate is (x)h',yh') The color value of the ith component of the pixel point of (1).
③ 4, calculate { SPhThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)p,SPq),Sim(SPp,SPq)=Simc(SPp,SPq)×Simd(SPp,SPq),Simc(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Simd(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe spatial similarity between the q-th regions in (j),
Figure BDA0000430473840000122
wherein, SPpRepresents SPhP-th area in (SP)qRepresents SPhThe q-th area in (1),
Figure BDA0000430473840000123
represents SPhQuantization region of the P-th region in { P } quantization region of the P-th region { Pp,i(xp,yp) The probability of occurrence of a pixel belonging to the k-th color in the pixel,
Figure BDA0000430473840000124
represents SPhQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth regionq,i(xq,yq) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to xp≤Wp,1≤yp≤Hp,WpRepresents SPhWidth of p-th area in (H)pRepresents SPhHeight of P-th area in (P) }, Pp,i(xp,yp) Represents { Pp,i(xp,yp) The position of the middle coordinate is (x)p,yp) Pixel point of1 ≦ x for the color value of the ith component ofq≤Wq,1≤yq≤Hq,WqRepresents SPhWidth of the q-th area in (H) } mqRepresents SPhHeight of the q-th area in (P) } Pq,i(xq,yq) Represents { Pq,i(xq,yq) The position of the middle coordinate is (x)q,yq) Min () is a minimum function,
Figure BDA0000430473840000125
represents SPhThe coordinate position of the center pixel point in the p-th region in (1),
Figure BDA0000430473840000126
represents SPhThe coordinate position of the central pixel point in the qth area in (1) is the symbol "iill" which is the euclidean distance symbol.
Fourthly, according to the { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheriThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.
In this embodiment, the specific process of step iv is:
fourthly-1, calculating { SPhColor contrast of each region in { SP } will be { SP }hColor contrast of the h-th area in (1) } is noted as
Figure BDA0000430473840000131
<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>
Figure BDA00004304738400001313
Wherein, SPhRepresents SPhH area in (SP)qRepresents SPhThe q-th area in (1),
Figure BDA00004304738400001314
represents SPhTotal number of pixel points included in the h-th area in (Sim)d(SPh,SPq) Represents SPhH area in the with { SP }hThe spatial similarity between the q-th regions in (j),
Figure BDA0000430473840000133
Figure BDA0000430473840000134
represents SPhThe coordinate position of the center pixel point in the h-th area in (1),
Figure BDA0000430473840000135
represents SPhThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,
Figure BDA0000430473840000136
represents SPhColor mean vector of h-th region in (i.e. { SP) }hAveraging the color vectors of all pixel points in the h-th area to obtain
Figure BDA0000430473840000137
Represents SPhThe color mean vector of the q-th region in (j).
Tetra-2, pair { SPhNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }hColor contrast of the h-th area in (1) } color contrast
Figure BDA0000430473840000138
The normalized color contrast obtained after normalization was recorded as
Figure BDA00004304738400001310
Wherein, NGCminRepresents SPhMinimum color contrast of M regions in (NGC) } NGCmaxRepresents SPhMaximum color contrast in M regions in (j).
Fourthly-3, calculating { SPhColor contrast based saliency value for each region in the will SPhThe h region in (1) has a significant value based on color contrast recorded as
Figure BDA00004304738400001311
<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Represents SPhSimilarity between the h-th and q-th regions in (1).
Fourthly-4, mixing the { SPhThe color contrast based saliency value for each region in the { SP } is taken as the saliency value for all pixel points in the corresponding region, i.e. for { SP }hH region of { SP } will behThe significant value of the h-th area based on the color contrast is taken as the significant value of all the pixel points in the area, so as to obtain { I }iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.
Fifthly, according to { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheri(x, y) } image saliency map based on region-space sparsity, denoted as { NSS (x, y) }, where NSS (x, y) represents the coordinate position in { NSS (x, y) }Is the pixel value of the pixel point of (x, y).
In this embodiment, the specific process of the fifth step is as follows:
fifthly-1, calculating { SPhSpatial sparsity of each region in { SP } will behThe spatial sparsity of the h-th region in (1) } is noted as
Figure BDA0000430473840000141
<math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Watch (A)Show SPhThe similarity between the h-th and q-th regions in (1),
Figure BDA0000430473840000143
represents SPhThe central pixel point in the h-th area in (I) } and (I)i(x, y) } euclidean distance between center pixel points.
Fifthly-2, for { SPhNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsityhSpatial sparsity of the h-th region in
Figure BDA0000430473840000144
The normalized space sparsity obtained after normalization is recorded as
Figure BDA0000430473840000145
Figure BDA0000430473840000146
Wherein NSSminRepresents SPhMinimum spatial sparsity, NSS, of M regions inmaxRepresents SPhThe greatest spatial sparsity of the M regions in (j).
Fifthly-3, calculating { SPhSignificant value based on spatial sparsity for each region in the { SP will behSignificant value based on spatial sparsity of the h-th region in (1) } is noted
Figure BDA0000430473840000151
<math> <mrow> <msub> <mrow> <mi>NS</mi> <msup> <mi>S</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> </mrow> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>.</mo> </mrow> </math>
Fifthly-4, mixing { SPhThe spatial sparsity-based saliency value for each region in the { SP } is taken as the saliency value for all pixel points in the corresponding region, i.e., for { SP }hH region of { SP } will behThe significant value of the h-th area based on space sparsity is used as the significant value of all pixel points in the area, so as to obtain { I }iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.
Sixthly, { Ii(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }iThe radical of (x, y) } or (y) or (c)Image saliency maps { NGC (x, y) } and { I } at regional color contrasti(x, y) } image saliency maps { NSS (x, y) } based on region space sparsity are fused to obtain { IiThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).
The method of the invention is used for extracting the saliency maps of five groups of images, namely Image1, Image2, Image3, Image4 and Image5 in a saliency object Image library MSRA provided by Microsoft Asian research institute. Fig. 2a shows an original Image of "Image 1", fig. 2b shows a real (Ground route) saliency map of an Image of "Image 1", fig. 2c shows an Image saliency map of an Image of "Image 1" based on a global color histogram, fig. 2d shows an Image saliency map of an Image of "Image 1" based on a regional color contrast, fig. 2e shows an Image saliency map of an Image of "Image 1" based on a regional spatial sparsity, and fig. 2f shows a final Image saliency map of an Image of "Image 1"; fig. 3a shows an original Image of "Image 2", fig. 3b shows a real (Ground route) saliency map of an Image of "Image 2", fig. 3c shows an Image saliency map of an Image of "Image 2" based on a global color histogram, fig. 3d shows an Image saliency map of an Image of "Image 2" based on a regional color contrast, fig. 3e shows an Image saliency map of an Image of "Image 2" based on a regional spatial sparsity, and fig. 3f shows a final Image saliency map of an Image of "Image 2"; FIG. 4a shows an original Image of "Image 3", FIG. 4b shows a real (Ground route) saliency map of an Image of "Image 3", FIG. 4c shows an Image saliency map of an Image of "Image 3" based on a global color histogram, FIG. 4d shows an Image saliency map of an Image of "Image 3" based on regional color contrast, FIG. 4e shows an Image saliency map of an Image of "Image 3" based on regional spatial sparsity, and FIG. 4f shows a final Image saliency map of an Image of "Image 3"; FIG. 5a shows an original Image of "Image 4", FIG. 5b shows a true (Ground route) saliency map of an Image of "Image 4", FIG. 5c shows an Image saliency map of an Image of "Image 4" based on a global color histogram, FIG. 5d shows an Image saliency map of an Image of "Image 4" based on regional color contrast, FIG. 5e shows an Image saliency map of an Image of "Image 4" based on regional spatial sparsity, and FIG. 5f shows a final Image saliency map of an Image of "Image 4"; fig. 6a shows an original Image of "Image 5", fig. 6b shows a real (Ground route) saliency map of an Image of "Image 5", fig. 6c shows an Image saliency map based on a global color histogram of an Image of "Image 5", fig. 6d shows an Image saliency map based on a regional color contrast of an Image of "Image 5", fig. 6e shows an Image saliency map based on a regional spatial sparsity of an Image of "Image 5", and fig. 6f shows a final Image saliency map of an Image of "Image 5". As can be seen from fig. 2a to fig. 6f, the image saliency map obtained by the method of the present invention can well conform to the features of the saliency semantics due to the consideration of the saliency change of the global and local regions.

Claims (5)

1. A region-based image saliency map extraction method is characterized by comprising the following steps:
recording the source image to be processed as { Ii(x, y) }, wherein I =1,2,3, 1 ≦ x ≦ W, 1 ≦ y ≦ H, W represents { I ≦ Hi(x, y) }, H denotes { IiHigh of (x, y) }, Ii(x, y) represents { IiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) }, wherein the 1 st component is an R component, the 2 nd component is a G component and the 3 rd component is a B component;
② first obtaining { Ii(x, y) } quantized image and global color histogram of quantized image, then according to { I }i(x, y) } obtaining { I } from the quantized imageiThe color type of each pixel point in (x, y) } is determined according to { I }iGlobal color histogram of quantized image of (x, y) } and { IiThe color type of each pixel point in (x, y) } is obtained to obtain { I }i(x, y) } is an image saliency map based on a global color histogram, and is denoted as { HS (x, y) }, wherein HS (x, y) represents a pixel value of a pixel point with a coordinate position (x, y) in { HS (x, y) }, and also represents { I }iThe coordinate position in the (x, y) } is the significant value of the pixel point of (x, y) based on the global color histogram;
(iii) using superpixel segmentation technique to divide { Ii(x, y) } into M non-overlapping regions, and then dividing { I }i(x, y) } is re-represented as a set of M regions, denoted as { SP }h}, recalculating { SPhSimilarity between the respective regions in (will) { SP }hThe similarity between the p-th and q-th regions in (SP) is denoted as Sim (SP)p,SPq) Wherein M is more than or equal to 1, SPhRepresents SPhIn the h-th area, h is more than or equal to 1 and less than or equal to M, p is more than or equal to 1 and less than or equal to M, q is more than or equal to 1 and less than or equal to M, p is not equal to q, SP is equal topRepresents SPhP-th area in (SP)qRepresents SPhThe q-th region in (1);
fourthly, according to the { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheri(x, y) } is an image saliency map based on regional color contrast, and is marked as { NGC (x, y) }, wherein the NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in the { NGC (x, y) };
fifthly, according to { SPhObtaining the similarity among all the areas in the { I }, and obtaining the { I } of the areas in the { I }, wherein the areas in the { I } are similar to each otheri(x, y) } is an image saliency map based on region space sparsity and is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) };
sixthly, { Ii(x, y) } global color histogram-based image saliency maps { HS (x, y) }, { I (I) }i(x, y) } region color contrast based image saliency maps { NGC (x, y) } and { Ii(x, y) } region-space sparsity-based image saliency map NSS (x,y) } to obtain { I }iThe final image saliency map of (x, y) } is denoted as { Sal (x, y) }, and the pixel value of the pixel point whose coordinate position is (x, y) in { Sal (x, y) } is denoted as Sal (x, y), and Sal (x, y) = HS (x, y) × NGC (x, y) × NSS (x, y).
2. The method for extracting the image saliency map based on the region as claimed in claim 1, wherein the specific process of the step (II) is as follows:
2- (1) pair ofiRespectively quantizing the color value of each component of each pixel point in (x, y) to obtain { I }i(x, y) } quantized image, denoted as { P }i(x, y) }, will { PiThe color value of the ith component of the pixel point with the coordinate position (x, y) in (x, y) is recorded as Pi(x,y),Wherein, the symbol
Figure FDA0000430473830000024
Is a rounded-down symbol;
2, calculating { Pi(x, y) }, denoted as { H (k) |0 ≦ k ≦ 4095}, where H (k) represents { P ≦ 4095}, where H (k) representsiThe number of all pixel points belonging to the kth color in (x, y) };
2-3 according to { Pi(x, y) calculating color values of respective components of each pixel in the (x, y) } image, calculating { I }i(x, y) } the color type of the corresponding pixel point will be { IiThe color type of the pixel point with the coordinate position (x, y) in (x, y) is recorded as kxy,kxy=P3(x,y)×256+P2(x,y)×16+P1(x, y) wherein P3(x, y) represents { P }iThe color value, P, of the 3 rd component of the pixel point with the coordinate position (x, y) in (x, y) } is2(x, y) represents { P }iThe color value, P, of the 2 nd component of the pixel with coordinate position (x, y) in (x, y) } is1(x, y) represents { P }iThe color value of the 1 st component of the pixel point with the coordinate position (x, y) in (x, y) };
② 4, calculating{Ii(x, y) } the global color histogram-based saliency value for each pixel point in the (x, y) } will be { IiThe significant value based on the global color histogram of the pixel point with the coordinate position (x, y) in (x, y) is marked as HS (x, y), <math> <mrow> <mi>HS</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mrow> <mo>(</mo> <mi>H</mi> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mi>D</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mi>xy</mi> </msub> <mo>,</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> D ( k xy , k ) = ( p k xy , 1 - p k , 1 ) 2 + ( p k xy , 2 - p k , 2 ) 2 + ( p k xy , 3 - p k , 3 ) 2 , wherein D (k)xyK) represents the k-th item in { H (k) |0 ≦ k ≦ 4095}xyThe euclidean distance between the seed color and the kth color,
Figure FDA0000430473830000031
p k xy , 2 = mod ( k xy / 16 ) ,
Figure FDA0000430473830000034
pk,2=mod(k/16),
Figure FDA0000430473830000035
Figure FDA0000430473830000036
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 1 st component corresponding to a seed color,
Figure FDA0000430473830000037
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyThe color value of the 2 nd component corresponding to the seed color,
Figure FDA0000430473830000038
denotes the k-th in { H (k) |0 ≦ k ≦ 4095}xyColor value of 3 rd component, p, corresponding to a colork,1Denotes a color value, p, of the 1 st component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,2Denotes a color value, p, of the 2 nd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}k,3Representing the color value of the 3 rd component corresponding to the k-th color in { H (k) |0 ≦ k ≦ 4095}, and mod () is a remainder taking operation function;
② 5 according to { IiThe significant value of each pixel point in (x, y) based on the global color histogram is obtained to obtain { I }i(x, y) } global color histogram based image saliency map, denoted as { HS (x, y) }.
3. The method according to claim 1 or 2, wherein { SP (SP-plus-SP) } in step (c)hSimilarity Sim (SP) between p-th and q-th regions inp,SPq) The acquisition process comprises the following steps:
③ 1, pair { SPhQuantizing the color value of each component of each pixel point in each region to obtain { SP }hQuantized region of each region in { SP } would behThe quantization region of the h-th region in (1) } is denoted as { Ph,i(xh,yh) Will { P }h,i(xh,yh) The position of the middle coordinate is (x)h,yh) The color value of the ith component of the pixel point is recorded as Ph,i(xh,yh) Suppose { Ph,i(xh,yh) The position of the middle coordinate is (x)h,yh) Has a pixel point of { IiThe coordinate position in (x, y) } is (x, y), then
Figure FDA0000430473830000039
Wherein x is more than or equal to 1h≤Wh,1≤yh≤Hh,WhRepresents SPhWidth of the H-th area in (H) } HhRepresents SPhHeight of h-th area in (1), sign
Figure FDA00004304738300000310
Is a rounded-down symbol;
③ 2, calculate { SPhColor histogram of quantized region of each region in { P }, will be { Ph,i(xh,yh) The color histogram of is noted as
Figure FDA0000430473830000041
Wherein,
Figure FDA0000430473830000042
represents { Ph,i(xh,yh) The number of all pixel points belonging to the kth color in the pixel;
③ 3, pair { SPhNormalizing the color histogram of the quantization area of each area to obtain a corresponding normalized color histogram, and performing normalization on the color histograms
Figure FDA0000430473830000043
The normalized color histogram obtained after normalization is recorded as <math> <mrow> <mo>{</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>|</mo> <mn>0</mn> <mo>&le;</mo> <mi>k</mi> <mo>&le;</mo> <mn>4095</mn> <mo>}</mo> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msub> <mi>H</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msub> <mi>H</mi> <msub> <mi>SP</mi> <msup> <mi>h</mi> <mo>&prime;</mo> </msup> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein,
Figure FDA0000430473830000045
represents SPhH-th region of { P } quantization region of the h-th regionh,i(xh,yh) The probability of occurrence of a pixel belonging to the k-th color in the pixel,
Figure FDA0000430473830000046
represents SPhQuantization region of h' th region in { P }h',i(xh',yh') All pixel points belonging to the k color in the pixel arrayNumber, 1 ≦ xh'≤Wh',1≤yh'≤Hh',Wh'Represents SPhWidth of H' th area in (H) }, Hh'Represents SPhHeight of h' th area in (P) } hh',i(xh',yh') Represents { Ph',i(xh',yh') The position of the middle coordinate is (x)h',yh') The color value of the ith component of the pixel point of (1);
③ 4, calculate { SPhThe similarity between the p-th and q-th regions in (1), denoted as Sim (SP)p,SPq),Sim(SPp,SPq)=Simc(SPp,SPq)×Simd(SPp,SPq),Simc(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe color similarity between the q-th regions in (j), <math> <mrow> <msub> <mi>Sim</mi> <mi>c</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>p</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>4095</mn> </munderover> <mi>min</mi> <mrow> <mo>(</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>p</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <msup> <mi>H</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mrow> <mo>(</mo> <mi>k</mi> <mo>)</mo> </mrow> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Simd(SPp,SPq) Represents SPhThe p-th region in (f) and (SP)hThe spatial similarity between the q-th regions in (j),
Figure FDA0000430473830000048
wherein, SPpRepresents SPhP-th area in (SP)qRepresents SPhThe q-th area in (1),represents SPhQuantization region of the P-th region in { P } quantization region of the P-th region { Pp,i(xp,yp) The probability of occurrence of a pixel belonging to the k-th color in the pixel,
Figure FDA00004304738300000410
represents SPhQuantization region of the qth region in { P }, a quantization region of the qth region of { P } is a quantization region of the qth regionq,i(xq,yq) The probability of appearance of pixel points belonging to the k-th color in the pixel is more than or equal to 1 and less than or equal to xp≤Wp,1≤yp≤Hp,WpRepresents SPhWidth of p-th area in (H)pRepresents SPhHeight of P-th area in (P) }, Pp,i(xp,yp) Represents { Pp,i(xp,yp) The position of the middle coordinate is (x)p,yp) The color value of the ith component of the pixel point is more than or equal to 1 and less than or equal to xq≤Wq,1≤yq≤Hq,WqRepresents SPhWidth of the q-th area in (H) } mqRepresents SPhHeight of the q-th area in (P) } Pq,i(xq,yq) Represents { Pq,i(xq,yq) The position of the middle coordinate is (x)q,yq) Min () is a minimum function,
Figure FDA0000430473830000051
represents SPhThe coordinate position of the center pixel point in the p-th region in (1),
Figure FDA0000430473830000052
represents SPhThe coordinate position of the central pixel point in the qth area in (1) is the symbol "iill" which is the euclidean distance symbol.
4. The method for extracting the image saliency map based on the region of claim 3, characterized in that the specific process of the step (iv) is as follows:
fourthly-1, calculating { SPhColor contrast of each region in { SP } will be { SP }hColor contrast of the h-th area in (1) } is noted as
Figure FDA0000430473830000053
<math> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>W</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <mo>|</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>m</mi> <msub> <mi>SP</mi> <mi>q</mi> </msub> </msub> <mo>|</mo> <mo>|</mo> <mo>,</mo> </mrow> </math>
Figure FDA00004304738300000513
Wherein, SPhRepresents SPhH area in (SP)qRepresents SPhThe q-th area in (1),
Figure FDA00004304738300000514
represents SPhTotal number of pixel points included in the h-th area in (Sim)d(SPh,SPq) Represents SPhH area in the with { SP }hThe spatial similarity between the q-th regions in (j), represents SPhThe coordinate position of the center pixel point in the h-th area in (1),
Figure FDA0000430473830000057
represents SPhThe coordinate position of the central pixel point in the qth area in (1), the symbol "iill" is the euclidean distance symbol,
Figure FDA0000430473830000058
represents SPhThe color mean vector of the h-th region in (j),
Figure FDA0000430473830000059
represents SPhThe color mean vector of the qth region in (j);
tetra-2, pair { SPhNormalizing the color contrast of each region in the { SP } to obtain the corresponding normalized color contrast, and aligning the { SP }hColor contrast of the h-th area in (1) } color contrast
Figure FDA00004304738300000510
The normalized color contrast obtained after normalization was recorded as
Figure FDA00004304738300000511
<math> <mrow> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <msub> <mi>NGC</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>-</mo> <msub> <mi>NGC</mi> <mi>min</mi> </msub> </mrow> <mrow> <msub> <mi>NGC</mi> <mi>max</mi> </msub> <mo>-</mo> <msub> <mi>NGC</mi> <mi>min</mi> </msub> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, NGCminRepresents SPhMinimum color contrast of M regions in (NGC) } NGCmaxRepresents SPhMaximum color contrast in M regions in (j);
fourthly-3, calculating { SPhColor contrast based saliency value for each region in the will SPhThe h region in (1) has a significant value based on color contrast recorded as
Figure FDA0000430473830000061
<math> <mrow> <msub> <msup> <mi>NGC</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NGC</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Represents SPhSimilarity between the h region and the q region in (1);
fourthly-4, mixing the { SPhThe significant value of each area based on the color contrast is taken as the significant value of all pixel points in the corresponding area, so as to obtain { I }iThe (x, y) } image saliency map based on area color contrast is denoted as { NGC (x, y) }, wherein NGC (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NGC (x, y) }.
5. The method according to claim 4, wherein the specific process of step (c) is as follows:
fifthly-1, calculating { SPhSpatial sparsity of each region in { SP } will behThe spatial sparsity of the h-th region in (1) } is noted as <math> <mrow> <msub> <mi>NSS</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <mi>D</mi> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math> Wherein, Sim (SP)h,SPq) Represents SPhThe similarity between the h-th and q-th regions in (1),represents SPhThe central pixel point in the h-th area in (I) } and (I)i(x, y) } euclidean distance between center pixel points;
fifthly-2, for { SPhNormalizing the space sparsity of each region in the { SP } to obtain corresponding normalized space sparsityhSpatial sparsity of the h-th region in
Figure FDA0000430473830000066
The normalized space sparsity obtained after normalization is recorded as
Figure FDA0000430473830000067
Figure FDA0000430473830000068
Wherein NSSminRepresents SPhMinimum spatial sparsity, NSS, of M regions inmaxRepresents SPhMaximum spatial sparsity in M regions in (j);
fifthly-3, calculating { SPhSignificant value based on spatial sparsity for each region in the { SP will behSignificant value based on spatial sparsity of the h-th region in (1) } is noted
Figure FDA0000430473830000069
<math> <mrow> <msub> <mrow> <mi>NS</mi> <msup> <mi>S</mi> <mrow> <mo>&prime;</mo> <mo>&prime;</mo> </mrow> </msup> </mrow> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>=</mo> <mfrac> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mrow> <mo>(</mo> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> <mo>&times;</mo> <msub> <msup> <mi>NSS</mi> <mo>&prime;</mo> </msup> <msub> <mi>SP</mi> <mi>h</mi> </msub> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>q</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <mi>Sim</mi> <mrow> <mo>(</mo> <msub> <mi>SP</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>SP</mi> <mi>q</mi> </msub> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow> </math>
Fifthly-4, mixing { SPhThe significant value of each region based on space sparsity is used as the significant value of all pixel points in the corresponding region, so as to obtain { I }iThe image saliency map based on the area space sparsity of (x, y) } is marked as { NSS (x, y) }, wherein NSS (x, y) represents the pixel value of a pixel point with a coordinate position (x, y) in { NSS (x, y) }.
CN201310651864.5A 2013-12-05 2013-12-05 Region-based image saliency map extracting method Active CN103632153B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310651864.5A CN103632153B (en) 2013-12-05 2013-12-05 Region-based image saliency map extracting method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310651864.5A CN103632153B (en) 2013-12-05 2013-12-05 Region-based image saliency map extracting method

Publications (2)

Publication Number Publication Date
CN103632153A true CN103632153A (en) 2014-03-12
CN103632153B CN103632153B (en) 2017-01-11

Family

ID=50213181

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310651864.5A Active CN103632153B (en) 2013-12-05 2013-12-05 Region-based image saliency map extracting method

Country Status (1)

Country Link
CN (1) CN103632153B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104133956A (en) * 2014-07-25 2014-11-05 小米科技有限责任公司 Method and device for processing pictures
CN104134217A (en) * 2014-07-29 2014-11-05 中国科学院自动化研究所 Video salient object segmentation method based on super voxel graph cut
CN104392233A (en) * 2014-11-21 2015-03-04 宁波大学 Image saliency map extracting method based on region
CN105512663A (en) * 2015-12-02 2016-04-20 南京邮电大学 Significance detection method based on global and local contrast
CN104050674B (en) * 2014-06-27 2017-01-25 中国科学院自动化研究所 Salient region detection method and device
CN106611427A (en) * 2015-10-21 2017-05-03 中国人民解放军理工大学 A video saliency detection method based on candidate area merging
CN106611178A (en) * 2016-03-10 2017-05-03 四川用联信息技术有限公司 Salient object identification method
CN106709512A (en) * 2016-12-09 2017-05-24 河海大学 Infrared target detection method based on local sparse representation and contrast

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102867313B (en) * 2012-08-29 2015-04-22 杭州电子科技大学 Visual saliency detection method with fusion of region color and HoG (histogram of oriented gradient) features
CN103218832B (en) * 2012-10-15 2016-01-13 上海大学 Based on the vision significance algorithm of global color contrast and spatial distribution in image

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050674B (en) * 2014-06-27 2017-01-25 中国科学院自动化研究所 Salient region detection method and device
CN104133956A (en) * 2014-07-25 2014-11-05 小米科技有限责任公司 Method and device for processing pictures
CN104134217A (en) * 2014-07-29 2014-11-05 中国科学院自动化研究所 Video salient object segmentation method based on super voxel graph cut
CN104134217B (en) * 2014-07-29 2017-02-15 中国科学院自动化研究所 Video salient object segmentation method based on super voxel graph cut
CN104392233A (en) * 2014-11-21 2015-03-04 宁波大学 Image saliency map extracting method based on region
CN104392233B (en) * 2014-11-21 2017-06-06 宁波大学 A kind of image saliency map extracting method based on region
CN106611427A (en) * 2015-10-21 2017-05-03 中国人民解放军理工大学 A video saliency detection method based on candidate area merging
CN106611427B (en) * 2015-10-21 2019-11-15 中国人民解放军理工大学 Saliency detection method based on candidate region fusion
CN105512663A (en) * 2015-12-02 2016-04-20 南京邮电大学 Significance detection method based on global and local contrast
CN106611178A (en) * 2016-03-10 2017-05-03 四川用联信息技术有限公司 Salient object identification method
CN106709512A (en) * 2016-12-09 2017-05-24 河海大学 Infrared target detection method based on local sparse representation and contrast
CN106709512B (en) * 2016-12-09 2020-03-17 河海大学 Infrared target detection method based on local sparse representation and contrast

Also Published As

Publication number Publication date
CN103632153B (en) 2017-01-11

Similar Documents

Publication Publication Date Title
CN103632153B (en) Region-based image saliency map extracting method
CN111915592B (en) Remote sensing image cloud detection method based on deep learning
US20240037926A1 (en) Segmenting objects by refining shape priors
Lopez-Antequera et al. Appearance-invariant place recognition by discriminatively training a convolutional neural network
CN110622177B (en) Instance partitioning
CN106951830B (en) Image scene multi-object marking method based on prior condition constraint
CN104392233B (en) A kind of image saliency map extracting method based on region
CN108564088A (en) Licence plate recognition method, device, equipment and readable storage medium storing program for executing
CN103020985B (en) A kind of video image conspicuousness detection method based on field-quantity analysis
WO2019071976A1 (en) Panoramic image saliency detection method based on regional growth and eye movement model
CN110827312B (en) Learning method based on cooperative visual attention neural network
CN101944178B (en) Significant region extraction method for intelligent monitoring
CN112541448B (en) Pedestrian re-identification method and device, electronic equipment and storage medium
CN102420985B (en) Multi-view video object extraction method
CN103824284A (en) Key frame extraction method based on visual attention model and system
CN102637298A (en) Color image segmentation method based on Gaussian mixture model and support vector machine
CN102799646B (en) A kind of semantic object segmentation method towards multi-view point video
CN110930384A (en) Crowd counting method, device, equipment and medium based on density information
CN103632372B (en) A kind of video saliency image extraction method
CN104050674B (en) Salient region detection method and device
Yanagisawa et al. Face detection for comic images with deformable part model
CN108647605B (en) Human eye gaze point extraction method combining global color and local structural features
CN103974074A (en) Education video and lantern slide synchronization method
Costea et al. Fast traffic scene segmentation using multi-range features from multi-resolution filtered and spatial context channels
CN104408712B (en) Information fusion-based hidden Markov salient region detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20191219

Address after: Room 1,020, Nanxun Science and Technology Pioneering Park, No. 666 Chaoyang Road, Nanxun District, Huzhou City, Zhejiang Province, 313000

Patentee after: Huzhou You Yan Intellectual Property Service Co., Ltd.

Address before: 315211 Zhejiang Province, Ningbo Jiangbei District Fenghua Road No. 818

Patentee before: Ningbo University

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20200702

Address after: 313000 Room 121,221, Building 3, 1366 Hongfeng Road, Wuxing District, Huzhou City, Zhejiang Province

Patentee after: ZHEJIANG DUYAN INFORMATION TECHNOLOGY Co.,Ltd.

Address before: Room 1,020, Nanxun Science and Technology Pioneering Park, No. 666 Chaoyang Road, Nanxun District, Huzhou City, Zhejiang Province, 313000

Patentee before: Huzhou You Yan Intellectual Property Service Co.,Ltd.

TR01 Transfer of patent right