CN106570498A - Salient region detection method and system - Google Patents

Salient region detection method and system Download PDF

Info

Publication number
CN106570498A
CN106570498A CN201610889100.3A CN201610889100A CN106570498A CN 106570498 A CN106570498 A CN 106570498A CN 201610889100 A CN201610889100 A CN 201610889100A CN 106570498 A CN106570498 A CN 106570498A
Authority
CN
China
Prior art keywords
image
input image
color
super
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610889100.3A
Other languages
Chinese (zh)
Other versions
CN106570498B (en
Inventor
王鹏
罗永康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN201610889100.3A priority Critical patent/CN106570498B/en
Publication of CN106570498A publication Critical patent/CN106570498A/en
Application granted granted Critical
Publication of CN106570498B publication Critical patent/CN106570498B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a salient region detection method and system. The method comprises the steps: carrying out the super-pixel segmentation of an input image, and obtaining an image super-pixel processing unit; measuring the topological background confidence of the input image based on the image super-pixel processing unit; calculating the contrast of the color and space position features of the image based on the topological background confidence of the input image, and obtaining the primary saliency value of the image; correcting the primary saliency value of the image through a compactness diffusion method, and obtaining a compactness saliency map; carrying out the saliency assignment through the topological background confidence of the input image and the compactness saliency map, and obtaining a single-scale saliency map of the image; finally carrying out the processing of the single-scale saliency map of the image through a multi-scale fusion method, and obtaining a salient object region of the image. According to the technical scheme of the invention, the method solves a technical problem how to quickly and effectively extract the salient region of the image.

Description

Salient region detection method and detection system
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a method and a system for detecting a salient region.
Background
The human visual system has an attention function, so that the human visual system can quickly lock key areas, only a part of information entering a visual field is selected for extracting and processing information of a subsequent deeper layer, and efficient visual perception is obtained. The significance detection model inspiring the human visual attention mechanism can provide a solution for a computer system to extract useful information quickly and efficiently. Based on the cognitive theory of visual attention and physiological models, researchers in the computer field have proposed many significance detection models. The visual saliency detection is used as a preprocessing process of a visual task, has a very wide application prospect, and can be applied to many different high-level complex visual processing tasks, such as object detection, visual target tracking, object recognition, image compression, robot cognition and the like.
In recent years, a plurality of frame background prior-based salient object detection methods are continuously proposed, and the methods obtain good detection effects. For example:
Xiaohui Li,Huchuan Lu,Lihe Zhang,Xiang Ruan,and Ming-HsuanYang.Saliency detection via dense and sparse reconstruction.In IEEEInternational Conference on Computer Vision(ICCV),pages2976–2983,2013；
Wangjiang Zhu,Shuang Liang,YichenWei,and Jian Sun.Saliencyoptimization from robust background detection.In IEEE Conference on ComputerVision and Pattern Recognition(CVPR),pages 2814–2821,2014；
Chuan Yang,Lihe Zhang,Huchuan Lu,Xiang Ruan,and Ming-HsuanYang.Saliency detection via graph-based manifold ranking.In IEEE Conferenceon Computer Vision and Pattern Recognition(CVPR),pages 3166–3173,2013。
however, these methods tend to focus more on global saliency in inferring saliency values using topological relationships of image regions and bounding box regions. Taking the existing frame background prior-based saliency detection method as an example, the following problems may exist: when the salient object region is similar to the image bounding box region, the local saliency of the salient object is erroneously suppressed; when the image background area is very dissimilar from the image border area, the saliency of the background area may be erroneously exaggerated, resulting in an erroneous saliency detection result.
In view of the above, the present invention is particularly proposed.
Disclosure of Invention
The embodiment of the invention provides a salient region detection method, which aims to solve the technical problem of how to quickly and effectively extract salient regions of images. In addition, the embodiment of the invention also provides a salient region detection system.
In order to achieve the above object, according to one aspect of the present invention, the following technical solutions are provided:
a salient region detection method, the method may comprise:
performing superpixel segmentation on an input image to obtain an image superpixel processing unit;
based on the image superpixel processing unit, measuring a topological background confidence of the input image;
calculating contrast on image color and spatial position features based on the topological background confidence of the input image to obtain an image primary significant value;
correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map;
carrying out significance value assignment by using the topological background confidence coefficient and the compactness significance map of the input image to obtain an image single-scale significance map;
and processing the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map, thereby obtaining an image salient object region.
Further, the measuring the confidence of the topological background of the input image based on the image superpixel processing unit may specifically include:
calculating a super-pixel extension area of the input image based on the image super-pixel processing unit;
calculating the length of a super pixel in the input image along the frame of the input image;
calculating the border connectivity of the input image superpixels based on the superpixel extension area and the length;
and calculating the confidence coefficient of the topological background of the input image based on the frame connection degree.
Further, the calculating a contrast between an image color and a spatial location feature based on the topological background confidence of the input image to obtain an image primary significant value may specifically include:
calculating the image color feature contrast based on the topological background confidence of the input image and according to:
wherein, said p and said piRepresents a super pixel, said i ═ 1.., N; the N represents the number of super pixels contained in the input image; b isiRepresenting the topological background confidence for the input image; d iscolor(p,pi) Representing said superpixel p and said piIn CIE L a bEuclidean distance over color space; the contastcolor(p) representing the image color feature contrast;
wherein, the wpos(p,pi) Is determined by the following formula:
wherein d ispos(p,pi) Represents said p and said piThe euclidean distance of the center position; the sigmaposRepresenting a contrast operating range adjustment coefficient;
calculating the spatial location feature contrast based on the topological background confidence of the input image and according to:
wherein, wcolor(p,pi) Representing said super-pixels p and piSimilarity in color characteristics; the contastpos(p) representing the spatial locality feature contrast;
wherein,
wherein, the sigmacolorAn adjustment coefficient representing the sensitivity of the spatial position feature contrast to the color feature; d iscolor(p,pi) Representing said super-pixels p and piIn CIE L*a*b*Euclidean distance over color space;
calculating the image primary saliency value based on the image color feature contrast and the spatial location feature contrast according to:
CS(p)＝contrastcolor(p)·exp(-k·contrastpos(p))
wherein k represents a weighting coefficient of the contribution of the image color feature and the spatial location feature to the super-pixel saliency value.
Further, the correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map may specifically include:
constructing a k-regular closed loop graph, wherein each connecting edge in the closed loop graph is measured by a correlation matrix;
and determining a manifold ordering function based on a non-normalized Laplace matrix by using a manifold smoothing constraint in graph theory based on the primary image significant value and the incidence matrix, thereby obtaining the compact significant map.
Further, the performing significance value assignment by using the topological background confidence coefficient and the compactness saliency map of the input image to obtain an image single-scale saliency map may specifically include:
performing significance value assignment by using the topological background confidence coefficient of the input image, the compact significance map and Euclidean distance of the super pixels on CIE L a b color space to obtain significance values of the super pixels of the input image;
and assigning the significant value of the input image superpixel to all pixels in the superpixel in the input image as the significant values of the pixels, thereby obtaining the image single-scale significant map.
Further, the correlation matrix is determined by:
wherein, the wijRepresents the strength of the connection; the n is a positive integer; d iscolor(pi,pj) Representing a super pixel piAnd pjIn CIE L*a*b*Euclidean distance over color space; the σ represents an adjustment coefficient; said i and said j take 1 … … N, said N representing the number of superpixels.
Further, the utilizing the topological background confidence of the input image with the compact saliency map and the superpixel is in CIE L*a*b*Performing significance value assignment on the euclidean distance in the color space to obtain a significance value of the super pixel of the input image, which may specifically include:
obtaining a saliency value of said input image superpixel using the following objective function:
wherein, said siRepresenting a super pixel piA significance value of; s isjRepresenting a super pixel pjA significance value of; the i, j is 1.·, N; the N represents the number of super pixels; b isiRepresenting the topological background confidence for the input image; f isiRepresenting a compact saliency map; said λ represents an adjustment factor; d iscolor(pi,pj) Representing said super-pixel piAnd pjIn CIEL*a*b*Euclidean distance over color space; the sigmacolorRepresents an adjustment coefficient; the musmIndicating the adjustment factor for noise suppression.
To achieve the above object, according to another aspect of the present invention, there is also provided a salient region detection system, including:
the segmentation module is used for carrying out superpixel segmentation on the input image to obtain an image superpixel processing unit;
the measuring module is connected with the segmentation module and used for measuring the confidence coefficient of the topological background of the input image based on the image super-pixel processing unit;
the calculation module is connected with the measurement module and used for calculating the contrast on image color and space position characteristics based on the topological background confidence of the input image to obtain an image primary significant value;
the correction module is connected with the calculation module and used for correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map;
the assignment module is respectively connected with the measurement module and the correction module and is used for assigning a significant value by utilizing the topological background confidence coefficient and the compactness significant map of the input image to obtain an image single-scale significant map;
and the fusion module is connected with the assignment module and used for processing the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map so as to obtain an image salient object region.
The embodiment of the invention provides a method and a system for detecting a salient region. The method comprises the steps of carrying out superpixel segmentation on an input image to obtain an image superpixel processing unit; secondly, measuring the confidence coefficient of the topological background of the input image based on the image super-pixel processing unit; then, based on the confidence coefficient of the topological background of the input image, calculating the contrast of the image color and the spatial position characteristics to obtain a primary significant value of the image; correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map; then, carrying out significance value assignment by using the topological background confidence coefficient and the compactness significance map of the input image to obtain a single-scale significance map of the image; and finally, processing the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map, thereby obtaining an image salient object region. By adopting the technical scheme, the global significance and the local significance of the image are considered in the detection process, and the beneficial effect of quickly and effectively extracting the significant region of the image is realized.
Drawings
Fig. 1 is a schematic flow chart of a salient region detection method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a salient region detection system according to an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention. The present embodiment is implemented on the premise of the technical solution of the present invention, and combines the detailed implementation manner and the specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
The prior art often focuses more on global saliency in the process of inferring a saliency value by using a topological relation between an image region and a frame region, because such methods approximate an image global background by using the frame region, and the frame region is not necessarily located in a local neighborhood of the image region, thereby possibly causing the saliency of a salient object region to be suppressed mistakenly or the saliency of a background region to be exaggerated mistakenly.
The basic idea of the embodiment of the invention is as follows: performing superpixel segmentation on an input image to obtain an image processing unit; then, calculating image background confidence according to the frame connection degree, and calculating an image significant value based on the background confidence; then, performing compactness diffusion on the obtained saliency map, correcting the image saliency, and obtaining a more accurate and more compact saliency map; then carrying out significance value optimization assignment by using the background confidence coefficient, the compactness significance map and the spatial neighbor information; and finally, obtaining a multi-scale saliency map by using a multi-scale fusion method.
Therefore, the embodiment of the invention provides a method for detecting a salient region. As shown in fig. 1, the method may be implemented through steps S100 to S150. Wherein:
s100: and carrying out superpixel segmentation on the input image to obtain an image superpixel processing unit.
Specifically, this step may perform superpixel segmentation on the input image using a superpixel segmentation method to obtain an image superpixel processing unit.
S110: and measuring the confidence of the topological background of the input image based on the image super-pixel processing unit.
Specifically, the step may include:
s111: based on the image super-pixel processing unit, a super-pixel extension area of the input image is calculated.
As an example, this step may calculate the super-pixel extension area of the input image according to the following formula:
wherein, p and piDenotes a super pixel, i 1.., N; n represents the number of super pixels included in the input image; sigmageoRepresents an adjustment coefficient; dgeo(p,pi) Representing superpixels p and piIn CIE L*a*b*Geodesic distance in color space; spanarea (p) represents the super pixel extended area of the input image.
For clarity, d will be referred to hereingeo(p,pi) Expressed as dgeo(p, q), which can be realized by the following formula:
wherein d iscolor(pi，pi+1) Representing a super pixel piAnd pi+1In CIE L*a*b*Euclidean distance over color space; n is a positive integer; dgeo(p, q) denotes superpixels p and q in CIE L*a*b*Geodesic distance in color space.
S112: the length of the super-pixels in the input image along the input image border is calculated.
As an example, this step may calculate the length of the super-pixel in the input image along the input image border according to the following formula:
wherein p isiDenotes a super pixel, i 1.., N; n represents the number of super pixels included in the input image; (. cndot.) represents an indicator function that has a value of 1 when a superpixel is on the input image border, and 0 otherwise; b represents a super pixel set on the input image frame; sigmageoRepresents an adjustment coefficient; dgeo(p,pi) Representing superpixels p and piIn CIE L*a*b*Geodesic distance in color space; lb (p) represents the length of a super-pixel in the input image along the input image border.
S113: and calculating the border connection degree of the super pixels of the input image based on the super pixel extension area and the length of the super pixels in the input image along the border of the input image.
By way of example, embodiments of the present invention may calculate the bounding box connectivity of an input image superpixel by:
wherein lb (p) represents the length of a super-pixel in the input image along the border of the input image; spanarea (p) represents the super pixel extended area of the input image; bc (p) represents the input image frame connectivity.
S114: and calculating the confidence coefficient of the topological background of the input image based on the connection degree of the frame.
As an example, the embodiment of the present invention may calculate the topological background confidence of the input image by the following formula:
wherein, BC (p)i) Representing the frame connection degree of the input image; sigmabcRepresenting influence weights of the border connection degree for adjusting the super pixels of the input image on the confidence coefficient of the topological background of the input image; biRepresenting the topological background confidence of the input image.
S120: and calculating the contrast of image colors and spatial position features based on the topological background confidence of the input image to obtain primary significant values of the image.
Specifically, the step may include:
s121: based on the confidence of the topological background of the input image, and calculating the image color feature contrast according to the following formula:
wherein, p and piDenotes a super pixel, i 1.., N; n represents the number of super pixels included in the input image; biRepresenting a topological background confidence of the input image; dcolor(p,pi) Representing superpixels p and piIn CIE L*a*b*Euclidean distance over color space; contastcolor(p) represents the image color feature contrast.
Wherein, wpos(p,pi) Is determined by the following formula:
wherein d ispos(p,pi) Denotes p and piThe euclidean distance of the center position; sigmaposIndicating the contrast operating range adjustment factor.
S122: based on the confidence of the topological background of the input image, calculating the spatial position feature contrast according to the following formula:
wherein d ispos(p,pi) Denotes p and piThe euclidean distance of the center position; biRepresenting a topological background confidence of the input image; w is acolor(p,pi) Representing superpixels p and piSimilarity in color characteristics; contastpos(p) represents the spatial location feature contrast.
Wherein,
wherein σcolorAn adjusting coefficient for representing the sensitivity of the spatial position feature contrast to the color feature; dcolor(p，pi) Representing superpixels p and piIn CIE L*a*b*Euclidean distance over color space.
S123: based on the image color feature contrast and the spatial position feature contrast, calculating an image primary saliency value according to the following formula:
CS(p)＝contrastcolor(p)·exp(-k·contrastpos(p))；
wherein k represents a weight coefficient of the contribution of the image color feature and the spatial location feature to the super-pixel saliency value; cotrastpos(p) represents spatial location feature contrast; contastcolor(p) represents the image color feature contrast.
S130: and correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map.
Specifically, in some optional embodiments, the step may further include:
s131: and constructing a k-regular closed loop graph, wherein each connecting edge in the closed loop graph is measured by a correlation matrix.
In some alternative embodiments, the correlation matrix is determined by:
wherein, wijRepresents the strength of the connection; n is a positive integer; dcolor(pi，pj) Representing a super pixel piAnd pjIn CIE L*a*b*Euclidean distance over color space; σ represents an adjustment coefficient; i and j are 1 … … N, N representing the number of superpixels.
Sigma is used for controlling the weight of the connecting edge, and can be 1/sigma in the concrete implementation process2＝10。
S132: and determining a manifold ordering function based on a non-normalized Laplace matrix by using manifold smoothing constraint in a graph theory based on the primary image significant value and the incidence matrix, thereby obtaining a compact significant graph.
In practical applications, the diffusion process of the manifold ordering function based on the non-normalized laplacian matrix can be calculated according to the following formula:
f*＝(D-αW)-1y；
wherein D represents a connection matrix, D ═ diag { D11,...,dnn}，dii＝Σjwij，wijRepresents the strength of the connection; y ═ y1,...,yn]，yi＝CS(pi)；CS(pi) Representing primary saliency value, p, of an imageiRepresenting superpixels, α representing adjustment coefficients, i, j, n being positive integers, f*A saliency map of the compactness is represented,f*may be a value obtained by a manifold ordered diffusion process, which may be used to represent the saliency value of each super-pixel in the image, i.e. the saliency value of each super-pixel in the image is fi＝fi *。
Preferably, α is 0.99.
Through step S130, the description capability of the model on the local saliency of the image is enhanced by using a compactness diffusion process, so that a more accurate and compact saliency map can be obtained.
S140: and carrying out significance value assignment by using the topological background confidence coefficient and the compact significance map of the input image to obtain the single-scale significance map of the image.
Specifically, the step may include:
s141: using the topological background confidence and compactness saliency map and superpixels of the input image in CIE L*a*b*And carrying out significance value assignment on Euclidean distance on the color space to obtain the significance value of the super pixel of the input image.
In some alternative embodiments, this step may obtain the saliency value of the input image superpixel using the following objective function:
wherein s isiRepresenting a super pixel piA significance value of; sjRepresenting a super pixel pjA significance value of; 1, N(ii) a N represents the number of superpixels; biRepresenting a topological background confidence of the input image; f. ofiRepresenting a compact saliency map; λ represents an adjustment factor, preferably λ takes 1-5.
Wherein,
wherein d iscolor(pi，pj) Representing a super pixel piAnd pjIn CIE L*a*b*Euclidean distance over color space; sigmacolorRepresents an adjustment coefficient; mu.ssmAnd the adjusting coefficient for suppressing the noise is used for adjusting the optimization problem under the complex scene and suppressing the noise in the scene. Preferably, musm＝0.1。
As an example, the saliency value s of a super-pixel of an input image may be obtained by performing an operation such as a derivation on the above-described objective functioni。
S142: and giving the significant value of the super pixel of the input image to all pixels in the super pixel in the input image as the significant values of the pixels, thereby obtaining the single-scale significant image of the image.
S150: and processing the single-scale saliency map of the image by using a multi-scale fusion method to obtain a scale saliency map, thereby obtaining a salient object region of the image.
In particular, in some alternative embodiments, the multi-scale saliency map may be obtained according to the following formula:
wherein r is 1, 2, 3, 4, which corresponds to superpixel segmentation generated according to different superpixel size settings; srRepresenting a single-scale saliency map generated under different-scale superpixel segmentation results; s represents a multi-scale saliency map.
The embodiment gives consideration to the global significance and the local significance of the image in the detection process, and the beneficial effect of quickly and effectively extracting the significant region of the image is achieved.
Although the foregoing embodiments describe the steps in the above sequential order, those skilled in the art will understand that, in order to achieve the effect of the present embodiments, the steps may not be executed in such an order, and may be executed simultaneously (in parallel) or in an inverse order, and these simple variations are within the scope of the present invention.
Based on the same technical concept as the method embodiment, the embodiment of the invention also provides a salient region detection system. As shown in fig. 2, the system 20 may include: segmentation module 21, metric module 22, calculation module 23, modification module 24, assignment module 25, and fusion module 26. The segmentation module 21 is configured to perform superpixel segmentation on the input image to obtain an image superpixel processing unit. The measurement module 22 is connected to the segmentation module 21 and is configured to measure the confidence of the topological background of the input image based on the image superpixel processing unit. The calculation module 23 is connected to the measurement module 22, and is configured to calculate contrast between image color and spatial location features based on the confidence of the topological background of the input image, and obtain a primary saliency value of the image. The correcting module 24 is connected to the calculating module 23, and is configured to correct the primary saliency value of the image by using a compactness diffusion method, so as to obtain a compactness saliency map. The assignment module 25 is connected to the measurement module 22 and the modification module 24, respectively, and is configured to perform significance value assignment by using the topology background confidence and the compactness saliency map of the input image, so as to obtain a single-scale saliency map of the image. The fusion module 26 is connected to the assignment module 25, and is configured to process the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map, so as to obtain an image salient object region.
Those skilled in the art will appreciate that the salient region detection system described above may also include some other known structures such as processors, controllers, memories, buses, etc., where the memories include, but are not limited to, random access memories, flash memories, read only memories, programmable read only memories, volatile memories, non-volatile memories, serial memories, parallel memories, or registers, etc., the processors include, but are not limited to, single core processors, multi-core processors, X86 architecture based processors, CPLD/FPGAs, DSPs, ARM processors, MIPS processors, etc., and the buses may include data buses, address buses, and control buses. Such well-known structures are not shown in fig. 2 in order to not unnecessarily obscure embodiments of the present disclosure.
It should be noted that, when the significant region detection system and the detection method provided in the foregoing embodiments perform significant region detection, the foregoing functional modules or steps are merely illustrated in terms of division, and in practical applications, the foregoing functions may be distributed to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined to complete all or part of the functions described above. The names of the modules or steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the scope of the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the processing steps closely related to the technical solution of the present invention are shown in the drawings, and thus other details not closely related to the present invention are omitted.
So far, the purpose, technical solutions and advantages of the present invention have been described in further detail with reference to preferred embodiments shown in the accompanying drawings, but it is obvious for those skilled in the art that the scope of the present invention is not limited to these specific embodiments. Without departing from the principle of the invention, those skilled in the art can make equivalent changes or substitutions to the related technical features within the spirit and principle of the invention, and the technical solutions after the changes or substitutions will fall within the protection scope of the invention.

Claims (8)

1. A salient region detection method, comprising:
performing superpixel segmentation on an input image to obtain an image superpixel processing unit;
based on the image superpixel processing unit, measuring a topological background confidence of the input image;
calculating contrast on image color and spatial position features based on the topological background confidence of the input image to obtain an image primary significant value;
correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map;
carrying out significance value assignment by using the topological background confidence coefficient and the compactness significance map of the input image to obtain an image single-scale significance map;
and processing the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map, thereby obtaining an image salient object region.
2. The method according to claim 1, wherein the measuring the topological background confidence of the input image based on the image superpixel processing unit specifically comprises:
calculating a super-pixel extension area of the input image based on the image super-pixel processing unit;
calculating the length of a super pixel in the input image along the frame of the input image;
calculating the border connectivity of the input image superpixels based on the superpixel extension area and the length;
and calculating the confidence coefficient of the topological background of the input image based on the frame connection degree.
3. The method according to claim 1, wherein the calculating a contrast between an image color and a spatial location feature based on the topological background confidence of the input image to obtain an image primary saliency value comprises:
calculating the image color feature contrast based on the topological background confidence of the input image and according to:
contrast c o l o r ( p ) = Σ i = 1 N d c o l o r ( p , p i ) w p o s ( p , p i ) b i
wherein, said p and said piRepresents a super pixel, said i ═ 1.., N; the N represents the number of super pixels contained in the input image; b isiRepresenting the topological background confidence for the input image; d iscolor(p,pi) Representing said superpixel p and said piIn CIE L*a*b*Euclidean distance over color space; the contastcolor(p) representing the image color feature contrast;
wherein, the wpos(p,pi) Is determined by the following formula:
w p o s ( p , p i ) = exp ( - d p o s 2 ( p , p i ) 2 σ p o s 2 )
wherein d ispos(p,pi) Represents said p and said piThe euclidean distance of the center position; the sigmaposRepresenting a contrast operating range adjustment coefficient;
calculating the spatial location feature contrast based on the topological background confidence of the input image and according to:
contrast p o s ( p ) = Σ i = 1 N d p o s ( p , p i ) w c o l o r ( p , p i ) b i
wherein, wcolor(p,pi) Representing said super-pixels p and piSimilarity in color characteristics; the contastpos(p) representing the spatial locality feature contrast;
wherein,
wherein, the sigmacolorAn adjustment coefficient representing the sensitivity of the spatial position feature contrast to the color feature; d iscolor(p,pi) Representing said super-pixels p and piIn CIE L*a*b*Euclidean distance over color space;
calculating the image primary saliency value based on the image color feature contrast and the spatial location feature contrast according to:
CS(p)＝contrastcolor(p)·exp(-k·contrastpos(p))
wherein k represents a weighting coefficient of the contribution of the image color feature and the spatial location feature to the super-pixel saliency value.
4. The method according to claim 1, wherein the correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map specifically comprises:
constructing a k-regular closed loop graph, wherein each connecting edge in the closed loop graph is measured by a correlation matrix;
and determining a manifold ordering function based on a non-normalized Laplace matrix by using a manifold smoothing constraint in graph theory based on the primary image significant value and the incidence matrix, thereby obtaining the compact significant map.
5. The method according to claim 1, wherein the performing significance value assignment by using the topological background confidence and the compact saliency map of the input image to obtain an image single-scale saliency map specifically comprises:
utilizing the topological background confidence level of the input image with the compact saliency map and the superpixel in CIE L*a*b*Carrying out significance value assignment on Euclidean distance on a color space to obtain a significance value of the super pixel of the input image;
and assigning the significant value of the input image superpixel to all pixels in the superpixel in the input image as the significant values of the pixels, thereby obtaining the image single-scale significant map.
6. The method of claim 4, wherein the correlation matrix is determined by:
W = [ w i j ] n × n , w i j = exp ( - d c o l o r ( p i , p j ) 2 σ 2 ) ;
wherein, the wijRepresents the strength of the connection; the n is a positive integer; d iscolor(pi，pj) Representing a super pixel piAnd pjIn CIEL*a*b*Euclidean distance over color space; the σ represents an adjustment coefficient; said i and said j take 1 … … N, said N representing the number of superpixels.
7. The method of claim 5, wherein the utilizing the topological background confidence of the input image with the compact saliency map and the superpixel is in CIE L*a*b*Performing significance value assignment on Euclidean distance on a color space to obtain a significance value of the super pixel of the input image, which specifically comprises the following steps:
obtaining a saliency value of said input image superpixel using the following objective function:
arg min s i λ Σ i = 1 N b i s i 2 + Σ i = 1 N f i ( 1 - s i ) 2 + Σ i , j h i j ( s i - s j ) 2
h i j = exp ( - d c o l o r 2 - ( p i , p j ) 2 σ c o l o r 2 ) + μ s m
wherein, said siRepresenting a super pixel piA significance value of; s isjRepresenting a super pixel pjA significance value of; the i, j is 1.·, N; the N represents the number of super pixels; b isiRepresenting the topological background confidence for the input image; f isiRepresenting a compact saliency map; said λ represents an adjustment factor; d iscolor(pi，pj) Representing said super-pixel piAnd pjIn CIE L*a*b*Euclidean distance over color space; the sigmacolorRepresents an adjustment coefficient; the musmIndicating the adjustment factor for noise suppression.
8. A salient region detection system, comprising:
the segmentation module is used for carrying out superpixel segmentation on the input image to obtain an image superpixel processing unit;
the measuring module is connected with the segmentation module and used for measuring the confidence coefficient of the topological background of the input image based on the image super-pixel processing unit;
the calculation module is connected with the measurement module and used for calculating the contrast on image color and space position characteristics based on the topological background confidence of the input image to obtain an image primary significant value;
the correction module is connected with the calculation module and used for correcting the primary saliency value of the image by using a compactness diffusion method to obtain a compactness saliency map;
the assignment module is respectively connected with the measurement module and the correction module and is used for assigning a significant value by utilizing the topological background confidence coefficient and the compactness significant map of the input image to obtain an image single-scale significant map;
and the fusion module is connected with the assignment module and used for processing the image single-scale saliency map by using a multi-scale fusion method to obtain a scale saliency map so as to obtain an image salient object region.
CN201610889100.3A 2016-10-12 2016-10-12 Salient region detecting method and detection system Active CN106570498B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610889100.3A CN106570498B (en) 2016-10-12 2016-10-12 Salient region detecting method and detection system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610889100.3A CN106570498B (en) 2016-10-12 2016-10-12 Salient region detecting method and detection system

Publications (2)

Publication Number Publication Date
CN106570498A true CN106570498A (en) 2017-04-19
CN106570498B CN106570498B (en) 2019-06-14

Family

ID=58532817

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610889100.3A Active CN106570498B (en) 2016-10-12 2016-10-12 Salient region detecting method and detection system

Country Status (1)

Country Link
CN (1) CN106570498B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805136A (en) * 2018-03-26 2018-11-13 中国地质大学（武汉） A kind of conspicuousness detection method towards waterborne contaminant monitoring
CN109284753A (en) * 2018-08-30 2019-01-29 深圳大学 A kind of localization method of liquid transmission line and application
CN109753957A (en) * 2018-12-07 2019-05-14 东软集团股份有限公司 Image significance detection method, device, storage medium and electronic equipment
CN110942095A (en) * 2019-11-27 2020-03-31 中国科学院自动化研究所 Method and system for detecting salient object area

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103745468A (en) * 2014-01-07 2014-04-23 上海交通大学 Significant object detecting method based on graph structure and boundary apriority
CN103914834A (en) * 2014-03-17 2014-07-09 上海交通大学 Significant object detection method based on foreground priori and background priori
CN105046701A (en) * 2015-07-08 2015-11-11 安徽大学 Multi-scale salient target detection method based on construction graph

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103745468A (en) * 2014-01-07 2014-04-23 上海交通大学 Significant object detecting method based on graph structure and boundary apriority
CN103914834A (en) * 2014-03-17 2014-07-09 上海交通大学 Significant object detection method based on foreground priori and background priori
CN105046701A (en) * 2015-07-08 2015-11-11 安徽大学 Multi-scale salient target detection method based on construction graph

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PENG WANG 等: "Salient region detection based on Local and Global Saliency", 《IEEE INTERNATIONAL CONFERENCE ON ROBOTICS & AUTOMATION (ICRA)》 *
YONGKANG LUO 等: "Sparse-Distinctive Saliency Detection", 《IEEE SIGNAL PROCESSING LETTERS》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805136A (en) * 2018-03-26 2018-11-13 中国地质大学（武汉） A kind of conspicuousness detection method towards waterborne contaminant monitoring
CN109284753A (en) * 2018-08-30 2019-01-29 深圳大学 A kind of localization method of liquid transmission line and application
CN109753957A (en) * 2018-12-07 2019-05-14 东软集团股份有限公司 Image significance detection method, device, storage medium and electronic equipment
CN109753957B (en) * 2018-12-07 2020-11-27 东软集团股份有限公司 Image significance detection method and device, storage medium and electronic equipment
CN110942095A (en) * 2019-11-27 2020-03-31 中国科学院自动化研究所 Method and system for detecting salient object area

Also Published As

Publication number Publication date
CN106570498B (en) 2019-06-14

Similar Documents

Publication Publication Date Title
CN111902825B (en) Polygonal object labeling system and method for training object labeling system
CN106570498B (en) Salient region detecting method and detection system
CN114842365B (en) Unmanned aerial vehicle aerial photography target detection and identification method and system
US9367762B2 (en) Image processing device and method, and computer readable medium
Lu et al. An alternative of lidar in nighttime: Unsupervised depth estimation based on single thermal image
Arnold et al. Map-free visual relocalization: Metric pose relative to a single image
Barrera et al. Multispectral piecewise planar stereo using Manhattan-world assumption
JP2018022360A (en) Image analysis device, image analysis method and program
Gee et al. Discovering Planes and Collapsing the State Space in Visual SLAM.
CN114155365B (en) Model training method, image processing method and related device
Li et al. Rts3d: Real-time stereo 3d detection from 4d feature-consistency embedding space for autonomous driving
Zhang et al. A new high resolution depth map estimation system using stereo vision and kinect depth sensing
CN107240130A (en) Remote Sensing Image Matching method, apparatus and system
CN115421158A (en) Self-supervision learning solid-state laser radar three-dimensional semantic mapping method and device
El Bouazzaoui et al. Enhancing RGB-D SLAM performances considering sensor specifications for indoor localization
Dong et al. Hierarchical superpixel-to-pixel dense matching
CN110992404B (en) Target tracking method, device and system and storage medium
CN117253044A (en) Farmland remote sensing image segmentation method based on semi-supervised interactive learning
Song et al. Small UAV based multi-viewpoint image registration for monitoring cultivated land changes in mountainous terrain
Plagemann et al. A nonparametric learning approach to range sensing from omnidirectional vision
JP7003617B2 (en) Estimator, estimation method, and estimation program
Zhao et al. Multi-look SAR image segmentation based on voronoi tessellation technique and EM/MPM algorithm
CN116934591A (en) Image stitching method, device and equipment for multi-scale feature extraction and storage medium
Kim et al. Ep2p-loc: End-to-end 3d point to 2d pixel localization for large-scale visual localization
Laaroussi et al. Dynamic mosaicking: combining A* algorithm with fractional Brownian motion for an optimal seamline detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant