CN106778776A - A kind of time-space domain significance detection method based on location-prior information - Google Patents

Abstract

The present invention relates to a kind of time-space domain significance detection method, belong to image processing field, and in particular to a kind of time-space domain significance detection method based on location-prior information.Main idea is that the well-marked target of video saliency map should be space and time continuous, by obtaining the foreground area positional information continuous in time comprising notable object, enhancing improves significance control methods, including prospect background contrast method and local foreground contrast method, so as to effectively suppress background area, and highlighted prospect well-marked target.The present invention proposes in frame of video Movement consistency to improve the differentiation degree of temporal motion information simultaneously, so as to obtain more accurately moving significance testing result.Additionally, merging the result of time domain and the detection of spatial domain significance to consider space-time domain information, the fusion that the present invention passes through time-space domain significance testing result, and then improve the accuracy that time-space domain significance is detected.

Description

A kind of time-space domain significance detection method based on location-prior information

Technical field

The present invention relates to a kind of time-space domain significance detection method, belong to image processing field, and in particular to one kind is based on The time-space domain significance detection method of location-prior information.

Background technology

In computer vision field, image/video significance is detected as target identification, video monitoring, mobile robot Preprocessing part widely studied.The research of image/video significance is generally basede on bottom-up quick significance Detection mode, this kind of method is driven by middle low-level feature, and the scope of application is wider, and arithmetic speed is very fast.

The significance research of current video time-space domain is relatively new for image significance research, the phase based on video with image Like property, existing image significance detection method can be applied independently in each two field picture of video, and such mode have ignored The time-domain information of video.It is that video significance detects topmost influence factor in view of time-domain information, spatial feature can be tied Close the significance that temporal signatures calculate independent frame.

Some algorithms of the prior art mainly have：(1) to original spatial feature on existing image significance model Dimension is extended, and adds motion feature, the notable position for predicting human eye prediction in dynamic scene；(2) using brightness, face The local contrast of color and motion vector generates final saliency map；(3) using the space-time remarkable degree based on various low-level features The video input of frame per second high is switched into low frame-rate video.These methods all have ignored the space-time expending of video saliency map, can The well-marked target in notable figure can be caused on a timeline and discontinuous.

The feature more high-dimensional in order to obtain video, region contrast is widely applied in significance detection, region pair It is zoning feature difference and weighted space position relationship than degree common practice, but such weighting scheme may lead Cause target decay or be difficult to differentiate between the similar color in foreground and background region.

In order to solve the two problems, it is necessary to be positioned to foreground location using location-prior information, believe by using position Breath, keeps the space-time consistency of time-space domain saliency map, and improves contrast algorithm, so as to improve the standard of video significance detection True property.

The content of the invention

In view of the shortcomings of the prior art, it is aobvious the invention provides a kind of time-space domain video based on foreground location prior information Work degree detection method, the method keeps video time-space domain saliency map by continuously acquiring video foreground zone position information Interframe continuity simultaneously improves prospect background contrast algorithm, and then highlight well-marked target and suppress background area.

A kind of significance detection method based on location-prior information, including：

Super-pixel collection segmentation step, the super-pixel collection that size is similar, color is highly consistent is divided into by frame of video It

Spatial domain significance calculation procedure, based on spatial relation weighted calculation super-pixelIt is super with other super-pixel Pixel importance degree characteristic value difference is obtainedSpatial domain contrast, the spatial domain contrast based on each super-pixel obtains the sky of video Domain work degree figureWherein, super-pixel importance degree is the contrast of its corresponding color and other super-pixel primary colors of whole frame；

Time domain significance calculation procedure, histogram is moved according to super-pixelHistogram is moved with whole frameBetween Difference obtain the motion discrimination of each super-pixelCalculated based on the feature difference updated between motion discrimination The time domain saliency map of video

Space-time remarkable figure fusion steps, the uniformity MCT based on time domain to spatial domain, the uniformity MCS in spatial domain to time domain, adopt With the options based on the two consistency coefficients and interaction item non-linear fusion Space-time Domain saliency map；Wherein, MCT and MCS The spatial domain for being respectively divided by all super-pixel in the saliency map of spatial domain by the product of time domain saliency map and spatial domain saliency map is notable The notable angle value summation of time domain of all super-pixel is obtained in angle value summation time domain saliency map.

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the super-pixel collection segmentation In step, the frame is divided into by the super-pixel collection that size is similar, color is highly consistent based on SLIC algorithms I is reduced using minimum variance quantizing algorithm_tThe low color of the middle frequency of occurrences, so as to quantify I_tColor.

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the spatial domain significance meter Calculate in step, super-pixel importance degree is calculated by color histogram contrast algorithm

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the spatial domain contrast Calculating includes that prospect background contrast is calculated, specially：

It is critical path to define shortest path of the super-pixel apart from background areaThe pass of background area super-pixel Key path is the super-pixel with the Euclidean distance between the most short background area super-pixel of its neighbor distance, and the super picture of foreground area The critical path of element is related with the area of foreground area,It is defined as follows.

In formula, F_tIt is foreground area, B_tIt is background area；

Spatial feature difference based on positional information isBetween super-pixel importance degree difference ψ (i, j) weighting When unit-step function ε (ψ (i, j)), i.e. ψ (i, j) are more than 0, ε (ψ (i, j)) value is 1, otherwise is then 0；ψ (i, j) is according to super picture Element is located at different regions, is defined as follows：

Prospect background contrast is the space characteristics difference weighted space position relationship of super-pixel, prospect background contrastIt is expressed as follows, in formula,It is the Euclidean distance between super-pixel；

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the spatial domain contrast Calculating includes that prospect local contrast is calculated, specially：Calculate super-pixel and super-pixel in other foreground areas in foreground area Super-pixel importance degree characteristic value difference summation and respective weight spatial relation, obtain prospect local contrast

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the spatial domain contrast Calculating includes that prospect local contrast is calculated, prospect local contrast is calculated, and calculates video spatial domain significance based on following formula Figure

Wherein, linear coefficient η is 0.5,It is prospect local contrast,For prospect background is contrasted Degree.

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the time domain significance meter Calculate in step, each frame Pixel-level motion vector field is extracted using LDOF optical flow methods, according to motion vector field, extract each super picture The motion histogram of elementHistogram is moved with whole frameMoved by calculating super-pixel level motion histogram and frame level Difference between histogram, obtains the motion discrimination of each super-pixel

Preferably, a kind of above-mentioned significance detection method based on location-prior information, is calculating motion discrimination Afterwards, add intraframe motion consistency constraint to update motion discrimination, specific steps include：Calculated according to SF methodsSky Between varianceUpdating motion discrimination isAccording to the renewal campaign discrimination for obtaining, use Prospect background contrast algorithm, calculates difference accumulation of whole each super-pixel of frame with the renewal motion discrimination of other super-pixel, So as to obtain time domain saliency map

Preferably, a kind of above-mentioned significance detection method based on location-prior information, the space-time remarkable figure melts Close in step, can be obtained according to SP methods, interaction itemAccording to MCT and MCS pairsWithFusion, fusion Mode is as follows：

OptionsCalculation be：The notable angle value of all super-pixel in time domain and spatial domain is calculated respectively Summation simultaneously weights each super-pixel to the Euclidean distance at foreground area center, obtains time domain significance distribution DIS^TIt is notable with spatial domain Degree distribution DIS^S, compare DIS^TAnd DIS^SSize,It is the corresponding saliency map of both middle smaller value.

Preferably, a kind of above-mentioned significance detection method based on location-prior information, is merged based on following formula and interacted Item and options obtain final time-space domain saliency map

In formula, λ_tEqual to sqrt (MCTMCS), using λ_tSpace-time Domain saliency map can preferably be merged.

The present invention can realize that carrying out accurate video significance to the complicated video of background compound movement extracts, so that Effectively solve the problems, such as that background technology is mentioned, can be as the pretreatment work part of video computer vision.

Brief description of the drawings

Fig. 1 is the inventive method flow chart.

Specific embodiment

Understand for the ease of those of ordinary skill in the art and implement the present invention, below in conjunction with the accompanying drawings and embodiment is to this hair It is bright to be described in further detail, it will be appreciated that implementation example described herein is merely to illustrate and explain the present invention, not For limiting the present invention.

The present invention is a kind of video time-space domain significance extracting method based on location-prior information, based on foreground location letter Breath, proposes enhancing prospect background contrast algorithm and prospect local contrast algorithm；

Its thinking is to combine intraframe motion uniformity to extract video high-level characteristic spatially and temporally, and based on former frame Time-space domain saliency map obtains the foreground area positional information of current video frame, based on the location-prior information, in extracting Layer feature is calculated using enhanced prospect background contrast algorithm and respectively obtains spatial domain, time domain saliency map.Finally by interaction Formula fusion obtains final time-space domain saliency map.

When implementing, the frame is divided into by the super-pixel that size is similar, color is highly consistent using SLIC algorithms first CollectionAnd reduce I using minimum variance quantizing algorithm_tThe low color of the middle frequency of occurrences；Then according to former frame Time-space domain saliency map extract foreground area F of UNICOM region of the notable angle value more than 0.4 as present frame_t, initial frame Foreground area F_tIt is defaulted as whole frame video image；According toMain color calculating super-pixelSuper-pixel importance degreeIt is then based on foreground location information, Utilization prospects background contrastsWith prospect local contrastWeighted calculation obtains spatial domain saliency mapThen the Pixel-level motion vector field per frame is extracted, one is gone forward side by side Step is extractedMotion histogramHistogram is moved with whole frameAccording to super-pixel level histogram and frame level histogram Difference is calculated motion discriminationThe renewal being distributed by the space for introducing reflection Movement consistency is transported Dynamic discrimination is obtainedBased on prospect background contrast method, the difference accumulation for updating motion contrast is calculated, obtained Time domain saliency mapIt is finally rightWithCarry out non-linear fusion and obtain final time-space domain significance Figure

Fig. 1 is the idiographic flow of the present embodiment, and the present embodiment uses MATLAB2012a as Simulation Experimental Platform, normal Tested on video significance detection video library SegTrack, FBMS and DS2.

It is further elaborated to of the invention below for above-described embodiment, flow of the invention includes：

Step 1：For a specific frame of video I_t, the frame is divided into size is similar, color is high using SLIC algorithms The consistent super-pixel collection of degreeI is reduced using minimum variance quantizing algorithm_tThe low color of the middle frequency of occurrences, from And quantify I_tColor.

Step 2：Color based on super-pixel has high similarity, and each super-pixel can be by its frequency of occurrences highest face Color is represented, then define the contrast that spatial feature super-pixel importance degree is its corresponding color and other super-pixel primary colors of whole frame Degree, super-pixel importance degree can be obtained by color histogram contrast (HC) algorithm

Step 3：Extract correspondence frame of video I_t-1Time-space domain saliency mapUNICOM's highlight regions as I_t's Foreground area F_t, I_tOther parts be background area B_t.SettingMiddle pixel gray value is bright values high more than 0.4.

Step 4：To I_tIn specific super-pixelCalculate it total with the super-pixel importance degree characteristic value difference of other super-pixel With simultaneously respective weight spatial relation, obtainSpatial domain contrast.In front of characteristic value difference and spatial relationship calculation Scape positional information is related, and spatial domain contrast algorithm is divided into prospect background contrast and local foreground contrast, implement including Following sub-step：

Step 4.1：Calculated for spatial domain prospect background contrast, defining shortest path of the super-pixel apart from background area is Critical pathThe critical path of background area super-pixel is the super-pixel with the most short background area of its neighbor distance Euclidean distance between super-pixel, and the critical path of foreground area super-pixel is related with the area of foreground area,It is fixed Justice is as follows.

Spatial feature difference based on positional information isBetween super-pixel importance degree difference ψ (i, j) weighting When unit-step function ε (ψ (i, j)), i.e. ψ (i, j) are more than 0, ε (ψ (i, j)) value is 1, otherwise is then 0；ψ (i, j) is according to super picture Element is located at different regions, is defined as follows：

Wherein alpha parameter is used to suppress the accumulation of the super-pixel for the super-pixel difference of background area of foreground area, so that Further suppress background.Prospect background contrast is the space characteristics difference weighted space position relationship of super-pixel, prospect background ContrastIt is expressed as follows,It is the Euclidean distance between super-pixel.

Step 4.2：Calculated for prospect local contrast, calculate the super picture of super-pixel and other super-pixel in foreground area Plain importance degree characteristic value difference summation and respective weight spatial relation, obtain prospect local contrastCalculate such as Under：

Step 4.3：It is linear to combine prospect background contrast and prospect local contrast result, obtain video spatial domain significance FigureLinear coefficient η is 0.5,It is expressed as follows

Step 5：Each frame Pixel-level motion vector field is extracted using LDOF optical flow methods, according to motion vector field, extracts every The motion histogram of individual super-pixelHistogram is moved with whole frameBy calculating super-pixel level motion histogram and frame Difference between level motion histogram, obtains the motion discrimination of each super-pixelAccording to Movement consistency, meter The space variance of the equal motion discrimination of reflection space distribution is calculated, so that renewal motion discrimination is obtained, based on renewal Feature difference between motion discrimination, prospect of the application background contrasts algorithm obtains time domain significance, its implement including Following steps：

Step 5.1：According to SF methods, calculateSpace varianceThen updating motion discrimination is

Step 5.2：According to the renewal campaign discrimination for obtaining, using prospect background contrast algorithm, calculating time domain each Super-pixel is accumulated with the difference of the renewal motion discrimination of other super-pixel, so as to obtain time domain saliency map

Step 6：Calculate the product of time domain saliency map and spatial domain saliency map)p It is respectively divided by the notable angle value summation in spatial domain of all super-pixel in the saliency map of spatial domainIn time domain saliency map The notable angle value summation of time domain of all super-pixelTime domain is obtained to the uniformity MCT in spatial domain and spatial domain then The uniformity MCS in domain.Based on the two parameters of consistency using options and interaction item non-linear fusion Space-time Domain significance Figure, implements including following sub-step：

Step 6.1：Interaction itemAccording to MCT and MCS pairsWithFusion, amalgamation mode is as follows：

Step 6.2：The summation of the notable angle value of all super-pixel in time domain and spatial domain is calculated respectively and weights each super picture Element obtains time domain significance distribution DIS to the Euclidean distance at foreground area center^TDIS is distributed with spatial domain significance^S.Compare DIS^T And DIS^SSize,It is the corresponding saliency map of both middle smaller value.

Step 6.3：Interaction item and options are merged, the formula for obtaining final time-space domain saliency map is as follows, Wherein, λ_tEqual to sqrt (MCTMCS).

It should be appreciated that the part that this specification is not elaborated belongs to prior art.

It should be appreciated that the above-mentioned description for preferred embodiment is more detailed, therefore can not be considered to this The limitation of invention patent protection scope, one of ordinary skill in the art is not departing from power of the present invention under enlightenment of the invention Profit requires under protected ambit, can also make replacement or deform, each falls within protection scope of the present invention, this hair It is bright scope is claimed to be determined by the appended claims.

Claims (10)

1. a kind of significance detection method based on location-prior information, it is characterised in that including：

Super-pixel collection segmentation step, by frame of video I_tIt is divided into the super-pixel collection that size is similar, color is highly consistent

Spatial domain significance calculation procedure, based on spatial relation weighted calculation super-pixelWith the super-pixel weight of other super-pixel Characteristic value difference is spent to obtainSpatial domain contrast, the spatial domain contrast based on each super-pixel obtains the spatial domain work degree of video FigureWherein, super-pixel importance degree is the contrast of its corresponding color and other super-pixel primary colors of whole frame；

Time domain significance calculation procedure, histogram is moved according to super-pixelHistogram is moved with whole frameBetween difference The different motion discrimination for obtaining each super-pixelRenewal campaign discrimination based on frame in consistency constraint, calculates The time domain saliency map of video

Space-time remarkable figure fusion steps, the uniformity MCT based on time domain to spatial domain, the uniformity MCS in spatial domain to time domain is using choosing Select item and interaction item non-linear fusion Space-time Domain saliency map；Wherein, MCT and MCS is by time domain saliency map and spatial domain significance The product of figure owns in being respectively divided by the notable angle value summation time domain saliency map in the spatial domain of all super-pixel in the saliency map of spatial domain The notable angle value summation of time domain of super-pixel is obtained.

2. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described In super-pixel collection segmentation step, the frame is divided into by the super-pixel collection that size is similar, color is highly consistent based on SLIC algorithmsI is reduced using minimum variance quantizing algorithm_tThe low color of the middle frequency of occurrences, so as to quantify I_tColor.

3. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described In the significance calculation procedure of spatial domain, super-pixel importance degree is calculated by color histogram contrast algorithm

4. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described The calculating of spatial domain contrast is calculated including prospect background contrast, specially：

It is critical path to define shortest path of the super-pixel apart from background areaThe critical path of background area super-pixel It is the super-pixel with the Euclidean distance between the most short background area super-pixel of its neighbor distance, and the pass of foreground area super-pixel Key path is related with the area of foreground area,It is defined as follows：

$D_S\left({\mathrm{sp}}_t^i\right)=\left\{\begin{array}{cc}sqrt\left(S_{area}\right(F_t\left)\right)& {\mathrm{sp}}_t^i\∈F_t\\ \underset{j}{\arg min}D({\mathrm{sp}}_t^i,{\mathrm{sp}}_t^j)& {\mathrm{sp}}_t^i,{\mathrm{sp}}_t^j\∈B_t\end{array}\right.---\left(1\right);$

In formula, F_tIt is foreground area, B_tIt is background area；

Spatial feature difference based on positional information isBetween super-pixel importance degree difference ψ (i, j) weighted units rank Jump function of ε (Ψ (i, j)), i.e. when ψ (i, j) is more than 0, ε (ψ (i, j)) value is 1, otherwise is then 0；ψ (i, j) is according to super-pixel position In different regions, it is defined as follows：

$\mathrm{\ψ}(i,j)=\left\{\begin{array}{ccc}SS\left({\mathrm{sp}}_t^i\right)-SS\left({\mathrm{sp}}_t^j\right)& if& {\mathrm{sp}}_t^i\∈F_t\\ \mathrm{\α}\·SS\left({\mathrm{sp}}_t^i\right)-SS\left({\mathrm{sp}}_t^j\right)& if& {\mathrm{sp}}_t^i\∈B_t\end{array}\right.---\left(2\right);$

Prospect background contrast is the space characteristics difference weighted space position relationship of super-pixel, prospect background contrastIt is expressed as follows, in formula,It is the Euclidean distance between super-pixel；

5. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described The calculating of spatial domain contrast is calculated including prospect local contrast, specially：Calculate super-pixel and foreground area in foreground area In other super-pixel super-pixel importance degree characteristic value difference summation and respective weight spatial relation, obtain prospect locally right Degree of ratio

6. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described The calculating of spatial domain contrast includes that prospect local contrast is calculated, prospect local contrast is calculated, and is regarded based on following formula calculating The empty saliency map of frequency

$S_S\left({\mathrm{sp}}_t^i\right)=\mathrm{\η}\·FBC\left({\mathrm{sp}}_t^i\right)+(1-\mathrm{\η})\·LC\left({\mathrm{sp}}_t^i\right)---\left(4\right);$

Wherein, linear coefficient η is 0.5,It is prospect local contrast,It is prospect background contrast.

7. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described In time domain significance calculation procedure, each frame Pixel-level motion vector field is extracted using LDOF optical flow methods, according to motion vector field, Extract the motion histogram of each super-pixelHistogram is moved with whole frameBy calculating super-pixel level motion Nogata Difference between figure and frame level motion histogram, obtains the motion discrimination of each super-pixel

8. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that described In time domain significance calculation procedure, specifically include：According to have equal motion distinguish angle value super-pixel collection center it Between difference, and weight correspondence super-pixel collective motion discrimination difference, be calculatedSpace varianceUpdate fortune Dynamic discrimination isAccording to the renewal campaign discrimination for obtaining, calculated using prospect background contrast Method, calculates difference accumulation of each super-pixel of time domain with the renewal motion discrimination of other super-pixel, notable so as to obtain time domain Degree figure

9. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that：

In the space-time remarkable figure fusion steps, according to SP methods, interaction itemAccording to MCT

With MCS pairsWithFusion, amalgamation mode is as follows：

$S_{\mathrm{int}}\left({\mathrm{sp}}_t^i\right)=\frac{MCT\·S_T\left({\mathrm{sp}}_t^i\right)+MCS\·S_S\left({\mathrm{sp}}_t^i\right)}{MCT+MCS}---\left(5\right);$

OptionsCalculation be：The summation of the notable angle value of all super-pixel in time domain and spatial domain is calculated respectively And each super-pixel to the Euclidean distance at foreground area center is weighted, obtain time domain significance distribution DIS^TWith spatial domain significance point Cloth DIS^S, compare DIS^TAnd DIS^SSize,It is the corresponding saliency map of both middle smaller value.

10. a kind of significance detection method based on location-prior information according to claim 1, it is characterised in that base In following formula fusion interaction itemAnd optionsObtain final time-space domain saliency map

$S\left({\mathrm{sp}}_t^i\right)={\mathrm{\λ}}_t\·S_{\mathrm{int}}\left({\mathrm{sp}}_t^i\right)+(1-{\mathrm{\λ}}_t)\·S_{sel}\left({\mathrm{sp}}_t^i\right)---\left(6\right);$

In formula, λ_tEqual to sqrt (MCTMCS).