CN105869178B - A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature - Google Patents
A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature Download PDFInfo
- Publication number
- CN105869178B CN105869178B CN201610261547.6A CN201610261547A CN105869178B CN 105869178 B CN105869178 B CN 105869178B CN 201610261547 A CN201610261547 A CN 201610261547A CN 105869178 B CN105869178 B CN 105869178B
- Authority
- CN
- China
- Prior art keywords
- pixel
- super
- energy function
- image
- spatial domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20064—Wavelet transform [DWT]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature that the present invention relates to a kind of, belongs to image procossing and technical field of computer vision.The present invention extracts wavelet field high yardstick target signature first, and calculate movement and spatial domain skirt response, by wavelet field, spatially and temporally edge feature merges to obtain Multiscale combination probability skirt response, then combination edge internal maps weight total variation energy functional model is established, the convex optimization of weight total variation functional model is calculated using alternating direction, and the unitary energy function item of markov random file and binary spatial domain energy function when defining super-pixel scale sky accordingly, obtain the posterior probability segmentation result of image sequence finally by figure cutting single-step iteration reasoning.The present invention can effectively changed complex target be split appearance color is inconsistent and shape from dynamic scene at any time.
Description
Technical field
The present invention relates to a kind of complex target dynamic scene non-formaldehyde finishing sides based on the convex optimization of Multiscale combination feature
Method belongs to image procossing and technical field of computer vision.
Background technology
Segmentation of Image Sequences is the technology of the band of position where semantic foreground target in each frame image of extraction, extensive use
In the computer visions such as target identification, tracking, image understanding and vision guided navigation and robot vision field.In reality, by
In illumination and the variation of environment and the movement of camera itself, cause that actual background is dynamic change and this variation is
Random；In addition, the feature of foreground target also has complexity under normal conditions, even if the appearance color of target is in synchronization
May also be inconsistent, the topological structure of target can also change at any time.Therefore, the automatic of complex target is divided under dynamic scene
Cutting technology becomes difficult point and hot issue that top scientific research institution of various countries competitively chases.
Segmentation of Image Sequences method main can be divided into two classes, that is, have measure of supervision and unsupervised approaches.Early stage
Image partition method needs the foreground target for being artificially labeled in certain frames.These methods very labor intensive, because this needs craft
Divide training set to handle image sequence.There is supervision Segmentation of Image Sequences method to usually require the language in user's selection key frame
Adopted region.It is main to show although these methods obtain segmentation result to a certain extent, but have some limitations
In the selection of the key frame of segmentation result heavy dependence mark.Unsupervised image division method is without using the image that manually marks
It is trained, but by the relevant range of Computer Automatic Extraction semanteme foreground target.
Recent image sequence non-formaldehyde finishing method has used dynamic background texture model, by comparing its current value
Determine whether a pixel belongs to background with past value, and the value for propagating a background pixel enters background extracting mould
Type.Gauss hybrid models are handled to estimate background distributions using Dirichlet, and are used them as to scene changes
A continuous newer model learning process input.This method is to be based on a typical hypotheses, i.e. background becomes
That changes is very steady and slow, and which limits image segmentation of this kind of algorithm under dynamic scene.
Fast target partition algorithm attempts the dynamic profile model of structure target, and assumes that background changes with time and be
Smooth.One advantage of this method is that it is possible to handle the Deja Vu in marking refinement stage on image block, so
And the only point on moving boundaries in target is initialized, a large amount of wrong report seed point may be will produce, especially in dynamic field
Jing Zhong.Subsequent choosing is done from the variation level of granularity with the method permission for building space-time dividing to combine layering clue by setting
It selects.Although this method is effective to processing layering shape clue, still there are one very strong limitations, cause the mistake of scene
Segmentation, can not solve the problems, such as the foreground segmentation task of itself.
Another kind of unsupervised Segmentation of Image Sequences method is ranked up using the structuring study in target assumption region.Make
With shape and action consistency class target it is interregional assume packet sequencings, introduced in the figure of weighted area in frame and interframe
Constraint look for maximum weighted super-pixel.Or cut using figure and realize segmentation with color clue, and region is by being based on one
The classification of the form clue of a multivariate model is sorted.It sorts although target assumption can be conducive to image block level, in reality
Such methods are all since there are the segmentations of the highly redundant in different images region, even if a moderate image sequence is also subject to choosing
It selects the higher storage of target assumption and calculates cost.
The present invention is subsidized by project of national nature science fund project (61461022,61302173) and is studied, and essentially consists in exploration
It is whole to estimate robust fusion algorithm with local feature multi-scale coupling mechanism and multiple dimensioned perceptual error, solve segmentation posteriority with
Really be distributed inconsistent problem, for vision computing platform under dynamic scene the accurate segmentation of complicated foreground target provide it is theoretical with
Method foundation.
Invention content
Based on problem above, the present invention provides a kind of dynamic scene non-formaldehyde finishings based on the convex optimization of Analysis On Multi-scale Features
Method, solves that appearance color is inconsistent and the topological complex object changed at any time of shape, when target be in due to
External environment change and camera motion caused by dynamic background when, other non-formaldehyde finishing technologies be also all difficult to foreground target into
The problem of row is effectively divided.
The technical scheme is that：A kind of complex target dynamic scene based on the convex optimization of Multiscale combination feature is without prison
Dividing method is superintended and directed, wavelet field high yardstick target signature is extracted first, and calculate movement and spatial domain skirt response, by wavelet field, sky
Domain and time domain edge Fusion Features obtain Multiscale combination probability skirt response, and it is total then to establish combination edge internal maps weight
Variation energy functional model calculates the convex optimization of weight total variation energy functional model using alternating direction, and defines super-pixel ruler
The unitary energy function item and binary spatial domain energy function of markov random file when degree is empty, finally by figure cutting single-step iteration
Reasoning obtains the posterior probability segmentation result of image sequence；
The specific step of the complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature
It is rapid as follows：
Step1, wavelet field high yardstick objective contour feature extraction；In image sequenceThe image I at each moment ttIn,
Maximal margin response of the foreground wavelet field high yardstick objective contour at each location of pixels (a, b) is verified first, and T is video
Totalframes；
In the step Step1, the maximal margin response definition at each pixel of wavelet field high yardstick objective contour is：
Bs t(a, b)=max (Hs t(a,b),Vs t(a,b),Ds t(a,b))
Wherein, Hs t、Vs tAnd Ds tIndicate image after wavelet transformation in the horizontal, vertical of the positions scale s (a, b) respectively
With it is diagonally opposed on wavelet coefficient.
Step2, the two images I for calculating adjacent momenttAnd It+1Variation light stream vector fieldPass through gradient
Operator calculates the gradient of variation light stream vector field amplitudeLinear normalization is carried out to optical flow gradient to handle to obtain
Movement edge
The structuring random forest F that Step3, foundation are made of N number of tree tree maximizes meter by the information gain of standard
The bifurcated function on each tree is calculated, N is the number set in random forest；By image ItInput structure random forest F carries out non-
Linear class prediction F (It), according to the prediction case p (I of each tree treet|Tree), statistical framework random forest is non-thread
Property prediction resultIt is normalized to obtain the spatial domain skirt response F of each pixel of imaget
(a,b)；
Step4, the response of high yardstick wavelet coefficient maximal margin is projected into pixel dimension, and linear combination fine dimension
Movement and spatial domain edge feature, fusion obtain Multiscale combination probability skirt response：
Wherein Bs t(a, b) is the maximal margin response at each pixel of wavelet field high yardstick objective contour；
Step5, by image sequenceOver-segmentation constitutes super-pixel set G, connection space neighborhood (a, b) ∈ εsAnd the time
Neighborhood (a, b, t) ∈ εt,t+1, markov random file when to build sky；Wherein εsConnect between all adjacent pixels of spatial domain
The set of line, εt,t+1The set of line between time-domain t moment and all adjacent pixels of t+1 moment two continuous frames images；
Step6, the unitary energy letter that markov random file is determined according to the convex optimum results of combination edge internal maps
Several Ut, the class label l of a super-pixel i ∈ G is judged by the display model of all picture framesiBe foreground or background can
It can property；
In the step Step6, markov random file is determined according to the convex optimum results of combination edge internal maps
Unitary energy function item UtThe step of it is as follows：
Step6.1, the projection M according to the internal maps seed point at combined probability edge on a fine scale, establish image
Foreground target in sequence maps weight total variation energy functional model：
Wherein first item is weight total variation item, and ▽ u are the gradient of alternately variable u；Section 2 is segment smoothing item, Mv-I
Indicate that seed point mapping matrix M subtracts each other with gray value of image again with the dot product for replacing variable v；Relative importance between the two is logical
Parameter ρ is crossed to balance；
Step6.2, the weight total variation segmentation problem of Pixel-level fine dimension is converted to constrained minimization problem：U*=
argminu,vH (u, v), constraints are u ∈ [0,1], v ∈ [0,1]；
Apply linear restriction with internal seeds point mapping matrix M between Step6.3, two alternating variables u and v, by multiplying
Sub- alternating direction convex optimized algorithm, iteration, which updates to obtain, successively minimizes variable u*；
Step6.4, according to convex optimum results u*, training gauss hybrid models foreground parameter ΘFWith context parameter ΘB, obtain
The unitary energy function item U of Markov random field model after convex optimization is smootht=Π (li|u*)；Π is to be similar to outside super-pixel
So；
Step6.5, the posterior probability for calculating current super-pixel RGB color vector, it is five to define super-pixel shape likelihood Π
The negative log-likelihood function of the color character of ingredient gauss hybrid models：
Wherein Pr is probability density function, ciIndicate super-pixel index, the class label l of pixel ii=0 or li=1,
Value is determined according to convex optimum results u*, and negative log-likelihood function is calculated further according to the gauss hybrid models of color clue
Πt(li|u*)。
Step7, markov random file binary spatial domain energy function V and binary time domain energy function W are determined, calculated separately
In spatial neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1Two super-pixel at both ends take class label respectively on line
li∈ L and ljThe possibility of ∈ L, L are the set of all class labels；
In the step Step7, markov random file binary spatial domain energy function V and binary time domain energy function are determined
The step of W, is as follows：
The consistency that Step7.1, binary item encode is embodied in room and time clue, is adjusted with the comparison of a standard
Function defines binary spatial domain energy function V：
Wherein dist indicates two super-pixel ci,cjGeometric centerWithBetween Euclidean distance；Πt(li)
It indicates to bear log-likelihood function Πt(li|U* abbreviation), σ representation space weight factors；
Step7.2, temporal target action being consistent property on image sequence is made by binary smooth potential function W,
Binary time domain energy function W is encoded using overlapping and appearance information：
Wherein β indicates the time weighting factor；
The specific calculation of Step7.3, percentage φ of two super-pixel with same pixel contacted by light stream
For：
Wherein # indicates the number of pixel.
Random field posterior probability classification approximate reasoning when Step8, super-pixel scale sky.By image sequenceOver-segmentation
Constitute super-pixel set G, connection space neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1.Define super-pixel Markov
Random field energy function E (L) solves foreground-background class label optimization class result L；
Step9, likelihood distribution of being classified by the posterior probability of energy function E (L) described in figure syncopation single-step iteration reasoning,
Output makes the label L that energy function minimizes for one*
L*=argminLE(L)
Export L*Obtain the final segmentation result of dynamic scene image sequence.
The beneficial effects of the invention are as follows：
1) present invention is merged using multi-scale transform domain, spatial domain and time-domain clue, obtains a kind of completely new target wheel
The combined probability edge extracting method of wide feature can effectively extract foreground in dynamic background image sequence on multiple scales
The maximal margin of objective contour responds.
2) Markov established using the convex optimization of combination edge internal maps total variation that the method for the invention proposes
Random field energy function, can effectively appearance color is inconsistent and shape at any time changed complex target from dynamic
It is split in scene.
Description of the drawings
Fig. 1 is the flow diagram of the present invention；
Fig. 2 is 6 frame complex target dynamic scene image sequence datas of the present invention for test；
Fig. 3 is the present invention at the maximization wavelet coefficient edge that scale 1 detects；
Fig. 4 is the invention detects that the simultaneously movement edge after normalized；
Fig. 5 is detected by structuring random forest and the spatial domain edge after normalized；
Fig. 6 is combination edge proposed by the present invention multiple features fusion result；
Fig. 7 is the foreground internal maps that the present invention is calculated by the combination edge feature proposed；
Fig. 8 is the segmentation result that the present invention obtains in panda image sequence data.
Specific implementation mode
Embodiment 1：As shown in figures 1-8, a kind of complex target dynamic scene based on the convex optimization of Multiscale combination feature without
Supervised segmentation method extracts wavelet field high yardstick target signature, and calculates movement and spatial domain skirt response first, by wavelet field,
Spatially and temporally edge feature merges to obtain Multiscale combination probability skirt response, then establishes combination edge internal maps weight
Total variation energy functional model calculates the convex optimization of weight total variation energy functional model using alternating direction, and defines super-pixel
The unitary energy function item of markov random file and binary spatial domain energy function, change finally by one step of figure cutting when scale sky
The posterior probability segmentation result of image sequence is obtained for reasoning；
The specific step of the complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature
It is rapid as follows：
Step1, wavelet field high yardstick objective contour feature extraction；In image sequenceThe image I at each moment ttIn,
Maximal margin response of the foreground wavelet field high yardstick objective contour at each location of pixels (a, b) is verified first, and T is video
Totalframes；
In the step Step1, the maximal margin response definition at each pixel of wavelet field high yardstick objective contour is：
Bs t(a, b)=max (Hs t(a,b),Vs t(a,b),Ds t(a,b))
Wherein, Hs t、Vs tAnd Ds tIndicate image after wavelet transformation in the horizontal, vertical of the positions scale s (a, b) respectively
With it is diagonally opposed on wavelet coefficient.
Step2, the two images I for calculating adjacent momenttAnd It+1Variation light stream vector fieldPass through gradient
Operator calculates the gradient of variation light stream vector field amplitudeLinear normalization is carried out to optical flow gradient to handle to obtain
Movement edge
The structuring random forest F that Step3, foundation are made of N number of tree tree maximizes meter by the information gain of standard
The bifurcated function on each tree is calculated, N is the number set in random forest；By image ItInput structure random forest F carries out non-
Linear class prediction F (It), according to the prediction case p (I of each tree treet|Tree), statistical framework random forest is non-thread
Property prediction resultIt is normalized to obtain the spatial domain skirt response F of each pixel of imaget
(a,b)；
Step4, the response of high yardstick wavelet coefficient maximal margin is projected into pixel dimension, and linear combination fine dimension
Movement and spatial domain edge feature, fusion obtain Multiscale combination probability skirt response：
Wherein Bs t(a, b) is the maximal margin response at each pixel of wavelet field high yardstick objective contour；
Step5, by image sequenceOver-segmentation constitutes super-pixel set G, connection space neighborhood (a, b) ∈ εsAnd the time
Neighborhood (a, b, t) ∈ εt,t+1, markov random file when to build sky；Wherein εsConnect between all adjacent pixels of spatial domain
The set of line, εt,t+1The set of line between time-domain t moment and all adjacent pixels of t+1 moment two continuous frames images；
Step6, the unitary energy letter that markov random file is determined according to the convex optimum results of combination edge internal maps
Several Ut, the class label l of a super-pixel i ∈ G is judged by the display model of all picture framesiBe foreground or background can
It can property；
In the step Step6, markov random file is determined according to the convex optimum results of combination edge internal maps
Unitary energy function item UtThe step of it is as follows：
Step6.1, the projection M according to the internal maps seed point at combined probability edge on a fine scale, establish image
Foreground target in sequence maps weight total variation energy functional model：
Wherein first item is weight total variation item, and ▽ u are the gradient of alternately variable u；Section 2 is segment smoothing item, Mv-I
Indicate that seed point mapping matrix M subtracts each other with gray value of image again with the dot product for replacing variable v；Relative importance between the two is logical
Parameter ρ is crossed to balance；
Step6.2, the weight total variation segmentation problem of Pixel-level fine dimension is converted to constrained minimization problem：U*=
argminu,vH (u, v), constraints are u ∈ [0,1], v ∈ [0,1]；
Apply linear restriction with internal seeds point mapping matrix M between Step6.3, two alternating variables u and v, by multiplying
Sub- alternating direction convex optimized algorithm, iteration, which updates to obtain, successively minimizes variable u*；
Step6.4, according to convex optimum results u*, training gauss hybrid models foreground parameter ΘFWith context parameter ΘB, obtain
The unitary energy function item U of Markov random field model after convex optimization is smootht=Π (li|u*)；Π is to be similar to outside super-pixel
So；
Step6.5, the posterior probability for calculating current super-pixel RGB color vector, it is five to define super-pixel shape likelihood Π
The negative log-likelihood function of the color character of ingredient gauss hybrid models：
Wherein Pr is probability density function, ciIndicate super-pixel index, the class label l of pixel ii=0 or li=1,
Value is determined according to convex optimum results u*, and negative log-likelihood function is calculated further according to the gauss hybrid models of color clue
Πt(li|u*)。
Step7, markov random file binary spatial domain energy function V and binary time domain energy function W are determined, calculated separately
In spatial neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1Two super-pixel at both ends take class label respectively on line
li∈ L and ljThe possibility of ∈ L, L are the set of all class labels；
In the step Step7, markov random file binary spatial domain energy function V and binary time domain energy function are determined
The step of W, is as follows：
The consistency that Step7.1, binary item encode is embodied in room and time clue, is adjusted with the comparison of a standard
Function defines binary spatial domain energy function V：
Wherein dist indicates two super-pixel ci,cjGeometric centerWithBetween Euclidean distance；Πt(li)
It indicates to bear log-likelihood function Πt(li|U* abbreviation), σ representation space weight factors；
Step7.2, temporal target action being consistent property on image sequence is made by binary smooth potential function W,
Binary time domain energy function W is encoded using overlapping and appearance information：
Wherein β indicates the time weighting factor；
The specific calculation of Step7.3, percentage φ of two super-pixel with same pixel contacted by light stream
For：
Wherein # indicates the number of pixel.
Random field posterior probability classification approximate reasoning when Step8, super-pixel scale sky.By image sequenceOver-segmentation
Constitute super-pixel set G, connection space neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1.Define super-pixel Markov
Random field energy function E (L) solves foreground-background class label optimization class result L；
Step9, likelihood distribution of being classified by the posterior probability of energy function E (L) described in figure syncopation single-step iteration reasoning,
Output makes the label L that energy function minimizes for one*
L*=argminLE(L)
Export L*Obtain the final segmentation result of dynamic scene image sequence.
Embodiment 2：As shown in figures 1-8, a kind of complex target dynamic scene based on the convex optimization of Multiscale combination feature without
Supervised segmentation method inputs the image sequence data containing dynamic scene information residing for target information and target, calculates multiple dimensioned
Wavelet Edge feature, structuring random forest spatial edge feature and movement gradient feature, before in dynamic image sequence
The combined probability edge of the maximal margin response extraction objective contour feature of scape objective contour；Structure foreground position on a fine scale
The mapping convex Optimized model of seed point total variation is set, the objective contour smoothly extracted in high yardstick of alternately more newly arriving by variable is special
Combined probability edge internal maps seed point is levied, the unitary of markov random file is established according to the target location mapping after smooth
Data capacity function and binary space, time energy function item utilize Ma Er when figure cutting an iteration completion super-pixel grade sky
Section husband random field posterior probability reasoning is to realize the Pixel-level segmentation of dynamic scene image sequence.
It is dynamic that the present invention overcomes the deficiencies of the prior art and provide a kind of complex target based on the convex optimization of Multiscale combination feature
State scene image sequence non-formaldehyde finishing method.The flow diagram of the present invention is as shown in Figure 1.It is described special based on Multiscale combination
The specific implementation step for levying the complex target dynamic scene image sequence non-formaldehyde finishing method of convex optimization is as follows：
Acquired dynamic image sequence data are inputted first, illumination variation, the foreground of scene environment when to obtaining image
It without specifically constraint and is required with image-forming conditions such as background contrasts, the movement of background and the movements of camera itself.Fig. 2 is
The present invention for test 6 frame complex target dynamic scene image sequence datas, wherein foreground target appearance color it is uneven and
And foreground shape changes over time, it is opposing stationary when between foreground and background there are apparent camera motion, when and opposite transport
Dynamic, background more spot is miscellaneous.
(1) then complete high yardstick objective contour feature extraction.In image sequencePer frame image ItIn verify foreground
High yardstick objective contour is responded in the maximal margin of wavelet field.It is defined according to high yardstick object edge proposed by the present invention：
Bs t(a, b)=max (Hs t(a,b),Vs t(a,b),Ds t(a,b))
Wherein, Hs t、Vs tAnd Ds tIndicate image after wavelet transformation in the horizontal, vertical of the positions scale s (a, b) respectively
With it is diagonally opposed on wavelet coefficient.
Extraction foreground high yardstick objective contour is responded in the maximal margin of wavelet field.The method of the invention is examined in scale 1
The maximization wavelet coefficient edge measured is as shown in Figure 3.
(2) the two images I of adjacent moment is calculatedtAnd It+1Variation light stream vector fieldPass through gradient operator
Calculate the gradient of variation light stream vector field amplitudeOptical flow gradient progress linear normalization is handled and is moved
EdgeThe method of the invention detects that the movement edge after simultaneously normalized is as shown in Figure 4.
(3) the structuring random forest F being made of N number of tree tree is established, is maximized and is calculated by the information gain of standard
Bifurcated function on each tree, N are the number set in random forest.By image ItInput structure random forest F carries out non-thread
Property class prediction F (It), according to the prediction case p (I of each tree treet|Tree), statistical framework random forest is non-linear
Prediction resultIt is normalized to obtain the spatial domain skirt response F of each pixel of imaget(a,
b).The method of the invention detects that the spatial domain edge after simultaneously normalized is as shown in Figure 5 by structuring random forest.
(4) response of high yardstick wavelet coefficient maximal margin is projected into pixel dimension, and the fortune of linear combination fine dimension
Dynamic and spatial domain edge feature, fusion obtain Multiscale combination probability skirt response：
Wherein Bs t(a, b) is the maximal margin response at each pixel of wavelet field high yardstick objective contour；
The results are shown in Figure 6 for combination edge proposed by the present invention multiple features fusion.
(5) the projection M according to the internal maps seed point at combined probability edge on a fine scale, is established in image sequence
Foreground target map weight total variation energy functional model:
Weight total variation item and segment smoothing item weight parameter ρ values between the two are 1.8.
(6) the weight total variation segmentation problem of Pixel-level fine dimension is converted to constrained minimization problem：
U*=argminu,vH (u, v), constraints are u ∈ [0,1], v ∈ [0,1].
Apply linear restriction with internal seeds point mapping matrix M between (7) two alternating variables u and v, is replaced by multiplier
Direction convex optimized algorithm carries out the proximal end project between alternately variable successively, and iteration, which updates to obtain, minimizes variable u*.This
Invention is described as shown in Figure 7 by combining the foreground internal maps that edge feature is calculated.
(8) according to convex optimum results u*, training gauss hybrid models foreground parameter ΘFWith context parameter ΘB, obtain convex excellent
Change the unitary data capacity function item of smooth rear Markov random field modelsΠ is to be similar to outside super-pixel
So.
(9) posterior probability of current super-pixel RGB color vector is calculated, definition super-pixel shape likelihood is five ingredient Gausses
The negative log-likelihood function of the color character of mixed model：
Wherein Pr is probability density function, ciIndicate super-pixel index, the class label l of pixel ii=0 or li=1,
Value is determined according to convex optimum results u*, and negative log-likelihood function is calculated further according to the gauss hybrid models of color clue
Πt(li|u*)。
(10) consistency of binary item coding is embodied in room and time clue, with Bayesian segmentation system and is defined
Space binary item V, the comparison adjustment item as a standard：
Wherein dist indicates two super-pixel ci,cjGeometric centerWithBetween Euclidean distance, parameter σ takes
Value is set as 2.8.
(11) temporal target action being consistent property on image sequence is made by binary smooth potential function W, used
Overlapping and appearance information carry out scramble time constraint：
Wherein, parameter beta value is set as 1.6.
(12) the percentage φ that there is same pixel by two super-pixel that light stream contacts is calculated：
Wherein # indicates the number of pixel.
(13) random field posterior probability classification approximate reasoning when super-pixel scale sky.By image sequenceOver-segmentation structure
At super-pixel set G, connection space neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1.Define super-pixel Markov
Random field energy function E (L) solves foreground-background class label optimization class result L.
(14) defeated by the classification likelihood distribution of the posterior probability of energy function E (L) described in figure syncopation single-step iteration reasoning
Go out makes the label L that energy function minimizes for one*
L*=argminLE(L)
Export L*Obtain the final segmentation result of dynamic scene image sequence.It is of the present invention to be based on Multiscale combination feature
Segmentation result such as Fig. 8 that the complex target dynamic scene non-formaldehyde finishing method of convex optimization obtains in panda image sequence data
It is shown.As seen from the figure, in due to dynamic background image caused by camera motion and environmental change, appearance color it is uneven and
The foreground panda semantic objects region that shape changes over time can access effective segmentation at all six moment.
The specific implementation mode of the present invention is explained in detail above in conjunction with attached drawing, but the present invention is not limited to above-mentioned
Embodiment within the knowledge of a person skilled in the art can also be before not departing from present inventive concept
Put that various changes can be made.
Claims (4)
1. a kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature, feature exist
In：Wavelet field high yardstick target signature is extracted first, and calculates movement and spatial domain skirt response, by wavelet field, spatially and temporally
Edge feature merges to obtain Multiscale combination probability skirt response, then establishes combination edge internal maps weight total variation energy
Functional model calculates the convex optimization of weight total variation energy functional model, and horse when defining super-pixel scale sky using alternating direction
The unitary energy function item and binary spatial domain energy function of Er Kefu random fields, obtain finally by figure cutting single-step iteration reasoning
The posterior probability segmentation result of image sequence；
The specific steps of the complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature are such as
Under：
Step1, wavelet field high yardstick objective contour feature extraction；In image sequenceThe image I at each moment ttIn, first
Maximal margin response of the foreground wavelet field high yardstick objective contour at each location of pixels (a, b) is verified, T is total frame of video
Number；
Step2, the two images I for calculating adjacent momenttAnd It+1Variation light stream vector fieldPass through gradient operator meter
Calculate the gradient of variation light stream vector field amplitudeLinear normalization is carried out to optical flow gradient to handle to obtain movement side
Edge
The structuring random forest F that Step3, foundation are made of N number of tree tree is maximized by the information gain of standard and is calculated often
Bifurcated function on a tree, N are the number set in random forest；By image ItInput structure random forest F carries out non-linear
Class prediction F (It), according to the prediction case p (I of each tree treet|Tree), statistical framework random forest is non-linear pre-
Survey resultIt is normalized to obtain the spatial domain skirt response F of each pixel of imaget(a,b)；
Step4, the response of high yardstick wavelet coefficient maximal margin is projected into pixel dimension, and the movement of linear combination fine dimension
With spatial domain edge feature, fusion obtains Multiscale combination probability skirt response：
WhereinFor the maximal margin response at each pixel of wavelet field high yardstick objective contour；
Step5, by image sequenceOver-segmentation constitutes super-pixel set G, connection space neighborhood (a, b) ∈ εsWith time neighborhood
(a,b,t)∈εt,t+1, markov random file when to build sky；Wherein εsThe line between all adjacent pixels of spatial domain
Set, εt,t+1The set of line between time-domain t moment and all adjacent pixels of t+1 moment two continuous frames images；
Step6, the unitary energy function item that markov random file is determined according to the convex optimum results of combination edge internal maps
Ut, the class label l of a super-pixel i ∈ G is judged by the display model of all picture framesiIt is the possibility of foreground or background；
Step7, markov random file binary spatial domain energy function V and binary time domain energy function W are determined, calculated separately in sky
Between neighborhood (a, b) ∈ εsWith time neighborhood (a, b, t) ∈ εt,t+1Two super-pixel at both ends take class label l respectively on linei∈L
And ljThe possibility of ∈ L, L are the set of all class labels；
Step8, super-pixel markov random file energy function E (L) is defined to solve foreground-background class label optimization class
As a result L；
Step9, by formula L*=argminLThe posterior probability classification likelihood of energy function E (L) described in E (L) single-step iteration reasoning, it is defeated
Go out L*Obtain final dynamic scene Segmentation of Image Sequences result.
2. the complex target dynamic scene non-formaldehyde finishing according to claim 1 based on the convex optimization of Multiscale combination feature
Method, it is characterised in that：In the step Step1, the maximal margin response at each pixel of wavelet field high yardstick objective contour
It is defined as：
Bs t(a, b)=max (Hs t(a,b),Vs t(a,b),Ds t(a,b))
Wherein, Hs t、Vs tAnd Ds tIndicate image after wavelet transformation in the horizontal, vertical and right of the positions scale s (a, b) respectively
Wavelet coefficient on angular direction.
3. the complex target dynamic scene non-formaldehyde finishing according to claim 1 based on the convex optimization of Multiscale combination feature
Method, it is characterised in that：In the step Step6, Markov is determined according to the convex optimum results of combination edge internal maps
The unitary energy function item U of random fieldtThe step of it is as follows：
Step6.1, the projection M according to the internal maps seed point at combined probability edge on a fine scale, establish image sequence
In foreground target map weight total variation energy functional model：
Wherein first item is weight total variation item,For the gradient of alternately variable u；Section 2 is segment smoothing item, and Mv-I is indicated
Seed point mapping matrix M subtracts each other with gray value of image again with the dot product for replacing variable v；Relative importance between the two passes through ginseng
ρ is counted to balance；
Step6.2, the weight total variation segmentation problem of Pixel-level fine dimension is converted to constrained minimization problem：U*=
argminu,vH (u, v), constraints are u ∈ [0,1], v ∈ [0,1]；
Apply linear restriction with internal seeds point mapping matrix M between Step6.3, two alternating variables u and v, is handed over by multiplier
For direction convex optimized algorithm, iteration, which updates to obtain, successively minimizes variable u*；
Step6.4, according to convex optimum results u*, training gauss hybrid models foreground parameter ΘFWith context parameter ΘB, obtain convex excellent
Change the unitary energy function item U of smooth rear Markov random field modelt=Π (li|u*)；Π is super-pixel shape likelihood；
Step6.5, the posterior probability for calculating current super-pixel RGB color vector, it is five ingredients to define super-pixel shape likelihood Π
The negative log-likelihood function of the color character of gauss hybrid models：
Wherein Pr is probability density function, ciIndicate super-pixel index, the class label l of pixel ii=0 or li=1, value
It is determined according to convex optimum results u*, negative log-likelihood function Π is calculated further according to the gauss hybrid models of color cluet(li|
u*)。
4. the complex target dynamic scene non-formaldehyde finishing according to claim 3 based on the convex optimization of Multiscale combination feature
Method, it is characterised in that：In the step Step7, markov random file binary spatial domain energy function V and binary time domain are determined
The step of energy function W, is as follows：
The consistency that Step7.1, binary item encode is embodied in room and time clue, with the comparison adjustment function of a standard
Define binary spatial domain energy function V：
Wherein dist indicates two super-pixel ci,cjGeometric centerWithBetween Euclidean distance；Πt(li) indicate negative
Log-likelihood function Πt(li|u*) abbreviation, σ representation space weight factors；
Step7.2, temporal target action being consistent property on image sequence is made by binary smooth potential function W, used
It is overlapped and encodes binary time domain energy function W with appearance information：
Wherein β indicates the time weighting factor；
Step7.3, it is by the specific calculation of percentage φ of two super-pixel with same pixel of light stream contact：
Wherein # indicates the number of pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610261547.6A CN105869178B (en) | 2016-04-26 | 2016-04-26 | A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610261547.6A CN105869178B (en) | 2016-04-26 | 2016-04-26 | A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105869178A CN105869178A (en) | 2016-08-17 |
| CN105869178B true CN105869178B (en) | 2018-10-23 |
Family
ID=56629177
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610261547.6A Active CN105869178B (en) | 2016-04-26 | 2016-04-26 | A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105869178B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846362B (en) * | 2016-12-26 | 2020-07-24 | 歌尔科技有限公司 | Target detection tracking method and device |
| JP7115846B2 (en) * | 2016-12-30 | 2022-08-09 | ダッソー システムズ | Generate Segmented Image Using Markov Random Field Optimization |
| CN108875756B (en) * | 2017-05-08 | 2021-01-05 | 深圳荆虹科技有限公司 | Behavior category acquisition method and device for video |
| CN107918759A (en) * | 2017-10-09 | 2018-04-17 | 大圣科技股份有限公司 | Automatic segmentation recognition method, electronic equipment and the storage medium of indoor object |
| CN108647649B (en) * | 2018-05-14 | 2021-10-01 | 中国科学技术大学 | Method for detecting abnormal behaviors in video |
| CN109063577B (en) * | 2018-07-05 | 2021-08-31 | 浙江大学 | Satellite image segmentation optimal segmentation scale determination method based on information gain rate |
| CN109165565A (en) * | 2018-08-01 | 2019-01-08 | 西安交通大学 | A kind of video object discovery and dividing method based on Coupled Dynamic Markov Network |
| CN109492530B (en) * | 2018-10-10 | 2022-03-04 | 重庆大学 | Robust visual object tracking method based on depth multi-scale space-time characteristics |
| CN110264482B (en) * | 2019-05-10 | 2022-09-09 | 河南科技大学 | Active contour segmentation method based on transformation matrix factorization of noose set |
| CN110232663B (en) * | 2019-05-16 | 2021-04-30 | 福建自贸试验区厦门片区Manteia数据科技有限公司 | Method and device for correcting automatic sketching model of organs at risk |
| CN110827318A (en) * | 2019-10-18 | 2020-02-21 | 天津大学 | Target tracking method based on fusion of multilayer semantic features and multi-response graph |
| CN111127479A (en) * | 2019-12-17 | 2020-05-08 | 昆明理工大学 | Level set image segmentation method based on curve area |
| CN111340852B (en) * | 2020-03-10 | 2022-09-27 | 南昌航空大学 | Image sequence optical flow calculation method based on optimized semantic segmentation |
| CN111950517A (en) * | 2020-08-26 | 2020-11-17 | 司马大大（北京）智能系统有限公司 | Target detection method, model training method, electronic device and storage medium |
| CN112164092B (en) * | 2020-10-13 | 2022-09-27 | 南昌航空大学 | Generalized Markov dense optical flow determination method and system |
| CN113420179B (en) * | 2021-06-24 | 2022-03-22 | 杭州电子科技大学 | Semantic reconstruction video description method based on time sequence Gaussian mixture hole convolution |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540047A (en) * | 2009-04-30 | 2009-09-23 | 西安电子科技大学 | Texture image segmentation method based on independent Gaussian hybrid model |
-
2016
- 2016-04-26 CN CN201610261547.6A patent/CN105869178B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540047A (en) * | 2009-04-30 | 2009-09-23 | 西安电子科技大学 | Texture image segmentation method based on independent Gaussian hybrid model |
Non-Patent Citations (3)
| Title |
|---|
| Bag of Textons for Image Segmentation via Soft Clustering and Convex Shift;Zhiding Yu et al.;《2012 IEEE Conference on Computer Vision and Pattern Recognition》;20120616;第781-788页 * |
| 基于多尺度结构张量的多类无监督彩色纹理图像分割方法;杨勇 等;《计算机辅助设计与图形学学报》;20140531;第26卷(第5期);第812-825页 * |
| 高分辨率SAR图像分割与分类方法研究;冯籍澜;《中国博文学位论文全文数据库（信息科技辑）》;20160315(第3期);正文第6-7页，第11页，第42页，第51页，第53页，第57页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105869178A (en) | 2016-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105869178B (en) | A kind of complex target dynamic scene non-formaldehyde finishing method based on the convex optimization of Multiscale combination feature | |
| CN110175566B (en) | Hand posture estimation system and method based on RGBD fusion network | |
| CN106897670B (en) | Express violence sorting identification method based on computer vision | |
| CN108197587B (en) | Method for performing multi-mode face recognition through face depth prediction | |
| CN106997597B (en) | It is a kind of based on have supervision conspicuousness detection method for tracking target | |
| CN106682598B (en) | Multi-pose face feature point detection method based on cascade regression | |
| CN104599275B (en) | The RGB-D scene understanding methods of imparametrization based on probability graph model | |
| CN106909902B (en) | Remote sensing target detection method based on improved hierarchical significant model | |
| CN109800794B (en) | Cross-camera re-identification fusion method and system for appearance similar targets | |
| CN112784736B (en) | Character interaction behavior recognition method based on multi-modal feature fusion | |
| CN109657612B (en) | Quality sorting system based on facial image features and application method thereof | |
| CN105528794A (en) | Moving object detection method based on Gaussian mixture model and superpixel segmentation | |
| CN111882586B (en) | Multi-actor target tracking method oriented to theater environment | |
| CN108021889A (en) | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information | |
| CN107066969A (en) | A kind of face identification method | |
| CN114758288A (en) | Power distribution network engineering safety control detection method and device | |
| CN110176016B (en) | Virtual fitting method based on human body contour segmentation and skeleton recognition | |
| CN103927511A (en) | Image identification method based on difference feature description | |
| CN104102904B (en) | A kind of static gesture identification method | |
| CN105279769A (en) | Hierarchical particle filtering tracking method combined with multiple features | |
| CN103886619A (en) | Multi-scale superpixel-fused target tracking method | |
| CN105574545B (en) | The semantic cutting method of street environment image various visual angles and device | |
| CN106570887A (en) | Adaptive Mean Shift target tracking method based on LBP features | |
| CN110334656A (en) | Multi-source Remote Sensing Images Clean water withdraw method and device based on information source probability weight | |
| CN104751111A (en) | Method and system for recognizing human action in video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: He Zifen Inventor after: Zhang Yinhui Inventor after: Wu Xing Inventor after: Zhang Yunsheng Inventor after: Wang Sen Inventor before: Zhang Yinhui Inventor before: He Zifen Inventor before: Wu Xing Inventor before: Zhang Yunsheng Inventor before: Wang Sen |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |