CN101777180B - Complex background real-time alternating method based on background modeling and energy minimization - Google Patents
Complex background real-time alternating method based on background modeling and energy minimization Download PDFInfo
- Publication number
- CN101777180B CN101777180B CN2009102437337A CN200910243733A CN101777180B CN 101777180 B CN101777180 B CN 101777180B CN 2009102437337 A CN2009102437337 A CN 2009102437337A CN 200910243733 A CN200910243733 A CN 200910243733A CN 101777180 B CN101777180 B CN 101777180B
- Authority
- CN
- China
- Prior art keywords
- background
- model
- value
- target
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 34
- 230000004927 fusion Effects 0.000 claims abstract description 21
- 230000000694 effects Effects 0.000 claims abstract description 11
- 238000012805 post-processing Methods 0.000 claims abstract description 6
- 230000011218 segmentation Effects 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000013461 design Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000012986 modification Methods 0.000 claims description 3
- 230000004048 modification Effects 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 230000036961 partial effect Effects 0.000 claims description 3
- 230000003628 erosive effect Effects 0.000 claims description 2
- 210000003746 feather Anatomy 0.000 claims description 2
- 238000002156 mixing Methods 0.000 claims description 2
- 238000012847 principal component analysis method Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 238000012821 model calculation Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 abstract description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011049 filling Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000013341 scale-up Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a complex background real-time alternating method based on background modeling and energy minimization, comprising the steps of: firstly, segmenting a current frame image into a foreground pixel set, a background pixel set and an unknown label pixel set by using a provided fusion background model; secondly, constructing a target energy function according to the color and contrast information under a dynamic diagram cutting frame, designing a data item of the fusion background model based on time continuity information and a contrast smooth item based on a local two-value mode, solving two-value labels of all pixels by using a diagram cutting algorithm minimization energy function; and seamlessly fusing segmented foreground targets into a virtual background by adopting postprocessing methods of boundary smooth, alpha value estimation and the like. Experimental results show that the method can better segment the targets in the complex background in real time and really synthesize the segmented targets into the virtual background. The invention has the characteristics of automatically segmenting the complex background in real time and replacing the segmented complex background into the virtual background to obtain a virtual effect sequence diagram with high quality.
Description
Technical field
The present invention relates to Flame Image Process and computer vision field, refer in particular to and a kind ofly in video flowing, complex background is carried out accurately cutting apart in real time the method with replacement.
Background technology
The video object dividing method can be divided into Interactive Segmentation and self-action partitioning algorithm on the whole at present.Interactive video target partitioning algorithm is meant in cutting procedure, is improved the effect of cutting apart the algorithm intervention by customer interaction information.Though the Interactive Segmentation technology has obtained very big progress, remains in some problems.As loaded down with trivial details alternately, workload is big, and exists certain craftsmenship alternately, and mutual quality directly affects final segmentation effect; Automatically partitioning algorithm need not manual intervention in cutting procedure, automatically accomplishes cutting apart of specific objective extraction.
If according to background complexity classification, automatic target cut apart the target that can be divided under the color background cut apart with complex background under target cut apart two kinds.Video Segmentation under the color background is meant that under color background, light homogeneous condition, carrying out cutting apart in real time with background of target replaces; Background replacement technology maturation and commercialization under the color background scratched diagram technology like blue screen and has been applied in virtual studio, the film making at present.But how color background realizes cutting apart automatically and detection of target in scene and the performer's dressing many restrictions being arranged under complicated natural background, this problem receives the increasing concern of Chinese scholars.
Video Segmentation system cost based on monocular under the complex background is cheap, and system constructing is convenient, has higher utility and application prospects in fields such as man-machine interaction, video editing, intelligent monitoring and digital entertainments.Though people are doing a large amount of effort aspect the Video Segmentation of monocular; Also obtained certain achievement; But general background replacement method can't be cut apart background in real time, exactly under complex background, the synthetic automatically higher virtual background video of accuracy.
Summary of the invention
In order to solve the in real time accurate background replacement problem under the complex background single camera; The purpose of this invention is to provide a kind of complex background real-time alternating method based on background modeling and energy minimization; Can be partitioned into prospect and background real-time and accurately; And will cut apart the real background that obtains and replace to virtual background picture, have the video effect of virtual background with generation.
To achieve these goals, the complex background real-time alternating method based on background modeling and energy minimization provided by the invention at first adopts the fusion background model that proposes that each two field picture three in the video is divided into background area, foreground area and zone of ignorance; Next utilizes color and spatial contrast degree feature construction target energy function, adopts the method for dynamically scheming to cut to ask for the target energy extreme value of a function, thereby each two field picture is divided into prospect and background in real time; Adopt edge smoothing method and post-processing approach such as my method transparency (alpha) value estimation at last, make the foreground target that is partitioned into seamlessly be fused in the virtual background based on frequency filtering; Specifically comprise the steps:
Step 1: system at first obtains video stream data; The background of the image of study video stream data; Set up this image based on the feature background model of brightness with based on the Gauss model of colourity; Two models are carried out linearity merge, obtain light is had antijamming capability, possesses the fusion background model of model modification characteristic, Pixel-level; According to merging background model, every two field picture pixel is divided into prospect, background and three types of labels of UNKNOWN TYPE;
Step 2: in the video present frame, system has confirmed that partial pixel possesses clear and definite foreground/background type affiliation according to step 1; Pixel for which UNKNOWN TYPE; Utilize to merge the time color characteristic of background model and based on local binary pattern (local binary pattern; Lbp) spatial contrast degree characteristic; Structure is based on the target energy function of space time information, to confirm the prospect or the background type of Unknown Label pixel through step 3;
Step 3:, obtain the segmentation tag of binaryzation according to cutting the extreme value that (graph cut) algorithm is asked for the target energy function of step 2 foundation based on max-flow and the theoretical dynamic figure of minimal cut; Therefore, all pixels of present frame have possessed clear and definite background or prospect label;
Step 4: the foreground target segmentation result is carried out aftertreatment: utilize area identification and morphologic filtering post-processing algorithm to remove the noise and the slot of segmentation result; And design obtains final background segment result based on the level and smooth prospect of the frequency filtering border of Fourier transform;
Step 5:, find the solution my method transparence value of prospect border to realize the feather effect on foreground target border for the foreground target that splits seamlessly is fused in the virtual background; And will cut apart the real background that obtains according to my method transparence value and replace to virtual background picture, have the video effect of virtual background with generation.
Wherein, step 1 structure fusion background model is described below:
Step 11: at first learn the video sequence that some frames are only had powerful connections;
Step 12: utilize Principal Component Analysis Method to set up feature background model, and pass through the method reconstructed background image of background compensation, to realize reaching the renewal of feature background model based on pixel intensity information;
Step 13: for each pixel is set up the single Gaussian Background model based on chrominance information, and the parameter of real-time update feature background model;
Step 14: feature background model and single Gaussian Background model to setting up carry out the linearity fusion, and according to pixel intensity information dynamic calculation fusion coefficients, obtain to merge background model;
Step 15: utilize above-mentioned fusion background model, current frame image is divided into prospect, background and three types of set of pixels of Unknown Label.
Wherein, step 2 energy object construction of function is described below:
Step 21: the classification results of the pixel that obtains according to step 15, the form that embodies of each item of target setting function;
Step 22: utilize the spatial contrast degree information of local two value models (lbp) calculating neighbor, to characterize the flatness between adjacent two pixels;
Step 23: according to fusion background model and local two-value model space contrast information, set the data item and level and smooth sex factor of gibbs energy (Gibbs) energy function, promptly obtain the energy object function based on time continuity information.
Wherein, the two-value segmentation procedure of all pixels of present frame is following:
Step 31:, cut algorithm with execution graph for each two field picture structure weighted graph;
Step 32: utilize the video relevance of frame segmentation result up and down,, upgrade the weight on limit in the weighted graph from the data item and level and smooth item that the previous frame segmentation result utilizes step 23 to ask for;
Step 33: according to dynamically scheming to cut the minimal cut set that algorithm is asked for current weighted graph, with the extreme value of definite target energy function and the two-value label of unknown pixel.
Wherein, step 4 pair foreground target segmentation result aftertreatment is undertaken by following process:
Step 41: utilize area identification and morphology closed operation algorithm, small size interference region among the foreground segmentation result that removal step 33 obtains is filled narrow being interrupted and little hole;
Step 42: the border of trying to achieve foreground target according to edge following algorithm;
Step 43: the boundary curve along foreground object obtains point sequence with interval sampling, and the Fourier that obtains the border through Fourier transform is described son;
Step 44: the cutoff frequency of design low-pass filter, the foreground target border that obtains with level and smooth step 42; Last inverse-Fourier transform obtains the frontier point coordinate after level and smooth, with the final border of the foreground target confirming to split.
Wherein, Step 5 is my method transparence value of the prospect boundary strip that obtains of calculation procedure 44 at first; Promptly utilize the expansion erosion algorithm to obtain the boundary strip of certain bandwidth,, and normalize in 0 to 1 scope according to my method transparence value of the level and smooth band of computation bound as a result of Gauss; Set the blending ratio of preceding background according to trying to achieve my method transparence value again, confirm each color of pixel value in the synthesising picture, the final video effect that generates with virtual background.
The present invention has following advantage compared with prior art:
1. according to color and contrast information structure target energy function, designed based on the fusion background model of time continuity information and the level and smooth item of contrast of local binary pattern (lbp).Like this, we make full use of color, contrast, space and time continuous property information, set up more robust, simple and direct target energy function.
2. cut the two-value label that algorithm minimization energy function is asked for all pixels through dynamic figure, improved the efficient of cutting apart of video current frame image; And the level and smooth prospect of design low-pass filter border, adopt area identification and my method transparency (alpha) value estimation to wait post-processing approach to make the foreground target that is partitioned into seamlessly be fused in the virtual background.
Conclusion is got up, and system building of the present invention is simple, algorithm possess full automation, in real time, result's characteristic accurately.
Description of drawings
Fig. 1 is the process flow diagram of complex background real-time alternating method of the present invention.
Fig. 2 is the real-time replacement process synoptic diagram of complex background of the present invention.
Fig. 3 (a) and Fig. 3 (b) they are the real-time segmentation result of complex background of the present invention, wherein:
The original video frame of Fig. 3 (a) input;
Fig. 3 (b) complex background segmentation result.
Fig. 4 (a)-Fig. 4 (d) is complex background replacement result of the present invention.
From the original video of Fig. 4 (a), Fig. 4 (c) row, be partitioned into prospect, and in Fig. 4 (b) Fig. 4 (d) row virtual scene that is added to respectively.
Embodiment
To combine accompanying drawing that the present invention is specified below, and it should be noted that described embodiment only is intended to be convenient to understanding of the present invention, and it is not played any qualification effect.
Fig. 1 is the process flow diagram of complex background real-time alternating method.According to flow sequence, the practical implementation process of each step of the inventive method is following:
1. read in video flowing
System at first obtains video stream data.Video data obtains the source mode can be divided into two kinds: the one, and the image sequence that camera collects in real time.The 2nd, the video file of gathering in advance.
2. background study, structure merges background model
(21) study N frame image sequence; General study 40-80 frame, the time of about 2s;
(22) set up characteristic background:, set up proper subspace based on monochrome information through study N frame background image sequence based on monochrome information.Be that dimension is that every two field picture of m * n is counted as a kind of random vector, obtain its quadrature K-L substrate with Karhunen-Loeve transformation, the corresponding wherein substrate of big eigenwert has constituted proper subspace W, has characterized the characteristic of static background.Rebuild present frame p background image B through proper subspace
p, calculate a new two field picture I
pDifference D with background image
p(E)=| I
p-B
p|, this difference D
pRepresented in a new two field picture that (E) pixel that calculates through feature background model belongs to the cost value of background;
(23) set up single Gaussian Background model:, set up Gauss model D based on chrominance information through study N frame background image sequence based on chrominance information
p(G);
(24) set up the background model that merges: characteristic background and Gaussian Background model to above-mentioned foundation carry out linearity fusion, D
p(E)=λ D
p(E)+(1-λ) D
p(G).Here, D
p(E) be the background model that merges, ask method shown in (22); D
p(G) be the Gauss model that (23) are set up; λ is an integration percentage, confirms according to the pixel intensity difference.
(25) utilize above-mentioned fusion background model, current frame image is divided into prospect, background and three types of set of pixels of Unknown Label:
Here, X
pThe label value of remarked pixel p.B representes background, and F representes prospect, and U representes to treat further to confirm as the UNKNOWN TYPE of foreground/background.D
p(E) be the background model that merges.T
b, T
fFor judging that pixel p belongs to the discrimination threshold of background, prospect.
3. the structure energy function is set data item and level and smooth item in the energy function
(31) calculate under the framework at markov-maximum a posteriori probability (MRF-MAP), adopt the objective function of gibbs Gibbs distribution energy function
for making up.Here D
i(x
i) be the data item of objective function, be expressed as summit i distributing labels x
iCost value; V
I, j(x
i, x
j) be the level and smooth item of objective function, be expressed as adjacent vertex i, j distributing labels x
i, x
jCost value; γ representes V
I, j(x
i, x
j) with respect to D
i(x
i) weight coefficient.Like this, the segmentation problem based on MRF just is converted into the pairing label problem of posteriority energy function Gibbs minimum value of asking;
(32) set the data item D in the energy function according to the feature background model that merges
i(x
i);
(33) according to neighbor i in local two value model lbp (local binary pattern) the calculating energy function, level and smooth the V of j
I, j(x
i, x
j).Do not relate to exponent arithmetic based on the lbp contrast value, calculate simply, and have the insensitive characteristics of illumination variation;
(34) set the scale-up factor γ in the energy function according to level and smooth importance with data item.
4. ask the extreme value of energy function
(41) we employing figure cuts the minimal value minE (X) that algorithm is found the solution target energy function (seeing shown in (31)).So at first construct weighted graph, each point among the figure corresponding each pixel of present frame, according to data item in the energy function and level and smooth the weight of setting each limit of weighted graph;
(42) method that adopts dynamic figure to cut accelerates that energy is minimizing to be asked for.Promptly at first to this fact of consecutive frame continuity in the video; Utilized the weight on the residual graph information updating present frame weighted graph limit of previous frame fully; Reduce the structure again of search tree in the present frame augmenting path algorithm, reduced the minimal cut path searching time; Find out the minimal cut in all cut sets according to max-flow-minimal cut algorithm again, promptly corresponding minimal value of trying to achieve energy function E (X) is for the partial pixel of UNKNOWN TYPE (U) has been confirmed two-value label (foreground/background).Like this, we have obtained the binary segmentation result of all pixels of present frame;
Adopt after dynamically figure cuts algorithm, for the picture of 320 * 240 sizes, every frame is cut algorithm required about 120 milliseconds (ms) by original execution graph and reduce to about 65ms sliced time, and splitting speed has improved one times nearly.Foreground target to be split in the present frame is more little, and it is remarkable more that splitting speed improves.
5. aftertreatment such as edge smoothing, area identification
(51) utilize the area identification algorithm that the result of cutting apart is carried out region labeling, remove the less isolated area of area;
(52) utilize the morphology closed operation to carry out the filling of said minuscule hole and fracture;
(53) according to edge following algorithm, try to achieve the prospect frontier point, list its plural expression-form;
(54) frontier point is carried out Fourier transform, set the cutoff frequency r of low-pass filter, to confirm the smoothness at edge;
(55) the frontier point planimetric coordinates after inverse-Fourier transform is tried to achieve smoothly.
6. ask for my method transparency (alpha) value on the prospect border that splits, the synthetic virtual image of replacement background
(61) the prospect border of confirming is corroded the expansive working of carrying out corresponding number of times afterwards, the long and narrow boundary strip that obtains having certain bandwidth several times.Bandwidth is relevant with the structural element size of morphological operation;
(62) it is level and smooth in boundary strip, to carry out Gauss; Like this in boundary strip interior pixel value by 0 and 255 two original value; Transfer the some discrete values between 0 to 255 to; And these discrete values are carried out the normalization in 0 to 1 scope, just obtained my method transparency (alpha) value of each pixel in the boundary strip.My method transparency (alpha) value of each background dot of the boundary strip outside is 0, and my method transparency (alpha) value of inboard each foreground point of boundary strip is 1;
(63) select virtual background or virtual video sequence, the background of cutting apart is replaced with virtual background image, virtual background is mixed with the foreground object of cutting apart, synthetic video sequence with virtual background according to my method transparency (alpha) value.
Fig. 2 is a background replacement algorithmic procedure synoptic diagram.System obtains current frame image information through reading in the image sequence that video or camera collection arrive; Utilize the fusion background model calculating pixel that proposes to belong to the probable value of background model, be divided three classes: background, prospect and unknown classification (remaining further to be differentiated to belong to prospect or background) with this each pixel with current frame image; Making up energy function, utilize and dynamically scheme to cut the pixel two-value label that algorithm is confirmed unknown classification, finally is that all pixels of present frame are distributed the two-value label of prospects or background; Ask for my method transparency (alpha) value of prospect boundary strip, cut apart the background that obtains, a synthetic frame virtual image with the virtual background replacement.
Fig. 3 is the test result in one section sequence image.Fig. 3 (a) has provided the original image of certain two frame in the sequence image.Fig. 3 (b) has provided the result of the background segment that the Fusion Model that proposes according to us and dynamic figure blanking method obtain, and wherein background is accurately differentiated, and foreground target is by complete extraction.
Fig. 4 is the background replacement result who has provided in the different sequence images.From the original video of Fig. 4 (a), Fig. 4 (c) row, be partitioned into prospect automatically real-time, and among the Fig. 4 (b) that is added to respectively, Fig. 4 (d) row virtual scene.
The invention provides a kind of complex background replacement algorithm based on background modeling and energy minimization.This algorithm has good real-time performance, is the input video of 320x240 for yardstick, is P42.8G at CPU, in save as on the machine of 512MB and test, algorithm speed can reach 16-18 frame/s, basic requirement of real time; This algorithm has carried out experimental verification in a large amount of sequence images simultaneously, and the result shows that algorithm has higher accuracy.
The above only is the embodiment among the present invention, but scope of the present invention should not described by this and limits.It should be appreciated by those skilled in the art,, all belong to claim of the present invention and come restricted portion in any modification that does not depart from the scope of the present invention or local replacement.
Claims (6)
1. the complex background real-time alternating method based on background modeling and energy minimization is characterized in that, at first adopts the fusion background model that proposes that each two field picture three in the video is divided into background area, foreground area and zone of ignorance; Next utilizes color and spatial contrast degree feature construction target energy function, adopts the method for dynamically scheming to cut to ask for the target energy extreme value of a function, thereby each two field picture is divided into prospect and background in real time; Adopt at last based on the edge smoothing method of frequency filtering and the post-processing approach of my method transparence value estimation, make the foreground target that is partitioned into seamlessly be fused in the virtual background; Specifically comprise the steps:
Step 1: system at first obtains video stream data; The background of the image of study video stream data; Set up this image based on the feature background model of brightness with based on the Gauss model of colourity; Two models are carried out linearity merge, obtain light is had antijamming capability, possesses the fusion background model of model modification characteristic, Pixel-level; According to merging background model, every two field picture pixel is divided into prospect, background and three types of labels of UNKNOWN TYPE;
Step 2: in the video present frame, system has confirmed that partial pixel possesses clear and definite foreground/background type affiliation according to step 1; Pixel for which UNKNOWN TYPE; Utilize to merge the time color characteristic of background model and based on the spatial contrast degree characteristic of local binary pattern; Structure is based on the target energy function of space time information, to confirm the prospect or the background type of Unknown Label pixel through step 3;
Step 3:, obtain the segmentation tag of binaryzation according to cutting the extreme value that algorithm is asked for the target energy function of step 2 foundation based on max-flow and the theoretical dynamic figure of minimal cut; Therefore, all pixels of present frame have possessed clear and definite background or prospect label;
Step 4: the foreground target segmentation result is carried out aftertreatment: utilize area identification and morphologic filtering post-processing algorithm to remove the noise and the slot of segmentation result; And design obtains final background segment result based on the level and smooth prospect of the frequency filtering border of Fourier transform;
Step 5:, find the solution my method transparence value of prospect border to realize the feather effect on foreground target border for the foreground target that splits seamlessly is fused in the virtual background; And will cut apart the real background that obtains according to my method transparence value and replace to virtual background picture, have the video effect of virtual background with generation.
2. a kind of complex background real-time alternating method based on background modeling and energy minimization according to claim 1 is characterized in that, step 1 makes up the fusion background model and is described below:
Step 11: at first learn the video sequence that some frames are only had powerful connections;
Step 12: utilize Principal Component Analysis Method to set up feature background model, and pass through the method reconstructed background image of background compensation, to realize reaching the renewal of feature background model based on pixel intensity information;
Step 13: for each pixel is set up the single Gaussian Background model based on chrominance information, and the parameter of real-time update feature background model;
Step 14: feature background model and single Gaussian Background model to setting up carry out the linearity fusion, and according to pixel intensity information dynamic calculation fusion coefficients, obtain to merge background model;
Step 15: utilize above-mentioned fusion background model, current frame image is divided into prospect, background and three types of set of pixels of Unknown Label.
3. a kind of complex background real-time alternating method based on background modeling and energy minimization according to claim 2 is characterized in that step 2 target energy construction of function is described below:
Step 21: the classification results of the pixel that obtains according to step 15, the form that embodies of each item of target setting energy function;
Step 22: utilize the spatial contrast degree information of neighbor in the local two-value Model Calculation target energy function, to characterize the flatness between adjacent two pixels;
Step 23: according to fusion background model and local two-value model space contrast information, set the data item and level and smooth sex factor of Gibbs energy flow function, promptly obtain the target energy function based on time continuity information.
4. a kind of complex background real-time alternating method based on background modeling and energy minimization according to claim 3 is characterized in that the two-value segmentation procedure of all pixels of step 3 present frame is following:
Step 31:, cut algorithm with execution graph for each two field picture structure weighted graph;
Step 32: utilize the video relevance of frame segmentation result up and down,, upgrade the weight on limit in the weighted graph from the data item and level and smooth item that the previous frame segmentation result utilizes step 23 to ask for;
Step 33: according to dynamically scheming to cut the minimal cut set that algorithm is asked for current weighted graph, with the extreme value of definite target energy function and the two-value label of unknown pixel.
5. a kind of complex background real-time alternating method based on background modeling and energy minimization according to claim 4 is characterized in that, step 4 pair foreground target segmentation result aftertreatment is undertaken by following process:
Step 41: utilize area identification and morphology closed operation algorithm, small size interference region among the foreground segmentation result that removal step 33 obtains is filled narrow being interrupted and little hole;
Step 42: the border of trying to achieve foreground target according to edge following algorithm;
Step 43: the boundary curve along foreground object obtains point sequence with interval sampling, and the Fourier that obtains the border through Fourier transform is described son;
Step 44: the cutoff frequency of design low-pass filter, the foreground target border that obtains with level and smooth step 42; Last inverse-Fourier transform obtains the frontier point coordinate after level and smooth, with the final border of the foreground target confirming to split.
6. a kind of complex background real-time alternating method according to claim 5 based on background modeling and energy minimization; It is characterized in that; Step 5 is my method transparence value of the foreground target boundary strip that obtains of calculation procedure 44 at first; Promptly utilize the expansion erosion algorithm to obtain the boundary strip of certain bandwidth,, and normalize in 0 to 1 scope according to my method transparence value of the level and smooth band of computation bound as a result of Gauss; According to the blending ratio of trying to achieve my method transparence value foregrounding and virtual background, confirm each color of pixel value in the synthesising picture again, final generation has the video effect of virtual background.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102437337A CN101777180B (en) | 2009-12-23 | 2009-12-23 | Complex background real-time alternating method based on background modeling and energy minimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102437337A CN101777180B (en) | 2009-12-23 | 2009-12-23 | Complex background real-time alternating method based on background modeling and energy minimization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101777180A CN101777180A (en) | 2010-07-14 |
| CN101777180B true CN101777180B (en) | 2012-07-04 |
Family
ID=42513635
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102437337A Expired - Fee Related CN101777180B (en) | 2009-12-23 | 2009-12-23 | Complex background real-time alternating method based on background modeling and energy minimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101777180B (en) |
Families Citing this family (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855634B (en) * | 2011-06-28 | 2017-03-22 | 中兴通讯股份有限公司 | Image detection method and image detection device |
| CN102496163B (en) * | 2011-11-03 | 2013-07-17 | 长安大学 | Background reconstruction method based on gray extremum |
| CN103136722B (en) * | 2011-11-25 | 2016-06-29 | 北京大学 | A kind of image partition method based on colour gamut analysis and system |
| CN102663398A (en) * | 2012-03-31 | 2012-09-12 | 上海博康智能信息技术有限公司 | Color image color feature extraction method and device thereof |
| CN102819841B (en) * | 2012-07-30 | 2015-01-28 | 中国科学院自动化研究所 | Global threshold partitioning method for partitioning target image |
| CN103686050A (en) * | 2012-09-18 | 2014-03-26 | 联想(北京)有限公司 | Method and electronic equipment for simulating call scenes, |
| CN103065327A (en) * | 2012-12-31 | 2013-04-24 | 合肥寰景信息技术有限公司 | Gait region partitioning algorithm based on novel space-time fusion |
| CN103500447B (en) * | 2013-09-18 | 2015-03-18 | 中国石油大学(华东) | Video foreground and background partition method based on incremental high-order Boolean energy minimization |
| JP5741660B2 (en) * | 2013-09-18 | 2015-07-01 | カシオ計算機株式会社 | Image processing apparatus, image processing method, and program |
| CN103559690B (en) * | 2013-11-05 | 2017-02-01 | 北京京东尚科信息技术有限公司 | Method for achieving smoothness of image edges |
| CN104933694A (en) * | 2014-03-17 | 2015-09-23 | 华为技术有限公司 | Method and equipment for segmenting foreground and background |
| EP3136717A4 (en) * | 2014-04-22 | 2017-12-13 | Nippon Telegraph and Telephone Corporation | Video display device, video projection device, dynamic illusion presentation device, video generation device, method thereof, data construct, and program |
| CN104902189A (en) * | 2015-06-24 | 2015-09-09 | 小米科技有限责任公司 | Picture processing method and picture processing device |
| CN104967855B (en) * | 2015-06-25 | 2018-01-16 | 华侨大学 | A kind of coding method suitable for monitor video |
| CN106558043B (en) * | 2015-09-29 | 2019-07-23 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of determining fusion coefficients |
| CN105608716B (en) * | 2015-12-21 | 2020-12-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105608699B (en) * | 2015-12-25 | 2019-03-29 | 联想(北京)有限公司 | A kind of image processing method and electronic equipment |
| CN105654513A (en) * | 2015-12-30 | 2016-06-08 | 电子科技大学 | Moving target detection method based on sampling strategy |
| CN106204426A (en) * | 2016-06-30 | 2016-12-07 | 广州华多网络科技有限公司 | A kind of method of video image processing and device |
| CN106254849B (en) * | 2016-08-08 | 2018-11-13 | 深圳迪乐普数码科技有限公司 | A kind of method and terminal that foreground object is locally replaced |
| CN106296683A (en) * | 2016-08-09 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of generation method of virtual screen curtain wall and terminal |
| CN106446749B (en) * | 2016-08-30 | 2019-08-13 | 西安小光子网络科技有限公司 | A kind of shooting of optical label and optical label decode relay working method |
| CN107979754B (en) * | 2016-10-25 | 2019-08-02 | 百度在线网络技术（北京）有限公司 | A kind of test method and device based on camera application |
| CN106846336B (en) * | 2017-02-06 | 2022-07-15 | 腾讯科技（上海）有限公司 | Method and device for extracting foreground image and replacing image background |
| CN108629815B (en) * | 2017-03-15 | 2021-02-26 | 北京京东尚科信息技术有限公司 | Image generation method and device |
| CN110023989B (en) * | 2017-03-29 | 2021-06-01 | 华为技术有限公司 | Sketch image generation method and device |
| CN107025457B (en) * | 2017-03-29 | 2022-03-08 | 腾讯科技（深圳）有限公司 | Image processing method and device |
| CN107194949B (en) * | 2017-05-18 | 2019-09-24 | 华中科技大学 | A kind of interactive video dividing method and system matched based on block and enhance Onecut |
| CN107301655B (en) * | 2017-06-16 | 2020-10-02 | 上海远洲核信软件科技股份有限公司 | Video moving target detection method based on background modeling |
| CN107920213A (en) * | 2017-11-20 | 2018-04-17 | 深圳市堇茹互动娱乐有限公司 | Image synthesizing method, terminal and computer-readable recording medium |
| JP7106282B2 (en) | 2018-01-30 | 2022-07-26 | キヤノン株式会社 | Image processing device, image processing method and program |
| CN108830780B (en) * | 2018-05-09 | 2020-09-01 | 京东数字科技控股有限公司 | Image processing method and device, electronic device and storage medium |
| CN109737961A (en) * | 2018-05-23 | 2019-05-10 | 哈尔滨理工大学 | A kind of robot optimization area Dian Dao paths planning method with probability completeness |
| CN108765278B (en) * | 2018-06-05 | 2023-04-07 | Oppo广东移动通信有限公司 | Image processing method, mobile terminal and computer readable storage medium |
| CN109118507B (en) * | 2018-08-27 | 2019-09-13 | 嵊州市万睿科技有限公司 | Shell bears air pressure real-time alarm system |
| CN111210450B (en) * | 2019-12-25 | 2022-08-09 | 北京东宇宏达科技有限公司 | Method and system for processing infrared image of sea-sky background |
| CN111080748B (en) * | 2019-12-27 | 2023-06-02 | 北京工业大学 | Automatic picture synthesizing system based on Internet |
| CN112070656B (en) * | 2020-08-10 | 2023-08-25 | 上海明略人工智能(集团)有限公司 | Frame data modification method and device |
| CN114078074A (en) * | 2020-08-11 | 2022-02-22 | 北京芯海视界三维科技有限公司 | Image processing device and terminal |
| CN112163589B (en) * | 2020-11-10 | 2022-05-27 | 中国科学院长春光学精密机械与物理研究所 | Image processing method, device, equipment and storage medium |
| CN112634312B (en) * | 2020-12-31 | 2023-02-24 | 上海商汤智能科技有限公司 | Image background processing method and device, electronic equipment and storage medium |
| CN112954398B (en) * | 2021-02-07 | 2023-03-24 | 杭州网易智企科技有限公司 | Encoding method, decoding method, device, storage medium and electronic equipment |
| CN113947523B (en) * | 2021-10-18 | 2022-06-14 | 杭州研极微电子有限公司 | Method and device for replacing background image |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101154265A (en) * | 2006-09-29 | 2008-04-02 | 中国科学院自动化研究所 | Method for recognizing iris with matched characteristic and graph based on partial bianry mode |
-
2009
- 2009-12-23 CN CN2009102437337A patent/CN101777180B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101154265A (en) * | 2006-09-29 | 2008-04-02 | 中国科学院自动化研究所 | Method for recognizing iris with matched characteristic and graph based on partial bianry mode |
Non-Patent Citations (4)
| Title |
|---|
| D.M.Greig et al.Exact Maximum A Posteriori Estimation for Binary Images.《Journal of the Royal Statistical Society.Series B(Methodological)》.1989,第51卷(第2期),271-279. * |
| JP特开2007-140805A 2007.06.07 |
| YURI BOYKOV et al.Graph Cuts and Efficient N-D Image Segmentation.《International Journal of Computer Vision》.2006,109-131. * |
| 樊亚春 等.利用图割实现背景自动切换.《小型微型计算机系统》.2009,第30卷(第29期),1834-1837. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101777180A (en) | 2010-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101777180B (en) | Complex background real-time alternating method based on background modeling and energy minimization | |
| CN102567727B (en) | Method and device for replacing background target | |
| Berjón et al. | Real-time nonparametric background subtraction with tracking-based foreground update | |
| CN109558806B (en) | Method for detecting high-resolution remote sensing image change | |
| CN106664417A (en) | Content adaptive background-foreground segmentation for video coding | |
| CN104408429A (en) | Method and device for extracting representative frame of video | |
| CN102542571B (en) | Moving target detecting method and device | |
| CN108369651B (en) | Method, system and non-transitory computer-readable storage medium for extracting sky area | |
| CN106548160A (en) | A kind of face smile detection method | |
| CN103198479B (en) | Based on the SAR image segmentation method of semantic information classification | |
| CN104166983A (en) | Motion object real time extraction method of Vibe improvement algorithm based on combination of graph cut | |
| CN110084782B (en) | Full-reference image quality evaluation method based on image significance detection | |
| CN109712247B (en) | Live-action training system based on mixed reality technology | |
| CN101510304B (en) | Method, device and pick-up head for dividing and obtaining foreground image | |
| CN108596923A (en) | Acquisition methods, device and the electronic equipment of three-dimensional data | |
| CN102034247A (en) | Motion capture method for binocular vision image based on background modeling | |
| CN102073866B (en) | Video super resolution method by utilizing space-time Markov random field model | |
| CN102609950A (en) | Two-dimensional video depth map generation process | |
| CN106447656B (en) | Rendering flaw image detecting method based on image recognition | |
| CN105046670A (en) | Image rain removal method and system | |
| US20230343017A1 (en) | Virtual viewport generation method and apparatus, rendering and decoding methods and apparatuses, device and storage medium | |
| CN102486827B (en) | Extraction method of foreground object in complex background environment and apparatus thereof | |
| CN110322479B (en) | Dual-core KCF target tracking method based on space-time significance | |
| CN109215047B (en) | Moving target detection method and device based on deep sea video | |
| Malladi et al. | Superpixels using morphology for rock image segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120704 Termination date: 20151223 |
|
| EXPY | Termination of patent right or utility model |