CN102005060B - Method and device for automatically removing selected images in pictures - Google Patents

Method and device for automatically removing selected images in pictures Download PDF

Info

Publication number
CN102005060B
CN102005060B CN2010105774848A CN201010577484A CN102005060B CN 102005060 B CN102005060 B CN 102005060B CN 2010105774848 A CN2010105774848 A CN 2010105774848A CN 201010577484 A CN201010577484 A CN 201010577484A CN 102005060 B CN102005060 B CN 102005060B
Authority
CN
China
Prior art keywords
image
module
border
boundary
original graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN2010105774848A
Other languages
Chinese (zh)
Other versions
CN102005060A (en
Inventor
徐小明
殷铭
刘玉亭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SHANGHAI JIETU SOFTWARE TECHN CO Ltd
Original Assignee
SHANGHAI JIETU SOFTWARE TECHN CO Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SHANGHAI JIETU SOFTWARE TECHN CO Ltd filed Critical SHANGHAI JIETU SOFTWARE TECHN CO Ltd
Priority to CN2010105774848A priority Critical patent/CN102005060B/en
Priority to PCT/CN2011/071073 priority patent/WO2012075729A1/en
Publication of CN102005060A publication Critical patent/CN102005060A/en
Application granted granted Critical
Publication of CN102005060B publication Critical patent/CN102005060B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation

Abstract

The invention discloses a method and device for automatically removing selected images in pictures, which have the advantages of high restoring speed and good restoring effect and have the technical scheme that the method comprises the following steps of: setting a boundary to be replaced on an original picture to shield the selected images; setting a boundary point located block using points on the boundary to be replaced as the center; finding a point p' with the highest block confidence degree in all boundary points, wherein the block of the boundary point corresponding to the point p' is phi p'; outwards expanding the boundary to be replaced to form an outwards expanded boundary; finding a boundary point located block phi q' with the maximum character similarity to the boundary point located block phi p' of the point p' in the outwards expanded boundary range, wherein the corresponding boundary point of the phi q' is q'; replacing the phi p' by using the phi q'; assigning the maximum value of the character similarity between eight adjacent pixels of the point q' and the point p'; and returning to the step of finding the point with the highest confidence degree to find the point with the highest confidence degree in the unprocessed boundary point located block until the processing on all boundaries to be processed is completed.

Description

Automatically remove the method and apparatus of selected image in the image
Technical field
The present invention relates to a kind of image processing method and device, relate in particular to the selected image that occurs in the image is carried out the method and apparatus that robotization removes.
Background technology
When image taking, through regular meeting in image, inevitably occur do not hope the local image preserved, this will bring very big loss to user's visual experience.Existing disposal route is to accomplish through manually removing, and concrete mode is in the local image that needs are removed, and through image editing tools, manually scratches the mode of figure, and local imagery zone is repaired.The very big labor intensive of this mode of operation.Increased the burden of post-processed.
Traditional reparation image technique is divided into two types.One type of image mending (inpainting) technology that is based on the several picture model; Such techniques make use PDE and physics thermal diffusion principle; Image is repaired; Having of representative: M.Bertalmio, A.L.Bertozzi etc. " Navier-stokes, fluid dynamics, and image and video inpainting "; C.Ballester, V.Caselles etc. " Avariational model for filling-in gray level and color images "; T.F.Chanand J.Shen " Non-texture inpainting by curvature-driyen diffusions (CDD) ".Such technology can be preserved the linear structure of image, but only is applicable to that the small scale of repairing in the image is damaged; Very bad for big slightly regional effect, usually occur significantly fuzzy at restoring area.Another kind of synthetic image completion (completion) technology of texture that is based on, wherein representational have: L.Liang, C.Liu etc. " Real-timetexture synthesis by patch-based sampling "; A.Efros and W.T.Freeman " Image quilting for texture synthesis and transfer "; M.Ashikhmin " Synthesizing natural textures "; Such technological merit is that speed is fast, but shortcoming can not well be preserved the image linear structure.Author afterwards improves this, such as " Object removal by exemplar-based inpainting " such as A.Criminisi in the thought of using for reference inpainting; D.Simakov etc. " Summarizing visual data using bidirectional similarity " .CHO, T.S. etc. " The patch transform and its appl ications to image editing "; KOMODAKIS, N. etc. " Image completion using efficient belief propagation viapriority scheduling and dynamic pruning "; KUMAR, " What is a goodnearest neighbors algorithm for finding similar patches in images such as N.? "; RUBINSTEIN, M. etc. " Improved seam carving for video retargeting "; WEXLER, Y etc. " Space-time completion of video ".This technology has effect preferably for blocks lost big in the blank map picture, also can preserve the structure of image, but speed is very slow, and for the picture of a 1200*1200 size, repairing the 200*200 zone usually needs a few minutes even half an hour.Therefore above-mentioned technology is removed or is repaired and use for local image, all has significant limitation.
So, how to remove automatically or repair and select imagery zone, be a urgent problem.
Summary of the invention
The objective of the invention is to address the above problem, a kind of method that removes selected image in the image automatically is provided, have the advantage that reparation speed is fast and repairing effect is good.
Another object of the present invention is to provide a kind of device that removes selected image in the image automatically.
Technical scheme of the present invention is: the present invention has disclosed a kind of method that removes selected image in the image automatically, comprising:
Border to be replaced on the original graph is set, to hide selected image;
Setting is with frontier point place piece that to wait to replace borderline point be the center;
In all frontier points, find out the highest some p ' of piece degree of confidence, the frontier point place piece of some p ' correspondence is ψ P '
Border to be replaced is extended out, form and extend out the border, back;
In the scope that extends out the circle, back, find out and put the frontier point place piece ψ of p ' P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q ', uses ψ Q 'Replace ψ P '
Relatively characteristic similarity maximal value between 8 adjacent pixels of a q ' and the some p ' is composed to a p ';
Return the step of searching the highest point of degree of confidence, in untreated frontier point place piece, search the highest point of degree of confidence, finish up to all boundary treatment to be replaced.
According to an embodiment who removes in the image method of selected image automatically of the present invention, in that being set, treating on the original graph also comprise before replacing the border:
Original graph is dwindled, so that follow-up process object is the original graph after dwindling.
According to an embodiment who removes the method for selected image in the image automatically of the present invention, original graph is dwindled through interpolation algorithm.
According to an embodiment who removes in the image method of selected image automatically of the present invention, finding a p ' and some q ' afterwards, find corresponding in the original graph two some p and q and this two some p and pairing ψ of q according to the original graph scale down pAnd ψ q, use ψ qReplacement ψ p, and the characteristic similarity maximal value that will put between 8 adjacent pixels and the some p of q is relatively composed to a p.
According to an embodiment who removes the method for selected image in the image automatically of the present invention; Treating on original graph is set replaced in the step on border; To wait to replace boundary marker is and the background colour various colors, so that become border to be replaced with background colour color Different Boundary.
The present invention has also disclosed a kind of device that removes selected image in the image automatically, comprising:
Boundary setting module is provided with the border to be replaced on the original graph, to hide selected image;
Frontier point place piece is provided with module, couples boundary setting module, the frontier point place piece that to be provided with to wait replacing borderline point be the center;
The degree of confidence processing module couples the degree of confidence processing module, in all frontier points, finds out the highest some p ' of piece degree of confidence, and the frontier point place piece of some p ' correspondence is ψ P '
The border extends out module, couples the degree of confidence processing module, and border to be replaced is extended out, and forms to extend out the border, back;
Replacement processing module couples this border and extends out module, in the scope that extends out the circle, back, finds out and put the frontier point place piece ψ of p ' P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q ', with frontier point place piece ψ Q 'Replacement frontier point place piece ψ P '
The assignment module; Couple replacement processing module and degree of confidence processing module; Relatively characteristic similarity maximal value between 8 adjacent pixels of a q ' and the some p ' is composed to a p '; Return the degree of confidence processing module and handle, in untreated frontier point place piece, search the highest point of degree of confidence, finish up to all boundary treatment to be replaced by the degree of confidence processing module.
According to an embodiment who removes the device of selected image in the image automatically of the present invention; This device comprises that also image dwindles module; Be coupled in before the boundary setting module, this image dwindles module original graph is dwindled, so that follow-up process object is the original graph after dwindling.
According to an embodiment who removes in the image device of selected image automatically of the present invention, dwindle to be provided with an interpolation process unit in the module at image, through the operation of the interpolation algorithm in the interpolation process unit original graph is dwindled.
According to an embodiment who removes the device of selected image in the image automatically of the present invention; In replacement processing module, also comprise the original graph map unit; This replacement processing module is finding a p ' and some q ' to start the original graph map unit afterwards, and the original graph map unit finds in the original graph corresponding two some p and q and this two some p and pairing ψ of q according to the original graph scale down pAnd ψ q, so that ψ qReplacement ψ p, and the characteristic similarity maximal value that in the assignment module, will put between 8 adjacent pixels and the some p of q is relatively composed to a p.
According to an embodiment who removes in the image device of selected image automatically of the present invention, in boundary setting module, will wait to replace boundary marker for the background colour various colors so that become border to be replaced with background colour color Different Boundary.
The present invention contrasts prior art has following beneficial effect: the present invention can remove the image of selection area in the image automatically; Make the supplier of raw video remove the image of selection area more easily, remove manpower, inefficiency, the too high problem of cost of too relying on thereby solved original image processing selection area image.
Description of drawings
Fig. 1 is the process flow diagram that removes first embodiment of the method for selected image in the image automatically of the present invention.
Fig. 2 is the process flow diagram that removes second embodiment of the method for selected image in the image automatically of the present invention.
Fig. 3 is the refinement process flow diagram of Fig. 2 embodiment.
Fig. 4 is the schematic diagram that removes first embodiment of the device of selected image in the image automatically of the present invention.
Fig. 5 is the schematic diagram that removes second embodiment of the device of selected image in the image automatically of the present invention.
Embodiment
Below in conjunction with accompanying drawing and embodiment the present invention is done further description.
Automatically remove first embodiment of the method for selected image in the image
Fig. 1 shows the embodiment that removes the method for selected image in the image automatically of the present invention.Seeing also Fig. 1, is the detailed description that removes each step of the method for selected image in the image automatically to present embodiment below.
Step S10: the border to be replaced on the original graph is set, to hide selected image.
Border to be replaced can be labeled as unified color, covers in the background colour on the original graph.That is to say that will wait to replace boundary marker is and the background colour various colors just can be judged as border to be replaced with background colour color Different Boundary like this.
Step S11: the frontier point place piece that to be provided with to wait replacing borderline point be the center.
Wait to replace borderline arbitrfary point around in certain pixel coverage, the piece that to be provided with this point be the center is used for carrying out follow-up contrast and replacement, the big I of piece is set arbitrarily.
Step S12: in all untreated frontier points, find out the highest some p ' of piece degree of confidence, the frontier point place piece of this time point p ' correspondence is ψ P '
Step S13: border to be replaced is extended out, form and extend out the border, back.
For example; Border to be replaced originally is that (r * m), then extending out the border, back is (r+areasize) * (m+areasize), and wherein areasize is a parameter; Be illustrated in bounds to be replaced and extend out big what pixels, will in this scope, search the piece image that supply to replace.
It should be noted that the mark of in step S10, setting that is used for waits to replace unique pixel rgb value (color just) in the bounds with to extend out in circle, the back scope each pixel rgb value of former figure inequality entirely.
Step S14: in the scope that extends out the circle, back, find out the some p ' place piece ψ the highest with current degree of confidence P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q ', uses ψ Q 'Replace ψ P '
In this step, accomplished the image replacement of some p ' place piece, but the pixel loss when preventing to replace at this moment need be carried out follow-up step, with guaranteeing that image that replacement obtains and on every side image merge fully.
Step S15: the characteristic similarity maximal value that will put between 8 adjacent pixels and the some p ' of q ' is relatively composed to a p '.
This step jumps to step S11 after finishing, and replaces the whole inquiries in border until all waiting and finishes.
In this step, also need through a kind of technological means will be treated frontier point place piece (ψ for example P ') get rid of outside round-robin seek scope next time, to guarantee when jumping to step S11, need not to search the border of having handled again.Such technological means for example is that the degree of confidence with a p ' is made as 0 (specifically can referring to the embodiment of Fig. 3).
In the present embodiment, between step S11 to S15, circulate, finish up to the boundary treatment to be replaced described in the step S10.
Automatically remove second embodiment of the method for selected image in the image
Fig. 2 shows second embodiment that removes the method for selected image in the image automatically of the present invention.Seeing also Fig. 2, is the detailed description to each step in the method for present embodiment below.
Step S20: original graph is dwindled, so that follow-up process object is the original graph after dwindling.
(n * n) is not if original graph itself just in this range of size, then needs to carry out convergent-divergent again, and with original graph substitution subsequent operation can to preset a range of size.
If original graph itself then adopts interpolation algorithm not in this range of size, it is zoomed in the range of size of n * n, and store the image that this dwindles separately.The purpose of minification is to improve the speed of computing, and when guaranteeing to search relevant matches point and zone, the traversal number of times that causes significantly reduces.
Step S21: judge the border to be replaced after original graph is dwindled.
Border to be replaced can be labeled as unified color, covers in the background colour on the original graph.That is to say that will wait to replace boundary marker is and the background colour various colors just can be judged as border to be replaced with background colour color Different Boundary like this.
Step S22: the frontier point place piece that to be provided with to wait replacing borderline point be the center.
Wait to replace borderline arbitrfary point around in certain pixel coverage, the piece that to be provided with this point be the center is used for carrying out follow-up contrast and replacement, the big I of piece is set arbitrarily.
Step S23: in all untreated frontier points, find out the highest some p ' of piece degree of confidence, the frontier point place piece of this time point p ' correspondence is ψ P '
Step S24: border to be replaced is extended out, form and extend out the border, back.
For example; Border to be replaced originally is that (r * m), then extending out the border, back is (r+areasize) * (m+areasize), and wherein areasize is a parameter; Be illustrated in bounds to be replaced and extend out big what pixels, will in this scope, search the piece image that supply to replace.
It should be noted that the mark that is used in step S21 waits to replace unique pixel rgb value (color just) in the bounds with to extend out in circle, the back scope each pixel rgb value of former figure inequality entirely.
Step S25: in the scope that extends out the circle, back, find out the some p ' place piece ψ the highest with current degree of confidence P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q ', finds corresponding in the original graph two some p and q and this two some p and pairing ψ of q according to the original graph scale down pAnd ψ q, use ψ qReplace ψ p
In this step, accomplished the image replacement of some p place piece, but the pixel loss when preventing to replace at this moment need be carried out follow-up step, with guaranteeing that image that replacement obtains and on every side image merge fully.
Step S26: the characteristic similarity maximal value that will put between 8 adjacent pixels and the some p of q is relatively composed to a p.
This step jumps to step S22 after finishing, and replaces the whole inquiries in border until all waiting and finishes.
In this step, also need through a kind of technological means will be treated frontier point place piece (ψ for example P ') get rid of outside round-robin seek scope next time, to guarantee when jumping to step S22, need not to search the border of having handled again.Such technological means for example is that the degree of confidence with a p ' is made as 0 (specifically can referring to the embodiment of Fig. 3).
In the present embodiment, between step S22 to S26, circulate, finish up to the boundary treatment to be replaced described in the step S21.
Automatically remove the refinement of second embodiment of the method for selected image in the image
Fig. 3 shows the further refinement flow process of above-mentioned Fig. 2 embodiment.Seeing also Fig. 3, below is the detailed description of each step in the refinement flow process of method of second embodiment.
Step S30: from original graph, choose the area to be repaired.
For example original graph is that (whether W * H), border to be repaired can be labeled as unified color, covers in background colour, be that the basis for estimation of waiting to replace the border is color and other color Different Boundary on it.
Step S31: judge that whether W and H are all greater than n.
N is a preset value, for example is 300 pixel values, and this is the value of an assurance operation efficiency that relatively is fit to.If W and H then carry out step S312, otherwise carry out step S314 all greater than the n pixel.
Step S312: the maximum ratio that obtains W and H and n is rate.
The formula that this step adopts be rate=max (W/n, H/n).
Step S314: with the rate assignment is 1.
Step S316: according to step S312 gained ratio, original image is introduced interpolation algorithm, original graph is dwindled, minification is rate.
The interpolation algorithm that this step adopts can be bilinearity, thereby reduces the loss of pixel.
Step S32: adopt formula
Figure BDA0000036718310000071
Figure BDA0000036718310000072
to dwindle computing, obtain W ' and H '.In this step, if according to step S314, then the rate value is 1, and W ' value equals W, and H ' value equals H.
Step S33: establish piece ψ P 'Size is t*t, max ψ P 'Be degree of confidence, max ψ P 'Value is 0.
Piece ψ in this step P 'For being the center with a p ', the length of side is the piece of t, and t is a natural number, and for example value is 10.
Step S34: whether judging point p ' is the area to be repaired of being demarcated among the step S30.
Whether the foundation of judging in this step is the pixel value of area to be repaired for the rgb value of point, and the pixel of the area to be repaired of for example in step S30, setting is green, and whether the rgb value that then is exactly judging point here is green.If green, then carry out step S342, if be not green, then carry out step S344.
Step S342: ask a p ' place piece ψ P 'Degree of confidence.
Piece ψ P 'The calculating of degree of confidence be a kind of known technology, for example:
P(p′)＝C(p′)D(p′)
C ( p ′ ) = Σ u ∈ ψ p ′ ∩ Ω C ( u ) | ψ p ′ |
D ( p ′ ) = | ▿ I 1 p ′ . n p ′ | α
Here | ψ P '| be ψ P 'Area.α is the image normalization factor, is defaulted as 255 here.Ω is the area to be repaired, n P 'For being orthogonal to the vector of unit length of p ' tangential direction. is p ' isophote.
The confidence value of C (u) expression point u, u representes certain pixel
The P that wherein obtains (p ') is a p ' place piece ψ P 'Degree of confidence.
Step S344: in the image that dwindles (being of a size of W ' * H '), seek next pixel.
If there is next pixel, then carries out step S345, otherwise carry out step S348 in this step.
Step S345: in the image that dwindles (being of a size of W ' * H '), p ' is made as next pixel.
After this step is finished, carry out step S34.
Step S346: the ψ that will in step S342, be obtained P 'Degree of confidence and max ψ P 'Degree of confidence compare.
In this step, if ψ P 'Degree of confidence greater than max ψ P 'Degree of confidence then carry out step S347, if ψ P 'Degree of confidence be less than or equal to max ψ P 'Degree of confidence then carry out step S35.
Step S348: accomplish removing of selected image, this method finishes.
Step S347: with ψ P 'The degree of confidence assignment give max ψ P 'Degree of confidence, carry out step S35 then.
In this step simultaneously with savepoint p ', in order to bring the computing that step S37 is carried out into.
Step S35:, then carry out step S36, otherwise get back to step S345 to search next pixel if pixel is last pixel of searching in the image (being of a size of W ' * H ') that dwindles.
Step S36: outside the most external rectangle of current boundary curve (r*m), extend out areasize pixel again, delimit flared region, be of a size of (r+areasize) * (m+areasize).
Boundary rectangle in this step (r*m) is the formed rectangle in outer (r*m) of the described area to be repaired of S30.
The value of areasize for example is made as 35 in this step, and this numerical value can improve the operation efficiency of subsequent step when bringing computing into.
Step S37: the q ' that sets up an office is the point in the zone except that the said area to be repaired of step S30 in the said flared region of step S36 (being of a size of (r+areasize) * (m+areasize)).If piece ψ Q 'For with a q ' being the piece zone at center, piece ψ Q 'Size be t*t, establish min ψ Q 'The Euclidean distance quadratic sum be infinitely great.
In this step, piece ψ Q 'The value of length of side t for example be made as 10, with piece ψ P 'The length of side identical.Be convenient to piece ψ in the subsequent step Q 'To piece ψ P 'Pixel replacement.
Step S38: judge in flared region, whether to also have piece ψ Q ', then carry out step S382 if exist, then do not carry out step S384 if do not exist.
Step S382: judging point q ' and the Euclidean distance quadratic sum of putting p ' With min ψ Q 'The size of Euclidean distance quadratic sum, if the Euclidean distance quadratic sum of some q ' and some p '
Figure BDA0000036718310000092
Less than min ψ Q 'The Euclidean distance quadratic sum, then carry out step S386, otherwise carry out step S384, wherein I Q ' rgbThe pixel value of expression q ' some RGB, I P ' rgbThe pixel value of expression p ' some RGB.
Step S386: will
Figure BDA0000036718310000093
Assignment is to min ψ Q 'The Euclidean distance quadratic sum.
Also will keep q ' point in this step, this is named a person for a particular job and brings subsequent step S387 or S388 into.
After this step is finished, with the judgement of proceeding step S38.
Step S384:, otherwise carry out step S388 if the rate value equals 1 and carries out step S387.
The rate value of rate value described in this step for obtaining among step S312 or the step S314.
Step S387: will put q ' place piece ψ Q 'Pixel copy a piece ψ at p ' place to P 'In.
This step will make a p ' place piece ψ P 'Pixel be replaced, accomplish in this piece removing of selected image.
This step will be carried out subsequent step S392 after accomplishing.
Step S388: in original graph (being of a size of W*H), find a little and point.This step is that the mapping relations employing formula of a linearity is p=rate*p ' and q=rate*q '.
The rate value of rate value described in this step for obtaining among step S312 or the step S314.
After finishing, this step will carry out step S389.
Step S389: will put q place piece ψ qPixel copy the piece ψ at some p place to pIn.
This step will make a p place piece ψ pPixel be replaced, accomplish in this piece removing of selected image.
After finishing, this step will carry out step S394.
Step S392: will put 8 adjacent pixels of q ' and European square of minimum value of p ' some comparison and compose to a p '.
After finishing, this step will proceed step S39.
Step S394: 8 adjacent pixels and the p point European square of minimum value relatively that will put q are composed to a p.
After finishing, this step will proceed step S39.
Step S39: the degree of confidence that will put p ' is made as 0.
This step makes judges the value that obtains among the subsequent step S346 always for not, thereby is able to carry out follow-up step S35.
After finishing, this step will carry out step S344.
Automatically remove first embodiment of the device of selected image in the image
Fig. 4 shows the principle that removes first embodiment of the device of selected image in the image automatically of the present invention.Seeing also Fig. 4, is the detailed description to the principle of the device of present embodiment below.
Automatically the device that removes in the image selected image of present embodiment comprises following module: boundary setting module 10, frontier point place piece are provided with module 11, degree of confidence processing module 12, border and extend out module 13, replacement processing module 14 and assignment module 15.Annexation between these modules is: boundary setting module 10 couples frontier point place piece module 11 is set; Frontier point place piece is provided with module 11 and couples degree of confidence processing module 12; Degree of confidence processing module 12 couples the border and extends out module 13; The border extends out module 13 and couples replacement processing module 14, and replacement processing module 14 couples assignment module 15, and the assignment module also couples degree of confidence processing module 12 except coupling replacement processing module 14.
Boundary setting module 10 is used to be provided with the border to be replaced on the original graph, to hide selected image.Border to be replaced can be labeled as unified color, covers in the background colour on the original graph.That is to say that will wait to replace boundary marker is and the background colour various colors just can be judged as border to be replaced with background colour color Different Boundary like this.
Frontier point place piece is provided with module 11 and is used to be provided with to wait replacing borderline some the frontier point place piece that is the center.Wait to replace borderline arbitrfary point around in certain pixel coverage, the piece that to be provided with this point be the center is used for carrying out follow-up contrast and replacement, the big I of piece is set arbitrarily.
In the degree of confidence processing module 12, in all untreated frontier points, find out the highest some p ' of piece degree of confidence, the frontier point place piece of this time point p ' correspondence is ψ P 'The calculating of piece degree of confidence is traditional technological means, and in the refinement of method second embodiment, detailed description is arranged.
The border extends out module 13 border to be replaced is extended out, and forms to extend out the border, back.For example; Border to be replaced originally is that (r * m), then extending out the border, back is (r+areasize) * (m+areasize), and wherein areasize is a parameter; Be illustrated in bounds to be replaced and extend out big what pixels, will in this scope, search the piece image that supply to replace.
It should be noted that in boundary setting module 10 mark of setting that is used for waits to replace unique pixel rgb value (color just) in the bounds with to extend out in circle, the back scope each pixel rgb value of former figure inequality entirely.
In the replacement processing module 14, find out the some p ' place piece ψ the highest in the scope on border after extending out with current degree of confidence P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q ', uses ψ Q 'Replace ψ P '
In this module, accomplished the image replacement of some p ' place piece, but the pixel loss when preventing to replace at this moment need be carried out the processing of subsequent module, with guaranteeing that image that replacement obtains and on every side image merge fully.
Assignment module 15 will be put 8 adjacent pixels of q ' and put the characteristic similarity maximal value that compares between the p ' and compose to a p '.
After assignment module 15 is handled, jump to degree of confidence processing module 12, in untreated frontier point place piece, search the highest point of degree of confidence by degree of confidence processing module 12, replace the whole inquiries in border until all waiting and finish.
In degree of confidence processing module 12, need will be treated through a kind of technological means frontier point belong to piece (ψ for example P ') get rid of outside round-robin seek scope next time, to guarantee the searching border of having handled again.Such technological means for example is that the degree of confidence with a p ' is made as 0 (specifically can referring to the embodiment of Fig. 3).
Automatically remove second embodiment of the device of selected image in the image
Fig. 5 shows the principle that removes second embodiment of the device of selected image in the image automatically of the present invention.Seeing also Fig. 5, is the detailed description to the principle of the device of present embodiment below.
Automatically the device that removes in the image selected image of present embodiment comprises following module: image dwindles module 20, boundary setting module 21, frontier point place piece and module 22, degree of confidence processing module 23, border are set extend out module 24, replacement processing module 25 and assignment module 26.Annexation between these modules is: image dwindles module 20 and couples boundary setting module 21; Boundary setting module 21 couples frontier point place piece module 22 is set; Frontier point place piece is provided with module 22 and couples degree of confidence processing module 23, and degree of confidence processing module 23 couples the border and extends out module 24, and the border extends out module 24 and couples replacement processing module 25; Replacement processing module 25 couples assignment module 26, and assignment module 26 also couples degree of confidence processing module 23 except coupling replacement processing module 25.In image dwindles module 20, be provided with interpolation process unit 200, in replacement processing module 25, be provided with original graph map unit 250.
Image dwindles module 20 original graph is dwindled, so that follow-up process object is the original graph after dwindling.
Dwindle in the module 20 at image, (n * n) is not if original graph itself just in this range of size, then needs to carry out convergent-divergent again, and with original graph substitution subsequent operation can to preset a range of size.
If original graph itself is then moved interpolation process unit 200 not in this range of size, interpolation process unit 200 adopts interpolation algorithm, original graph is zoomed in the range of size of n * n, and stores the image that this dwindles separately.The purpose of minification is to improve the speed of computing, and when guaranteeing to search relevant matches point and zone, the traversal number of times that causes significantly reduces.
Boundary setting module 21 is used to judge the to be replaced border of original graph after dwindling processing, to hide selected image.Border to be replaced can be labeled as unified color, covers in the background colour on the original graph.That is to say that will wait to replace boundary marker is and the background colour various colors just can be judged as border to be replaced with background colour color Different Boundary like this.
Frontier point place piece is provided with module 22 and is used to be provided with to wait replacing borderline some the frontier point place piece that is the center.Wait to replace borderline arbitrfary point around in certain pixel coverage, the piece that to be provided with this point be the center is used for carrying out follow-up contrast and replacement, the big I of piece is set arbitrarily.
In the degree of confidence processing module 23, in all untreated frontier points, find out the highest some p ' of piece degree of confidence, the frontier point place piece of this time point p ' correspondence is ψ P 'The calculating of piece degree of confidence is traditional technological means, and in the refinement of method second embodiment, detailed description is arranged.
The border extends out module 24 border to be replaced is extended out, and forms to extend out the border, back.For example; Border to be replaced originally is that (r * m), then extending out the border, back is (r+areasize) * (m+areasize), and wherein areasize is a parameter; Be illustrated in bounds to be replaced and extend out big what pixels, will in this scope, search the piece image that supply to replace.
It should be noted that in boundary setting module 21 mark of setting that is used for waits to replace unique pixel rgb value (color just) in the bounds with to extend out in circle, the back scope each pixel rgb value of former figure inequality entirely.
In the replacement processing module 25, find out the some p ' place piece ψ the highest in the scope on border after extending out with current degree of confidence P 'Compare the maximum frontier point place piece ψ of characteristic similarity Q ', ψ Q 'Corresponding frontier point is q '.
Find some p ' and some q ' afterwards, finding in the original graph piece ψ of corresponding two some p and q and these two somes correspondences by original graph map unit 250 according to the original graph scale down pAnd ψ qUse ψ qReplace ψ p
In this module, accomplished the image replacement of some p place piece, but the pixel loss when preventing to replace at this moment need be carried out the processing of subsequent module, with guaranteeing that image that replacement obtains and on every side image merge fully.
Assignment module 26 will be put 8 adjacent pixels of q and put the characteristic similarity maximal value that compares between the p and compose to a p.
After assignment module 26 is handled, jump to degree of confidence processing module 23, in untreated frontier point place piece, search the highest point of degree of confidence by degree of confidence processing module 23, replace the whole inquiries in border until all waiting and finish.
In degree of confidence processing module 23, need will be treated through a kind of technological means frontier point belong to piece (ψ for example p) get rid of outside round-robin seek scope next time, to guarantee the searching border of having handled again.Such technological means for example is that the piece degree of confidence with a p is made as 0 (specifically can referring to the embodiment of Fig. 3).
The foregoing description provides to those of ordinary skills and realizes or use of the present invention; Those of ordinary skills can be under the situation that does not break away from invention thought of the present invention; The foregoing description is made various modifications or variation; Thereby protection scope of the present invention do not limit by the foregoing description, and should be the maximum magnitude that meets the inventive features that claims mention.

Claims (10)

1. one kind removes the method for selecting image in the image automatically, comprising:
Border to be replaced on the original graph is set, to hide selected image;
Setting is with frontier point place piece that to wait to replace borderline point be the center;
In all frontier points, find out the highest some p' of piece degree of confidence, the frontier point place piece that some p' is corresponding is ψ P '
Border to be replaced is extended out, form and extend out the border, back;
In the scope that extends out the circle, back, find out and put the frontier point place piece ψ of p' P'The frontier point place piece ψ that the characteristic similarity of comparing is maximum Q', ψ Q'Corresponding frontier point is q', uses ψ Q'Replace ψ P'
Relatively characteristic similarity maximal value between 8 adjacent pixels of a q' and the some p' is composed to a p';
Return the step of searching the highest point of degree of confidence, in untreated frontier point place piece, search the highest point of degree of confidence, finish up to all boundary treatment to be replaced.
2. the method that removes in the image selected image automatically according to claim 1 is characterized in that, also comprises before replacing the border in that treating on the original graph is set:
Original graph is dwindled, so that follow-up process object is the original graph after dwindling.
3. the method that removes selected image in the image automatically according to claim 2 is characterized in that, through interpolation algorithm original graph is dwindled.
4. the method that removes in the image selected image automatically according to claim 2 is characterized in that, after finding a p' and some q', finds corresponding in the original graph two some p and q and this two some p and pairing ψ of q according to the original graph scale down pAnd ψ q, use ψ qReplacement ψ p, and the characteristic similarity maximal value that will put between 8 adjacent pixels and the some p of q is relatively composed to a p.
5. the method that removes selected image in the image automatically according to claim 1; It is characterized in that; Treating on original graph is set replaced in the step on border, will wait to replace boundary marker to be and the background colour various colors, so that become border to be replaced with background colour color Different Boundary.
6. one kind removes the device of selecting image in the image automatically, comprising:
Boundary setting module is provided with the border to be replaced on the original graph, to hide selected image;
Frontier point place piece is provided with module, couples boundary setting module, the frontier point place piece that to be provided with to wait replacing borderline point be the center;
The degree of confidence processing module couples frontier point place piece module is set, and in all frontier points, finds out the highest some p' of piece degree of confidence, and the frontier point place piece that some p' is corresponding is ψ P'
The border extends out module, couples the degree of confidence processing module, and border to be replaced is extended out, and forms to extend out the border, back;
Replacement processing module couples this border and extends out module, in the scope that extends out the circle, back, finds out and put the frontier point place piece ψ of p' P'The frontier point place piece ψ that the characteristic similarity of comparing is maximum Q', ψ Q'Corresponding frontier point is q', with frontier point place piece ψ Q'Replacement frontier point place piece ψ P'
The assignment module; Couple replacement processing module and degree of confidence processing module; Relatively characteristic similarity maximal value between 8 adjacent pixels of a q' and the some p' is composed to a p'; Return the degree of confidence processing module and handle, in untreated frontier point place piece, search the highest point of degree of confidence, finish up to all boundary treatment to be replaced by the degree of confidence processing module.
7. the device that removes selected image in the image automatically according to claim 6; It is characterized in that this device comprises that also image dwindles module, is coupled in before the boundary setting module; This image dwindles module original graph is dwindled, so that follow-up process object is the original graph after dwindling.
8. the device that removes selected image in the image automatically according to claim 7 is characterized in that, dwindles being provided with an interpolation process unit in the module at image, through the operation of the interpolation algorithm in the interpolation process unit original graph is dwindled.
9. the device that removes selected image in the image automatically according to claim 7; It is characterized in that; In replacement processing module, also comprise the original graph map unit; This replacement processing module is finding a p' and point to start the original graph map unit after the q', and the original graph map unit finds two some p and q and this two some p and pairing ψ of q of correspondence in the original graph according to the original graph scale down pAnd ψ q, so that ψ qReplacement ψ p, and the characteristic similarity maximal value that in the assignment module, will put between 8 adjacent pixels and the some p of q is relatively composed to a p.
10. the device that removes in the image selected image automatically according to claim 6 is characterized in that, in boundary setting module, will wait to replace boundary marker for the background colour various colors so that become border to be replaced with background colour color Different Boundary.
CN2010105774848A 2010-12-08 2010-12-08 Method and device for automatically removing selected images in pictures Expired - Fee Related CN102005060B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN2010105774848A CN102005060B (en) 2010-12-08 2010-12-08 Method and device for automatically removing selected images in pictures
PCT/CN2011/071073 WO2012075729A1 (en) 2010-12-08 2011-02-18 Method and device for removing selected image in picture automatically

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010105774848A CN102005060B (en) 2010-12-08 2010-12-08 Method and device for automatically removing selected images in pictures

Publications (2)

Publication Number Publication Date
CN102005060A CN102005060A (en) 2011-04-06
CN102005060B true CN102005060B (en) 2012-11-14

Family

ID=43812398

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010105774848A Expired - Fee Related CN102005060B (en) 2010-12-08 2010-12-08 Method and device for automatically removing selected images in pictures

Country Status (2)

Country Link
CN (1) CN102005060B (en)
WO (1) WO2012075729A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521786B (en) * 2011-12-01 2013-12-18 中国科学院自动化研究所 Method for removing watermarks of photos based on color detection and fast matching method
US10296646B2 (en) 2015-03-16 2019-05-21 International Business Machines Corporation Techniques for filtering content presented in a web browser using content analytics
CN111524076B (en) * 2020-04-07 2023-07-21 咪咕文化科技有限公司 Image processing method, electronic device, and computer-readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1731449A (en) * 2005-07-14 2006-02-08 北京航空航天大学 A method of image restoration
CN101355648A (en) * 2008-06-26 2009-01-28 天津市亚安科技电子有限公司 Method for reducing image noise and enhancing image
CN101360246A (en) * 2008-09-09 2009-02-04 西南交通大学 Video error masking method combined with 3D human face model
CN101571950A (en) * 2009-03-25 2009-11-04 湖南大学 Image restoring method based on isotropic diffusion and sparse representation

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7167519B2 (en) * 2001-12-20 2007-01-23 Siemens Corporate Research, Inc. Real-time video object generation for smart cameras
CN100562065C (en) * 2005-08-15 2009-11-18 索尼株式会社 Camera head, denoising device, noise-reduction method
US7720283B2 (en) * 2005-12-09 2010-05-18 Microsoft Corporation Background removal in a live video
CN101266685A (en) * 2007-03-14 2008-09-17 中国科学院自动化研究所 A method for removing unrelated images based on multiple photos
CN101482968B (en) * 2008-01-07 2013-01-23 日电（中国）有限公司 Image processing method and equipment
CN102005036B (en) * 2010-12-08 2012-05-09 上海杰图软件技术有限公司 Method and device for automatically removing tripod afterimage in panoramic image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1731449A (en) * 2005-07-14 2006-02-08 北京航空航天大学 A method of image restoration
CN101355648A (en) * 2008-06-26 2009-01-28 天津市亚安科技电子有限公司 Method for reducing image noise and enhancing image
CN101360246A (en) * 2008-09-09 2009-02-04 西南交通大学 Video error masking method combined with 3D human face model
CN101571950A (en) * 2009-03-25 2009-11-04 湖南大学 Image restoring method based on isotropic diffusion and sparse representation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
张晓峰
李景辉
李景辉;张晓峰;马燕.纹理合成在图像修复中的应用研究.《计算机工程》.2009,第35卷(第7期),206-208. *
马燕.纹理合成在图像修复中的应用研究.《计算机工程》.2009,第35卷(第7期),206-208.

Also Published As

Publication number Publication date
CN102005060A (en) 2011-04-06
WO2012075729A1 (en) 2012-06-14

Similar Documents

Publication Publication Date Title
CN104376535B (en) A kind of rapid image restorative procedure based on sample
CN103971338B (en) Variable-block image repair method based on saliency map
CN105741231B (en) The skin makeup treating method and apparatus of image
US8023768B2 (en) Universal front end for masks, selections, and paths
CN109345480B (en) Face automatic acne removing method based on image restoration model
CN102005060B (en) Method and device for automatically removing selected images in pictures
CN105354810A (en) Method, system and shooting terminal for removing speckles of image
US8116590B2 (en) Online image processing methods utilizing user's satisfaction loop
CN105139338A (en) Multi-dimensional lookup table generation method and device and image scaling processing method and device
CN101458821A (en) Method for animation processing image and video
CN102005036B (en) Method and device for automatically removing tripod afterimage in panoramic image
CN104952089A (en) Image processing method and image processing system
CN106327449B (en) A kind of image repair method, device and calculate equipment
CN105139345A (en) Automatic searching method of high-quality non-standard Gamma curve
CN108389259B (en) Random center aggregation image mesh tone method and system
CN106296605B (en) A kind of image mending method and device
CN110060267A (en) A kind of certificate photo changes background method and device
CA2339480A1 (en) Method and apparatus for removing defects from digital images
Huang et al. A patch-based image inpainting based on structure consistence
CN102592295B (en) A kind of method and apparatus of image procossing
CN114820340A (en) Lip wrinkle removing method, system, equipment and storage medium based on image processing
Chavda et al. Survey on image inpainting techniques: Texture synthesis, convolution and exemplar based algorithms
CN110473295B (en) Method and equipment for carrying out beautifying treatment based on three-dimensional face model
CN111080512B (en) Cartoon image generation method and device, electronic equipment and storage medium
CN109285166B (en) Overlapping and conglutinating chromosome automatic segmentation method based on full convolution network

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20110406

Assignee: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd.

Assignor: SHANGHAI JEITU SOFTWARE Co.,Ltd.

Contract record no.: 2014310000051

Denomination of invention: Method and device for automatically removing selected images in pictures

Granted publication date: 20121114

License type: Common License

Record date: 20140321

LICC Enforcement, change and cancellation of record of contracts on the licence for exploitation of a patent or utility model
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20121114

Termination date: 20211208

CF01 Termination of patent right due to non-payment of annual fee