The detailed description of preferred embodiment
Various embodiments of the present invention use the interpolation that is instructed to change view data in the target, thereby mixed result images is provided.Can change aiming field based on the interpolater that is instructed that relies on or be independent of source images.When in the aiming field that source region is blended into image, the interpolation that is instructed can eliminate or minimize the obvious seam between the remainder of the zone of inserting and image.In addition, can use various interpolations to instruct in mixed result images, to produce different-effect.This type of interpolation instructs can include, but not limited to identical guidance, level and smooth guidance, filter guide, transparent guide, mask instructs and the weighted mean combination of various guidances.
Fig. 1 has described the source images in one embodiment of the present of invention, target image and mixed result images.Source images 102 comprises the source region 100 that contains the sun.The user wishes source region 100 is inserted and is blended in the target image 104, and 108 places do not comprise the sun that is inserted to the latter in the position at first.In first operation, the user defines source region 100 by such as pull out profile around source region 100.Can use the replacement method of definition source region 100, include but not limited to, for clone's paintbrush is provided with anchor point, selects element in image, the chromatic characteristic in assigned source zone or specified coordinate etc. in source images 102.
Generally speaking, digital picture is represented by pixel value.For example, the digital picture of black and white can be represented that wherein each vector element is corresponding to single pixel by the pixel brightness value vector.On the contrary, the Digital Image Data of coloured image can be represented with a plurality of values of each pixel.For example, in RGB (red, green, blue) color space, each pixel is presented by each the brightness value in 3 color channels.Also conceive other color space, included but not limited to LC1C2 and CIE-lab space.
Should be appreciated that only source region 100 to be duplicated and paste the border or the seam 110 that can cause the circumsolar irregular blue colour of sky in the target image 104 usually, the darker night sky background 112 of its coverage goal image 104.Compare with target image, seam is usually expressed as the sudden change in color, brightness, feature, texture or the like.So, produce result images more true to nature, the mixed observability that can blur or eliminate the seam 110 in the result images 106 of the source region that inserts 104.Therefore, as shown in result images 106, source region 100 is inserted into and is blended in the target image 104, to produce effect true to nature.
Fig. 2 has described the notional mixed of two one dimension functions in one embodiment of the present of invention.The conceptual demonstration of the mix operation that relates to one dimension " image " is provided about this description of Fig. 2.But, should be appreciated that and in category of the present invention, conceived various mix operation, include but not limited to, with respect to the described two-dimentional mix operation of other accompanying drawing herein.
The first luminance function A of expression one dimension target " image " has been shown in aim curve Figure 200.The second luminance function B of the one-dimensional source " image " in the target image of indicating to be inserted into curve Figure 200 has been shown in source curve map 202.The mixed result's's " image " of expression one dimension mixed result function has been shown among result curve Figure 20 4.
The border of dotted line 206 expression source regions and target area.In another embodiment of the present invention, source region and aiming field can have diverse size and dimension (for example, rectangle, circle, irregular shape), and the both can be changed by system, thereby for mixed more compatible.
In one embodiment of the invention, the user wishes to use mix operation that source region is merged in the aiming field, thereby the view data of aiming field boundary is by mixed true to nature to show.In shown embodiment, this conceptually illustrates by the frontier point of the source region of the first luminance function A being linked the second luminance function B.So, Hun He result function shows the combination of two luminance functions.In order to eliminate, minimize or reduce the appearance of seam, each forms the boundary condition of function by mixed, thereby when each approximation of function frontier point 208 and 210, each value of forming function equates.
Fig. 3 shows mixing of two two dimensional image zones in one embodiment of the invention.Describe for the concept nature that the one dimension of expander graphs 2 is mixed, two-dimentional mixed example is provided among Fig. 3.Target image 300 illustrates the dark water surface.Among two dimension brightness curve Figure 20 2 the corresponding brightness data that are associated with target image 300 have been shown, wherein brightness can comprise that monochromatic brightness is (such as the white in the black white image, the perhaps single color channel in the coloured image), perhaps certain other outward appearance signal (for example, the component of LC1C2 or polymerization value).For example, note when target image 300 when locating to become more shallow near the top, the brightness of the white shown in brightness curve Figure 30 2 has increased.
In intermediate image 304, the source region 312 that is included in the inverted image of the sun on the different waters surface is inserted in this image.In an illustrated embodiment, the target area is defined as by the zone of the overlapping image of source region 312.Before mixed, to the difference of small part because of the color of water, the seam between the remainder of source region and target image is very obvious, as shown in intermediate image 304.This also can find out in non-mixed brightness curve Figure 30 6, and the brightness data that wherein relates to source region 312 is visible in fits and starts at 314 places.
In mixed result images 308, source region is blended in the target area, so that the photorealism of sun inverted image on the water surface to be provided.The variation of image data illumination in the obviously visible source region in mixed Figure 31 of brightness curve as a result 0, wherein the brightness data of the source region that is inserted is by mixed, with brightness data, thereby reduced total " brightness " of source region at the boundary of this first area coupling target image.
Fig. 4 has described the source images in one embodiment of the present of invention, target image and mixed result images.Target image 400 (with coloured image) illustrates light green sea, place, seabeach.Source images 402 (with coloured image) illustrates two children in the swimming pool that water seems light blue.For the purpose of this discussion, and for characteristics of image being mapped to algorithm described herein, target image 400 is known as " f
*", the target area 404 in the target image 400 is known as " Ω ".The border 412 of target area 404 is known as " Ω ".Similarly, source images 402 is known as " g
*", the view data of source region 416 is known as g.In mixed result images 410, source region 416 is blended in the target area 404, comprises image-region 418 and 420 (the blended image data f of Ω inside, and f with generation
*Remainder, i.e. the part of target image outside the target area) synthetic result images.Attention: mixed result images is to be inserted in the target image by the source region that will comprise each child in order to generate.
In one embodiment of the invention, mix operation is to use following function minimization to realize as the interpolater (or " minimizing device ") that is instructed:
min
f∫∫
Ω|f-g|
2,
It has boundary condition f|
Ω=f
*|
Ω, wherein
It is gradient operator.In addition, can use other interpolater, such as the maximization function and be optimized to set the goal, the function of threshold value or scope.
Euler-Lagrange (Euler-Lagrange) equation that is associated, the condition that interpolater satisfied that is promptly instructed is considered to have Poisson (Poisson) equation of Di Li Cray (Dirichlet) boundary condition:
Δ f=Δ g on the Ω, its f|
Ω=f
*|
Ω,
Wherein
It is Laplce (Laplacian) operator.
The interpolating method that is instructed can more briefly be expressed as the interpolater that is instructed by following function minimization:
min
f∫∫
Ω|f-v|
2,
It has boundary condition f|
Ω=f
*|
ΩWherein v represents that interpolation instructs, and f represents " interpolation type ".Integration illustrates that this function minimization is the unique solution of following Poisson equation:
△ f=div v on the Ω, its f|
Ω=f
*|
Ω,
Wherein
Be v=[u, v] divergence.
The various selections of v allow to mix with the whole bag of tricks, as shown in following example interpolation guidance table, instruct although also conceived other interpolation in category of the present invention.
Interpolation instructs title | Symbol | Interpolation instructs definition |
Identical | I | ?v=f
* |
Smoothly | S | ?v=0 |
Filter | F | ?v=F(g) |
Transparent | T | v=1
[|f*|>|g|]f
*+1
[|g|>|f*|]g
|
Mask | M | v=1
[|f|>τ]f
|
The weighted mean combination | W | V=α I+ (1-α) F (for example) |
For example, identical interpolation instructs I to cause no change (therefore, f=f in the target image
*).Alternatively, smoothly interpolation instructs S to change target image.The Euler-Lagrange equation that instructs S to be associated with level and smooth interpolation is:
f=0 on the Ω, its f|
Ω=f
*|
Ω,
And be considered to have the Laplace's equation of Dirichlet boundary conditions.Unique solution with Laplace's equation of Dirichlet boundary conditions often is known as " film interpolation type (membrane interpolant) ".The discrete form of head it off causes image-region very level and smooth or fuzzy among the Ω of territory in each color channel.Level and smooth interpolation instructs S the gradient of this inside, territory to be minimized (" the empty guidance "), and causes being positioned at the very level and smooth interpolation of the borderline color in territory.Level and smooth interpolation instructs S to be used by the computer graphical region-filling algorithm, comprises the filling algorithm of polygon or arbitrary shape.
Example effect transparent and that the mask interpolation instructs illustrates and discusses with respect to Fig. 7 and 8 respectively.
Fig. 5 has described the hybrid system in one embodiment of the present of invention.The target 504 of the view data of image editing system 500 reception such as target images etc.The selection that target selection module 508 receives the target area in the target 504.As previously mentioned, other method in the zone that the target area can be by paste operation, clone operations, outline operation or select target image defines.
In at least one embodiment, image editing system 500 also receives such as sources such as source images 502.The selection that the source selects module 506 to receive the source region in the source 504.But, should be appreciated that relating to the alternative embodiments such as embodiment that level and smooth interpolation instructs or the mask interpolation instructs such as those does not need source or source to select module, is independent of any source because this interpolation instructs.
Territory definition module 510 receives one or more selections, and objective definition territory (may also active territory).In one embodiment, the territory definition can be the form of the scope of X and Y coordinates on the image, although conceived other method of field of definition in category of the present invention, includes but not limited to the array and the zone definitions algorithm of picture position.
Image data collection device 511 is collected in the input picture in this article respectively by function f
*, the f on the Ω
*The target image of representing with g, aiming field, the view data that is comprised in may also active territory.Blended computation module 512 receiving target territory definition, destination image data, may also have source region information.In addition, blended computation module 512 is used these inputs, based on being derived from or taking from the interpolation that is stored in the information in the interpolation guide datastore 515 and instruct, calculates the view data of mixed result images 516.Output module 514 receives the view data of mixed result images 516, and this module presents mixed result images 516 on display, perhaps this image is outputed to memory device or Another application program.
Fig. 6 has described the operation that mixes in one embodiment of the present of invention.Receive the selection that operation 600 receives source (for example, source region).Another receives the selection of operation 602 receiving targets (for example, target area).Defining operation 604 defines the territory, source based on the selection to the source.Another defining operation 606 comes the objective definition territory based on the selection to target.
Collecting operation 608 is collected in herein respectively by function f
*, the f on the Ω
*The target image of representing with g, aiming field, the view data that is comprised in may also active territory.Provide operation 610 by store or provide the algorithm of hard coded such as visit data, the interpolater that is instructed is provided.Calculating operation 612 calculates mixed result images from collected view data and the interpolater that is instructed.Output function 614 outputs to display or other output device with mixed result images.
Fig. 7 has described to be derived from one embodiment of the present of invention the mixed result that transparent interpolation instructs.Use the filtration interpolation to instruct F that source images 700 is blended in the target image 702, to produce mixed result images 704.Yet, it is evident that from result images 704 between each letter of word " BLEND " and inner interval is not presented practically.On the contrary, used and be called transparent second interpolation and instruct, produced mixed result images 706.Transparent algorithm provides in last table.
The replacement of transparent algorithm is represented:
Transparent interpolation instructs the image gradient (selecting from source image data or from destination image data) of each the choice of location maximum of T in the target area.In this way, Hun He result images comprised with aiming field in the obvious characteristics of source region or target area of the given position view data of mating most.
Fig. 8 has described to be derived from one embodiment of the present of invention the mixed result that the mask interpolation instructs M.In the mask interpolation instructs, the gradient optimization to satisfy preassigned, is included but not limited to, be lower than threshold value, be higher than threshold value, in given range and satisfy and set the goal.
As implied above, the mask interpolation instructs M to filter destination image data, thereby only generates notable attribute in the target area of mixed result images.For example, use the mask interpolation as shown in preceding table to instruct M, low (perhaps, alternatively, height) gradient region is weakened the image data illumination in the source region 804 of source images 800, to produce the effect of the similar cartoon shown in the mixed result images 802.
Alternatively, can revise the mask interpolation instructs M that band filter range is provided:
Fig. 9 shows and is blended in one embodiment of the present of invention in territory, the source dimension target image in addition.Show target image 900 with target area of being defined by object boundary Ω 902.Mixed regional T 906 is defined by mixed border T 904 in inside, is externally defined by object boundary Ω 902.The interpolater that is instructed can be provided, to provide a certain amount of mixed among the regional T between object boundary Ω 902 and mixed border T 904:
min
f∫∫
Ω|f-v|
2+α∫∫
T|f-f
*|
2,
It has boundary condition f|
Ω=f
*|
ΩThe big I of mixed effect is controlled by weighting parameters α in this zone.
Be used to realize the example hardware of Figure 10 of the present invention and the universal computing device that operating environment comprises computing machine 20 forms, comprise processing unit 21, system storage 22 and the various system components that are used for comprising system storage are coupled to the system bus 23 of processing unit 21.Can only have one or an above processing unit 21 is arranged, thereby the processor of computing machine 20 comprises single CPU (central processing unit) (CPU), or a plurality of processing units of so-called parallel processing environment.Computing machine 20 can be the computing machine of conventional computing machine, distributed computer or any other type; The present invention is not limited by this type of.
System bus 23 can be any one in the various bus structure, any the local bus that comprises memory bus or Memory Controller, peripheral bus and use various bus architectures.System storage also can abbreviate storer as, and comprises ROM (read-only memory) (ROM) 24 and random-access memory (ram) 25.Comprise such as when starting, helping basic input/output (BIOS) 26 to be stored among the ROM 24 in that personal computer 20 inner each interelement transmit the basic routine of information.Computing machine 20 also comprises the hard disk drive 27 that is used for reading or writing the hard disk (not shown), be used for reading or writing the disc driver 28 of moveable magnetic disc 29 and be used for reading or writing CD drive 30 such as removable CDs 31 such as CD ROM or other light media.
Hard disk drive 27, disc driver 28 and CD drive 30 are linked system bus 23 by hard disk drive interface 32, disk drive interface 33 and CD drive interface 34 respectively.These drivers and the computer-readable medium that is associated thereof provide the non-volatile memories of computer-readable instruction, data structure, program module and other data for computing machine 20.It will be appreciated by those skilled in the art that such as any storage such as tape cassete, flash card, digital video disc, Bei Nuli magnetic tape cassette, random-access memory (ram), ROM (read-only memory) (ROM) and can also can be used in this exemplary operation environment by the computer-readable medium of other type of the data of computer access.
The some program modules that comprise operating system 35, one or more application program 36, other program module 37 and routine data 38 can be stored on hard disk, disk 29, CD 31, ROM 24 or the RAM 25.The user can be by coming input command and information in personal computer 20 such as input equipments such as keyboard 40 and positioning equipments 42.Other input equipment (not shown) can comprise microphone, operating rod, game mat, satellite dish, scanner or the like.These and other input equipment normal open overcoupling is linked processing unit 21 to the serial port interface 46 of system bus, but also availablely waits other interface to connect such as parallel port, game port or USB (universal serial bus) (USB).The display device of monitor 47 or other type is also via linking system bus 23 such as interfaces such as video adapters 48.Except that monitor, computing machine generally includes such as other peripheral output device (not shown) such as loudspeaker and printers.
Personal computer 20 can be used logic such as one or more remote computers such as remote computers 49 and be connected in the networked environment and move.These logics connections are by being coupled to computing machine 20 or realizing as its a part of communication facilities.Remote computer 49 can be another computing machine, server, router, network PC, client computer, peer device or other common network node, and generally include above-mentioned with respect to personal computer 20 described many or whole elements, although in Figure 10, only show memory storage devices 50.The logic of describing among Figure 10 connects and comprises Local Area Network 51 and wide area network (WAN) 52.This type of network environment is common in office, enterprise-wide. computer networks, Intranet and the Internet, and they are the type of network.
When being used for the lan network environment, computing machine 20 is linked LAN (Local Area Network) 51 by the network interface or the adapter 53 that belong to a class communication facilities.When being used for the WAN network environment, computing machine 20 generally includes the modulator-demodular unit 54 that belongs to a class communication facilities or is used for setting up any other device of communication by wide area network 52.Modulator-demodular unit 54 can be internal or external, and it links system bus 23 via serial port interface 46.In networked environment, program module or its part described with respect to personal computer 20 can be stored in the remote memory storage devices.Should be appreciated that it is exemplary that the network that illustrates connects, and also can use other device and the communication facilities of setting up communication link at intercomputer.
In one embodiment of the invention, select module, territory definition module, blended computation module and output module can be merged into the part of operating system 35, application program 36 or other program module 37.Interpolation guide datastore and view data can be stored as routine data 38.
Each embodiment of the present invention described herein is implemented as the logic step in one or more computer systems.Logical operation of the present invention is implemented as: the sequence of steps that the processor that carry out in one or more computer systems (1) is realized, and the machine module of (2) one or more inside computer system interconnection.This realization is to depend on the selection problem that realizes that performance of computer systems of the present invention requires.Therefore, the logical operation of forming various embodiments of the present invention described herein is known as operation, step, object or module under different occasions.
Above instructions, example and data provide the structure of exemplary embodiment of the present and the complete description of use.Can not depart from spirit of the present invention and category because making many embodiments of the invention, so the present invention resides in the appending claims.