CN104834459A

CN104834459A - Providing drawing assistance using feature detection and semantic labeling

Info

Publication number: CN104834459A
Application number: CN201510030165.8A
Authority: CN
Inventors: H·温尼莫勒; 谢君; W·W-M·李; A·P·赫兹曼
Original assignee: Adobe Systems Inc
Current assignee: Adobe Inc
Priority date: 2014-02-07
Filing date: 2015-01-21
Publication date: 2015-08-12
Anticipated expiration: 2035-01-21
Also published as: GB2524871A; GB201501968D0; DE102015000377A1; GB2524871B; CN104834459B

Abstract

All embodiments of the invention relate to provide drawing assistance using feature detection and semantic labeling. The Method for providing drawing assistance to a user sketching an image include geometrically correcting adjusting user strokes to improve their placement and appearance. In particular, one or more guidance maps indicate where the user ''should'' draw lines. As a user draws a stroke, the stroke is geometrically corrected by moving the stroke toward a portion of the guidance maps corresponding to the feature of the image the user is intending to draw based feature labels. To further improve the user drawn lines, parametric adjustments are optionally made to the geometrically-corrected stroke to emphasize ''correctly'' drawn lines and de-emphasize ''incorrectly'' drawn lines.

Description

Feature detection and semantic tagger is used to provide drawing auxiliary

the cross reference of related application

The application be by reference content is all incorporated into this, be filed on February 3rd, 2014 and name be called " Geometrically and Parametrically Modifying User Inputto Assist Drawing ", the 14/171st, the part continuation application of No. 760 U.S. Patent applications.

Technical field

One or more embodiment of the present invention relates generally to provides drawing auxiliary to user.More specifically, one or more embodiment of the present invention relates to based on semantic tagger information adjustment user stroke with the system and method for aided drawing.

Background technology

Drawing is a kind of important communication and expression-form.Regrettably, drawing new hand is sometimes irresolute to drawing, because they feel that they lack creation for generation of the works being satisfied with visual quality and technical skills.Drawing auxiliary routine is attempted attempting creating when high-quality is drawn drawing new hand assisting them.Conventional drawing auxiliary routine often provides guidance mode.Although individual coaching can help user to produce the drawing of improvement, the pattern of their frequent minimizings or minimum user.In addition, individual coaching, visual cues and other interaction paradigm may make user feel less entitlement and the individual satisfaction to gained works.

In order to provide auxiliary and the drawing of correcting user, following guidance may be favourable, and this guidance is such as the result of edge detection filter, and this edge detection filter provides user should with the idea should not drawing what lines.But the result of edge detection filter presents various shortcoming, these shortcomings hinder them to become effective guidance of the drawing for correcting user.Such as, artist has drafting lines and selects and leave some features wittingly and do not draw to create the degree of depth and perspective.In contrast, edge detection filter produces sufficient edge usually.Therefore, the result of edge detection filter often teach for being used as effectively to draw instruct for have very much noise and inconsistent.

Along relevant lines, when image has the position of high pixel density, edge detection filter may produce the region of high rim concentration degree.Attempt during depicted features, may being difficult to or can not determining which edge should be used as the guidance for user's stroke in such region user.Like this, conventional drawing auxiliary routine may use an edge with a part for correcting user stroke, and uses another edge with another part of correcting user stroke.The possibility of result of such correction causes the drawing of distortion.

New hand it has been generally acknowledged that they do not have suitable drawing skill level without creativity or they.Drawing new hand is often total makes them lack confidence and the challenge of the low-qualityer works of generation.These challenges can comprise the understanding etc. lacked light, the degree of depth, perspective and the element relation in drawing.Overcome these challenges because new hand is unable, so the frequent quality defect of the drawing created by new hand, this makes new hand be unwilling to use draw conduct interchange and expression-form.

The challenge that drawing new hand faces how to draw to it seems that artistic lines are to produce high quality results.New hand does not often be sure of the placement of lines, shape and curvature.This does not be sure of to make new hand draw lentamente along the many corrections of single stroke.Such stroke usually seems uneven and otherwise lacks the dynamic appearance performed by the speed controlled and curvature of consummate artistical stroke.

Another challenge for drawing new hand is that they draw the content of they " known " instead of content that they " see " at large.Such as, when describing face, both eyes and face are often plotted as closed almond shape by drawing new hand.This is formed with housebroken artist and contrasts, and these artists are often emphasized and omit other lines to pass on CONSTRUCTED SPECIFICATION and lighting effect on some lines.The expression that drawing new hand is unable sees and pass on profile and lighting effect often to cause the simplification of object and produce low-qualityer net result.

Use the drawing of computing machine, flat computer and mobile device and describe to become increased popularity.Such equipment provides to comprise and is easy to obtain and wieldy many advantages.Regrettably, computing equipment may aggravate drawing new hand facing challenges when drawing.Such as, use low fidelity input equipment (such as touch screen or touch panel) may make the mistake placements, offset or uneven lines.In fact, " slightly point " even if problem may make more experienced artist also be difficult to draw in " correctly " place.

These and other shortcoming may exist about using the drawing of touch screen or other low fidelity device especially.

Summary of the invention

Embodiments of the invention provide benefit with following system and method and/or overcome one or more problem in the aforementioned or other problem in this area, and these system and methods use semantic tagger to provide intelligent drawing to assist.Especially, in one or more embodiment, drawing backup system detects the feature of the image that user is drawing.When user draws stroke, move stroke by the feature of image intending to draw towards user and carry out geometrically correcting user stroke.

In one or more embodiment, drawing backup system and method use one or more to instruct mapping (guidance map) as the basis being used for correcting user stroke.Instruct and map that wherein and how can provide should the instruction of the feature of drawing image to user.In certain embodiments, such instruction can be explicit, as by suitably visual (such as, user can trace (trace) on guidance maps) that map guidance.In other embodiments, this instruction can be implicit expression.In other words, user does not see that guidance maps.Replace, when user draws, as described below with user's ND mode correcting user stroke to aim at instructing the feature that maps.Additionally, instructing one or more guidance in mapping to map can by the position using semantic label indicate the feature detected in the picture.Semantic label can help to ensure to intend the feature of drawing instead of feature mobile subscriber stroke near another towards user during trimming process.

In order to improve the stroke of user further, alternatively, drawing backup system and method can provide parameter adjustment alternatively.Lines that parameter adjustment is emphasized " correctly " draws and de-emphasize the lines that (de-emphasize) " improperly " draw.Additionally, parameter adjustment can pass on CONSTRUCTED SPECIFICATION and lighting effect.

System and method disclosed herein can not allow almost not or do not have skilled user to create the pleasant drawing of vision.Especially, the system and method for one or more embodiment can help the lines of correcting user due to rawness, do not be sure of and/or use the geometry caused by low fidelity input equipment to place.In one or more embodiment, system and method disclosed herein can provide such benefit and maintain the individual pattern of user to produce drawing, and these draw helps provide the individual satisfaction of entitlement sense and high level to user.

To set forth in the following description and partly from by clear this description or supplementary features and the advantage of exemplary embodiment of the present invention can be understood by putting into practice such exemplary embodiment.Can realize by the means pointed out especially in the following claims and combination and obtain the feature and advantage of such embodiment.These and other feature is known becoming from the following description and the appended claims more completely or can be understood by the exemplary embodiment of practice as hereafter set forth.Foregoing summary is not summarize widely, and it is not intended to mark key element or indicates scope of the present invention.In fact, the aspect of embodiments of the invention is designated the preorder of the following embodiment presented by foregoing summary.

Accompanying drawing explanation

In order to describe can obtain of the present invention above that record with other advantage and the mode of feature, by show by referring to illustrated specific embodiments of the invention in the accompanying drawings above describe briefly of the present inventionly more particularly to describe.It should be noted that each figure not drawn on scale and the unit of analog structure or function generally runs through each figure for exemplary object is represented by similar label.These accompanying drawings should be understood and only describe exemplary embodiments of the present invention and the scope that therefore can not be regarded as limiting it, will describe by using accompanying drawing feature and details and the present invention is described, in the accompanying drawings:

Fig. 1 illustrates the drawing backup system according to one or more embodiment of the present invention;

Fig. 2 A illustrates the computing equipment of image of will draw on it according to the display user of one or more embodiment of the present invention;

The computing equipment of Fig. 2 A that the guidance that Fig. 2 B illustrates the image of the display Fig. 2 A according to one or more embodiment of the present invention maps;

The computing equipment of Fig. 2 A that the feature guidance that Fig. 2 C illustrates the image of the display Fig. 2 A according to one or more embodiment of the present invention maps, this feature instructs the feature tag mapping and have the different characteristic that instruction detects in the picture;

Fig. 2 D illustrate according to one or more embodiment of the present invention with the computing equipment of Fig. 2 A indicating the guidance showing Fig. 2 B together with the feature tag of different characteristic that detects in the picture to map;

Fig. 2 E illustrates the computing equipment of Fig. 2 A of two user's strokes of drawing on the image being presented at Fig. 2 A according to one or more embodiment of the present invention;

Fig. 2 F illustrates the level and smooth version of one of the user's stroke of Fig. 2 E according to one or more embodiment of the present invention and this user's stroke;

Fig. 2 G illustrates according to the computing equipment of Fig. 2 A of one or more embodiment of the present invention and image and two strokes geometrically corrected;

The version that Fig. 2 H adjusts with illustrating the parameter of one of the stroke geometrically corrected of Fig. 2 G according to one or more embodiment of the present invention;

Fig. 3 A illustrates and maps according to another guidance of the image of one or more embodiment Fig. 2 A of the present invention;

Fig. 3 B illustrates artist's sketch of the image of Fig. 2 A according to one or more embodiment of the present invention;

Fig. 4 A illustrates another image will drawn on it according to the user of one or more embodiment of the present invention;

The sketch that the user that Fig. 4 B illustrates the image of Fig. 4 A according to one or more embodiment of the present invention draws;

Fig. 4 C illustrates the sketch drawn according to the user adjusting Fig. 4 A with passing through geometrically correction and parameter of one or more embodiment of the present invention and the output formed is drawn;

The output that Fig. 5 A illustrates Fig. 4 C is drawn;

Fig. 5 B illustrates another output formed according to the adjusting user's stroke with passing through geometrically correction and parameter of one or more embodiment of the present invention and draws;

Fig. 5 C illustrate according to one or more embodiment of the present invention by geometrically with parameter adjust user's stroke and another the outputs drawing that formed;

Fig. 6 A illustrates the image of Fig. 4 A;

Fig. 6 B illustrates the image of Fig. 6 A according to one or more embodiment of the present invention, and this image has the feature tag of the different characteristic that instruction detects in the picture;

The guidance that Fig. 6 C illustrates the image of Fig. 4 A according to one or more embodiment of the present invention maps;

Fig. 6 D illustrates and maps according to the guidance of Fig. 6 C of one or more embodiment of the present invention, and this guidance maps the feature tag with the different characteristic that instruction detects in the picture;

Fig. 7 illustrate according to one or more embodiment of the present invention by use semantic tagger geometrically correcting user stroke the process flow diagram of a series of actions of drawing in auxiliary method is provided;

Fig. 8 illustrate according to one or more embodiment of the present invention by use semantic tagger geometrically correcting user stroke the process flow diagram of a series of actions of drawing in auxiliary other method is provided; And

Fig. 9 illustrates the block diagram of the example calculation equipment according to one or more embodiment of the present invention.

Embodiment

One or more embodiment of the present invention comprises the drawing backup system that a kind of user of help creates original drawing or sketch.Especially, drawing backup system provides as hypograph, and user can draw in face on this image.When user draws stroke, drawing backup system moves stroke by the feature of image intending to draw towards user and geometrically corrects stroke.Drawing backup system can use one or more to instruct to map as the benchmark of correcting user stroke or guidance.Instruct and map to provide wherein and/or how should place their instruction of stroke to user.

In order to help to ensure that correctly correcting user stroke (namely, the feature of the image drawn instead of feature mobile subscriber stroke near another is intended towards user), at least one instructing in mapping instructs the semanteme or the feature tag that map and can comprise the position of one or more feature of indicating image.When user draws stroke, drawing backup system can detect user and intend to draw which feature, and then uses the feature of plan to correct stroke as guidance.Therefore, even if semantic or feature mark can allow system and method still to know to make what geometry adjustment when user is drawing the intensive position of feature.Like this, one or more embodiment can provide intelligent drawing to assist, the intention of this intelligent drawing aid identification user and the stroke of correspondingly correcting user.

System and method disclosed herein can not allow almost not or do not have skilled user to create the pleasant drawing of vision.As described above, system and method disclosed herein can make geometry correction to the stroke of user.As used herein, term " geometry correction " or " geometrically correct " refer to towards the stroke such as instructing " correctly " of mapping definition location and shape to draw user by one or more or lines (or its part) and move, reorientate and/or be again shaped.Such as, geometry correction can to due to lack control or do not be sure of and mistake place and/or mistake be shaped user draw stroke or lines move, reorientate and/or be again shaped.In addition, geometry correction can aim at by the mistake compensated caused by " slightly pointing " problem the consummate artist assisting and use low fidelity input equipment (such as touch screen, mouse or touch panel).

System and method disclosed herein can provide geometry correction and also maintain the Style Attributes of original stroke of user.Especially, although the stroke of geometrically correcting user, the pattern that still can maintain original stroke of user is at least in part selected or characteristic (such as stroke quality, length, shape and curvature).Maintain the individual pattern of user can help to provide entitlement sense and to exporting the correction of drawing and higher-qualityly exporting the high level individual satisfaction of drawing for creating to user.

Except the stroke of geometrically correcting user, one or more embodiment of the present invention can help the another way of the drawing providing vision pleasant to be adjust the stroke of user by parameter further.As used herein, whether term " parameter adjustment " or " parameter ground adjustment " refer to based on as instructed " correctly " mapping definition to draw by one or more or " improperly " drafting lines are emphasized or de-emphasize them.Parameter adjustment can help reduce their counter productive by alleviating or otherwise de-emphasizing the lines that mistake is drawn or mistake is placed.Along relevant lines, the lines that parameter adjustment correctly can be drawn by darkening or the opacity increasing them help emphasize them.Additionally, parameter adjustment can help to pass on the degree of depth, CONSTRUCTED SPECIFICATION and lighting effect.

In order to help to provide for exporting the entitlement sense of drawing to user, in one or more embodiment, system revises each input stroke when user draws stroke.In other words, when user draws stroke, system does not show user's stroke, but in fact, stroke that is that system display geometrically corrects and that adjust to parameter alternatively.In such embodiments, to correct and adjustment process is ND for user, this can reduce the cognition of user to the quantity of any geometry correction and parameter adjustment.In an alternative embodiment, user can check when drawing input stroke that they are to obtain the correction that performs system and the view of adjustment immediately.Additionally or alternatively, system can be provided for checking to user the option that the input of original stroke is drawn, and draws to draw with the output be made up of the stroke geometrically corrected with adjust to parameter alternatively can compare input.

Along phase liny, in trimming process in the ND embodiment of user, not to user provide and on rear end instruction map with correcting user stroke.In other words, user can not check to instruct and maps.Alternatively, system can be provided for checking the option instructing mapping to user, can compare, input is drawn, output is drawn and guidance maps.

Fig. 1 illustrates the embodiment of illustrative plot backup system 100.Drawing backup system 100 can include but not limited to instruct mapping generator 101, property detector 102, re-sampler 104, smoother 106, geometrical corrector 108, parameter regulator 110 and memory module 112.Each parts in the parts 101-112 of drawing backup system can use any suitable communication technology to intercom mutually.As illustrated by fig. 1, drawing backup system 100 can comprise one or more computing equipment (that is, client device 118, server apparatus 120).

To recognize, although the parts 101-112 of drawing backup system in FIG 100 is shown as separation, any parts in parts 101-112 can be combined into less parts (being such as combined into single parts) as served specific embodiment or be divided into more multi-part.Parts 101-112 can comprise software, hardware or the two.Such as, parts 101-112 can be included in computer-readable recording medium stores and one or more instruction that can be performed by the processor of one or more computing equipment (such as, client device 118 and/or server apparatus 120).When being performed by one or more processor, the computer executable instructions of drawing backup system 100 can make computing equipment 118,120 perform drawing householder method described herein.Alternatively, parts 101-112 can comprise the hardware for performing certain function or one group of function, such as dedicated treatment facility.Additionally or alternatively, parts 101-112 can comprise the combination of computer executable instructions and hardware.

In addition, the parts 101-112 of drawing backup system 100 can such as be implemented as independent application, the module for application, the plug-in unit for the application for comprising image procossing application, for one or more built-in function that (such as image procossing application) call can be applied by other and/or be cloud computing model.Therefore, the parts of drawing backup system 100 may be implemented as independent application, such as desktop type or Mobile solution.Alternatively or additionally, the parts of drawing backup system 100 may be implemented within and include but not limited in any image procossing application of ADOBEPHOTOSHOP, ADOBE PHOTOSHOP ELEMENTS and ADOBEILLUSTRATOR." ADOBE ", " PHOTOSHOP ", " ELEMENTS " and " ILLUSTRATOR " are the registered trademark of Adobe Systems Incorporated in the U.S. and/or other country or trade mark.

Fig. 2 A to Fig. 2 H and provide the general introduction of the drawing householder method that drawing backup system 100 can perform about describing.Fig. 2 A one or more drawing accessory illustrated in drawing backup system 100 or drawing accessory 101-112 may be implemented within the client device 202 on it.In other words, client device 202 is examples for the computing equipment of the part that can form drawing backup system 100.In embodiment in fig. 2, client device 202 comprises the flat computer with touch screen 204.The flat computer with touch screen 204 is only can as the part of drawing backup system 100 by the client device 118 of a type used.Such as, in an alternative embodiment, client device 202 can comprise the dissimilar computing equipment of any number, such as referring to the computing equipment that Fig. 8 describes.

In order to assisted user and the threat reducing the blank page, drawing backup system 100 can provide image 206 in user interface 207.Drawing backup system 100 can provide image 206 by instructing client end equipment 202 via display device (such as, touch screen 204) display image 206.Image 206 can comprise the image that user selects from the image library 114 of memory module 112.Image library 114 can comprise multiple images that user can select to draw.Additionally or alternatively, user can such as drive from this locality, the Internet or be implemented the camera importing/upload images of equipment thereon from drawing backup system 100.When user imports or otherwise select the image 206 do not preserved in image library 114, drawing backup system 100 can automatically add it for using in the future to image library 114.

As illustrated by fig. 1, comprise image library 114 and instruct the memory module 112 of mapping library 116 completely or partially can be positioned at client device 118 place.Alternatively, comprise image library 114 and instruct the memory module 112 of mapping library 116 can be positioned at server apparatus 120 place.In such embodiments, client device 118 can communicate to fetch image 206 or instruct to map with server apparatus 120 via network.

Image 206 shown in Fig. 2 A is faces of performer.Challenge is often drawn and important uniquely in face-to-face communication by face.Like this, with reference to the user's drawing correcting face, example embodiment of the present invention is described here.The drawing that the invention is not restricted to correct face will be recognized according to disclosure herein content.In fact, drawing backup system 100 can draw any in fact image with additional semantic information by assisted user.

As mentioned previously, drawing backup system 100 can use one or more to instruct mapping as the guidance for correcting user stroke.The guidance that drawing backup system 100 can produce or fetch for image 206 maps.As used herein, term " instructs and maps " and refers to following data, the idealized version of one or more part of this data representative image.Instruct the benchmark mapping and be provided for geometry correction and parameter adjustment.Instruct playing up of point, lines or the edge mapping one or more part that can comprise representative image.Alternatively, mapping is instructed can to comprise collecting of data point.In addition, in one or more embodiment, instruct mapping to comprise semantic tagger, this semantic tagger is provided for the context instructing the part (such as, point, lines or edge) mapped.Under any circumstance, during geometry correction, drawing backup system 100 to or towards the stroke instructing the lines that map or edge to move (such as, reorientate and/or be again shaped) user.In other words, instruct mapping can indicating user " should " draw what lines and user " should " draw lines wherein.The exemplary public opinion instructing the user of result, artist's sketch of image, the result of feature detection algorithm and the image 206 mapping the edge detection filter comprised to image applications to draw.

Instruct mapping generator 101 can produce one or more from the image 206 selected and instruct mapping.Such as, the edge that Fig. 2 B illustrates the image 206 of Fig. 2 A instructs an embodiment of mapping 208.Instructing mapping 208 to produce edge, instructing mapping generator 101 then edge detection filter can be run to the image 206 of Fig. 2 A.Edge instructs mapping 208 to provide the stroke of user geometrically to be corrected and/or the adjustment (such as, lines or shape) that is adjusted to of parameter ground.Alternatively, edge instruct map can comprise artist's sketch of image, public opinion that the user of image draws or another instruct, this another instruct one or more feature comprising indicating image 206 to be positioned at lines or the edge of where.

In one or more embodiment, instruct mapping generator 101 can run extended pattern Gauss difference (XDoG) wave filter with edge setting to image 206 and instruct mapping 208 to produce edge.At Holger deng the XDoG:An eXtendedDifference-of-Gaussians Compendium including Advanced Image of people, describing the suitable example of XDoG wave filter in 36 (6) STYLIZATION COMPUTERS & GRAPHICS 740-753 (2012), by quoting completely, the full content of the document being incorporated into this.Those skilled in the art will recognize that, other edge detection filter (such as Canny edge detector or Sobel detecting device) also can provide suitable edge to instruct mapping.But, as shown in by Fig. 2 B, have XDoG wave filter that edge arranges be applicable to producing with artist may draft plan as 206 time the edge that is similar to of the lines drawn instruct and map.Especially, edge instructs mapping 208 all features of lines representative image 206.Such as, as shown in by arrow 210, not used for the edge in the left side of the face of performer.Similarly, the edge not used for the part of the bottom of the eyes of performer in mapping 208 is instructed at edge.

One or more embodiment additional data of the present invention supplements edge and instructs mapping 208.Especially, drawing backup system 100 can be provided by the place instructing mapping 208 to omit from edge at edge or lines them to come supplementary edge and instruct mapping 208.Alternatively, drawing backup system 100 can provide another to instruct mapping (such as feature instructs and maps), and this another guidance maps the omission edge that mapping 208 is instructed in consideration.Therefore, additional guidance maps the basis that can be provided for correcting the user stroke corresponding with the feature instructing mapping 208 to omit from edge.

As shown in by Fig. 2 B, edge instructs mapping 208 to comprise intersection, intensive or merge the region at edge.Such as, the right side of nose, the profile of right eye and iris as all combined shown in arrow 212.Those skilled in the art may be difficult to know what geometry correction is the user's stroke in the region indicated by arrow 212 make, because feature is not defined by recognizing according to disclosure herein content.Such as, drawing backup system 100 can be split and mark edge with semantic markup information (that is, from the feature tag that feature detection process obtains) and instructs mapping 206.Such as, drawing backup system 100 can as following more specifically illustrate use markov random file, pattern zoning or nearest-neighbors searching algorithm with segmenting edge instruct map.

In order to instruct mapping 208 to provide context to edge, one or more embodiment comprises the method instructing the edge mapped with feature tag mark edge.Feature tag can allow drawing backup system 100 to reduce in the region that feature merges or otherwise intersects or disambiguation and correcting user stroke intelligently.

Especially, property detector 102 can analysis chart as 206 with the feature of identification image 206.Such as, property detector 102 can application image feature detection algorithm with the various features of identification image 206.Fig. 2 C illustrates the result of the face trackers algorithm applied to image 206.As shown in by Fig. 2 C, identify and be associated with the various features of image 206 with feature tag 211a-211f.Especially, property detector 102 be associated with the profile of face and feature tag 211a, the profile of eyes and feature tag 211b, iris profile and feature tag 211c, eyebrow and feature tag 211d, nose and feature tag 211e and mouth and feature is even removes 211f.

Use one or more feature in the feature identified by property detector 102, the feature instructing mapping generator 101 can produce or fetch for image 206 instructs mapping 213.As used herein, term " feature instructs and maps " refers to following point and/or lines, and these points and/or lines identify the feature of the image detected by property detector 102.Feature instructs playing up of point, lines or the edge mapping one or more part that can comprise representative image.Alternatively, feature instructs to map and can comprise collecting of data point.In one or more embodiment, feature instructs mapping 213 to get rid of the feature of the image that property detector 102 does not identify.Such as, when property detector 102 comprises application face detection algorithms, feature instructs mapping 213 to comprise mark face, eyes, iris, eyebrow, the point of profile of nose and lip and/or lines, as shown in by Fig. 2 C.But the feature shown in Fig. 2 C instructs mapping 213 not comprise the mark hair of performer or the point of shirt and/or lines.Alternatively, feature instructs the feature mapping and only can comprise and instruct from edge and map and omit.Additionally, in an embodiment, property detector 102 can to the multiple detection algorithm of image applications.Such as, property detector 102 can also apply texture recognition algorithm except face detection algorithms.Texture recognition algorithm can identify the position (such as, shirt comparison is than skin comparison hair) of different texture.

In addition, property detector 102 is not limited to detect face feature.Such as, the feature of the image of property detector 102 scene that also can detect and mark or alternatively detect and mark except face or object.Such as, property detector 102 can detect the parts (such as, the car body of the spoke of bicycle, wheel and framework or automobile, window and tire) of landscape (mountain range, trees, lake, sky, water body, street, building etc.) or object.In addition, property detector 102 also can comprise gesture detector, clothing detecting device, people's detecting device or any other and can be provided for available detection algorithm or the parts of the data/feature identification of semantic tagger.

Feature instructs mapping 213 mapping can be instructed to provide supplementary to another when being provided for the benchmark of geometry correction.Such as, during the edge instructing mapping 208 not comprise for the part of feature or feature at edge, feature instructs mapping 213 can be used for supplementary edge and instructs mapping 208.Such as, Fig. 2 D illustrates edge and instructs the edge mapping and omit for the left side of the face of performer, as shown in by arrow 222.As further described below, during geometry correction, drawing backup system 100 can to or instruct mobile (such as, reorientate and/or the be again shaped) user of the lines 214 (see Fig. 2 C) corresponding with the left side of the face of performer of mapping 213 to be intended as the stroke in the left side of the face of performer towards feature.

Under any circumstance, once property detector 102 has identified the different characteristic of image 206, instruct mapping generator 101 just linked character label 211a-211f and edge can instruct the edge corresponding with the feature of mark of mapping 208.Such as, Fig. 2 D illustrates the feature tag 211a-211f instructing the edge corresponding with the feature of image 206 of mapping 208 to associate with edge.Specifically, mapping generator 101 linked character label 211a and edge is instructed to instruct the edge of the profile of the formation face of mapping 208, linked character label 211b and form the edge of profile of eyes, linked character label 211c and form the edge of profile of iris, linked character label 211d and form the edge of eyebrow, linked character label 211e and form the edge of nose and linked character label 211f and the edge forming mouth.

Referring now to Fig. 2 E, user can use input equipment (that is, touch screen 104) to draw user's stroke 216,218.Such as, user can offset to press with touch screen 204 and point or stylus and along touch screen 204 moveable finger or stylus with the feature of drawing image 206.Drawing backup system 100 can detect user's stroke 216,218.As used herein, term " stroke " refers to that lines at least partially.Such as, stroke can comprise the whole lines of the starting point contacted with input equipment by user, end point that user departs from from input equipment and the path definition between starting point and end point.Alternatively, stroke only can comprise a part for the lines of the starting point contacted with input equipment by user, end point that user departs from from input equipment and the path definition between starting point and end point.

Fig. 2 E also illustrates user interface 207 can comprise one or more control or option 220.In one or more embodiment, user interface 207 can comprise the option 220a for using as the original stroke inputted by user.Additionally, user interface 207 can comprise the option 220b for level and smooth user's stroke.When selecting to be used for the option 220b of level and smooth user's stroke, re-sampler 104 can resampling user stroke 216 to determine the point defining stroke 216.As shown in by Fig. 2 F, smoother 106 then can by carrying out level and smooth user's stroke 216 to create level and smooth user's stroke 217 to the some matching lines determined during resampling.

One or more embodiment of the present invention can provide multiple to user pattern or drawing option otherwise.Especially, drawing backup system can provide various stroke/hard-tipped pen/brush pattern.Such as, Fig. 2 E illustrates exemplary stroke option 220c-220f.User can select ink 220c, the brush 220d of the stroke of drawing on image 206 for user, hard-tipped pen 220e or gradually thin 220f.Use stroke option 220 can provide the another kind of mode for their drawing of user individual.To recognize, the present invention is not limited to the stroke option 220c-220f shown in Fig. 2 C.Drawing backup system 100 can provide additional or alternative brush/stroke option or other image processing tool, the image processing tool such as provided by ADOBEPHOTOSHOP.

Drawing backup system 100 can determine that user intends which feature of drawing image 206.As shown in by Fig. 2 G, user intends the right side of stroke 216 as the face of the performer in image 206.Drawing backup system 100 can linked character label 211a and user's stroke 216.Similarly, drawing backup system 100 can linked character label 211a and user's stroke 218.

Drawing backup system 100 can compare user's stroke 216 and instruct mapping 208 to determine that edge instructs the edge 222 (see Fig. 2 B) that instructs recently of mapping 208 to have the feature tag 211a (i.e. feature tag 211a) identical with user's stroke to utilize with edge.As shown in by Fig. 2 G, geometrical corrector 108 can by towards instructing edge 222 mobile subscriber's stroke 216 to carry out geometrically correcting user stroke 216 to create the stroke 224 geometrically corrected.Especially, geometrical corrector 108 can towards or instruct the location of lines 222 (Fig. 2 B) of instructing of mapping 208 to move (such as, reorientate and/or be again shaped) user's stroke 216 to edge.

Drawing backup system 100 can compare user's stroke 218 and edge instruct mapping 208 with determine edge instruct mapping 208 there is the feature tag 211a mated with the feature tag of user's stroke (i.e. feature tag 211a), nearest with user's stroke 218 part (such as, instruct reflect edge) (see Fig. 2 B).In this case, nearest the instructing edge can refer to guide margin edge 222 or instruct edge 225 (see Fig. 2 B) with user's stroke with same characteristic features label 211a.When instructing both edges 222 and 225 away from user's stroke 218, drawing backup system 100 can use feature to instruct mapping 213 (Fig. 2 C) to supplement edge and instruct mapping 208.Specifically, drawing backup system 100 can determine that feature instructs the lines nearest with user's stroke 218 or the point of mapping 213, and these lines or point will be characteristic line 214 in this case.As shown in by Fig. 2 G, Geometry rectification device 108 can by generating towards characteristic line 214 mobile subscriber stroke 218 stroke 226 geometrically corrected.Especially, geometrical corrector 108 can towards or instruct the location of the characteristic line 214 (Fig. 2 C) of mapping 213 to move (such as, reorientate and/or be again shaped) user's stroke 218 to feature.

Instruct the alternative of mapping 208 as supplementary edge, feature instructs mapping 213 can be provided for separately the guidance of correcting user stroke.Especially, can towards feature instruct the part of mapping 213 (that is, lines, edge, point) mobile subscriber's stroke with instruct any correction correcting user stroke independently mapping (such as edge instructs mapping 208) based on another.Therefore, those skilled in the art will recognize according to disclosure herein content, can separately or instruct to map with other to use feature to instruct in combination to map.Especially, in one or more embodiment, when attempting correcting user stroke, drawing backup system 100 individually can check that each guidance maps.Then drawing backup system 100 can be used and determine which uses instruct the more senior decision mechanism (via voting scheme, weighting scheme etc.) mapped.Such as, when attempting correcting user stroke 218, drawing backup system 100 can determine that edge instructs mapping 208 not comprise neighbouring edge to be provided for the guidance of user's stroke 218.Similarly, drawing backup system 100 can determine that feature instructs mapping 213 to comprise neighbouring part to be provided for the guidance of user's stroke 218.Like this, drawing backup system 100 choice for use feature can instruct mapping 213 instead of edge to instruct mapping 208 when correcting user stroke 218.

To recognize according to disclosure herein content, the guidance of any number or combination maps the basis that can be provided for the stroke of correcting user.In addition, guidance mapping of the present invention is not limited to edge and instructs mapping and feature guidance mapping.In fact, embodiments of the invention are gone back or are replaced characteristic sum edge and instruct mapping can also comprise other guidance except characteristic sum edge instructs mapping.Such as, as more specifically illustrated referring to Fig. 3 A, one or more embodiment can use shadowed to instruct mapping.

More embodiments can comprise importance degree and instruct mapping.Importance degree instructs to map to indicate and will be given the region of the object of more or less weights by during trimming process.Such as, in the context of face, importance degree instructs to map and can indicate region around eyes, eyebrow and mouth than nose and lower jaw by weighting higher.Such importance degree instructs to map and then can to map and the similar mode of the mode that describes maps to aim at the image of actual face by using feature to instruct according to above to instruct about linked character label and edge.Under these circumstances, feature instructs mapping can define the spatial mappings of general face (and feature) to concrete face, comprises its spatial offset from template.

Geometrically correcting user stroke 216,218 time, geometrical corrector 108 can keep the pattern element of user's stroke 216,218 or lines change.Such as, as shown in by Fig. 2 F, user's stroke 216 comprises input change 216a and 216b.The stroke 224 geometrically corrected can comprise change 224a, 224b based on input change 216a and 216b as shown in fig. 2h.In other words, the stroke 224 geometrically corrected accurately is not aimed at instructing lines 222 due to the reservation of the change to user's stroke 216.Can help to ensure that each drawing is unique to the reservation of the pattern of user, because do not have two users will provide same subscriber stroke 216.

In order to improve the stroke that user has drawn further, alternatively, parameter adjuster 110 can adjust the stroke 224 that geometrically corrects with the lines 228 adjusted with creating parameter, as shown in by Fig. 2 H in parameter ground.When execution parameter adjusts, parameter regulator 110 emphasizes the lines being regarded as having " correctly " lines characteristic.Such as, analyzed in order to parameter adjustment lines characteristic can comprise lines location (such as, locating), orientation, shape or other lines characteristic any.Additionally, when execution parameter adjustment, parameter regulator 110 de-emphasizes the lines being regarded as having " incorrect " lines characteristic.

More specifically, what the lines 224 that more geometrically correct of parameter regulator 110 and edge instructed mapping 208 instructs lines 222.When comparing, parameter regulator 110 detect the stroke 224 geometrically corrected from change 224a, 224b of instructing lines 222 to depart from.Parameter regulator 110 also detect the lines 224 geometrically corrected with instruct lines 222 to aim at or the part 224c of substantial registration.Parameter regulator 110 then can emphasize the lines geometrically corrected with the part 224c instructing lines 222 good alignment.Such as, Fig. 2 H part 228c corresponding with the part 224c of the lines 224 geometrically corrected of showing the lines 228 adjusted with emphasizing parameter.Similarly, parameter regulator 110 de-emphasize not with the lines instructing lines 222 to aim at or its part 224a, 224b.Such as, Fig. 2 H shows part 228a, the 228b corresponding with part 224a, the 224b of the stroke 224 geometrically corrected that de-emphasize the lines 228 geometrically adjusted.

Similarly, the stroke 226 that parameter regulator 110 more geometrically corrects instructs mapping 208 with edge.In doing so, parameter regulator 110 can determine to instruct at edge the edge of not corresponding with the stroke 226 geometrically corrected (that is, close) in mapping 208.Like this, user's stroke 218 and the stroke geometrically corrected 226 can be identified as and be placed by mistake by geometry adjuster 110.Therefore, parameter regulator 110 de-emphasizes the stroke 226 geometrically corrected.

Parameter regulator 110 can use one or more technology in multiple technology to provide and emphasize.Emphasize to provide, parameter regulator 110 can revise one or more visible characteristic of the stroke 224,226 geometrically corrected.Such as, drawing backup system 100 can revise line roughness, lines opacity, line density and/or line color.To recognize according to disclosure herein content, parameter adjustment can help to consider illumination, the degree of depth, sketch the contours more senior pattern effect with other.

Especially, when using advanced tutorials to map, parameter adjustment can help in the stroke 224,226 geometrically corrected, be incorporated to the senior pattern element instructing and map.Such as, the part that edge instructs mapping 208 to show eyes and lower jaw is not intactly sketched the contours to provide artistic effect.As edge instructs mapping 208 indicates, even if the stroke of user is aimed at characteristics of image, this does not still mean that and should depict lines.Therefore, when user draws the stroke not instructing mapping 208 to comprise at edge, drawing backup system can by stroke recognition for being placed by mistake or otherwise being drawn improperly.Like this, when making parameter adjustment, parameter adjuster 110 can de-emphasize any lines (such as stroke 218) not instructing mapping 208 to comprise at edge.In this way, drawing backup system 100 can produce there is new hand and do not draw at large illumination, the degree of depth, sketch the contours the drawing with other Advanced Effect.Additionally, the effect that drawing backup system 100 minimum user can be helped to draw content that they " know " is such, this is different from the content of in reference picture 206 " description ".

The each stroke can drawn for user repeats the above geometry correction about user's stroke 216 and 218 description and parameter adjustment.In addition, can substantially occur in real time each correction/adjustment in the correction/adjustment of user's stroke 216,218.In other words, drawing backup system corrects and/or adjusts them when can work as draw/play up user on client device 202 stroke 216,218.As mentioned previously, bearing calibration and process is allowed can to reduce user to the cognition corrected cause user to draw output map to feel like this for user is ND.

Except the lines sketched the contours are drawn in correction or adjustment, one or more embodiment also can be revised or correct shadowed or fill lines.Especially, drawing backup system can provide shadowed to instruct mapping as the benchmark for correcting shadowed lines.Show the example that suitable shadowed instructs mapping 308 in figure 3 a.The shadowed of Fig. 3 A instructs mapping 308 to be apply the result with the XDoG wave filter that luminosity response is arranged to the image 206 of Fig. 2 A.As shown, shadowed instructs mapping 308 can provide instruction to the location and configuration that can accept shadowed.Especially, Fig. 3 B illustrates artist's sketch 310 of the image 206 of Fig. 2 A.As instructed shown in mapping 308 by comparison arts man sketch 310 and shadowed, shadowed instructs mapping 308 can comprise the region 312 of the shadowed similar to the region 314 of the shadowed of artist's sketch 310.

Similar to geometry correction described above, the shadowed stroke that mapping 308 can be instructed to move (such as, reorientate and be again shaped) drawn by user based on shadowed.To recognize according to disclosure herein content, the definite layout of shadowed lines is with to sketch the contours the important degree of lines lower.Total darkness (combination as line density and thickness) can be controlled to ensure appropriate shadowed.In one example, drawing backup system 100 uses XDoG skirt response to instruct mapping 308 to retrain shadowed stroke in reasonable border, and backup system 100 of drawing uses shadowed to instruct mapping 308 to move stroke and amendment rugosity and/or opacity partly to realize the gross density of wishing.

In order to distinguish sketching the contours lines and shadowed or fill between lines, user interface 207 can comprise shadowed option 316 as shown by figure 3 a.When non-selected shadowed option 316, the stroke of user is regarded as sketching the contours lines and is corrected as described above.When selecting shadowed option 316, the stroke of user is considered as shadowed or filling and uses the guide data of amendment and method of adjustment and process to be corrected.

To recognize according to disclosure herein content, user's stroke can be beautified, corrects, aims at and be shifted to drawing backup system to represent source images more accurately.The quantity of the correction that drawing backup system performs and degree can depend on the accuracy of user's stroke.Fig. 4 A to Fig. 4 C illustrates the validity of drawing backup system described herein and method.Especially, Fig. 4 A illustrates source images 406.The input that Fig. 4 B illustrates user's stroke of being drawn on image 406 by user draws 410.Fig. 4 C illustrate by geometrically correct and parameter adjust input drawing 410 user's stroke and be created output drawing 416.As by comparing input drawing 410 and exporting drawing 416 and being illustrated, geometrically to correct and the output of parameter ground adjustment is drawn and 416 can be provided comparatively for the remarkable improvement inputting drawing 410.

In addition, geometrically correct and parameter adjust the stroke of user time, drawing backup system 100 can maintain the pattern character of original stroke of user.Especially, although geometrically correction and parameter ground adjust the stroke of user, the pattern that also can maintain original stroke of user is at least in part selected or characteristic (such as stroke character, length, shape and curvature).The individual pattern maintaining user can leave the individual satisfaction of entitlement sense and the high level of drawing for output to user.

To recognize according to disclosure herein content, from the patterned sets of stroke maintaining user, stroke option 220 (Fig. 2 E) is even if can help to ensure that each drawing is based on different during identical image is also still in one or more.Such as, Fig. 5 A to Fig. 5 C illustrates three output drawing (that is, the drawing of geometrically correction and the adjustment of parameter ground) 416,516,518 of image 406.As shown in by Fig. 5 A to Fig. 5 C, each output drawing 416,516,518 provides pleasant drawing attractive in appearance and retains different pattern element.

Fig. 1 to Fig. 5 C provides some details of geometry correction about one or more embodiment of the present invention and parameter tuning process.Now the additional of each sub-processes or alternative details will be provided for.

Stroke resampling

When user draws stroke, drawing backup system 100 can resampling stroke.Stroke decomposition can be become to provide the point of the framework of stroke by resampling stroke.Then the stroke point of resampling is used as reference mark.Then drawing backup system 100 can use reference mark smoothly or geometrically to correct stroke.

In one or more embodiment, drawing backup system 100 is based on curvature and distance resampling user stroke.The sampling density that mode based on curvature can increase the sampling density in the segmentation of higher curvature and reduce in the segmentation of low curvature.Such as, circle can have many samplings (such as, 24) and straight line can by two sampling representatives.

Especially, stroke point x is imported into for each of ratio s _i, m ₁, m ₂, m ₃at x _ithe stroke point of new resampling before.(if with undefined) tolerance z _ibe greater than 1, then get x _ifor new sampled point: x _i← m _k+1.If tolerance z _ibe less than 1, then skip x _iand check x _i+1.Tolerance z _ibe defined as foloows:

z_{i} = \frac{t_{1} β_{i} Δ_{θi}}{2 π}

Wherein r is at x _iwith m _kbetween distance; And Δ _{θ i}at θ <x _i, m _k> and θ <m _k-1, m _kabsolute orientation difference between >.Threshold value t ₁and t ₂control the sampling density being used for curve and distance respectively.

In an alternative embodiment, other resampling technology can be used.Such as, other suitable resampling technology includes but not limited to even distance samples or time-sampling.

Stroke is level and smooth

As more early mentioned, user can select the option for level and smooth user's stroke.In an alternative embodiment, can automatically level and smooth all user's strokes.Under any circumstance, level and smooth user's stroke can reduce or eliminate uneven and uncertain user's stroke.Therefore, level and smooth user's stroke can improve the gross mass of user's stroke and export the gross mass of drawing.

In order to level and smooth user's stroke, curve approximation can be fit to the reference mark determined during stroke resampling process.Alternatively, curve approximation can be fit to the point geometrically corrected.The shape of stroke can be caught to point is smoothing and reduces uneven shake, as shown in by Fig. 2 D.In one or more embodiment, splines is applied to a little with level and smooth user's stroke.Such as, each stroke can be interpolated by carrying out matching with cubic B-Spline curve approximation to point.In an alternative embodiment, other smoothing technique can be used.Such as, other suitable smoothing technique can include but not limited to that running mean is level and smooth or Laplce is level and smooth.

Feature detection

To recognize according to disclosure herein content, feature detection process can depend on the image being selected drafting by user.Such as, when user selects to draw up the image of face, face detection algorithms can be served and be provided feature detection.By completely to quote full content is incorporated into this, the Deformable Model Fitting by RegularizedLandmark Mean-Shift of Jason M.Saragih, describe suitable face detection algorithms in 91INT ' L J.COMPUTER VISION 200-215 (2010).When image comprises the something except face, further feature detection algorithm can be served and be provided feature detection.Usually, feature detection algorithm can extract standard terrestrial reference about the different parts of image according to preset order.Additionally or alternatively, user manually can define the feature of image.

Fig. 6 A illustrates another image 606 of face.Fig. 6 B illustrates the feature tag 611a-611f determined by applying face detection algorithms to image 606.Especially, Fig. 6 B illustrates feature tag 611a and associates with the profile of face, and feature tag 611b associates with the profile of eyes, feature tag 611c associates with the profile of iris, feature tag 611d associates with eyebrow, and feature tag 611e associates with nose, and feature tag 611f associates with mouth.To recognize, feature detection and feature tag are not limited to feature detection shown in Fig. 6 B or other figure and feature tag.Such as, the feature on the left side of face can have the feature tag different from the feature on the right side of face.In addition, supplementary features (such as ear or hair) can comprise feature tag.

Instruct and map

As mentioned above, drawing backup system 100 one or more can be used to instruct map as user " should " draw what lines and " should " place the benchmark of lines wherein.Drawing backup system 100 can be fetched and be wished that the guidance of the previous generation of the image drawn up maps for user, or alternatively, drawing backup system 100 can generate to instruct and map.The exemplary result (see 208 of Fig. 2 B) instructing mapping to comprise the edge detection filter to image applications.

In one or more embodiment, instruct the result mapping and comprise to image applications XDoG wave filter.Fig. 6 C illustrates the XDoG wave filter using and have edge setting and another edge be created instructs mapping 608.As shown, edge instructs mapping 608 not comprise sufficient edge.In fact, edge instructs mapping 608 to comprise the edge corresponding with the selectional feature that artist can draw up.

With reference to Fig. 6 D, drawing backup system 100 can be split and mark edge with semantic markup information (that is, from the feature tag 611a-611f that feature detection process obtains) and instructs mapping 608.Such as, drawing backup system 100 can use markov random file, pattern zoning or nearest-neighbors searching algorithm to instruct mapping 608 with segmenting edge.

Such as, in one embodiment, once split the edge that edge instructs mapping 608, drawing backup system 100 has just performed nearest-neighbors search is used to guide edge each nearest unique point instructing point with mark.Then drawing backup system 100 can be determined for instructing the great majority at edge to instruct point and identify which feature tag (based on nearest unique point).Drawing backup system 100 is then for each guidance point instructed on edge or as a whole for instructing edge to associate the feature tag identified for great majority guidance point.As mentioned above, drawing backup system 100 can use feature instruct mapping edge instruct mapping 608 as user " should " draw what lines and " should " place the benchmark of lines wherein.Drawing backup system 100 separately or can instruct with another and map (such as edge instructs and maps) and use feature to instruct in combination to map.Drawing backup system 100 can use various ways or its combination to carry out generating feature and instruct mapping.Such as, because face has strong vertically symmetry, map to search edges matched on the either side of face/mappings or feature to increase discovery for the chance of useful data of user for correcting so drawing backup system 100 can use feature to instruct.Especially, once drawing backup system 100 has been split and marked edge instruct mapping 608, then drawing backup system 100 can if any defining vertical axis of symmetry 612 shown in Fig. 6 D.If edge instructs the side of mapping 608 to omit marginal information, then drawing backup system 100 can search drain message on the opposite side of vertical axis of symmetry 612.In other words, drawing backup system 100 can reflect (mirror) and instruct edge/point on every side of vertical axis of symmetry.Such as, as shown in by the arrow 210 of Fig. 2 D, edge instructs mapping 208 to omit the edge in the left side of the lower jaw for user.In order to supplementary edge instructs mapping 208, drawing backup system 100 can reflectance signature point 211a can be corrected into provide the stroke 218 being intended as lower-left jaw of user on the right side of vertical pivot point.

Additionally or alternatively, as shown in by Fig. 6 B, unique point itself can instruct mapping 613 by defined feature.Such as, drawing backup system 100 can use unique point or can to unique point matched curve and resampling lines with identification characteristics point.

Geometry correction

When user draws lines, drawing backup system 100 can determine the feature of image that user intends to draw.Such as, drawing backup system 100 can compare user's stroke and map to determine to instruct the lines/edge nearest with user's stroke on locating and orienting also with same characteristic features label mapped with guidance.Detect that user intends to draw instruct lines time, drawing backup system can by towards or move (such as, reorientate and/or be again shaped) user's stroke to the lines that the guidance that user intends to draw maps and carry out geometrically correcting user stroke.

In order to determine the feature tag of user's stroke, drawing backup system 100 can perform nearest-neighbors search with the nearest unique point of mark for each input point of user's stroke.Then drawing backup system 100 can be determined most of input point for user's stroke and identify which feature tag (based on nearest unique point).Drawing backup system 100 then for user's stroke each input point or as a whole user's stroke is associated to the feature tag that most of input point is identified.

Especially, for each user's input, geometry correction is modeled as energy minimization process, wherein energy function is defined as at least partly:

E＝αE _guidance+βE _face+γE _variation

Edge instruct map, feature instructs and to map and the change mapped of instructing from edge of stroke of user can the basis of forming energy function.In other words, energy function can help to ensure instruct the stroke at the mutatis mutandis family of mapping pair according to edge and retain the change/pattern of original user's stroke.Input point x successively in given each user's stroke ₁, x ₂... x _n, unique point f ₁, f ₂... f _nwith guidance point g ₁, g ₂... g _n, energy function allows the some p determining geometrically to correct ₁, p ₂... p _n.

E _guidancethe stroke of this help aligning user and edge instruct and map and be defined as:

Σ_{i = 1}^{n} \frac{1}{d_{i}} {| p_{i} - g_{x_{i}}^{*} |}^{2}

Wherein x _iedge instruct map in the closest approach with same characteristic features label.Nearest-neighbors search can be used to determine (X, Y, curvature) can be used to define each point.Therefore, instruct recently a little in discovery time, can comparison and location and curvature difference.Therefore, if equally instruct a little in two, interval from input point, then can select to have with the immediate curvature of the curvature of input point instruct point (as long as they all have the feature tag identical with input stroke).The curvature of point can be defined as wherein φ is the inclination angles of lines at the tangent line of point.Kd sets implementation and also can be used for accelerating nearest-neighbors search procedure to realize in real time in real time or substantially correcting.

As by for E _guidanceformula shown in, E _guidancethe Distance geometry curvature difference between the point geometrically corrected and the guidance point of mark can be reduced. at x _iwith between Euclidean distance.In other words, d _imake E _guidanceand user stroke (or defining the point of this stroke) and instruct the distance between the nearest lines/edge of wave filter (or defining the point at this lines/edge) to be inversely proportional to.D _ireduce the geometry correction instructing in mapping user's stroke or its part do not had close to corresponding lines/edge at edge.Instruct mapping 208 for example, referring to the image 206 of Fig. 2 A and the edge of Fig. 2 B, if user draws user's stroke of the lower-left jaw being used for performer, then instruct in mapping 208 at edge and there is no corresponding lines.D _ican help to ensure not correct the user stroke corresponding with the lower-left jaw of performer in large quantities towards the guidance point of mark.In addition, more specifically illustrating as following, can mapping be instructed geometrically to correct and during parameter adjustment, de-emphasize user's stroke in guidance maps without corresponding lines by feature based.

E _facethe stroke of this help aligning user and feature instruct and map and be defined as:

Σ_{i = 1}^{n} \frac{1}{d_{f_{i}}} {| p_{i} - f_{x_{i}}^{*} |}^{2}

Wherein x _ifeature instruct map in the closest approach with same characteristic features label.With similar, nearest search can be used determine (X, Y, curvature) can be used to define each point.Therefore, when finding nearest unique point, can comparison and location and curvature difference.Therefore, if from input point equally two, interval unique point, then can select to have the unique point with the immediate curvature of the curvature of input point, as long as it has the feature tag identical with input point.Kd sets implementation and also can be used for accelerating nearest-neighbors search procedure to realize in real time in real time or substantially correcting.

at x _iwith between Euclidean distance.In other words, make E _faceand user stroke (or defining the point of this stroke) and instruct the distance between the nearest lines/edge of wave filter (or defining the point at this lines/edge) to be inversely proportional to. be increased in the geometry correction that feature instructs in mapping user's stroke or its part do not had close to corresponding lines/edge.Instruct mapping 208 for example, referring to the image 206 of Fig. 2 A and the edge of Fig. 2 B, if user draws user's stroke of the lower-left jaw being used for performer, then instruct in mapping 208 at edge and there is no corresponding lines.As described above, d _ican help to ensure not correct the user stroke corresponding with the lower-left jaw of performer in large quantities towards the guidance point of mark.On the other hand, can help to ensure as indicated by the arrow 214 of Fig. 2 C towards the unique point geometrically correcting user stroke with feature tag 211a.

E _guidanceand E _facethese to or at least instruct the location that user's " should " depicts the lines of the same characteristic features label at stroke place that has mapped to move input stroke (that is, each point of user's stroke) towards edge or feature.Section 3 E _variationserve back balance E _guidanceand E _faceto help the pattern of the original stroke by retaining user to ensure, each output drawing of identical image does not map identical definitely with guidance for these.E _variationbe defined as:

Σ_{i = 2}^{n} {| (p_{i} - p_{i - 1}) - (x_{i} - x_{i - 1}) |}^{2}

E _variationthis can be minimized in the point of proximity difference before and after correction.In other words, E _variationthis can by ensureing that neighbor point synchronizing moving helps the shape maintaining original user's stroke.Especially, each user's stroke is divided into multiple point.The pattern of original user's stroke is defined in distance between the point of composition stroke and the shape of therefore regulation stroke.E _variationthe distance that this help remains between the neighbor point in the stroke geometrically corrected is identical or similar with the distance between the corresponding point of proximity in original user's stroke.

Weights α, β and γ can control aim at the stroke of user and edge instructs map, feature instruct map and maintain original user's stroke characteristic/shape between trade off.Especially, along with increase α and β and reduce γ, geometry correction process by the less person kept in the characteristic of original stroke of user and the stroke of closer aiming at user map with instructing.On the other hand, along with minimizing α and β and increase γ, geometry correction process can keep the more person in the characteristic of original stroke of user and the stroke of aiming at user with more keeping off maps with instructing.In one or more embodiment, automatically select weights α, β and γ to optimize the perception exporting and draw.In an alternative embodiment, user can revise how α, β and γ correct them drawing with adjustment.

Parameter adjustment

To recognize, due to E _variationdraw with user the content that they " know ", be offset so the stroke geometrically corrected still may instruct from edge to map or otherwise be shifted.As the part of parameter tuning process, drawing backup system can determine that " correctly " depicts stroke which geometrically corrects and which lines " improperly " has depicted.Especially, the stroke that drawing backup system can more geometrically correct and edge instruct and map to determine that they instruct mapping to depart from how many from edge.Then based on edge instruct map depart from or aim at quantity, accompanying drawing backup system can be emphasized to instruct with edge the lines that map and aim at and de-emphasize the lines instructing from edge and map and depart from.

For each some p in the stroke that geometrically corrects _idepart from or or error can be defined as at p _iand its distance between edge instructs the point in mapping.More particularly, to depart from or error can be defined as:

e = dist (p_{i}, g_{i}^{*}) = \sqrt{{(X_{p_{i}} - X_{g_{p_{i}}^{*}})}^{2} + {(Y_{p_{i}} - X_{g_{p_{i}}^{*}})}^{2} + η {({Curvature}_{p_{i}} - {Curvature}_{g_{p_{i}}^{*}})}^{2}}

Front two projects (that is, ) represent at p _iand its distance between edge instructs the closest approach in mapping.Section 3 (that is, measure curvature difference between two points.Therefore, but Section 3 ensures exceedingly not emphasize to instruct with edge the close stroke geometrically corrected instructing the edge mapped differently to bend from edge in the edge mapped.

Parameter adjustment can use one or more technology in multiple technology to provide and emphasize.Such as, drawing backup system can revise line roughness.Thick lines show in the picture more strongly, and therefore when making parameter adjustment, drawing backup system can increase the width of the lines placed and reduce the width of lines that mistake places.Additionally or alternatively, parameter adjustment can adjust the opacity of lines.Such as, it is drawn more transparent that drawing backup system can ensure that the lines placed are plotted as the lines of opaque and wrong placement.Adjust opacity cannot time, drawing backup system can by emulation scratch brushing medium (brushing medium) simulate opacity change.Such as, Stroke decomposition can be become indivedual lines (prompting bristle brush contacts with the light of paper) and reduce the opacity of the perception of total stroke thus by drawing backup system.In a still further embodiment, drawing backup system can use color to apply to be emphasized or de-emphasizes.Such as, drawing backup system can play up with another color or shade (such as, dull gray) lines placed with the lines of a color (such as, light gray) Rendering errors placement.

Can emphasize based on the error amount e application of the stroke for geometrically correcting or de-emphasize.Such as, if the error of lines is below predetermined quantity or threshold value, then parameter adjustment can emphasize lines.Especially, along with error amount reduces, emphasize to increase.On the other hand, if error amount is more than predetermined threshold, then parameter adjustment can de-emphasize lines.In fact, along with error amount increases, de-emphasize and can increase.If error amount is in or close to predetermined quantity, then parameter adjustment can be applied and minimumly be emphasized or do not apply to emphasize and de-emphasize.

Drawing backup system can (that is, along whole stroke) or (that is, along the change of single stroke) application parameter adjustment partly globally.Such as, the lines 228 geometrically corrected of Fig. 2 H illustrate apply partly emphasizing and de-emphasizing.In one or more example, the error amount of assigning partly can not generate good visualization effect when local error changes continually along identical stroke.In such example, level and smooth local error can be carried out along identical stroke by optimizing with contiguous consistance punishment.Global error can be calculated as the mean value of local smoothing method error and then be used for application and emphasize or de-emphasize.

Fig. 1 to Fig. 6 D, corresponding word and example provide for providing by geometry adjustment draw auxiliary multiple different system and equipment based on semantic tagger.In addition to the foregoing, also according to the process flow diagram comprised for realizing action in the method for particular result or step, embodiments of the invention can be described.Such as, Fig. 7 and Fig. 8 illustrates the process flow diagram of the illustrative methods according to one or more embodiment of the present invention.

Fig. 7 illustrate use semantic tagger with to draft plan as 206,406 user the process flow diagram of auxiliary a kind of illustrative methods 700 of drawing is provided.Method 700 comprises the action 702 generating and maps for the guidance of image, and this guidance maps and has the feature tag that mark instructs the part corresponding with the feature of image of mapping.Especially, action 702 can relate to generate edge instruct mapping 208,608 and feature instruct in mapping 213 one or more instruct map.

Such as, action 702 can relate to and instructs mapping 208,608 to image applications edge detection filter to generate edge.Such as, action 702 can relate to instructs mapping 208,608 to image 206,406 application extension type Gaussian difference value filter to produce edge, and edge instructs mapping 208,608 to comprise to sketch the contours of or multiple edges 222 of the otherwise feature of indicating image 206,406.

Action 702 can also relate to one or more feature in identification image.Action 702 can relate to different characteristic in mutual differentiate between images 206,406 or different characteristic type.Such as, when image 206,406 comprises face, action 702 can relate to applies face detection algorithms to identify eyes, iris, mouth, the profile of face, nose and eyebrow to image 206,406.When image comprises outdoor scene, action 702 can relate to application characteristic detection algorithm to identify sky, water body, mountain range, trees etc.In a still further embodiment, action 702 can relate to user's input or the usage data storehouse of the feature receiving identification image, the characteristics of image of this Database Identification user or computer identity.

The feature that action 702 can relate to synthetic image 206,406 instructs mapping 213,613, and Feature Mapping 213,613 comprises feature tag 211a-211f, 611a-611f, the location of the feature of the mark of these feature tag indicating images 206,406.Such as, action 702 can relate to edge and instructs mapping 208,608 to carry out filtering only to comprise the edge 222 with feature tag 211a-211f, 611a-611f.Then action 702 can relate to reflective edges on every side of vertical symmetrical lines 612.Additionally or alternatively, action 702 can relate to and uses the unique point of the determination in image 206 to instruct mapping 213,613 with defined feature.Such as, action 702 can use result to instruct mapping 213,613 as feature to unique point matching lines 214 and then.

Action 702 can relate to and uses feature to instruct mapping 213 using the semantic tagger providing edge and instruct mapping 208,608, supplement edge instruct mapping 208,608 and/or as instructing mapping separately when omitting the edge corresponding with one or more feature of image 206.Additionally, action 702 can relate to linked character label and edge and instructs the edge corresponding with the feature of mark in mapping.Such as, action 702 can relate to the feature comparing in image 206,406 mark and instructs mapping 208,608 and correlation tag 211a-211f, 611a-611f and edge to instruct the edge 222 corresponding with the feature identified of mapping 208,608 with edge.Especially, action 702 can relate to use markov random file, pattern zoning or nearest-neighbors searching algorithm carry out segmenting edge and instruct mapping 208,608.Action 702 can also relate to edge 222 as nearest with the feature identified from image 206,406 in mark as shown in Fig. 2 D and Fig. 6 D and feature tag 211a-211f, 611a-611f of linked character and the edge of mark.

Method 700 comprises the action 704 detecting the stroke 216,218 of being drawn by user.Especially, action 704 can relate to the activation and movement that detect input equipment (such as, mouse, product or touch panel).Action 704 also can relate to the detection starting point of stroke 216,218, the direct of travel of stroke 216,218 and the end point for stroke 216,218.Action 702 can also relate to the location and curvature of detecting stroke 216,218.

Fig. 7 illustrates the action 706 that method 700 comprises linked character label and stroke.Such as, action 706 can relate to the feature and stroke 216,218 and feature tag 211a-211f, the 611a-611f and the stroke 216,218 that associate the feature that stroke 216,218 corresponds to that compare mark in image 206,406.Such as, action 706 can also relate to mark and stroke 216, feature tag 211a-211f, 611a-611f of 218 nearest features and linked character and stroke 216,218.

Additionally, method 700 comprise mark instruct map have the feature tag 211a-211f, the 611a-611f that associate mate with feature tag 211a-211f, 611a-611f of stroke 216,218, and stroke 216,218 nearest parts action 708.Such as, action 708 can relate to mark edge instruct mapping 208,608 have with stroke 216,218 identical feature tag 211a-211f, 611a-611f, with stroke 216,218 nearest edges 222.More specifically, action 708 can relate to determine edge instruct mapping 208,608 comprise matching characteristic label 211a-211f, 611a-611f, in Distance geometry curvature with stroke 216,218 nearest edges 222.

Alternatively or additionally, action 708 can relate to identification characteristics instruct mapping 213,613 have the feature tag 211a-211f, the 611a-611f that associate mated with feature tag 211a-211f, 611a-611f of stroke 216,218, with stroke 216,218 nearest lines 214.More specifically, action 708 can relate to determine feature instruct mapping 213,613 comprise matching characteristic label 211a-211f, 611a-611f, in Distance geometry curvature with stroke 216,218 nearest lines 214.

Fig. 7 also illustrates method 700 and comprises the action 710 that the part of mark that instruction maps generates the stroke geometrically corrected.Such as, action 710 can relate to by use edge instruct mapping 208,608 have the feature tag 211a-211f, the 611a-611f that associate mate with feature tag 211a-211f, 611a-611f of stroke 216,218, and stroke 216,218 nearest marks edge 222 move stroke 216,218, reorientate or be again shaped to generate the stroke 224,226 geometrically corrected as instructing.Such as, action 710 can relate to the location and the curvature difference that instruct the corresponding edge 222 of mapping 208,608 from edge that reduce or minimize stroke 216,218.Additionally or alternatively, action 710 also relate to except using the edge 222 of mark use feature instruct mapping 213 have the feature tag 211a associated mate with the feature tag 211a of stroke 218, and the lines 214 of the nearest mark of stroke 218.

In addition to the foregoing, action 710 can also relate to change 216a, 216b of instructing the edge 222 of the mark of mapping 208 from edge of determining stroke 216.In order to maintain the pattern of user, action 710 can also relate to and to maintain change 216a, 216b when geometrically correcting stroke 216 at least in part.Such as, action 710 can relate to and reducing in change 216a, 216b of stroke 216 and the difference between change 224a, 224b of stroke 224 that geometrically corrects.

In addition to the foregoing, method 700 can adjust the stroke geometrically corrected with relating to parameter alternatively.Such as, what method 700 can relate to the stroke 224 determining geometrically to correct instructs departing from of the edge 222 of the mark of mapping 208 from edge.Especially, method 700 can relate to the location and curvature difference determined between the edge 222 that the stroke 224 geometrically corrected and edge instruct the mark of mapping 208.In addition, method 700 can relate to by instructing departing from of the edge 222 of the mark of mapping 208 to emphasize or de-emphasize the stroke 224 geometrically corrected and adjust the stroke 224 geometrically corrected with carrying out parameter from edge based on the stroke 224 geometrically corrected.Such as, method 700 can relate to the stroke 224 emphasizing geometrically to correct with the part 224c that aims at of edge 222 of mark and de-emphasize the part 224a departed from, the 224b that have from the edge 222 identified of the stroke 224 geometrically corrected.Emphasize that the part of the stroke 224 geometrically corrected can relate to and increase in width, opacity or density one or multinomial, and the part de-emphasizing the stroke 224 geometrically corrected can relate to and reduces in width, opacity or density one or multinomial.

Method 700 can relate to the stroke played up and geometrically correct.Method 700 can play up the stroke 224,226 geometrically corrected when user draws stroke 216,218.In other words, method 700 can be played up the stroke 224,226 that geometrically corrects instead of or replace and play up stroke 216,218.In such embodiments, drawing backup system does not play up stroke 216.218。

Referring now to Fig. 8, illustrate and use semantic tagger to provide the process flow diagram of another auxiliary illustrative methods 800 of drawing to the user of draft plan picture.Method 800 also comprises the action 802 of one or more feature in detected image.Action 802 can relate to different characteristic in mutual differentiate between images 206,406 or different characteristic type.Such as, when image comprises face, action 802 can relate to image applications face detection algorithms to identify eyes, iris, mouth, the profile of face, nose and eyebrow.When image comprises outdoor scene, action 802 can relate to application characteristic detection algorithm to identify sky, water body, mountain range, trees etc.In a still further embodiment, action 802 can relate to user's input or the usage data storehouse of the feature receiving identification image, the characteristics of image of this Database Identification user or computer identity.Action 802 can also relate to generation or identification characteristics point, the feature that these unique point label detection arrive.

Additionally, method 800 comprises the action 804 that linked character label and edge instruct the guidance point corresponding with the feature detected in mapping.Such as, action 804 can relate to and compares the feature that detects in image 206,406 and instruct mapping 208,608 and correlation tag 211a-211f, 611a-611f and edge to instruct the guidance point g corresponding with the feature of mark of mapping 208,608 with edge _i.Especially, action 804 can relate to use markov random file, pattern zoning or nearest-neighbors searching algorithm carry out segmenting edge and instruct mapping 208,608.Action 804 can also relate to the search of execution nearest-neighbors is used to guide edge 222 each guidance point g with mark _inearest unique point f _i.Then action 804 can relate to determines for instructing the great majority at edge 222 to instruct some g _iand identify which feature tag 211a-211f, 611a-611f (based on nearest unique point f _i).Action 804 also relates to association and instructs some g for great majority _iand feature tag 211a-211f, 611a-611f of mark and to instruct on edge 222 each instructs some g _i.

As shown, method 800 comprises the action 806 of mark input point, and these input points define the stroke of being drawn by user.Especially, action 806 relates to mark input point x _i, these input point x _idefine the input stroke 216,218 of being drawn on image 206,406 by user.More specifically, action 806 can relate to analysis stroke to identify the input point x of definition stroke 216,218 _i.Analysis stroke 216,218 can comprise sampling or resampling stroke 216,218 inputs stroke point x to identify _i.Additionally, analyze stroke 216,218 can comprise and determine that stroke 216,218 is at input point x _iin the location of each input point and local orientation.

Additionally, method 800 comprises the action 808 of linked character label and input point.Such as, action 808 can relate to the feature and input point x that compare and detect in image 206,406 _iand correlation tag 211a-211f, 611a-611f and input point x corresponding to feature with mark _i.Action 808 can relate to the search of execution nearest-neighbors with each input point x of mark for user's stroke 216,218 _inearest unique point f _i.Then action 808 can relate to the most of input point x determined for stroke 216,218 _iand identify which feature tag 211a-211f, 611a-611f (based on nearest unique point f _i).Action 800 also relates to association for most of input point x _iand feature tag 211a-211f, 611a-611f of mark and each input point x of input stroke 216,218 _i.

Fig. 8 illustrate method 800 comprise mark edge instruct map have the feature tag associated mate with the feature tag of the input point of stroke, and the action 810 of the nearest guidance point of the input point of stroke.Such as, action 810 can relate to mark and has and input point x _iidentical feature tag 211a-211f, 611a-611f, with input point x _inearest guidance point g _i.More specifically, action 810 can relate to and determines to have equally and be associated with input point x _ifeature tag coupling association feature tag 211a-211f, 611a-611f, in distance with input point x _inearest guidance point g _i.

Along relevant lines, method 800 comprises the action 812 of that mark has the feature tag associated mated with the feature tag of the input point of stroke, nearest with the input point of stroke unique point.Such as, action 812 can relate to mark and has and input point x _iidentical feature tag 211a-211f, 611a-611f, with input point x _inearest unique point f _i.More specifically, action 812 can relate to and determines to have equally and be associated with input point x _ifeature tag coupling association feature tag 211a-211f, 611a-611f, in Distance geometry curvature with input point x _inearest unique point f _i.

Fig. 8 also illustrates method 800 and comprises the action 814 determining the point geometrically corrected of stroke based on the guidance point of mark and the unique point of mark.Especially, action 814 can relate to the some p calculated for geometrically correcting _ilocating and orienting.Action 814 can relate to minimizing or be minimized in the some p geometrically corrected _iwith guidance point g _ibetween difference.Action 814 can relate to minimizing or be minimized in the some p geometrically corrected _iwith unique point f _ibetween difference.

Action 806 can also comprise to be determined at contiguous input point x _ibetween change and reduce at contiguous input point x _ibetween change with contiguous input point x _ithe point p that corresponding vicinity geometrically corrects _ibetween change between difference.Therefore, the some p geometrically corrected is determined _ithe point p that aligning geometrically corrects can be related to _iwith guidance point g _iwith unique point f _ithe two and maintain the change of stroke 216,218.

Method 800 additionally comprises the action 816 of stroke or the curve geometrically corrected from the dot generation geometrically corrected.More specifically, method 816 relates to by the some p geometrically corrected _imatching lines generate the stroke or curve 224,226 that geometrically correct.Such as, action 816 can relate to cubic B-Spline curve approximation or the some p that uses least square regression technology or another lines fitting algorithm to carry out matching geometrically to correct _i.

Embodiments of the invention can comprise like that as discussed in more detail below or utilize the special or multi-purpose computer comprising computer hardware (as such as one or more processor and system storage).Embodiment within the scope of the invention also comprises physics for carrying or store computer executable instructions and/or data structure and other computer-readable medium.In some specific embodiment, can be based, at least in part, on embody in non-transient computer-readable medium and one or more process in process described herein is implemented in the instruction that can be performed by one or more computing equipment (any media content access device such as, in media content access device described herein).Generally speaking, processor (such as, microprocessor) receive instruction and perform those instructions from non-transient computer-readable medium (such as, storer etc.), perform one or more process thus, one or more process comprised in process described herein.

Computer-readable medium can be general or any usable medium that can access of dedicated computer system.The computer-readable medium storing computer executable instructions is non-transient computer-readable recording medium (equipment).The computer-readable medium of load capacity calculation machine executable instruction is transmission medium.Therefore, for example unrestricted, embodiments of the invention can comprise at least two differently different types of computer-readable mediums: non-transient computer-readable recording medium (equipment) and transmission medium.

Non-transient computer-readable recording medium (equipment) comprises RAM, ROM, EEPROM, CD-ROM, solid-state driving (" SSD ") (such as, based on RAM), the storer of flash memory, phase transition storage (" PCM "), other type, other optical disk storage apparatus, disk storage device or other magnetic storage apparatus or other medium any, this other medium any can be used for storing the program code devices of wishing with the form of computer executable instructions or data structure and can being accessed by general or special purpose computer.

" network " is defined as being supported in one or more data link transmitting electronic data between computer system and/or module and/or other electronic equipment.By network or another communication connection (hardwire, wireless or hardwire or wireless combination), to computing machine transmission or when providing information, connection is considered as transmission medium by computing machine rightly.Transmission medium can comprise the program code devices and the network can accessed by general or special purpose computer and/or data link that can be used for wishing with the conveying of the form of computer executable instructions.Also above combination should be comprised in the scope of computer-readable medium.

In addition, when arriving various computer system part, can from transmission medium to non-transient computer-readable recording medium (equipment) (or contrary) automatically delivery form be the program code devices of computer executable instructions or data structure.Such as, can buffering and then finally transport through to computer system RAM and/or to the less volatile computer storage medium (equipment) in computer system the computer executable instructions or data structure that network or data link receive in the RAM in Network Interface Module (such as, " NIC ").Therefore, should be appreciated that and the computer system part of transmission medium can be utilized to comprise non-transient computer-readable recording medium (equipment) in also (or even mainly).

Computer executable instructions is such as included in when processor performs and makes multi-purpose computer, special purpose computer or dedicated treatment facility perform the instruction and data of certain function or one group of function.In certain embodiments, computer executable instructions is performed on a general-purpose computer multi-purpose computer to be transformed into special purpose computer implementation unit of the present invention.Computer executable instructions can be such as scale-of-two, intermediate format instructions (such as assembly language) or even source code.Although describe subject content with architectural feature and/or the distinctive language of method action, will understand, the subject content defined in the following claims may not be limited to the action of feature described above or description.In fact, the characteristic sum action of description is published as the exemplary forms implemented the claims.

Those skilled in the art will recognize that, can have perhaps eurypalynous computer system configurations (comprise personal computer, desktop computer, laptop computer, message handling device, handheld device, multicomputer system, based on microprocessor or programmable consumer electronic devices, network PC, small-size computer, mainframe computer, mobile phone, PDA, board, pager, router, switch etc.) network computing environment in put into practice the present invention.Also can put into practice the present invention in distributed system environment, wherein be executed the task by local and remote both computer systems of network linking (by hard wired data link, wireless data link or the combination by hardwire and wireless data link).In distributed system environment, program module can be arranged in local and remote both memory storage device.

Also embodiments of the invention can be implemented in cloud computing environment.In this manual, " cloud computing is defined for the model of supporting to access the network as required of the shared pool of configurable computational resource.Such as, cloud computing can be used in fairground to give the general and convenience of the shared pool to configurable computational resource, the access according to demand.Can promptly allocate via virtual and discharge alternately with low amounts management work or service provider and the shared pool of the configurable computational resource that then correspondingly stretches.

Cloud computing model can be made up of various characteristic (as such as according to the service etc. of the converging from service, wide network insertion, resource of demand, rapid elasticity, measurement).Cloud computing model also can expose various service model, as such as software namely serve (" SaaS "), platform namely serves " PaaS " and namely foundation structure serve " IaaS ".Also different deployment model (such as specific cloud, group's cloud, mixed cloud etc.) can be used to dispose cloud computing model.In this manual and in detail in the claims, " cloud computing environment " is the environment wherein using cloud computing.

Fig. 9 illustrates the example calculation equipment 900 that can be configured to perform one or more process in process described above in form of a block diagram.To recognize, client device 118 (or even server apparatus 120) can comprise the implementation of computing equipment 900.As shown in Figure 9, computing equipment can comprise processor 902, storer 904, memory device 906, I/O interface 908 and communication interface 910.Although figure 9 illustrates parts shown in Example Computing Device 900, Fig. 9 not to be intended to restriction.Additional or alternate means can be used in other embodiments.In addition, in certain embodiments, computing equipment 900 can comprise the parts more less than parts shown in Fig. 9.The parts of the computing equipment 900 shown in Fig. 9 will be described now with additional detail.

In a particular embodiment, processor 902 comprises the hardware for performing instruction (such as forming the instruction of computer program).Exemplarily but not by restriction, in order to perform instruction, processor 902 can be fetched from internal register, internally cached, storer 904 or memory device 906 (or get read) instruction and they are decoded and performs.In a particular embodiment, processor 902 can comprise for data, instruction or address one or more is internally cached.Exemplarily but not by restriction, processor 902 can comprise one or more instruction cache, one or more data cache and one or more translation look aside buffer (TLB).Instruction in instruction cache can be the copy of the instruction in storer 904 or memory storage 906.

Computing equipment 900 comprises the storer 904 being coupled to processor 902.Storer 904 may be used for storing data, metadata and the program for being performed by processor.Storer 904 can comprise in the data storage device of volatibility and nonvolatile memory (such as random access memory (" RAM "), ROM (read-only memory) (" ROM "), solid-state disk (" SSD "), flash memory, phase transition storage (" PCM ")) or other type one or multinomial.Storer 904 can be inner or distributed memory.

Computing equipment 900 comprises memory device 906, and this memory device 906 comprises the memory storage for storing data or instruction.Exemplarily but not by restriction, memory device 906 can comprise non-transient storage medium described above.Memory device 906 can comprise the driving of hard drive (HDD), disk drive, flash memory, CD, photomagneto disk, tape or USB (universal serial bus) (USB) or two or the combination of more in these.Memory device 906 can comprise in due course can remove or non-ly to remove (or fixing) medium.Memory device 906 can be inner or outside at computing equipment 900.In a particular embodiment, memory device 906 is non-volatile solid state memory.In a particular embodiment, memory device 906 comprises ROM (read-only memory) (ROM).In due course, this ROM can be the ROM of mask programming, programming ROM (PROM), erasable PROM (EPROM), electric erasable PROM (EEPROM), electricity can change (ROM) (EAROM) or flash memory or two or the combination of more in these.

Computing equipment 900 also comprise provide for allow user to computing equipment 900 provide input (such as user's stroke), from computing equipment 900 receive export and otherwise to transmit from computing equipment 900 data one or more input or export (" I/O ") equipment/interface 908.These I/O equipment/interfaces 908 can comprise the combination of mouse, keypad or keyboard, touch screen, camera, photoscanner, network interface, modulator-demodular unit, other known I/O equipment or such I/O equipment/interface 908.Can with stylus or finger activated touch screen.

I/O equipment/interface 908 can comprise one or more equipment for presenting output to user, include but not limited to graphics engine, display (such as, display screen), one or more output driver (such as, display driver), one or more audio tweeter and one or more audio driver.In certain embodiments, equipment/interface 908 is configured to provide graph data to present for user to display.Graph data can represent one or more graphic user interface and/or other graphical content any as served specific implementation mode.

Computing equipment 900 can also comprise communication interface 910.Communication interface 910 can comprise hardware, software or the two.Communication interface 910 can be provided for communicating one or more interface of (as such as packet-based communication) between other computing equipment 900 of computing equipment and one or more or one or more network.Exemplarily but not by restriction, communication interface 910 can comprise for Ethernet or other is based on wired network of network interface controller (NIC) or network adapter or for the wireless NIC (WNIC) that communicates with wireless network (such as WI-FI) or wireless adapter.

The present disclosure any suitable network of imagination and any suitable communication interface 910.Exemplarily but not by restriction, computing equipment 900 can with one or more part of self-organizing network, personal area network (PAN), LAN (Local Area Network) (LAN), wide area network (WLAN), Metropolitan Area Network (MAN) (MAN) or the Internet or these every in two or more communicate.One or more part of one or more network in these networks can be wired or wireless.Exemplarily, computing equipment 900 can with wireless PAN (WPAN) (as such as BLUETOOTH WPAN), WI-FI network, WI-MAX network, cellular phone network (as such as global system for mobile communications (GSM) network) or other suitable wireless network or its combined communication.Computing equipment 900 can comprise any suitable communication interface 910 for any network in these networks in due course.

Computing equipment 900 can also comprise bus 912.Bus 912 can comprise the hardware of the parts of the computing equipment 900 that intercouples, software or the two.Exemplarily but not by restriction, bus 912 can comprise Accelerated Graphics Port (AGP) or other graphics bus, enhancement mode Industry Standard Architecture (EISA) bus, front side bus (FSB), HYPERTRANSPORT (HT) interconnection, Industry Standard Architecture (ISA) bus, peripheral parts interconnected (PCI) bus, PCI-fast (PCIe) bus, serial advanced technology attachment (SATA) bus, VESA this locality (VLB) bus or another suitable bus or it combines.

In instructions above, describe the present invention with reference to concrete example embodiment of the present invention.Describe various embodiment of the present invention and aspect with reference to details discussed here, and accompanying drawing illustrates various embodiment.Above instructions and accompanying drawing illustrate the present invention and can not be interpreted as limiting the present invention.Many details are described to provide the thorough understanding to various embodiment of the present invention.

Can the present invention be embodied with other concrete form and not depart from its Spirit Essence or fundamental characteristics.The embodiment described will only be regarded as example and unrestricted in all respects.Such as, by less or more multi-step/action executing method described herein or steps/actions can be performed according to different order.Additionally, can repeat or perform parallel to each other or perform steps/actions described herein concurrently with different instances that is identical or similar steps/action.Scope of the present invention is therefore by claims instead of by describing instruction above.Fall in implication and equivalent scope that potentiality require change and will covered in their scope.

Claims

1. use semantic tagger to provide an auxiliary method of drawing to the user of rendering image, comprising:

The guidance generated for described image by one or more processor maps, the described feature tag instructing mapping to comprise one or more part corresponding with one or more feature of described image that the described guidance of mark maps;

Detect the stroke of being drawn by user;

Feature tag is associated with described stroke;

Identify described instruct map have the feature tag associated mate with the described feature tag of described stroke, and the nearest part of described stroke; And

By described one or more processor use described instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the described part of the nearest mark of described stroke generate the stroke geometrically corrected.

2. method according to claim 1, wherein generates described guidance mapping and comprises by generating edge guidance mapping to described image applications edge detection filter.

3. method according to claim 2, wherein generates described guidance mapping and also comprises:

To described image applications feature detection algorithm to identify one or more feature in described image; And

The edge corresponding with the described feature of mark in mapping is instructed in feature tag and described edge to associate.

4. method according to claim 2, wherein generates described guidance mapping and also comprises by carrying out generating feature guidance mapping to described image applications feature detection algorithm with one or more feature identified in described image.

5. method according to claim 4, also comprises and uses described feature to instruct mapping to map by using described feature to instruct the characteristic curve of mapping to instruct the holiday mapped to carry out supplementary described edge guidance for described edge.

6. method according to claim 4, wherein use described instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the described part of the nearest mark of described stroke generate the described stroke geometrically corrected and comprise:

Determine described feature instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the nearest characteristic curve of described stroke;

Determine described edge instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the nearest edge of described stroke;

Determine in described edge or described characteristic curve which be provided for the better guidance of described stroke; And

Described stroke is moved towards the described edge of determination or described characteristic curve that are provided for better guidance described in described stroke.

7. method according to claim 2, wherein said edge detection filter comprises extended pattern Gaussian difference value filter.

8. method according to claim 1, wherein generates described guidance mapping and comprises by carrying out generating feature guidance mapping to described image applications feature detection algorithm with one or more feature identified in described image.

9. method according to claim 7, wherein:

Described image comprises the photograph of face; And

Comprise to described image applications face detection algorithms to described image applications feature detection algorithm.

10. method according to claim 1, also comprises location difference between the described part by determining the mark mapped in the described stroke geometrically corrected and described guidance and curvature difference determines that the described stroke geometrically corrected instructs departing from of the described part of the mark mapped from described.

11. methods according to claim 9, also comprise in the following one or multinomial:

The part that the described part de-emphasizing the mark from described guidance mapping of the described stroke geometrically corrected departs from; Or

Emphasize the described stroke geometrically corrected with the described part instructing the described part of the mark mapped to aim at.

12. methods according to claim 1, are also included in the described stroke played up when described user draws described stroke and geometrically correct.

13. methods according to claim 1, also comprise:

From the described change instructing the described edge of the mark mapped to determine described stroke; And

Described change is maintained at least in part when geometrically correcting described stroke.

14. 1 kinds of systems, comprising:

At least one processor; And

At least one non-transient computer-readable recording medium, at least one non-transient computer-readable recording medium described stores instruction, and described instruction makes described system when being performed by least one processor described:

The guidance generated for described image maps, the described feature tag instructing mapping to comprise one or more part corresponding with the feature of described image that the described guidance of mark maps;

Detect the stroke of being drawn by user;

Feature tag is associated with described stroke;

Towards described instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the described part of the nearest mark of described stroke use described stroke to generate the stroke geometrically corrected.

15. systems according to claim 14, wherein said instruction makes described system by generating or the multinomial described guidance mapping generated for described image in the following when being performed by least one processor described:

Instructed by the feature of the described image generated to described image applications feature detection algorithm and map, described feature detection algorithm identifies the location of one or more feature of described image; Or

Instructed by the edge of the described image generated to described image applications edge detection filter and map.

16. systems according to claim 15, what wherein said guidance mapped the have described feature tag associated mate with the described feature tag of described stroke, and the described part of the nearest mark of described stroke comprise in the following one or multinomial:

Described feature instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the nearest line of described stroke; Or

Described edge instruct map have the described feature tag associated mate with the described feature tag of described stroke, and the nearest edge of described stroke.

17. systems according to claim 15, wherein said instruction also makes described system when being performed by least one processor described:

Determine that described stroke instructs the first distance of the described line mapped apart from described feature;

Determine that described stroke is apart from the described second distance instructing the described edge mapped; And

The described edge of described line and the described edge guidance mapping that map is instructed to give weights based on second distance described in described first Distance geometry determined to described feature when generating the described stroke geometrically corrected.

18. 1 kinds use semantic tagger to provide auxiliary method of drawing to the user of rendering image, comprising:

Detect one or more feature in described image;

Feature tag is associated with instructing the guidance point corresponding with the described feature detected in mapping;

Mark defines the input point of the stroke of being drawn by described user;

Feature tag is associated with the described input point of described stroke;

Identify described instruct map have the feature tag associated mate with the described feature tag of the described input point of described stroke, and the described input point of described stroke nearest instruct a little;

That mark has the feature tag associated mated with the described feature tag of the described input point of described stroke, nearest with the described input point of described stroke unique point;

By the described point that geometrically correct that instruct the described unique point of point and mark determine described input point of one or more processor described based on mark; And

By generating to the described some matching lines geometrically corrected the stroke geometrically corrected.

19. methods according to claim 18, the described described point geometrically corrected instructing the described unique point of point and mark to determine described input point wherein based on mark comprises:

Reduce the distance difference between the described point geometrically corrected and mark described is instructed a little and curvature difference; And

Reduce the distance difference between the described point geometrically corrected and the described unique point of mark and curvature difference.

20. methods according to claim 19, wherein instruct the guidance point corresponding with the described feature detected in mapping to associate to comprise with described by feature tag:

Perform nearest-neighbors search is used to guide edge each nearest unique point instructing point with mark;

Mark is used for and the described each feature tag instructing a little nearest each nearest unique point instructing edge;

Determine to identify which feature tag for the described each most of described nearest unique point a little that instructs at edge that instructs; And

The each guidance on edge is instructed a little to associate with described the described feature tag identified for described most of described unique point.