WO2001059709A1 - Internet-based method and apparatus for generating caricatures - Google Patents

Abstract

An Internet-based system is described for generating line art caricatures or other caricatures based upon photographs selected by a user. The caricatures may be employed to personalize computer messages, such as birthday greetings and holiday greetings (100). Alternatively, the caricatures may be employed as Avatars for use within chat rooms or multi-player computer games (100). The caricatures are created (110) by automatically locating image zones (126) within the selected photograph, then separately converting features within the zones into vector-based caricature representations (128) using conversion techniques optimized for use with each particular zone. After the separate features have been independently converted into a vector-based representation, the features are combined and then a caricature is rendered (116). Thereafter, the user may choose to modify the caricature to reflect a particular selected artistic style (142), such as a style reminiscent of popular comic book, television or motion picture cartoons. Alternatively, the user may choose to merge the caricature with pre-stored caricatures (242) corresponding to particular cartoon characters, celebrities or historical figures.

Description

INTERNET-BASED METHOD AND APPARATUS FOR GENERATING

CARICATURES

BACKGROUND OF THE INVENTION

1. Field of the Invention The invention generally relates to e-commerce web sites and in particular to an e-commerce web site for automatically converting an input image, such as a user photograph, into a line art caricature or other caricature.

2. Description of Related Art

Millions of e-mail messages or other computer mail messages are transmitted via the Internet every day. As the speed and bandwidth of the Internet has increased, and as the speed of modems used by individual users has increased, it is becoming increasingly feasible to send a photograph of the sender along with the computer message to thereby personalize the message. This is particularly desirable for holiday greetings, birthday greetings and the like. Likewise, it has become feasible for participants of chat rooms, multi-user computer games, and the like, to employ visual Avatars for use in representing themselves within the chat rooms or computer games. Accordingly, some Internet users now scan a photograph of themselves into their computer to create a bit-mapped image for forwarding along with computer messages or for use as an Avatar. Often, however, users are unsatisfied with the bit-mapped photographic image, and would prefer to modify the image as needed to generate an image with which they are more comfortable. In other cases, users might want to provide an amusing caricature of themselves, rather than a simple bit-mapped photograph. To this end, a user may wish to convert his or her picture into a cartoon caricature having a particular artistic style, perhaps one reminiscent of popular comic book, television, or motion picture animation styles. Also, bit-mapped images are typically fairly large images requiring considerable resources to upload and download via the Internet. Accordingly, many users might prefer to convert their bit-mapped photograph into a more compact image format, such as vector-based format, before sending via the Internet.

Accordingly, it would be desirable to provide an Internet web site for permitting users to easily convert bit-mapped photographs into vector-based caricature images, to selectively enhance or modify aspects of the caricature images, or to convert the caricature into a cartoon having a selected artistic style. Heretofore, as far as the inventors are aware, no Internet web sites are available providing this combination of features Moreover, an attempt to create such a web site using conventional automated image processing techniques may not be feasible Conventional techniques for converting a bit-mapped photograph into a vector-based image typically process the entire input photograph to extract the vectors, often resulting in an output image that emphasizes shadows or other artifacts in the photograph unrelated to the actual features of the object of the photograph, such as the eyes, nose and mouth of a person pictured in a photograph Hence, such conventional techniques may not be suitable for use in a web site for converting an input photograph into a caπcature image Conventional techniques for enhancing or modifying aspects of a photograph also typically process the entire input image resulting in similar problems For example, one technique that may be employed for modifying aspects of an image is to morph the image together with another image However, typical morphing techniques are applied uniformly throughout the images, resulting in a final image wherein distinctive facial features, such as those of the eyes, nose, and mouth, are often considerably distorted In this regard, portions of the eyes of one image may be morphed with portions of the nose of another image resulting in a highly distorted final image These problems are even more significant when morphmg a photograph with a cartoon character, typically resulting in a highly comical but rather unsatisfactory final image Moreover, conventional automated morphing techniques typically require considerable processing power and hence may be relatively slow or expensive to perform, particularly for use withm a heavily accessed Internet site Conventional morphing techniques can be fairly complex and many Internet users may not wish to learn techniques needed to control a potentially complex morphmg process Hence, such conventional techniques may not be suitable for use in a web site to selectively enhance or modify aspects of caπcature images or to convert caπcatures into cartoons Also, conventional morphing techniques typically require that one input image be morphed with another image and hence may not be suitable for generating a cartoon caπcature merely having a certain selected artistic style

Furthermore, as the speed and bandwidth of the Internet increases, it becomes increasingly feasible to employ animation in connection with computer messages, chat room Avatars, and the like Problems associated with conventional techniques for generating and processing photographs, Avatars, and the like are even more significant in circumstances wherein the images need to be animated

Accordingly, it is desirable to provide an Internet web site for permitting users to easily convert bit-mapped photographs or other images into vector-based caπcature images, to selectively enhance or modify aspects of the caricature images, or to convert the caricatures into cartoon caricatures having a selected artistic style. It is also desirable to provide improved image processing techniques for use with such a web site. In particular, it would be desirable to provide a technique for easily and efficiently converting a bit-mapped image into a caricature image, which retains distinctive facial features of the bit-mapped image. It also would be desirable to provide a technique for efficiently converting the caricature into a cartoon which does not result in a highly distorted final image and which permits control of the style of animation. It also would be desirable to provide a technique for efficiently combining a pair of caricature images together in a manner that does not result in a highly distorted final image. It is to these ends that aspects of the present invention are primarily directed.

SUMMARY OF THE INVENTION

In accordance with a first aspect of the invention, an interactive network web site is provided that includes an image selection unit for receiving a user selection of an image and a caricature generation unit for automatically generating a caricature image representative of the selected image. The term caricature as used herein is not intended to imply that the caricature is necessarily an exaggerated or distorted representation of the selected image. For example, the caricature may be a realistic line art representation of the selected image.

In an exemplary embodiment of the first aspect of the invention, the interactive network web site is an Internet site. The selected image is a bit-mapped photograph either received via the Internet or selected from a database of stored images. The selected image is converted to a vector-based caricature for embedding within a computer message, such as a holiday or birthday greeting. The caricature may also be used as an Avatar within chat rooms or multi-player computer games or for other purposes. By using a caricature, rather than a simple bit-mapped photograph, the image of the user is therefore far more distinctive than one merely containing an ordinary photograph. The use of a caricature is particularly advantageous for those who do not wish to use a photograph of themselves, perhaps because the photograph is unflattering or because the user is otherwise uncomfortable with transmitting a photograph, unmodified, via the Internet. Also, a vector-based image typically requires fewer bytes than a bit-mapped photograph and hence can be easier to transmit, store and display via the Internet. Also, in the exemplary embodiment of the first aspect of the invention, the caricature, once generated, can be modified to adopt a distinctive artistic style drawn from pre-stored artistic styles, such as styles based on popular television or motion picture cartoon characters or original artistic styles, or can be merged with another caricature. In this manner, the user can generate a more amusing and memorable caricature for use via the Internet.

In accordance with a second aspect of the invention, a method is provided for use by a computing system to convert an input image into a computer representation of a caricature image. Initially, a set of image zones is automatically located within the input image. Portions of the input image lying within each image zone are separately converted into individual respective line representations. The individual respective caricature representations are then combined to yield a combine caricature representation.

In an exemplary embodiment of the second aspect of the invention, the input image is a bit-mapped photograph of a face. The set of image zones automatically located within the input image are facial image zones such as the left eye, the right eye, the nose and the mouth. By first locating image zones within the bit-mapped photograph, then separately converting portions of the input image lying within the image zones into caricature representations, the resulting combined caricature representation can more effectively retain the distinctive features of the photograph than is achieved using conversion techniques of the type which uniformly process the entire input image. Hence, a more satisfactory caricature image may be generated. Preferably, separate conversion algorithms are used within the separate image zones, each specifically adapted for processing the distinctive features of the respective image zone. Also, preferably, the resulting caricature representation is defined by geometrical nodes associated with each image zone, which collectively define the features within the image zone. Hence, the resulting image representation is highly compact and can be easily processed, perhaps to generate multiple caricatures each adopting a different distinctive artistic style.

In accordance with a third aspect of the invention, a method is provided for use by a computing system to render a caricature image adopting a selected artistic style. In accordance with the method, a caricature image representation is input and an artistic style is selected. Then, an image is rendered based on the caricature image representation while applying the selected artistic style.

In an exemplary embodiment of the third aspect of the invention, the caricature image representation is a vector-based caricature representation generated from a user image or other image. The selected artistic style is representative of popular television or motion picture cartoon characters or an original artistic style. The image rendered by the computing system based on the vector-based caricature and the selected artistic style is a cartoon caricature. The artistic style defines general features for use in generating the cartoon caricature, such as whether the eyes, nose and mouth of the caricature should be large or small. The artistic style also defines colors or line drawing textures for use in rendering the caricature, such as whether lines should be bold and curved or thin, rough and angular. By rendering a caricature so as to adopt a selected artistic style, a memorable line caricature is thereby created for use via the Internet or for other applications.

In accordance with a fourth aspect of the invention, a method is provided for use by a computing system to combine computer representations of first and second images. Initially, a set of corresponding image zones within the first and second images are located. The image zones of the first image are separately modified based on characteristics of corresponding image zones of the second image to yield separately modified image zones. Then, the separately modified image zones are combined to yield a computer representation of a modified image.

System embodiments are also provided. Other objects, features and advantages of the invention will be apparent from the descriptions provided below in combination with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an Internet-based method for generating a computer message having a caricature incorporated therein.

FIG. 2 illustrates a method for generating a caricature based on an input bit-mapped photograph for use within the method of FIG. 1. FIG. 3 illustrates a method of modifying the caricature generated by the method of FIG.

2 based on a pre-stored artistic style also for use within the Internet-based method of FIG. 1.

FIG. 4 graphically illustrates the generation of a caricature based on a bit-mapped photograph performed by the method of FIG. 2 and further graphically illustrates modification of the caricature based on an artistic style as performed by the method of FIG. 3. FIG. 5 illustrates a method of merging a caricature generated by the method of FIG. 2 with another caricature also for use within the Internet-based method of FIG. 1. FIG. 6 illustrates a system for performing the methods of FIGS. 1 - 5. DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

With reference to the figures, preferred and exemplary embodiments of the invention will now be descnbed. The techniques of the invention will pπmaπly be descnbed with reference to an Internet-based method for generating a computer message having a line art cancature or other caπcature embedded therein. The cancature is generated by converting a photograph mto a cancature image. Then, depending upon the programming of the system and the commands entered by a user, the cancature may be modified so as to adopt a selected artistic style or merged with another caπcature. However, pπnciples of the invention are applicable for use in other systems and to achieve other ends. The techniques are not limited for use with interactive web sites but may be employed in connection with stand-alone computers or computer kiosks. The techniques descnbed herein for converting a photograph into a cancature need not be performed in conjunction with generating a computer message. The techniques for modifying a cancature so as to adopt a selected artistic style need not be performed in conjunction with images denved from onginal bit-mapped photographs. Accordingly, it should be understood that the examples descnbed herein are provided for illustrative purposes only and are not to be taken as limiting the scope of the invention as set forth in the accompanying claims.

Initially, the Internet-based method for generating a computer message having a cancature embedded therein will be descnbed with reference to FIG. 1. Techniques for converting a bitmapped photograph mto a cancature will be described with reference to FIGS. 2 and 4. Techniques for modifying the cancature to adopt a selected artistic style will be descnbed with reference to FIG. 3 and 4. Techniques for merging the cancature with another cancature will be descnbed with reference to FIG. 5. Then a system performing the techniques of FIGS. 1 - 5 will bnefly be summanzed with reference to FIG. 6.

The method of FIG. 1 is performed by an interactive network web site computer system connected to the Internet and accessible as an Internet web site by Internet users via a web browser. Initially, at step 100, the user selects a general format for the computer message from a group of predetermined message formats stored within the computer system. Each computer message format includes space for a picture, such as a picture of a person, couple, house, pet, boat, car, skyline, honzon, landscape etc. Thousands of individual computer formats are preferably provided, categoπzed according to the general purpose of the computer message. Examples of general categones include holiday greetings, birthday greetings, reminders, anniversary greetings, vacation greetings, and wedding invitations. Within each general category, numerous specific formats are provided corresponding to the category. For example, for holidays, separate formats are provided for Christmas, Easter, Chanukah, Fourth of July, etc. For birthday greetings, specific formats are provided for different age groups as well as for different genders. Once the user has selected a format, then, at step 102, the user inputs or selects specific text or sound recordings for incorporation into the format. For example, the user may modify a prewritten birthday greeting to personalize the greeting or may provide entirely new text to replace prewritten greetings. The user may recite a message into a microphone of the user's computer, which digitizes the voice, sends it to the web site computer for incorporation within the message. At step 104, the computer system embeds the text or voice recording, if any, provided by the user into the computer message format.

As noted, each message format includes space for a picture. At step 106, the system determines the subject mater of picture and the particular camera angle required for the picture to be inserted into the message, based upon the pre-stored format of the message. For example, for a message format with space for a facial photograph, the system determines whether a front view, side view, left angle, right etc. is required, then prompts the user to input a bit-mapped photograph having the prescribed camera angle. The user inputs the photograph at step 108 by transmitting the image via the Internet from a computer of the user to the web site computer system. The bit-mapped photograph is initially generated using a digital camera or other digital input device or by scanning a photograph using a digital scanner or by uploading or downloading an image from a computer database. For example, if the user's computer has a digital camera mounted thereto, the system may prompt the user to take a digital photograph using the digital camera for immediate transmission via the Internet. Alternatively, the user selects from among photographs previously input into the system and stored therein or from among previously converted vector-based images. To this end, each individual user may initially download numerous photographs into the computer system for storage therein to provide a library of images. The library may include various pictures taken at different angles of the user, his or her friends or family, pets, houses, boats etc. In any case, once the user has input a bit-mapped photograph or selected from among pre-stored photographs then, at step 110, the system generates a caricature of the photograph and displays the caricature for the user by transmitting the caricature via the Internet to the user's computer. The caricature is a vector-based image. Geometric nodes defining the vectors are stored by the system. Depending upon the programming of the computer system and upon any commands entered by the user, the generation of the caricature is performed entirely automatically or is performed in conjunction with specific user commands. At step 112, subject to the control of the user, the caricature may be modified to adopt the characteristics of a selected artistic style to yield a cartoon caricature or may be merged with another caricature. At step 114, the user has the opportunity to review the caricature that has been generated and to accept or refuse the caricature. Assuming that the user is satisfied with the caricature, then the system embeds the caricature within the selected computer message format, at step 116, and then sends the computer message at step 118.

Thus, FIG. 1 illustrates an Internet-based technique for embedding caricatures within computer messages, particularly within holiday greetings, birthday greetings, and the like. In this manner, the user may personalize the greeting to a much greater extent than can be achieved merely by incorporating text greetings. Moreover, by generating and sending a caricature, rather than a simple bit-mapped photograph, the message received by the recipient is likely to be far more distinctive than one merely containing a simple bit-mapped photograph. The creation of a caricature is particularly advantageous for those who do not wish to send a simple photograph, perhaps because the photograph is unflattering or because they are simply uncomfortable with transmitting their photograph via the Internet. Also, as will be more fully explained below, in generating the caricature, the user has the opportunity to adjust or modify features to enhance the visual representation as desired.

Although described with reference to a system which first converts bit-mapped images into caricatures then modifies the caricature based on a selected artistic style, the general techniques of FIG. 1 alternatively may be performed to just create an initial caricature, or to just modify a caricature based on an artistic style, or to just merge a caricature image with another. Also, the system may be configured to convert among different types of image formats, such as from vector-based to vector-based rather than only from bitmapped to vector-based. The final caricatures or other final images need not be employed only in conjunction with computer messages, but may be employed as Avatars for use in chat rooms, multi-player computer games, and the like or may merely be printed out and not used via the Internet at all. Also, the final caricature for embedding in the message may be animated. Or the final caricature need not be embedded within a message at all. As will be described more fully below, the caricatures are internally represented as a set of geometrical nodes that facilitate animation. For the purposes of animation, the input image may be an entire sequence of images of a person, object or animal in motion, with each image converted into a caricature to achieve an animated sequence. The input image may comprise motion capture data from which animation can be generated. Depending upon the implementation, the system may incorporate the animated caricature within a message story vignette, perhaps to show the caricature blowing out the candles of a birthday cake. Also, 3-D caricatures may be generated by, for example, processing multiple views of the same person taken from different angles. Furthermore, the computer network of FIG. 1 need not be the Internet, but may be an Intranet or other limited computer network. Moreover, when implemented in connection with the Internet, many of the steps of the invention may be performed within the user's computer rather than within the web site server, perhaps using plug- ins or the like. Also, aspects of the method need not be implemented via a computer network at all, but instead may be implemented within a stand-alone computer or kiosk. To this end, software performing the steps of the method may be stored on CD-ROM's, floppy disks or the like for loading into user computers. Also the method may be implemented via a channel rather than a web site. As can be appreciated, numerous other modifications or applications of the general techniques of FIG. 1 may be provided consistent with the general principles of the invention.

Techniques for converting a bit-mapped image into a vector-based caricature will now be described for an example involving a facial image with reference to FIG. 2 in conjunction with FIG. 4. The steps of FIG. 2 may be modified as needed to process other images, such as houses, cars, boats, etc. Initially, at step 120, the system verifies the integrity of the bit-mapped image (item 121 of FIG.4) input by the user. To this end, the system verifies that the exposure, contrast, focus and other image quality features are all within acceptable pre-determined ranges. If the image fails the integrity test, the user is prompted to input an alternative image or to otherwise correct the image. The system preferably incorporates various tools for altering the image, such as changing the contrast and the like, so that the user may attempt to remedy any defects in the image. Alternatively, the system may automatically perform steps directed to remedying the image defects on behalf of the user, then present the corrected image to the user for verification that the image is acceptable.

At step 122, the system then analyzes the bit-mapped image to extract the outline of the face (item 123 of FIG. 4) contained within the image. This is performed by analyzing the differences between two or more adjacent pixels, such as differences in brightness, contrast, color and the like. One exemplary technique that may be employed by the system is to process the bit- mapped image using the following filter applied to each block of nine adjacent bits:

-1 -1 -1

-1 8 -1

-1 -1 -1

The aforementioned filter operates to highlight changes in contrast within the image. Any part of the image where an absolute filtered value is higher than a predetermined threshold is defined as an "edge" of the image. Additional information regarding edge detection may be found within a chapter entitled Noise Thresholding in Edge Images by John Schlag, (pp. 105- 106) of Graphics Gems edited by Andrew S, Glassner, Academic Press (1990). Having identified the edge of the input image, the system next locates predetermined image zones such as the eyes, nose, mouth, eyebrows, forehead, hair profile, hairline, jaw line, chin, cheekbones, and ears within the image by applying a facial image zone template to the outline. (For other image types, such as houses or boats, other image zones are defined.) More specifically, within step 124, the system inputs an image zone template (item 125 of FIG. 4) having predetermined image zones. In the example, of FIG. 4, only a few of the image zones are illustrated. At step 126, the system adjusts the size and location of the image zone template to fit the outline of the bit-mapped image. This may be achieved by fitting an face perimeter template, such as an oval or round template, to the of the image while simultaneously adjusting the sizes and locations of internal image zones in accordance with the adjusted size and location of the oval face perimeter template. This may be performed by minimizing an average distance between points on the oval perimeter and points within the outline derived from the bit-mapped image. When fitted properly, the individual image zones of the template fit over the corresponding facial features in the bit-mapped image. In other words, the right eye zone template fits over the right eye of the image; the mouth zone template fits over the mouth of the image, and so on. To ensure proper fit, the technique employed to fit the oval perimeter to the face is preferably performed so as to be relatively unaffected by any portions of the outline that may represent beard or hair, rather than facial outline. To this end, left and right portions of the oval template are more closely fitted to left and right portions of the bit-mapped image outline as compared with the top and bottom portions of the oval perimeter and top and bottom portions of the image outline. Also, the forehead zone may be independently fitted first, with the other facial zones thereafter fitted below the hairline. Note that the operations of step 126 are performed internally within the computer system using digitized data not requiring actual super- positioning of a graphic template over the facial image. The graphic template illustrated in FIG. 4 is provided to symbolically illustrate the operations performed by the computer and not to imply that a graphic template is required.

Thus, the system automatically identifies the locations of pre-determined features of the input image without requiring further user input. This is a significant advantage over systems that would require the user to separately identify facial features, which may be a laborious process. However, for those users who wish to assist the computer system in identifying the facial features, the system may be programmed to display the bit-mapped image along with a graphical image zone template and permit the user to manually superimpose one above the other.

Having identified the location of the various image zones within the bit-mapped image, the system then, at step 128, independently processes portions of the bit-mapped image lying within each respective image zone to generate a vector-based representation of features within the zone. To this end, the system analyzes the bit-mapped image lying within each respective image zone and identifies pre-determined geometric nodes therein. The nodes include specific geometrical points, tangents and the like. Within item 131 of FIG.4, five exemplary nodes (A) - (E) are identified within the bit-mapped image of the right eye. As can be seen, the nodes correspond to the left and right corners of the eye, the center of the pupil, and horizontal tangents lying along the top and bottom portions of the eye. To provide a more detailed representation of the eye, a greater number of nodes may be employed. (The eyebrow of item 131 is analyzed separately from the eye; resulting in separate nodes, not specifically show in FIG. 4.) Also, in step 128 the bits defining the edge of the face are processed to generate a vector representation of the outline of the face. The characteristics of particular nodes for identification within each zone within step 128 are pre-determined and are stored within the system. Hence, during step 128, the system need only locate the position or other characteristic of each predetermined node within the image zone based on the identifying characteristics of the nodes. The nodes, once identified, are then saved by the system for use in generating a caricature. A caricature of the eye, for example, can be subsequently generated by drawing an ellipse connecting the left and right corner nodes and the upper and lower tangent nodes and by drawing a pupil having the size and location identified by the pupil node. In this manner, the original bit-mapped image is converted to a compact set of geometrical nodes from which the caricature can be generated subsequently. An exemplary caricature is shown as item 135 of FIG. 4.

As noted, the conversion of the bit-mapped representation to a vector-based representation is performed separately and independently for the image zones using separate conversion techniques appropriate to the features within those zones. For example, the conversion algorithm for identifying the geometrical nodes within the right eye is optimized for identifying distinctive features of the eye, such as the left and right corners, upper and lower eyelid tangents and the size and center of the pupil. The conversion algorithm for converting the bit-mapped image corresponding to the mouth into a caricature image of the mouth is optimized for identifying features of the mouth.

For the example involving the eye, an exemplary conversion algorithm for use in locating the pupil of the eye is as follows:

Above equation is used to find the pupil by finding the best circle fitting a set of points by minimizing the integral square error (E). Values x, and y, represent the locations of bits corresponding to the pupil. Variable r represents the radius of the pupil and variables x_c and y_c represent the location of the center of the pupil. Variables r, x_c, and y_c are determined by minimizing E over all of bits x, and y.. In this manner, the size and position of the pupil is identified and the bit-mapped image of the pupil is thereby converted into a vector representation defined by solely by r and center location x_c, and y_c, which define node (e) representing the pupil.

Hence, a graphic representation of the pupil thereafter may be generated merely by displaying a filled circle of radius r and center location x_c, and y_c. The same conversion algorithm may be applied for use with other round image zone features as well, such as automobile wheels within a picture of an automobile being converted into a caricature. To approximate the outline of the eye, or any other generally elliptical feature, the following equation may be employed to first determine the semi -major and semi-minor axis of the elliptical feature: o

The semi-major (a) and semi-minor (b) axes of the ellipse are determined for a given set of n points on the average where the coordinates x_t and y • are calculated with respect to a system of coordinates having its center at the center of the ellipse, and the axes parallel to the ellipse axes. Once the semi-major and semi-minor axes are determined, other equations may be employed to determine the location of the center of the ellipse with respect to other features of the caricature. Then, the left and right and top and bottom points on the ellipse are determine for storage as nodes (a) - (d). In this manner, the size, shape and position of an elliptical facial feature are identified and the bit-mapped image of the feature is thereby converted into a vector representation. Hence, a graphic representation of the elliptical facial feature thereafter may be generated merely by displaying an ellipse of the determined size, shape and location. For the eye, as an alternative, other conversion algorithms maybe employed for directly locating tangents (b) and (d) and corners (a) and (c) which do not require first determining the semi-major and semi- minor axes. Also, more complex conversion algorithms may be employed to generate a greater number of nodes per facial feature so as to more precisely approximate the feature.

Thus, within each image zone, one or more conversion algorithms are employed, with the particular conversion algorithms selected based on the features expected to be found in the image zone. In this manner, the overall bit-mapped image to caricature image conversion algorithm is optimized for each image zone. For an image zone expected to contain circular and elliptical features, the above conversion algorithms may be employed. For an image zone expected to contain curved lines, such an image zone covering the eyebrows, conversion algorithms adapted to locating curved lines are employed. For an image zone expected to contain square features, such image zone applied to the windows of a house, conversion algorithms adapted to locating squares are employed. Each conversion algorithm converts portions of the image into one or more geometrical nodes, which may be represented by numerical values such as object center locations, radii, lengths, tangent points, etc. The geometrical nodes are stored along with an indication of the type of node, i.e. whether the node defines a circle, a portion of an ellipse, a straight line etc. As can be appreciated, numerous different algorithms may be employed depending upon the subject matter of the image and the particular image zone contained therein. No attempt is made herein to describe or identify the many possible conversion algorithms. Those skilled in the art can readily develop conversion algorithms adapted for particular image types and image zones. General information pertaining to the generation of a vector-based representation may be found within Vectorization of Digital images Using Algebraic Curves, Zhang et al., Computing and Graphics, Vol. 22, No. 1, pp. 99-101 (1998).

Following the steps of FIG. 2, the caricature is generated by drawing an image using all the geometric nodes identified in step 128 with all the image zones (item 135 of FIG. 4) corresponding to the original input image. When generating the caricature, the system preferably applies different textures to different image features. For example, when drawing the eyebrow, the system applies a predetermined rough texture. Hence, although the eyebrow may be internally represented using only a set of nodes defining a simple geometrical arc, the eyebrow drawn in the caricature is a more realistic eyebrow. In general, every feature of the image as represented by the geometrical nodes may be processed using textures when generating the caricature. When generating the caricature, the system also preferably applies different colors to the different image features. The colors may be determined by sampling corresponding portions of the input image. As one example, the pupil may be sampled to determine the color of the eye, then that color is employed when generating the pupil in the caricature. Alternatively, rather than applying colors and textures, a simple line art caricature can be generated.

The caricature is displayed to the user to verify that the character is acceptable to the user. If acceptable, the caricature may thereafter be embedded into the aforementioned computer message or perhaps employed as an Avatar for use within a chat room, multi-player computer game or the like. If the caricature is unacceptable, the user may generate a new caricature based upon a newly input photograph or, depending upon the programming of the system, may adjust various parameters used in generating the caricature in an attempt to provide a more acceptable caricature. For example, the system may provide tools to permits the user to manually control the identification of the location of the eyes, nose and mouth if it appears that the automatic determination performed by the system is not accurate. Preferably, also, the user is provided with tools for controlling the textures use in generating the caricature from the geometrical nodes so as to provide greater control over the resulting caricature. Also, the user is preferably provided with tools to permit the caricature to be modified so as to incorporate a wig, hat, etc, or if the caricature only covers the face, to supenmpose the face on a cartoon body or within a cartoon vehicle. Other tools are provided to permit the user to animate the caπcature, perhaps to make the eyes blink or the mouth move. Since the cancature is internally represented by nodes, it is easy to animate the caπcature by sequentially rendenng the cancature while changing node locations. For example, an eye can be made to blink by alternately changing the spacing between nodes (b) and (d) of item 131 of FIG. 4. As another example, the mouth can be made to open and close by alternately changing the spacing between nodes define top and bottom portions of the lips. If a voice recording is provided for use with the caπcature, the movement of the mouth may be synchronized with the voice to provide the illusion that the caπcature is speaking the words of the voice recording.

Thus, FIG. 2 illustrates a method for converting a bit-mapped image into a cancature. Although descnbed pnmanly with reference to the conversion of the facial images, the general techniques of FIG. 2 may be applied to converting other bit-mapped images into cancatures, such as converting the photograph of a house into a caπcature of the house or converting the photograph of a person's body to a cancature. To this end, a separate set of image zones are defined, corresponding to standard features of homes. For example, an image zone corresponding to the doors and windows may be employed within a generally rectangular outline corresponding to the outline of the house.

In many cases, the user may be satisfied with the cancature generated via the steps of FIG. 2 and will perform no further processing. In other cases, the user may wish to modify the caπcature to adopt a predetermined artistic style or to merge the caπcature with another caπcature. To modify the caricature to adopt a selected artistic style, the steps of FIG. 3 are performed. To merge the caπcature with another, the steps of FIG. 5 are performed. Referπng first to FIG. 3, at step 140, the system inputs an artistic style (represented by item 141 of FIG. 4) selected by the user for use in modifying the caπcature. Exemplary artistic styles include styles associated with popular television programs, movies, comic books etc. The artistic styles may also be representative of animation techniques, such as cardboard cutout animation or pen and ink animation. One specific style is the Japanimation style associated with many Japanese cartoons. Each style is represented in vector form using the same types of geometnc nodes as with the cancature, but sized and shaped so as to correspond with the particular artistic style. Hence, if the artistic style calls for characters with large bulbous eyes, nose and mouth, the nodes associated with the style are thereby pre-set to be wide apart (item 141 of FIG. 4). However, whereas a particular caricature may be represented using a large number of nodes per image zone so as to fairly closely represent the original image, the style typically is defined using relatively few image nodes, so as to merely convey the general characteristics of cartoon characters of the artistic style rather than to correspond closely to a specific cartoon character. Each style is further represented by a set of drawing textures, color palettes and other rendering parameters for use in generating a caricature. Thus, for a style of animation wherein the characters appear as cardboard cutouts, the drawing textures cause the caricature to be rendered in a manner reminiscent of cardboard cut-outs. For a style of animation wherein the characters are drawn using thick, bold curvy lines, with vivid primary colors, the drawing textures cause the caricature to be rendered using bold, curvy lines and the color palette specifies primary colors. For a style of animation wherein the characters are drawn using thin, rough lines, with pastel colors, the drawing textures specify thin, rough lines and the color palette specifies pastel colors. As can be appreciated, numerous sets of textures, colors and other rendering features may be defined for the various artistic styles, each selected to convey the distinctive features of the associated style. The information specifying each artistic style is initially input by an artist or other operator of the system, who specifies the particular geometric nodes, color palettes, drawing textures and the like for each style. Within item 141 of FIG.4, note that the computer system need not actually store the image of the cartoon shown therein. The image of the cartoon is provided in the figure merely to help to illustrate the idea of a general artistic style. Within step 142, the system then adjusts the nodes of the caricature generated based upon the user input photograph with the nodes of the selected artistic style. For example, the system extracts the image corresponding to the right eye of the user caricature (item 147 of FIG.4) and adjusts the image based on the nodes of the artistic style (item 145 of FIG.4) to generate a more bulbous right eye (item 149 of FIG.4). A variety of techniques may be employed to perform the actual adjustment operation. In one example, the system first determines, for each node within the user caricature, a new location for the node by averaging the node locations from the user caricature with those of the artistic style based upon a predetermined biasing or weight factor which biases the average either toward the node location of the user caricature or to the node location of the artistic style. For example, the blending process may be fully biased towards the artistic style by setting the new node locations for the user caricature to be equal to the node locations of the artistic style. As a result, within a particular facial zone, the adjusted caricature image will more closely conform to the artistic style, rather than to the original user image. In another example, the biasing is weighted heavily in favor of the user caricature such that the adjusted facial feature perhaps only slightly adopts the characteristics of the artistic style. Preferably, the biasing factors are input by the artist or other operator along with the other information defining the artistic style. To this end, the artist or other operator may subjectively evaluate the degree of distinctiveness of various features within the artistic style and assign very distinctive features a high biasing level and less distinctive features a lower biasing level. In this manner, when the adjustment procedure is performed, highly distinctive features of the artistic style are evoked within the adjusted image. Depending upon the programming of the system, the user may be permitted to adjust the weight factors as well. Also, as noted above, the artistic style may provide fewer nodes per facial feature than the user caricature provides. If so, the adjustment process thus far described is modified as needed to determine new locations for each node of the caricature based on the fewer node locations of the artistic style.

Thus, following step 142, the system has generated a new set of node locations corresponding to the pre-determined image zones. Next, at step 144, the system automatically renders a new caricature based on the new set of node locations using the aforementioned artistic style textures, color palettes and the like to yield a final composite image (item 151 of FIG. 4).

If the original caricature included colors, the new caricature can be rendered using the original colors modified using the color palette of the artistic style. As can be seen from FIG.4, the final image retains characteristics drawn from both the initial user photograph and the selected artistic style. The final image is displayed to the user and, if satisfactory, is employed by the user perhaps as part of the aforementioned computer message or perhaps as an Avatar for use in a chat room or multi-game user computer game. If unacceptable, the user may select a different artistic style and repeat the process illustrated in FIG. 3 or, if permitted by the system, may attempt to selectively control the rendering process to achieve a more satisfactory result. Depending upon the programming of the system, the user caricature may be automatically rendered using many different artistic styles, with thumbnail sketches of each resulting caricature simultaneously displayed so that the user may select one.

Thus, what has been described with reference to FIG. 3 is a technique for generating a new caricature based on the original caricature but rendered in a manner consistent with a selected artistic style. With reference to FIG. 5, techniques for merging the original caricature with another caricature will be described. At step 240, the system inputs a new caricature selected by the user for merging with the caricature previously generated by the system based on the user photograph The new cancature may be representative, for example, of a celebnty, histoπcal figure or cartoon character Such cancatures are preferably input mto the system by an artist or other operator The artist or other operator inputs the specific geometnc nodes corresponding to the facial features within each zone of the caπcature Preferably, a relatively large number of nodes are input to permit the cancature to closely resemble the particular celebnty, histoncal figure or cartoon character. Alternatively, the new caπcature may be another cancature generated by the user, perhaps from a different input photograph. In either case, each cancature is stored within the system in vector form using the aforementioned geometπcal nodes Within step 242, the system then generates new node locations based on the nodes of the user cancature and the nodes of the selected caπcature This may be performed using the same method of FIG. 3 to average node locations from the user cancature with those of the other cancature based upon a predetermined biasing or weight factors The system generates a final image of the merged caricature based on the new node locations for display to the user and, if satisfactory, the new caπcature is employed by the user as part of the aforementioned computer message or perhaps as an Avatar for use in a chat room or multi-game user computer game. If unacceptable, the user may choose to select a different pre-stored cancature for creating a merged character and repeat the process illustrated in FIG.5 or, if permitted by the system, may attempt to selectively control the merging process to achieve a more satisfactory result. Depending upon the programming of the system, the user caricature may be automatically merged with many different pre-stored cancatures, with thumbnail sketches of each resulting merged cancature simultaneously displayed so that the user may select one.

Thus, what has been descnbed with reference to FIG. 5 is a technique for merging a cancature denved from a photograph with a pre-stored cancatuie Alternatively, the general techniques of FIG. 5 may be employed to merge two separate cancatures provided by the user Alternatively, the system may be programmed to enable the user to select two pre-stored cancatures for merging together The system is not restncted to merging facial cancatures together. Rather, the technique may also be applied to merging other cancatures together, such as merging the cancature of a real house with that of a cartoon house. Also, the general technique of performing merging operations, on a zone-by-zone basis, may be applied to other merging techniques as well, including those not limited to the merging of cancatures together. The techniques of FIGS. 1 - 5 have been descnbed with respect to flow charts. Each block within the flowcharts illustrated both a method step and an apparatus element for performing the corresponding method step Accordingly, FIGS.1 - 5 also represent an image processing system

The high-level features of the system are also represented withm FIG. 6

Bnefly, FIG. 6 illustrates functional components of a web site 300 performing the techniques of FIG. 1 - 5 The system includes an image selection unit 301 for inputting an image provided by a user via an input unit 302 or for allowing the user to select a pre-stored image from an image database 303 The system also includes a cancature generation unit 304 for converting the input image into a cancature, perhaps for transmission via a transmission unit 306 such as a modem over the Internet 308 or other computer network Caπcature generation unit 304 converts a selected image into a cancature image using an image to geometncal node conversion unit 310, which extracts a set of geometncal nodes from the selected image, and a cancature rendenng unit 312, which renders a cancature based on the extracted geometncal nodes The nodes are extracted from the selected image on a zone-by-zone basis from set of predefined image zones, such as the eyes, nose and mouth The zones are located within the image using an image zone locator unit 315 The geometncal nodes are stored in a node storage unit 313 The web site also includes an artistic style conversion unit 316 which may be used to modify the cancature to adopt an artistic style selected from an artistic style database 318 The system also includes a cancature merge unit 320 which may be used to merge a caπcature generated by caπcature generation unit 304 with a pre-stored caπcature stored in vector-based format within node database 313 Specific operations performed by the vanous components illustrated in FIG. 6 are descnbed above with reference to FIGS. 1 - 5

The descnption of the exemplary embodiments is provided to enable any person skilled in the art to make or use the invention Vanous modifications to the embodiments will be readily apparent and genenc pπnciples defined herein may be applied to other embodiments Thus, the invention is not intended to be limited to the specific embodiments shown and descnbed herein but is to be accorded the widest scope consistent with the pπnciples and novel features disclosed herein

Claims

1. An interactive network web site comprising: an image selection unit for receiving a user selection of an image; and a caricature generation unit for automatically generating a caricature image representative of the selected image.

2. The interactive network web site of claim 1 wherein the image selection unit includes an input unit for inputting an image provided by the user.

3. The interactive network web site of claim 1 further including a selected image verification unit for determination whether the selected image meets a predetermined threshold of image integrity.

4. The interactive network web site of claim 3 wherein the input image verification unit determines whether the selected image meets a predetermined threshold by analyzing one or more of the exposure, contrast and focus of the image.

5. The interactive network web site of claim 1 wherein the image selection unit includes storage unit for storing images for selection by the user.

6. The interactive network web site of claim 1 wherein the image selected by the user is a bit-mapped image and wherein the caricature generation unit generates a vector representation of the bit-mapped input image.

7. The interactive network web site of claim 1 wherein the caricature generation unit locates a set of image zones within the selected image, separately converts portions of the selected image lying within each image zone into respective caricature image portions, and combines the caricature image portions to yield a resulting combined caricature image.

8. The interactive network web site of claim 7 further including: an artistic style storage unit for storing information representative of a plurality of artistic styles an artistic style selection unit for receiving a user selection of one of the artistic styles; and an artistic style conversion unit for converting the caricature image based on the selected artistic style.

9. The interactive network web site of claim 8 wherein the information representative of an artistic style includes information defining colors and textures.

10. The interactive network web site of claim 1 further including a message generation unit for generating a message for transmission via the interactive network, for embedding the caricature image into the message, and for transmitting the message.

11. The interactive network web site of claim 10 wherein the message generation unit receives a user selection of a message format from a set of pre-stored message formats and embeds the caricature image into the selected message format.

12. The interactive network web site of claim 11 wherein the pre-stored image formats include one or more formats directed to holiday greetings, birthday greetings, and reminder messages.

13. The interactive network web site of claim 10 wherein the message generation unit inputs digitized speech provided by the user and embeds the digitized speech in the message.

14. The interactive network web site of claim 1 further including a caricature merge unit for merging the caricature image with another caricature image to create a merged caricature image.

15. The interactive network web site of claim 14 wherein the caricature merge unit locates a set of corresponding image zones within the caricature images, separately merges portions of the caricature images within each image zone into respective merged image portions, and combines the merged image portions to yield a resulting combined merged caricature image.

16. The interactive network web site of claim 1 wherein the web site is connected to the Internet.

17. A method performed by an interactive network web site comprising the steps of: receiving a user selection of an image; and automatically generating a caricature image representative of the selected image.

18. The method of claim 17 wherein the step of receiving a user selection of an image includes the step of inputting an image provided by the user.

19. The method of claim 17 further including the step of analyzing the selected image to determine whether the image meets a predetermined threshold of image integrity and rejecting the image if it falls below the threshold.

20. The method of claim 19 wherein the step of analyzing the input image to determine whether the image meets a predetermined threshold includes the steps of analyzing one or more of the exposure, contrast and focus of the image.

21. The method of claim 17 wherein the step of receiving a user selection of an image includes the step of displaying stored images for selection by the user.

22. The method of claim 17 wherein the image selected by the user is a bit-mapped image and wherein the step of automatically generating a caricature image is performed to generate a vector representation of the bit-mapped input image.

23. The method of claim 17 wherein the step of automatically generating a caricature image locates a set of image zones within the selected image, separately converts portions of the input image lying within each image zone into respective caricature image portions, and combines the caricature image portions to yield a resulting combined caricature image.

24. The method of claim 17 further wherein the cancature image is a line art image.

25. The method of claim 17 further compnsing the steps of stoπng information representative of a plurality of artistic styles artistic style selection unit for receiving a user selection of one of the artistic styles, and converting the cancature image based on the selected artistic style.

26. The method of claim 25 wherein the information representative of an artistic style includes information defining colors and textures.

27. The method of claim 17 further including the steps of generating a message for transmission via the interactive network, for embedding the cancature image into the message, and for transmitting the message.

28. The method of claim 27 wherein the step of generating a message includes the steps of receiving a user selection of a message format from a set of pre-stored message formats and embedding the cancature image into the selected message format.

29. The method of claim 28 wherein the pre-stored image formats include one or more formats directed to holiday greetings, birthday greetings, and reminder messages.

30. The method of claim 27 further including the step of inputting digitized speech provided by the user and embedding the digitized speech in the message

31. The method of claim 17 further including the step of merging the cancature image with another cancature image to create a merged cancature image.

32. The method of claim 31 wherein the step of merging the cancature image with another caπcature image locates a set of corresponding image zones within the cancature images, separately merges portions of the cancature images within each image zone into respective merged image portions, and combines the merged image portions to yield a resulting combined merged cancature image.

33. A method, performed by a computing system, for converting an input image mto a computer representation of a cancature image compnsing the steps of: locating a set of image zones withm the input image; and separately converting portions of the input image lying withm each image zone into individual respective cancature representations; and combining the individual respective cancature representations to yield a combined cancature representation.

34. The method of claim 33 wherein the input image is a facial image and wherein the set of image zones are facial image zones.

35. The method of claim 33 wherein the facial image zones include the left eye, the nght eye, the nose and the mouth.

36. The method of claim 33 wherein the step of locating a set of image zones within the input image includes the steps of: associating a set of discrete image zone templates with the input image, with each discrete image zone template corresponding to a respective one of the image zones; and converting portions of the input image corresponding to each discrete image zone template into a respective cancature image

37. The method of claim 36 wherein the set of discrete image zone templates are constrained with respect to an image template outline and wherein the step of associating the set of image zone templates with the input image includes the steps of: extracting the outline of the input image; adjusting a size and location of the image template outline to match the outline of the input image while adjusting sizes and locations of the discrete image zone templates in accordance with the adjusted size and location of the image template outline; and determining the final adjusted locations of the image zone templates with respect to the input image.

38. The method of claim 33 wherein the input image is a bit-mapped image.

39. The method of claim 38 wherein the step of separately converting portions of the input image lying within each image zone into individual respective cancature representations includes the step of, for each respective template zone, applying a corresponding predetermined bit-map to cancature conversion algonthm to the portion of the bit-map lying withm the respective template zone.

40. The method of claim 39 wherein at least some of the corresponding predetermined bitmap to cancature conversion algonthms differ from one another.

41. The method of claim 40 wherein the input image is a facial image, the image zones of the facial image include the left eye, the nght eye, the nose and the mouth and wherein a first conversion algonthm is applied for the left and nght eye template zones, a second conversion algonthm is applied to the nose template zone and a third conversion algonthm is applied to the mouth template zone.

42. The method of claim 41 wherein the first conversion algoπthm is adapted for converting a bit-mapped representation of an eye to a cancature representation of an eye, the second conversion algonthm is adapted for converting a bit-mapped representation of a nose to a cancature representation of a nose, and the third conversion algonthm is adapted for converting a bit-mapped representation of a mouth to a cancature representation of a mouth.

43. The method of claim 33 wherein the caπcature image is a line art image.

44. A method, performed by a computing system, for rendenng a cancature image compπsing the steps of: inputting a caπcature image representation; selecting an artistic style; and rendenng an image based on the caricature image representation while applying the selected artistic style.

45. The method of claim 44 wherein the selected artistic style includes a specification of line textures and color palettes.

46. The method of claim 45 wherein the step of rendering the image comprises the steps of: locating a set of image zones associated with the caricature image; identifying geometrical artistic style characteristics for use with each respective image zone; separately modifying portions of the caricature image representation within the image zones based on the geometrical artistic style characteristics identified for use with each respective image zone to yield separately modified image zone portions; combining the separately modified image zone portions.

47. The method of claim 44 wherein the step of separately modifying portions of the caricature image within the image zone based on the geometrical artistic style characteristics identified for use with each respective image zone includes the step of, for each image zone, identifying a set of geometric features within the image zone and then adjusting the geometric features based on a corresponding set of geometric features of the artistic style for that zone.

48. The method of claim 47 wherein different zones are assigned different weight factors and wherein the images lying within the image zones of the caricature image are adjusted based on the corresponding set of geometric features of the artistic style to a degree based on the weight factor.

49. The method of claim 48 wherein the geometric features include predetermined geometric points and geometric tangents defining vectorized caricature images.

50. The method of claim 47 wherein the images lying within the image zones of the caricature image are adjusted by averaging locations of the corresponding geometric points within the caricature image and the artistic style to yield a set of average locations of geometric features.

51. The method of claim 44 wherein the artistic style is selected from a set of predetermined artistic styles representative of different types of art.

52. A method for combining computer representations of first and second images comprising the steps of: locating a set of corresponding image zones within the first and second images; separately modifying image zones of the first image based on characteristics of corresponding image zones of the second image to yield separately modified image zones; and combining the separately modified image zones to yield a modified image.

53. The method of claim 52 wherein the first and second images are caricature images.

54. The method of claim 53 wherein the first and second caricature images are facial images and wherein the set of image zones are facial image zones.

55. The method of claim 52 wherein the step of separately modifying image zones of the first image based on characteristics of corresponding image zones of the second image includes the step of, for each image zone, identifying corresponding sets of geometric features within the first and second images and then adjusting the images lying within the image zone of the first image based on the corresponding sets of geometric features from the second image.

56. The method of claim 55 wherein different zones are assigned different weight factors and wherein the images lying within the image zones of the first image are adjusted to a degree based on the weight factor.

57. The method of claim 52 wherein the image zones contain caricature images and the sets of features include one or more of predetermined geometric points or geometric tangents within the caricature images.

58. The method of claim 55 wherein the images lying within the image zones of the first image are adjusted by averaging locations of the corresponding geometric points within the first and second facial images to yield a set of average locations of geometric features, then modifying the images lying within the image zones of the first image based on the average locations of the geometric features of the second image.

59. The method of claim 52 wherein in the first image is a bit-mapped image and the second image is a caricature image and wherein the method further includes the steps of : extracting the outline of the bit-mapped image; extracting the outline of the caricature image; conforming the outline of the caricature image to that of the bit mapped image to yield a modified caricature outline; and combining the modified caricature outline with the modified image generated by combining the separately modified image zones to yield a combined modified image.

60. The method of claim 52 wherein the first image is a bit-mapped photograph and the second image is a cartoon.