WO2014094199A1

WO2014094199A1 - Facial movement based avatar animation

Info

Publication number: WO2014094199A1
Application number: PCT/CN2012/086739
Authority: WO
Inventors: Yangzhou Du; Wenlong Li; Xiaofeng Tong; Wei Hu; Yimin Zhang
Original assignee: Intel Corp
Current assignee: Intel Corp
Priority date: 2012-12-17
Filing date: 2012-12-17
Publication date: 2014-06-26
Anticipated expiration: 2015-06-17
Also published as: US9466142B2; CN104782120A; US20140218371A1; CN104782120B; US20170193684A1

Abstract

Avatars are animated using predetermined avatar images that are selected based on facial features of a user extracted from video of the user. A user's facial features are tracked in a live video, facial feature parameters are determined from the tracked features, and avatar images are selected based on the facial feature parameters. The selected images are then displayed are sent to another device for display. Selecting and displaying different avatar images as a user's facial movements change animates the avatar. An avatar image can be selected from a series of avatar images representing a particular facial movement, such as bunking. An avatar image can also be generated from multiple avatar feature images selected from multiple avatar feature image series associated with different regions of a user's face (eyes, mouth, nose, eyebrows), which allows different regions of the avatar to be animated independently.

Description

FACIAL MOVEMENT BASED AVATAR ANIMATION

BACKGROUND

[000 1] Users can be represented in software applications ard various platforms, such as gaming or social me dia platforms, by an avatar. Some of these avatars can be animated.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] FIG. 1 is a diagram of an e xemplary environment in which te chnologies de scribed here in can be implemented

[0003] FIG. 2 illustrates two ex mplary avatar images series for animating an avatar's face.

[0004] FIG. 3 illustrate s two ex emplary avatar feature image serie s and an avatar background.

[0005] FIG. 4 shows a graph illustrating exemplary line ar and nonlinear relationships between a facial feature parameter and an index to an avatar image series or an avatar feature image series.

[0006] FIG. 5 is a block diagram of a first e xe mplary c omputing device for animating an avatar.

[0007] FIG. fj is a flowchart of a first exemplary avatar animation method.

[0008] FIG. 7 is a flowchart of a second exemplary avatar animation method.

[0009] FIG. S is a block diagram of a second exemplary computing device for animating an avatar.

[0010] FIG. 9 is a block diagram of an exe mplary proc essor core that can execute instructions as part of implementing technologies described herein

DETAILED DESCRIPTION

[001 1] The tec hnologies de scribed he rein alio w avatars to be animate d in real time base d on a user's fac ial or head moveme nts (hereinafter, c olle ctively re ferred to as "facial movements"). A user's facial features are tracked in live video of the user, and facial feature parameters deternuned from the tracked features are mapped to predetermined avatar images. The disclosed avatar ammation technologies consume less power than approaches that comprise ge re rating or deforming a complex 3D avatar mo del based on a user's facial move ments and rendering an avatar image based on sue h a model . Acc ordingly, the battery life of mobile computing devic s employing the disclosed technologies can be extended relative to devices that use more computationally intensive approaches.

[0012] Reference is now made to the drawings, wherein like refer rce numerals are used to re fer to like e lements throughout . In the fb Do wing de scription, for purpose s of e xplanation, numerous specific details are set forth in order to provide a thorough understanding thereof. It maybe evident, however, that the novel embodiments can be practiced without these specific details. In other instances, known structures and devices are shown in block diagram form in order to facilitate a description the le of. The intention is to cover all modifications, equivalents, and alternatives within the scope of the claims.

[0013] FIG. 1 is a diagram of an e xemplary environme nt 100 in which technolo gies described here in can be implemented. The environment 100 comprises a first user 1 10 using a first computing device 120 to communicate with a second user 130 using a second computing device 140 via a network (or cloud) 150 via a video call or videoconference . The first and se co nd computing device s 120 and 1 40 can b e any computing device, such as a mobile device (e.g., smartphone, laptop or tablet computer), desktop computer or server; and the network 150 can be any type of net work such as a Local Area Network (LAN), Wide Area Network (WAN) or the Internet.

[0014] A user interface 160 of a vide o call application running on the first c omputing device 120 can comprise an avatar image 170 of the second user 130 that occupies a majority of the user interface lo^"0. Optionally, the user interface 160 can also comprise an avatar image 130 o f the user 1 10 that occupies a smalle r portion of the interfac . The avatar images 170 and 1 SO are based on facial feature s of the users 110 and 130 extracte d from live video of the use rs to alio w for real time c ommunication bet we en the partie s. That is, video frames are made available for video avatar animation proc essing as soon as the y are ge nerated by a video capture device. In some embodiments, where real time communication between users is not re quire d, an avatar can be animated based on store d video captured at any previous time. [0015] ideo can be provided to a computing devic by, for example, a camera integrated into the omputing devic , such as a front-facing camera integrated into a smartphone (e.g., cameras 190 and 195) or tablet computer, or an external video capture device communicatively coupl d with the computing devic , sue has a wireless camcorder coupled to a laptop or a web camera coupled to a desktop computer.

[0016] In general, avatars are animated based on facial feature parameters detennined from facial features tracked in video of a user. Exemplary facial features include the position of the upper and lower lips, upper and lower eyelids, mouth corners, l ft and right eyebrows (inside end, middle and outside end), chin, left and right ears, nose tip and left and right nose wing. Exemplary f cial feature param ters mclude the degree of head rotation, degree of head tilt, distance between uppe r and lo er lips, distanc e bet wee n mouth c orners, distance between upper lip and nose tip, distance between nose wing and nose tip, distance between upper and lower eyelids, distance between eyebrow tips and distance between eyebrow tip and e yebro w middl . Facial features and facial feature parame ters can b e tracked and determined in addition to those listed above.

[0017] Determined facial fe ature parameters are used to select predetermined avatar images for animating an avatar. The avatar images are pre de termined in that the y have be en generate d before a use r^* s facial feature parameters are dete rmine d fio m a vide o. The prede termined avatar imag es can come pre -installed on a pure hased co mputing de vie e or installe d at the device after purchase . Avatar image s can be installed after purchase by, for example, downloading a conrn unication application that supports avatar animation using techniques de scribed herein or by do wrL ading avatar image s separately. In addition, avatar images can be provided by another computing device. For example, avatar images can be provided to a omputing devic by a remote computing device with which the computing device is in communication. For instance, avatar images can be provided by a remote computing devic as part of setting up a video call or during the video call.

[0018] Predetermined avatar image s can take various forms. For example, they can be simple cartoon images or images generated from sophisticated 3D models created using professional rendering engines. The predetermined avatar imag s are also typically stored locallyat a computing device. This allows for the quick provision of avatar images to other re source s at the computing device and to other computing devices, sue has remote computing device s participating in a video call with the c omputing devic e . [0019] In some embodiments, an avatar is animated by se lec ting an avatar image fro m one or more serie s of avatar images base d at least in part on facial features paramete rs determined from video of a user. As a user's facial features change in a video due to t e user's changing facial movements, different avatar images are selected and displayed, resulting in an animated avatar whose appearance corresponds to the facial movements of the user. The manners in which an avatar can be animated canbe based on the series of avatar images available to a computing device.

[0020] FIG. 2 illustrates two ex emplary avatar image series 200 and 210 for animating an avatar. Series 200 canbe used to animate the opening and closing o fan avatar's mouth ard series 210 canbe used to animate an avatar b nking. Series 200 and 210 each comprise ten avatar image s having corre sponding indie es 220 and 230, respectively. An avatar image series (or avatar feature image series, as discussed below) can comprise any number of images. The number of images in a series canbe based on, for example, a desired level of animation s moo time ss and a desired amount o f memory that avatar image se ries can oc cupy . In general, the facial features tracked in a particular implementation of the disclosed technologies can b e based on the avatar images serie s available to the devic e . For e xample, if a computing devic e only has acce ss to the serie s 200 and 10, the device may only track facial features relating to the mouth and eyes.

[002 1] Sel ting an avatar image in a series can comprise mapping o e or more facial feature parameters to an image index . For example, the distanc e bet wee n a use r's uppe r and lower lips can be mappe d to one of the indices 220 of serie s 200. V arious mapping approaches can be used. One exemplary mapping comprises normahzing a facial feature parameter to a range of zero to one and then performing a linear mapping of the normalized parameter to a series index . For example, the distance between a user¹ s upper and lower hps can be normalized to a range of ze ro to one and then rounded up to the nearest te nth to generate an index for the series 200. In some embodiments, an index canbe generated based on two or more facial features parameters. For example, a combination of the distance between a user's upper and lower hps and the distance betw en the corners of the user's mouth can be mapped to an index of the series 200.

[0022] Accordingly, an avatar can b e animate d to open and close its mouth by succe ssively displaying adj ace nt avatar imag es in the series 200 in inc reasing order by index value, and then succe ssively in decreasing order by index value . As used herein, the term "adjacen ' as it refers to an avatar image in a series means an image that is the next or preceding image in the animation sequence represented by the series. For example, with reference to avatar image series 200, avatar image 240 is adjacent to avatar images 250 and 2o^~0. However, it is not necessary that adjacent avatar images be displayed in succession. For example, a displayed avatar animation could comprise avatar image 2o^~0 (index =7) being displayed immediately after avatar image 250 (index=5) is displayed.

[0023] Additional avatar images series can be used to animate an avatar with facial mov ments other Unblinking and opening and closing its mouth. For example, avatar images series can be used to make an avatar yawn, smile, wink, raise its eyebrows, frown, etc.

[0024] In some embodiments, an avatar is animated using one avatar image serie s at a time . For example, if a c omputing device has ac cess to avatar image serie s 200 and 210, the avatar can be made to blink or to open and c lose its mouth, but cannot b e made to do both simultaneously. In other embodiments, an avatar image series can animate multiple facial move ments. For example , the series 200 and 210 can be combined to create an avatar image series c omprising 100 avatar images corre spending to combinatio ns of the ten mouth imag es inthe series 200 with the ten eye images in the series 210. Suc an expanded avatar image series can be used to animate an avatar that can blink and open and close its mouth at the same time. Avatar image serie s can animate more than two facial expressions.

[0025] If a c omputing devic e has ac cess to more than one avatar image s series that animate the same region of the lace (e .g., two series that can animate the mouth - one series that animates a smile and one series that animates a frown), the computing device can select which series to use to animate the avatar base d on determine d facial feature parameters . For example, a vertical distance between the mouth comers and the lower hp canbe used to determine if the user is smiling or frowning.

[0026] In some embodiments, multiple serie s of avatar feature images can b e used to animate various re gions of an avatar independe ntly Each avatar feature image series corresponds to one region of an avatar's face (eyes, eyebrows, nose, mouth, etc.). In such en±iodiments, an avatar image canbe generated by combining the avatar feature images selected from the avatar feature image series. An avatar feature image canbe selected from a series base d on facial feature parameters associated with the facial region corresponding with the serie s. In some e mbodiments, sele cted avatar feature image s can b e combined with an avatar background (e.g., an image of a face missing one or more parts) to generate an avatar image. Utilising multiple series of aratar feature images allows for independent animation of multiple regions of an avatar's face without the need for a single avatar image series containing a large number of images.

[0027] FIG. 3 illustrates two ex emplary avatar feature image serie s 300 and 310 and an avatar background 320. The series 300 and 310 can be used to irde end ntly animate the mouth and eye s of an avatar. The se ries 300 can be use d to animate an avatar's mouth ope ring and closin , and the series 310 can be use d to animate an avatar's eyes blinking . The avatar background 320 comprises an image of an avatar face without the eyes and mouth. An avatar image to be presented in an animation can be generated by selecting an avatar feature image fio m se rie s 310 based o n, for example, the distance bet we en a user' s upper and lower eyelids, selecting an avatar feature image from series 300 based on, for example, the distance between the upper and lower lips, and combining the selected avatar feature images with the avatar background 320. For instanc e, if, at a particular moment in time in a vide o, the determined distance between a user's upper and lower lips is mapped to index six of the series 300 and the d termined distance between the user's upper and lower eyelids is mapped to index ten of the series 310, se lec ted avatar ie ature images 330 and 340 can be co mbined with the avatar background 320 to generate the avatar image 350.

[0028] In some embodiments, separate avatar feature image series can be used to independently animate the left and right eyebrows and the left and right eyes. Moreover, a single image series can comprise images used to animate different facial moveme ts for a facial region. For example, a single avatar feature image series for animating an avatar's mouth can b e use d to making the avatar yawn, smile, grin, fro wn, or make the mouth move ments associate d with a language ' s phone mes . Such a more complex avatar feature image series can correspond to a plurality of facial features parameters.

[0029] FIG. 4 shows a graph 400 illustrating exemplary linear and nonlinear r lationships, via curves 410 and 420, re spe ctively, be tween a facial feature parameter (o r a co mbination of multiple facial feature parameters) and an index to an avatar image series or an avatar feature image series. In some embodime ts, the mapping can be nonlinear to emphasize movement of avatar features. For example, onsider the avatar feature image series 300 used for animating the opening and closing of an avatar's mouth. Although the images range from a close d mout (imag e 3 o^"0) to a fully opened mouth (imag e 370), during typical conversation a user's mouth move me nts may not vary over this entire range . Thus, a linear mapping can re suit in the user' s mouth move ments being mapp d to a re latively narrow range of indice s, such as two throug h six, in se rie s 300. Using a non- linear mapping, such as the mapping re p¾se tedby curve 420, the mouth movements made bya user during a typical

conversation can be made to map to a wider rang e of ndie es ( e .g . one through eight in series 300) to emphasize mouth movement in the avatar.

[0030] Avatar image intbrmatio n se nt by one c omputing device to another c omputing device for the purposes of animating an avatar can take various forms. For example, in some embodiments, avatar image information can comprise an avatar image for each name in an avatar animation sequ nce. The avatar images can be sent in a known image file format (e.g., .jpg, .tiff, bmp) or other format. In some embodiments, if the receiving computing device has acc ss to an avatar image series associated with the avatar to be animated, the avatar image information can comprise an index into an avatar image series. The re eiving device can then retrieve the appropriate avatar image from the series fbr display at the re eiving device. Avatar image information can also omprise identifiers of av tar image series.

[003 1] In othe r embodiments, avatar image intormatio n can comprise one or more indices to one or more avatar feature image series, and the receiving computing device can combine the avatar feature ima es associated with the indices into an avatar image for display at a receiving computing device display Avatar image information can also comprise an indicator for an avatarbackground to be combined with avatar feature images. In various embodiments, avatar image information can comprise facial feature parameters determined from video and the receiving computing device can map the received parameters to an avatar image or one or more avatar feature images.

[0032] In some embodiments, avatar image mformation can e sent to a second computing device via an intermediate computing device, sue has a cloud-based server. For example , a cloud-based server that is part of a vide oconfere ncing service can re ceive avatar image information from a computing device being used by a first participant to the vide oconfere nee, and distribute the avatar image information to other participants to the vide oconfere nee.

[0033] In some embodiments, a user's avatar is animated at a display of the c omputing device that tracked the user's facial features video and/or determined facial feature parameters from the tracked facial features. For example, with r ferenc to FIG. 1, the first computmg device 120 generates video of the user 110 with camera 1 0, tracks the user's facial features, determines facial feature parameters based on the tracked features and pre sents an animated avatar 1 SO of the user 110 in the user interfac e 160 of the first device . In some embodiments, avatar image information is not sent to another computing device. For example, a gaming console may animate a user's game avatar based on a user's facial movements captur d by a depth camera connected to the gaming console. The gaming console can pre se nt the animated game avatar in a display connec ted to the gaming c onsole, such as a television.

[0034] FIG. 5 is a block diagram of a first e xe mplary c omputing device 500 for animating an avatar. The computing devi 500 comprises a display 510, a communication module 520 to send avatar image information to another computing device, a facial feature tracking module 530 to track facial features in video of a user, a facial feature parameter module 540 to determine facial feature parameters from tracked facial features, an avatar image generation module 550, and an avatar image series store 5D^"0 to store avatar image series and/or avatar feature images series.

[0035] The avatar image ge neration module 550 can select avatar images or avatar feature images from ima e series base d on determined facial feature parameters. The avatar image generation module 550 can also select multiple avatar feature images from multiple avatar feature image series and combine the selected avatar feature images with an avatar background to generate an avatar image. The avatar image generation module 550 can further provide the selected or g erated avatar image for display, at, for example, the computing device 500 or anot er computing device. Optionally, the computing devi e can comprise a video camera 570 to capture video of a user. In some enitoodrments, the computing device 500 receives video from an external video source 580, such as a web camera or a cloud-based video source .

[0036] It is to be understood that FIG . 5 illustrates one example of a set of module s that can be included in a computing device . In other e mbodiments, a c omputing device can have more or fewer modules than those shown in FIG. 5. Further, modules shown as separate in FIG. 5 c an be combined into a sing le mo dule, or a single module shown in FIG . 5 can be split into multiple modules. Moreover, any of the modules shown in FIG. 5 can be part of the operating system of the computing device 500, one or more software applications

independ nt of the operating system, or operate at another software layer. The modules shown in FIG. 5 can be implemented in software, hardware, firmware or combinations thereof. A computer device referred to as being programmed to perform a method can be programmed to perform the method via software, hardware, firmware or combinations thereof. [0037] FIG. 6 is a flowchart of a first exemplary avatar aitimation method 600 . The method o^"00 can be performed by. for example, a smartphone mnrdng a video call application in whic h the user is re presente d by an avatar on the display o f remote computing device being used by another party to the call. The avatar tracks the user's facial movem nts and the smartphone store s multiple avatar feature image se rie s to animate the avatar' s eyebro s, le ft and right ey s and mouth independently. At a process act 6 10, one or more pr edetermined avatar feature images are selected from one or more pluralities of predetermined avatar feature images using a first computing device based at least in part on one or more facial feature parameters determined from video of a user. In the example, the smartphone sele ts an avatar feature image for the eyebrows, the left eye, the right eye ard the mouth, based on facial feature parameters determined from video of the user captured by the smartphone 's integrated camera. At a pro ess act 62ΰ, an avatar image is generated based at least in part on the one more selected pre determined avatar feature images. In the example, the smartphone combines the selected eyebrow, left eye, right eye and mouth avatar feature imag s with an avatar background image associated with the user to generate an avatar image . At a process act 630, avatar image ir-formation is provided for display. In the example, the avatar image is pirovided to smartphone display re sources for display of the avatar in a portion of the smartphon 's display so that the user can see how his avatar is animate d for other peine s to the call. The smartphone also pirovides the avatar image to the computing device of the other party to the call.

[0038] FIG. 7 is a flowchart of a second exemplary avatar animation method 700. The method 700 can be performed by, for example, a tablet computer executing a video call application in which the user is represented by an avatar on the display of a remote computing device operated by the other party to the video call. The avatar tracks the user's facial movements and the smartphone stores multiple avatar image series used to animate the avatar. The various avatar imag e series animate various facial move ments of the user, sue h as smiling, frowning andbhnking. At a process act 71 0, a predetermined avatar image is selected from a plurality of prede termined avatar image s at a first computing device base d at least in part on one or more facial feature parameters determined from video of a user. In the example , an avatar imag e is selected from an avatar image series use d to make the avatar smile, based on facial features parameters determined by video of the user captured by the tablet computer's inte grate d camera. At a ptocess act 720, the sele cte d avatar image is displayed at a display of the first computing devic e or avatar imag e information is se nt to a second computing device. In the example, the tablet sends the selected avatar image to the second computing device.

[0039] The tec hnologies de scribed here in have at least the following exe mplary advantages. The use of predetermined avatar images to animate an avatar provides a lower power avatar animation option relative to animation approaches that generate or deform a 3D avatar model on the fly based on tracked facial featur s or that use a sophisticated 3D rendering engine to ge rate the avatar image to be presented at a display By avoiding such computationally expensive approaches, the technologies described herein can also generate avatar animation more quickly. Although avatar animation has b een discussed primarily in the context of video call applications, the described tec hnologies can be use din any scenarios where avatars are or can be animated, such as in gaming applications (e.g., console -based application or massively multi player orJine role -playing games).

[0040] The tec hnologies de scribe d he rein can be performed by any o fa variety of computing devices, including mobile devices (sue has smartphones, handheld computers, tablet computers, laptop computers, media players, portable gaming consoles, cameras and video recorders), non-mobile devices (sue has desktop computers, servers, stationary gaming consoles, smart televisions) and embedded devi es (sue has devices incorporated into a vehicle) . As used herein, the term "computing devices" includes computing systems and includes devices comprising multiple discrete physical components.

[00 1] FIG. 8 is a block diagram of a second exemplary computing device 800 for animating an avatar. Generally, components shown in FIG. 8 can communicate with other shown components, although not all connections are shown, for ease of illustration. The device 800 is a multiprocessor system comprising a first processor 802 and a second processor 804 and is illustrated as comprising point-to-point (P-P) interconnects. For example, a point-to-point (P-P) interface 806 of the processor 802 is coupled to a point-to- point interface 807 of the processor 804 via a point-to-point interconnection 805. It is to be understood that any or all of the point-to-point interconnects illustrate d in FIG . 8 can be alternatively implemented as a multi-drop bus, and that any or all buses illustrated in FIG. 8 could be replaced by point-to-point interconnects.

[0042] As shown in Figure 8, the processors 802 and 804 are multicore processors.

Processor 802 comprises processor cores 808 and 809, and processor 804 comprises proce ssor c ore s 81 0 and 81 1 . Pro cessor c ores 808-811 can ex ecute computer-e xecutable instructions in a manner similar to that discussed below in connection with FIG. 9, or in other manners.

[0043] Proce ssors SO 2 and 804 farther co mprise at least one share d cache memory 81 2 ard 814, re spe ctively. The shared caches 812 and 81 4 can store data (e .g ., instructions) utilized by one or more components of the processor, such as the processor cores 808-809 ard SlO-811. The sharedcaches 812 and 814 can be part of a memory hierarchy for the device 800. For example, the shared cache 812 can locally store data that is also stored in a memory 816 to alio w for faste r ac ess to the data by co mponents of the pro cessor 802. In some e mbodiments, the share d caches 812 and 814 can comprise multiple cache layers, sue h as level 1 (LI), level 2 (L2), level 3 (L3), level 4 (L4), and/or other caches or cache layers, such as a last level cache (LLC).

[0044] Although the device 800 is shown with two processors, the device 800 can comprise only one processor or more than two processors. Further, a processor can comprise one or more processor cores. A processor can take various forms such as a central processing unit, a controller, a graphics processor, an accelerator (such as a graphics accelerator or digital signal pro cessor (DSP)) or a field programmable gate array (FPGA). A processor in a device can be the same as or different from other processors in the device. In some embodiments, the device 800 can comprise one or more processors that are reterogeneous or asymmetric to a first processor, accelerator, FPGA, or any other processor. There can be a variety of differen s between the processing elements in a system in terms of a spectrum of metrics of merit including arc tectural, nucroarchitectural, thermal, power consumption characteristics and the like . These differences can effe ctively manifest themselves as asymmetry and heterogeneity amongst the processors in a system. In some embodiments, the processors 802 and 804 reside in the same die package.

[0045] Processors 802 and 804 further comprise memory controller logic (MC) 820 and 822. As shown in FIG. 8, MCs 820 and 822 control me morie s 816 and 8 18 coupled to the proce ssors 802 and 804, respe ctively. The memories 816 and 818 can comprise various tjj e s of memories, such as vol tile memory (e.g., dynamic random access memories (DRAM), static random access memory (SRAM)) or non-volatile memory (e.g., flash memory). While MCs 820 and 822 are illustrate das being integrated into the processors 802 ard 804, in alternative embodiments, the MCs can be logic external to a processor, ard can comprise one or more layers of a memory hierarchy. [0046] Proce ssors SO 2 and 804 are couple d to an Input/Output (ISO) subsyste m 830 via P- P interconnections 832 and 834. The point-to-point interconnection 832 connects a point-to- point interface 836^" of the proc ssor 802 with a point-to-point interf ce 838 of the I/O subsyste m 830, and the point-to-point interconnection 834 connects a point-to-point interface 840 of the processor 804 with a point-to -point interface 842 of the I/O subsystem 830.

Input/Output subsystem 830 further includes an interface 850 to couple I/O subsystem 830 to a graphics engine 852. which can be a high-performance gmpnics engine. The I/O subsystem 830 and the graphics engine 852 are coupled via a bus 854. Alternately, the bus 844 could be a pint-to-point interconn ction

[0047] Input/Output subsystem 830 is further coupled to a first bus 860 via an interface 8o^"2. The first bus 8o^"0 can be a Peripiheial Component Interconnect (PCI) bus, a PCI Express bus, another third generation I/O interconnection bus or any other type of bus.

[0048] Various I/O devices 864 can be coupled to the first bus 860. A bus bridge 870 can couple the first bus 860 to a second bus 880. In some embodiments, the second bus 880 can be a low pin count (LPC) bus. Various devices can be coupled to the second bus 880 including, for example, a keyboard/mouse 882, audio ISO devices 888 and a storage device 890, such as a hard disk drive, solid-state dnve or other storage device for storing computer- executable instructions (code) 892. The code 892 comprises computer- ecutable

instructions for performing technologies described herein. Additional components that can be coupled to the second bus 880 include co muncation device(s) 884, which can provide for c ommunication bet wee n the device 800 and one or mo κ wired or wireless networks 880^" (e.g. Wi-Fi, cellular or satellite networks) via one or more wired or wireless communication links (e .g., wire, cable, Ethernet connection, radio-frecjuency(RF) channel, infrared channel. Wi-Fi channel) using one or more communication standards (e.g., IEEE 802.1 1 standard and its supplements).

[0049] The device 800 can comprise removable memory such flash memory cards (e.g., 3D (Secure Digital) cards), memory sticks, Subscriber Identity Module (SIM) cards). The memory in de vie e 800 (including caches 812 and 81 4, memories 816 and 818 and storage device 890) can store data and/or compiler-executable instructions for executing an operating system 894 and application programs 896. Example data includes web pages, text messages, images, sound files, video data, avatar image series, avatar feature images series, avatar backgrounds or other data sets to b e se nt to and/or rec eive d from one or more network servers or other devic es by the device 800 via one or more wire d or wire less networks, or for use by the devic 800. The device 800 can also have access to external memory (not shown) such as external hard drives or cloud-based storage.

[0050] The operating system 894 can control the allocation and usage of the compore nts illustrated in FIG. 8 and support one or more application programs 896. The application programs 296 can include common mobile computing devic applications (e.g., mail applications, calendars, ontact managers, web browsers, messaging applications) as well as other computing applications, such as a video call application 897 that animates user avatars based o n a use r ^Js facial features .

[005 1] The devic e 800 can support various input de vie es, sue h as a touch sere en, microphone, camera, physical keyboard, proximity s nsor and trackball, and one or more output devices, such as a speaker and a display. Othe r possible input and output devices include pieaoe lectric and other haptic I/O devic es. Any of the input or output devices can be internal to, external to or removably attachable with the device 800, such as an external depth camera or a connected television. External input and output devices can communicate with the device 800 via wired or wireless connections.

[0052] In addition, the computing device 800 can provide one or more natural user interfaces (NUIs) . For example, the o perating syste m 292 or applicatio us 294 can comprise speech recognition logic as part of a voice user interface that allows a user to operate the device 800 via voic e co nunands . Further, the device 800 c an comprise input device s and logic that allows a user to interact with the devic e 800 via a bo dy, hand or face ge stures. For example, a user's hand gestures can be detected and interpreted to provide input to a gaming application.

[0053] The device 800 can further comprise one or more wireless modems (which could comprise communication devices 884) coupled to one or more antennas to support communication between the system 800 and external devic s. The wireless modems can support various wireless communication protocols and technologies sue has Near Field Communication (NFC), Wi-Fi, Bluetooth, 4G Long Term Evolution (LTE), Code Division Multiplexing Access (CDMA), Universal Mobile Telecommunicatio System (UMTS) and Global Syste m for Mobile Telecommunication (GS M) . In additio n. the wire less modems can support communication with one or more cellular networks for data and voice

communications within a single cellular network, between cellular networks, or between the mobile computing devi e and a public switched telephone network (PSTN). [0054] The devic e 800 can further include at le ast one input/output port ( which can be, for example, a USB port, IEEE 994 (Fir Wire) port and/or RS-232 port) comprising physical connectors, a power supply, a satellite navigation system receiver such as a GPS receiver, a gyroscope, an ac elero mete r and a c ompass. A GPS re ce iver can be coupled to a GPS antenna. The device 800 can furthe r include one or more additional antennas coupled to one or more additional receivers, transmitters and/or transceivers to enable additional functions.

[0055] It is to be understood that FIG. 8 illustiate s o nly one exe mplary computing device architecture . Computing devices based on alternative architectures can be used to implement technologies described he rein. For example, instead of the processors 802 ard 804, and the graphics engine 852 being located on discrete integrated circuits, a computing device can comprise a SoC (system-on-a-chip) integrated circuit incorporating multiple proc ssors, a graphics engine and additional components. Further, a computing device can connect elements via bus co nfi gurations different fio m that sho wn in FIG. 8. Moreover, the illustrated components in FIG. 8 are not requited or all-inclusive, as shown components can be removed and other components added in alternative enitoodiments.

[0056] FIG. 9 is a block diagram of an exe mplary processo r core 900 to execute computer-executable instructions for implementing technologies described herein The processor core 900 can be a core for any type of processor, such as a microprocessor, an embedded processor, a digital signal processor (DSP) or a network processor. The processor core 900 can be a single -threaded core or a multithreaded core in that it may include more than one hardware thread conte t (or 'logical processor") per core.

[0057] FIG. 9 also illustrate s a memory 910 couple d to the processor 900. The memory 910 can be any memory de scribed here in or any other me mory known to those of skill in the art. The memory 910 can store computer-executable instruction 915 (code) executable by the processor core 900.

[0058] The processor core comprises front-end logic 920 that receives instructions from the memory910. An instruction can be processed by one or more decoders 930. The dec oder 930 can generate as its output a micro operation sue h as a frxe d width micro operation in a predefined formal, or generate other instructions, microinstructions, or control signals, which reflect the original code instruction. The fiont-end logic 920 further comprises register renaming logic 935 and scteduling logic 940, which generally allocate resourc s and queues operations corresponding to converting an instruction for execution. [0059] The processor core 900 further co mprise s e xecution logic 950, whic h comprise s one or more execution units (EUs) 9 o^"5-l through 9o^"5-N. Some processor core embodiments can include a number of execution units dedicated to specific functions or sets of functions. Other n iodiments can include only one ecution unit or ore execution unit that can perform a particular function. The execution logic 950 performs the operations specified by code instructions. After completion of execution of the operations specified by the code instructio s, back-end logic 970 retires instructions using retirement logic 975. In some n iodiments, the processor core 900 allows out of order execution but requires in-order retirement of instructions. Retirement logic 970 can take a variety of forms as known to those of skill in the art (e.g., re -order buffers or the like).

[0060] The processor core 900 is transformed during execution of instructions, at least in terms of the output generated by the decoder 930, hardware registers and tables utilised by the register reraming logic 935, and any registers (not shown) modified by the execution logic 950. Although not illustrated in Figure 9, a processor can include other elements on an integrated chip with the processor core 900. For example, a proc ssor may include additional elements sue has memory control logic, one or more graphics engines, I/O control logic and/or one or more caches.

[006 1] Referring back to FIG. 1, the network or cloud 150 can provide various cloud- based service s that can b e used to implement technologie s describ ed herein. For example, av tar image series or avatar feature ima es series or applications that employ avatar animation techniques described herein can be provided by cloud-based services.

[0062] Any of the disclosed methods can be implemented as computer-executable instructio ns or a c omputer program product . Sue h instructions can cause a c omputer to perform any of the disclosed me thods . Generally as used here in, the term "computer" refers to any computing device or system described or mentioned herein, or any other computing device. Thus, the term "computer-executable instruction" refers to mstructions that can be executed by any computing device described or mentioned herein, or any other computing device.

[0063] The computer-e xe cutable instructions or co mputer program products as well as any data created and used during implementation of the dis losed technologies can be stored on one or more tangible computer- readable storage media, sue has optical media discs (e.g., DVDs, CDs), volatile memory components (e.g., DRAM, SRAM), or non-volatile memory components (e.g., flash memor disk drives). Computer-readable storage media can be contame d m computer-readable storage devices such as solid-state drives. USB flash drives, ard memory modules. Alternatively, the computer- xecutable instructions maybe p rformed by specific hardware components that contain hardwired logic for p rforming all or a prtion of disclosed methods, or by any combination of omputer-readable storage media and hardware components.

[0064] The computer-e xe cutable instructions can be part of, for example, a dedicate d software applicatb n or a so ft ware application that is acce sse d via a web bro wser or other software application (sue has a remote omputing application). Su h software can be executed, for example, on a single computing device or in a network environment using one or more network computers. Further, it is to be understood that the discbsed technology is not limited to any specific computer language or program. For instanc , the disclosed technobgies can be implemented by software written in C++, Java, Perl, JavaScript, Adobe Flash, or any other suitable programming language. Likewise, the discbsed t chnobgies are not hmited to any particular computer or type of hardware. Certain details of suitable computers and hardware are kno wn and ne ed not be set forth in detail in this discbsure .

[0065] Furthermore, any of the software -biase d embodiments (co mprising, for example, computer-executable instructions for causing a computer to perform any of the discbsed methods) can be uploaded, downloaded or re motely accessed through a suitable

communication means. Such suitable communication means include, for example, the Internet the World Wide Web, an intranet, cable (including fiber optic cable), magnetic communications, electromagnetic communications (mcluding RF, microwave, andinfiared communications), electronic communications, or other such commurdcat n means.

[006 ] As used in this applicatbn and in the c laims, a fist of ite ms j oined by the term "and/or" c an mean any combinat n of the listed items. For example, the phrase "A, B and/or C" can mean A; B; C; A and B; A and C ; B ard C; or A, B andC . As used in this applicatbn and in the claims, a list of items joined by the terms "at least one of or "one or more of can mean any combinatbn of the listed terms. For example, the phrases "at least one of A, B or C" or "one or more of A, B or C" can mean A; B, C; A and B; A and C; B and C; or A, B and C.

[0067] The discbsed methods, apparatuses and systems are not to be construedas limiting in any way I nstead, the present disc bsure is dire cte d ta ward all novel and nonobvbus features and aspe cts of the various discb sed e mbodiments, alone and invar bus combinations and subcombinations with one anoth r. The discbsed methods, apparatuses, and systems are not limited to any spe ific aspect or feature or combination thereof, nor do the disclosed e n odiments require that anyone or more specific advantages be pres nt or probl ms be solved.

[0068] Theorie s of operation, scie ntific principle s or other the oretic al de scriptions pre sented here in in re ferenc e to the apparatuses or methods of this disclosure have be en provided for the purposes of better understanding and are not intended to be hmiting in scop . The apparatuses and methods in the appende d claims are not hmited to tho se apparatuses and methods that function in the manner described by such theories of operation.

[0069] Although the operations of some of the disclosed methods are describedina particular, sequential order for convenient presentation, it is to be understood that this manner of description encompasses rearrangement, unless a particular ordering is required by specific language set forth herein. For example, operations described sequentially may in some cases be rearrang d or performed concurrently. Moreover, for the sake of simplicit the attached figures may not show the various ways in which the disc losed methods can be use d in conjunction with other methods.

[0070] The following e samples pertain to furthe r embodiments.

[007 1] Example 1 . An avatar animation method, comprising : selecting one or more prede ermined avatar feature images from one or more pluralities of pred termined avatar feature images using a first computing device based at least in part on one or more facial feature parameters determined from video of a user; ge renting an avatar image based at least in part on the one more selected predetermined avatar feature images; and providing avatar image information for display.

[0072] Example 2. The method of Example 1, further comprising displaying the avatar image at a display of the first computing device.

[0073] Example 3. The method of Example 1, wherein the providing comprises sending avatar image information to a se ond computing device .

[0074] Example 4. The methodof Example 1, wherein the selecting comprises, for respective ones of the one or more facial feature parameters, mapping the respective facial feature parameter to the sel cted predetermined avatar feature image belo ging to the plurality of predetermined avatar featured images associated with the respective facial feature parameter.

[0075] Example 5. The methodof Example 4, wherein the mapping is a nonlinear mapping. [0076] Example ό^". The method of Example 1, wherein the avatar image is further based at least in part on an avatar background.

[0077] Example 7. The methodof Example 1, further comprising displaying the avatar image at a display of the second computing device.

[0078] Example 8. An avatar ardmation method, comprising: selecting one or more prede ermined avatar feature images from one or more pluralities of predetermined avatar feature imag s using the computing device based at least in part on one or more facial feature parameters determined from video of a user; geneiating an avatar image based at le st in part on the one more selected pr determined avatar feature images; and providing avatar image information for display.

[0079] Example 9. The method of Example 8, further comprising displaying the avatar image at a display of the first computing device.

[0080] Example 10. The method of Example 8, wherein the providing comprises sending avatar image information to a se cond c omputing d vie e .

[008 1] Example 1 1 . The method of Example 10, further comprising displaying the avatar image at a display of the second computing device.

[0082] Example 12. The method of Example 8, wherein the selecting comprises, for respective ones of the one or more facial feature parameters, mapping the respective facial feature parameter to the sel cted predetermined avatar feature image belonging to the plurality of Redetermined avatar featured images associated with the respective facial feature parameter.

[0083] Example 13. The method of Example 8, wherein the mapping is a nonlinear mapping.

[0084] Example 14. The method of Example 8, wherein the avatar image is further based at least in part on an avatar background.

[0085] Example 15. The method of Example 8, further comprising displaying the avatar image at a display of the s cond computing device.

[0086] Example lfj. A method of distributing computer-executable instructions for causing a computing device to perform an avatar animation method, the method comprising: sending computer-readable instructions to a computing device for causing the computing device to a perform an avatar animation method, the method comprising: sel cting one or more predetermined avatar f ature images from one or more pluralities of predetermined avatar feature imag s using a first computing device based at least in part on one or more facial feature parameters determined from video of a user; ge eiating an avatar image based at least in part on the one more selected pred termined avatar feature images; and providing avatar

- IS - image information for display, and storing the computer-readable instructions at the computing device.

[0087] Example 17. One or more omputer-readable storage media storing computer- exe utable instructions for causing a computing devic to perform anyone of the avatar animation methods of claims l-lo^".

[0088] Example IS. At least one computing device programmed to perform anyone of the avatar animation methods of claims 1 -16.

[0089] Example 19. At least one computing device comprising a means to perform anyone of the methods of claims 1 -16.

[00 0] Example 20. An avatar animation method, comprising: tracking one or more facial features in video of a user; and sending avatar image information to a second computing device.

[009 1] Example 21 . The method of Example 20, wherein the avatar image mfbrmation comprises one or more indices to one or more avatar feature image series or one or more indices to one or more avatar image series.

[0092] Example 22. The method of Example 20, where the avatar image information further comprises an avatar background image or an indicator of an avatar background image.

[0093] Example 23. One or more computer-readable storage media storing computer- exe utable instructions for causing a computing devi e to perform anyone of the avatar animation methods of claims 20-22.

[0094] Example 24. At least one computing device programmed to perform anyone of the avatar animation methods of claims 20-22.

[0095] Example 25. A computing device, comprising: a facial feature tracking module to tiack one or more facial features in video of a user; a fecial feature parameter module to determine one or more facial feature parameters from the one or more tracked fecial features; ard an avatar image generation module to select an avatar image from one or more avatar image series based on the determined one or more fecial feature parameters and provide the selected avatar image for display.

[0096] Example 26. A computing device, comprising: a fecial feature tracking module to tiack one or more facial featuK s in vide o o f a user; a fecial feature parame te r module to determine one or more facial feature parameters from the one or more tracked fecial features; ard an avatar image g neration module to select one or more avatar feature images from one or more avatar feature images series based on the determined one or more fecial feature parameters, generate an avatar image based on the one or more selected avatar feature images, and provide the generated avatar image for display.

Claims

CLAIMS We claim:

1. An avatar animation method, comprising :

selecting one or more predetermined avatar feature images from one or more pluralities of predetermined avatar feature images using a first computing device based at least in part on one or more facial feature parameters determined from video of a user;

generating an avatar image base d at least in part on the one more sele cte d

pred termined avatar feature imag s; and

providing avatar image information for display.

2. The method of claim 1 , further comprising displaying the avatar irnag e at a display of the first computing device.

3. The method of claim 1 , wherein the pro viding comprise s sending avatar image information to a se ond computing devi e.

4. The method of claim 1, wherein the selecting comprises, for respective ones of the one or more facial feature parameters, mapping the respective facial feature parameter to the selected pr determined avatar feature image be longing to the plurality of predetermined avatar featured images associated with the respective facial feature parameter.

5. The method of claim 4, wherein the mapping is a nonlinear mapping .

6. The method of claim 1, wherein the avatar image is further based at least in part on an avatar background.

7. The method of claim 1, further comprising displaying the avatar image at a display of the se ond computing device.

8. An avatar animation method, comprising : selecting one or more pr edete rmined avatar feature image s from o ne or more pluralities of predetermined avatar feature images using the computing device based at least in part on one or more facial feature parameters determined from video of a user;

generating an avatar image base d at least in part on the one mo κ se lec ted

predetermined avatar feature images; and

providing avatar image information for display.

9. The method of claim 8, further comprising displaying the avatar image at a display of the first computing device.

10. The method of claim 8, wherein the providing comprises sending avatar image information to a se ond computing device.

11 . The method of claim 10, further comprising dispfeying the avatar image at a display of the second computing device.

12. The method of claim 8, wherein the selecting co mprises, for respective one s of the one or more fecial feature parameters, mapping the respective fecial feature parameter to the selected predetermined avatar feature image be longing to the plurality of predetermined avatar featured images associate d with the respective facial feature parameter.

13. The method of claim 8, wherein the mapping is a nonlinear mapping.

14. The method of claim 8, wherein the avatar image is further based at least in part on an avatar background.

15 . The method of claim 8, further comprising displaying the avatar image at a display of the se ond computing device.

16 . A method of distributing computer-executable instructions for causing a computing device to perform an avatar animation method, the method comprising:

sending computer-readable instructions to a computing device for causing the computing device to perform an avatar animation method, the method comprising: selecting one or more predetermined avatar feature images from one or more pluralities of predetermined avatar feature images using a first computing device based at least in part on one or more facial feature parameters determined from video of a user;

generating an avatar image base d at least in part on the one mo κ se lec ted predetermined avatar feature images; and

providing avatar image information for display, and

storing the computer-readable instructio s at the computing device .

17. One or more computer- readable storage media storing computer-executable instructio ns for causing a c omputing device to perform any one of the avatar animation methods of claims 1-16.

IS . At least one computing device programmed to perform anyone of the avatar animation methods of claims l-lo^".

19. At least one c omputing device comprising a me ans to perform any one of the methods of claims l-lo^".

20. An avatar animation method, comprising :

tracking one or more facial features in video of a user; and

sending avatar image ir-formatio n to a sec ond c omputing devic e .

21 . The method of claim 20, wherein the avatar image information comprises one or more indices to one or more avatar feature image series or one or more indices to one or more avatar image series.

22. The method of claim 20, where the avatar image information further comprises an avatar background image or an indicator of an avatar background image.

23. One or more computer- readable storage media storing computer-executable instructions for causing a computing device to perform anyone of the avatar animation methods of claims 20-22.

24. At least one c omputing device programmed to perform any one of the avatar animation methods of claims 20-22.

25. A computing device, comprising:

a facial feature tracking module to track one or more facial features in video of a user; a facial feature parameter module to determine one or more facial feature parameters from the one or more tracked facial features; and

an avatar image generation module to select an avatar imag from one or more avatar image series based on the determined one or more facial feature parameters and provide the selected avatar image for display.

2o^~ . A computing device, comprising:

a facial feature tracking module to track one or more facial features in video of a user;

a facial feature parameter module to determine one or more facial feature parameters from the one or more tracked facial features; and

an avatar image generation module to select one or more avatar feature images from one or more avatar feature imag s series based on the determined one or more facial feature parameters, generate an avatar image based on the one or more selected avatar feature images, and provide the generated avatar image for display.