TWI682669B - Communication using interactive avatars - Google Patents

Abstract

Generally this disclosure describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar; initiating communication; detecting a user input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting at least one of the animation command and the avatar parameters.

Description

Communication technology using interactive avatars (5) Field of invention

The following disclosure is about video communication, and more specifically, about video communication using interactive avatars.

Background of the invention

The increase in the variety of functionalities available in mobile devices has created a desire for users to communicate via video in addition to simple calls. For example, the user can initiate a "video call", "video conference", etc., where the camera and microphone in the device capture the user's audio and video, which is transmitted in real time to one or more other recipients, such as Other mobile devices, desktop computers, video conferencing systems, etc. Video communication may involve the transmission of real-quality data (eg, depending on the camera technology, the specific video codec used to process the captured image data, etc.). Considering the bandwidth limitations of existing 2G/3G wireless technology and the still limited bandwidth of emerging 4G wireless technology, many device users can make parallel video calls that exceed the available bandwidth in the existing wireless communication infrastructure, which can negatively affect The quality of video calls.

According to one embodiment of the present invention, one or more non-transitory computer-readable storage devices with instructions stored thereon are specifically proposed. When the instructions are executed by at least one processor of a first computing device, the It includes the following operations: enabling the selection of a first avatar; identifying one or more facial features of a user of the first computing device; generating information to be transmitted to a second computing device to cause the The selected first avatar is to be animatedly displayed on a display of the second computing device, wherein the information is based on the one or more facial features of the identified user of the first computing device; and The animation of the selected first avatar is enabled based on a user input command, wherein the user input command is different from the one or more facial features, and when the user of the first computing device During control, the user input command is generated by a user input device.

100‧‧‧device-to-device system/system

102, 112, 102’‧‧‧ device/remote device

104, 114, 104’/114’‧‧‧ camera

106, 116, 106’, 116’‧‧‧ microphone

107, 117‧‧‧ speaker

108、118‧‧‧Touch-sensitive display/display

108’, 118’‧‧‧ monitor

110、120‧‧‧avatar

112’‧‧‧device/remote device

122‧‧‧ Internet

124, 124’‧‧‧ Server

126‧‧‧System

128‧‧‧ virtual space

200‧‧‧Camera, audio and touch screen frame module

202‧‧‧Face detection and tracking module/Face detection/Tracking module/Face detection module

204‧‧‧Feature extraction module

206‧‧‧Audio conversion module

208‧‧‧Touch detection module

210‧‧‧gesture detection module

212‧‧‧Avatar Selection Module

214‧‧‧Avatar Control Module

216‧‧‧ system

218‧‧‧Feedback Avatar

220‧‧‧Communication module

222‧‧‧ processor

300, 304‧‧‧WiFi connection

302‧‧‧Internet

306‧‧‧Enterprise AP

308‧‧‧Gateway

310‧‧‧Firewall

312‧‧‧Media and signal path

314‧‧‧ AP

400‧‧‧Flowchart

402~428‧‧‧Operation

The features and advantages of the various embodiments of the claimed subject matter will be apparent below as the detailed description proceeds and after referring to the drawings. The same numbers in the drawings designate the same parts, and among them: FIG. 1A illustrates various kinds according to the present disclosure. Example device-to-device system of an embodiment; FIG. 1B illustrates an example virtual space system according to various embodiments of the present disclosure; FIG. 2 illustrates an example device according to various embodiments of the present disclosure; FIG. 3 illustrates at least at least one according to the present disclosure. An example system implementation scheme of an embodiment; and FIG. 4 is a flowchart of example operations according to at least one embodiment of the present disclosure.

Although the following detailed description is made with reference to illustrative embodiments, those skilled in the art will appreciate many alternatives, modifications, and variations of these embodiments.

Detailed description of the preferred embodiment

Generally, this disclosure describes systems and methods for using interactive avatars for video communication. In contrast to moving images, using avatars generally reduces the amount of data to be transmitted, and therefore avatar communication requires less bandwidth. Interactive avatars are configured to enhance the user experience by changing the display of the selected avatar based on user input. In addition, user voice can be captured and converted to generate avatar voice. The avatar voice can then be related to the user's voice, but can obscure the user's identity. Audio conversion may include, for example, pitch shifting and/or time extension.

In one embodiment, the application in the device coupled to the camera, microphone, and speaker is launched. The application can be configured to allow users to select avatars for display on remote devices, in virtual spaces, etc. The devices can then be configured to initiate communication with at least one other device, virtual space, etc. For example, communication can be established via 2G, 3G, 4G cellular connections. Alternatively or additionally, communication can be established via the Internet via a WiFi connection. After the communication is established, the camera can be configured to start capturing images and/or the distance to the object, and the microphone can be configured to start capturing sounds, such as user voice, and convert the user voice into a user voice signal.

It can then be determined whether user input is detected. User input can be captured by user input devices. User input includes touch events captured by a touch-sensitive display and gestures captured by a camera, for example, a depth camera configured to capture the distance from an object, and/or a web camera. Therefore, the user input device includes a touch-sensitive display and/or a camera. If user input is detected, the user input can be recognized. For touch events, the user input identifier may be related to the touch type and one or more touch locations. For gestures (such as an open hand), the user input identifier may be related to the gesture identifier. The animation commands can then be recognized based on user input. The animation command corresponds to the desired response associated with the user input, such as changing the color of the appearance of the displayed avatar in response to a single tap on the appearance of the displayed avatar.

The avatar parameters can then be generated. Avatar parameters can be generated based on facial detection, head movement, and/or animation commands. The avatar parameters may therefore include passive components based on, for example, face detection and head movement, and interactive components based on animation commands. The avatar parameters can be used to animate the avatar on at least one other device, in a virtual space, etc. In one embodiment, avatar parameters may be generated based on face detection, head movement, and animation commands. In this embodiment, the resulting animation includes passive animation based on face detection and head movement, which is modified by interactive animation based on animation commands. Therefore, avatar animations may include passive animations based on, for example, face detection and head movement, and interactive animations based on user input.

At least one of the animation command and the avatar parameters can then be transmitted. In one embodiment, at least one of remote animation commands and remote avatar parameters is received. The remote animation command may cause the device to determine avatar parameters based on the remote animation command in order to animate the displayed avatar. The remote avatar parameter may cause the device to animate the displayed avatar based on the received remote avatar parameter.

Audio communication can be accompanied by avatar animation. After the communication is established, the microphone can be configured to capture audio input (sound), such as user voice, and convert the captured sound into a corresponding audio signal (such as user voice signal). In one embodiment, the user's voice signal can be converted into an avatar voice signal, which can then be encoded and transmitted. The received avatar voice signal can then be converted back to sound (eg avatar voice) through the speaker. The avatar voice can therefore be based on the user's voice and can save the content but can change the spectrum data associated with the captured voice. For example, conversion includes but is not limited to pitch shift time extension and/or conversion playback rate.

User input devices (such as touch-sensitive displays and/or cameras) can be configured to capture user input, and these user inputs are configured to animate avatars based on user input on at least one other device. User-driven animations (based on animation commands) may additionally be animations based on facial expressions and/or head movements. Animation commands may include, but are not limited to, changes in positioning of the avatar display, distortion of facial features, changing features to convey emotions, and so on. The animation commands can therefore modify avatar animations similar to those based on face detection/tracking and/or modify avatar animations in addition to those based on face detection/tracking. The animation commands can generate animations with limited time and can instantiate the resulting animations on the display of the local user based on input from a remote user.

Therefore, limited bandwidth video communication systems can be implemented using avatars. The audio can be converted and the video can be animated based on the detected user input and the recognized animation commands, thereby using avatar communication to enhance the user experience. In addition, avatars can be used to preserve anonymity, including audio conversion as described in this article.

FIG. 1A illustrates a device-to-device system 100 consistent with various embodiments of the present disclosure. System 100 may generally include devices 102 and 112 that communicate via network 122. The device 102 includes at least a camera 104, a microphone 106, a speaker 107, and a touch-sensitive display 108. The device 112 includes at least a camera 114, a microphone 116, a speaker 117, and a touch-sensitive display 118. The network 122 includes at least a server 124.

The devices 102 and 112 may include various hardware platforms capable of wired communication and/or wireless communication. For example, devices 102 and 112 may include, but are not limited to, video conferencing systems, desktop computers, laptop computers, tablet computers, smart phones (eg, iPhones®, Android-based phones, Blackberries®, Symbian®-based phones , Palm®-based phones, etc.), cellular phones, etc. The cameras 104 and 114 include any device for capturing digital images of the environment including one or more people, and may have sufficient resolution for appearance analysis and/or gesture recognition as described herein. For example, the cameras 104 and 114 may include still-life cameras (for example, cameras configured to capture still photos) or video cameras (for example, cameras configured to capture moving images composed of multiple frames). The cameras 104 and 114 may be configured to operate using light in the visible spectrum or using light in other parts of the electromagnetic spectrum that are not limited to the infrared spectrum, ultraviolet spectrum, and so on. In one embodiment, cameras 104 and 114 can be configured to detect depth, that is, the distance of the camera from the object and/or points on the object. The cameras 104 and 114 may be incorporated into the devices 102 and 112, respectively, or may be separate devices that are configured to communicate with the devices 102 and 112 via wired or wireless communication. Specific examples of the cameras 104 and 114 may include wired (for example, universal serial bus (USB), Ethernet, FireWire, etc.) or wireless (for example, WiFi, Bluetooth, etc.) web cameras, such as those that can communicate with computers, Video monitors, etc. are associated; depth camera; mobile device camera (eg, cell phone or smart phone camera, for example integrated into the example device previously discussed); integrated laptop camera; integrated tablet camera ( For example, iPad®, Galaxy Tab® and similar cameras) and so on.

Devices 102 and 112 may further include microphones 106 and 116 and speakers 107 and 117. The microphones 106 and 116 include any device configured to sense (ie, capture) sound and convert the sensed sound into corresponding audio signals. Microphones 106 and 116 may be integrated within devices 102 and 112, respectively, or may interact with these devices via wired or wireless communication, such as described in the examples above with cameras 104 and 114. The speakers 107 and 117 include any devices that are configured to convert audio signals into corresponding sounds. The speakers 107 and 117 may be integrated inside the devices 102 and 112, respectively, or may interact with these devices via wired or wireless communication, such as described in the examples above with respect to the cameras 104 and 114. The touch-sensitive displays 108 and 118 include any devices that are configured to display text, still images, moving images (such as video), user interfaces, graphics, etc. and are configured to sense such things as taps, swipes, etc. Wait for touch events. Touch events may include touch type and touch location. Touch-sensitive display The devices 108 and 118 may be integrated within the devices 102 and 112, respectively, or may interact with these devices via wired or wireless communication, such as described in the examples above with cameras 104 and 114. In one embodiment, displays 108 and 118 are configured to display avatars 110 and 120, respectively. As mentioned herein, an avatar is defined as a graphical representation of a user in two dimensions (2D) or three dimensions (3D). Avatars do not have to resemble the face of the user, and so although avatars can be realistic representations, they can also take the form of drawings, cartoons, sketches, etc. In system 100, device 102 may display an avatar 110 representing a user of device 112 (eg, a remote user), and likewise, device 112 may display an avatar 120 representing a user of device 102. In this way, users can see the representations of other users without having to exchange large amounts of information related to device-to-device communication using moving images. In addition, the avatar can be animated based on user input. In this way, the user can interact with the display of the local and/or remote avatar, thereby enhancing the user experience. The resulting animation can provide a larger range of animation than when it is possible to use only face detection and tracking. In addition, users can actively choose these animations.

As mentioned herein, avatar audio (ie, sound) is defined as converted user audio (sound). For example, the voice input may include the user's voice, that is, the user's voice, and the corresponding avatar audio may include the converted user's voice. Avatar audio can be related to user audio. For example, the avatar voice may correspond to pitch shift, time extension, and/or other conversion of user voice. The avatar speech may be similar to human speech or may correspond to cartoon characters and so on. In the system 100, the device 102 may emit an avatar audio representing a remote user of the device 112, and similarly, the device 112 may emit a The avatar audio of the audio captured by the device 102 (for example, the voice of the local user of the device 102). In this way, the user can hear representations of other users' voices that can be converted.

The network 122 may include various second-generation (2G), third-generation (3G), and fourth-generation (4G) cellular-based data communication technologies, Wi-Fi wireless data communication technologies, and so on. The network 122 includes at least one server 124 configured to establish and maintain a communication connection when using these technologies. For example, the server 124 can be configured to support Internet-related communication protocols, such as the Session Initiation Protocol (SIP), which is used to establish, modify, and terminate two-party (unicast) and multi-party (multicast) conversations; interactive connections Established Protocol (ICE), which is used to present a framework that allows protocols to be established on top of bitstream connections; Network Access Translator (NAT) Conversational Utilities Protocol (STUN), which allows applications operated via NAT Program to discover the existence of other NATs; IP addresses and ports, which are allocated for user datagram protocol (UDP) connections of applications to connect to remote hosts; use relay to traverse NAT (TURN), which allows NAT Or the components behind the firewall receive data via Transmission Control Protocol (TCP) or UDP connections, etc.

FIG. 1B illustrates a virtual space system 126 consistent with various embodiments of the present disclosure. System 126 may use device 102, device 112, and server 124. Device 102, device 112, and server 124 may continue to communicate in a manner similar to that illustrated in FIG. 1A, but user interaction may occur in virtual space 128 instead of user interaction in a device-to-device format. As mentioned herein, the virtual space can be defined as a digital simulation of the physical location. For example, the virtual space 128 may be similar to an outdoor location, like a city, Roads, sidewalks, fields, forests, islands, etc., or indoor locations, like offices, houses, schools, shopping malls, shops, etc. The user represented by the avatar may appear to interact with the virtual space 128 as in the real world. The virtual space 128 may exist on one or more servers coupled to the Internet, and may be maintained by a third party. Examples of virtual spaces include virtual offices, virtual meeting rooms, virtual worlds like Second Life®, massive multiplayer online role-playing games (MMORPG) like World of Warcraft®, massive multiplayer online reality like The Sims Online® Life games (MMORLG) and so on. In the system 126, the virtual space 128 may contain multiple avatars corresponding to different users. Instead of the displayed avatars, the displays 108 and 118 may display a virtual space (VS) 128 in an enclosed (eg, smaller) style. For example, the display 108 may display a perspective view of what the avatar of the user of the device 102 "sees" in the virtual space 128. Similarly, the display 118 may display a perspective view of what the avatar of the user of the device 112 "sees" in the virtual space 128. Examples of content that an avatar may see in the virtual space 128 include, but are not limited to, virtual structures (such as buildings), virtual vehicles, virtual objects, virtual animals, other avatars, and so on.

FIG. 2 illustrates an example device 102 according to various embodiments of the present disclosure. Although only the device 102 is described, the device 112 (eg, a remote device) may include resources configured to provide the same or similar functions. As previously discussed, device 102 is shown to include camera 104, microphone 106, speaker 107, and touch-sensitive display 108. The camera 104, microphone 106, and touch-sensitive display 108 can provide input to the camera, audio, and touch screen frame module 200, and the camera, audio, and touch screen frame module 200 The output to the speaker 107 (eg, audio signal) can be provided. The camera, audio, and touch screen frame module 200 may include conventional, proprietary, known, and/or later developed audio and video processing codes (or instruction sets). The audio and video processing codes are usually clearly defined and It is operable to control at least the camera 104, the microphone 106, the speaker 107, and the touch-sensitive display 108. For example, the camera, audio, and touch screen frame module 200 enables the camera 104, microphone 106, speaker 107, and touch-sensitive display 108 to record images, distance to objects, sound, and/or touch, and can process images, sound, and audio signals And/or touch, the image and/or sound can be copied, the audio signal can be provided to the speaker 107, and so on. The camera, audio and touch screen frame module 200 may vary depending on the device 102, and more particularly depending on the operating system (OS) operating in the device 102. Example operating systems include iOS®, Android®, Blackberry® OS, Symbian®, Palm® OS, etc. The speaker 107 can receive audio information from the camera, audio, and touch screen frame module 200, and can be configured to copy local sound (for example, to provide audio feedback of the user's voice, which is converted or unconverted) And remote voice (for example, voice (converted or unconverted) of another party or parties involved in a phone, video call, or interaction in a virtual location).

The face detection and tracking module 202 can be configured to identify and track the head, face, and/or face area within the image provided by the camera 104. For example, the face detection module 204 may include conventional, proprietary, known, and/or later developed face detection codes (or instruction sets), hardware, and/or firmware, which are usually clearly defined and Operable to receive standard format images (such as but not limited to RGB color images) and at least to some extent identify Face. The face detection and tracking module 202 can also be configured to track the detected face through a series of images (such as a video frame at 24 frames per second) and determine the head based on the detected face position. Known tracking systems that can be used by the face detection/tracking module 202 can include particle filtering, average variation, Kalman filtering, etc., each of which can use edge analysis, square and variance analysis, feature point analysis, Histogram analysis, skin color analysis, etc.

The feature extraction module 204 can be configured to recognize the position and/or shape of features (eg, facial indicators (such as eyes, eyebrows, nose, mouth, etc.) in the face detected by the face detection module 202 ). In one embodiment, the avatar animation may be directly based on the sensed facial movements (eg, changes in facial features) without facial expression recognition. The corresponding feature points on the face of the avatar can follow or imitate the movement of the face of the real person. This is called "expression imitation" or "performance-driven facial animation". The feature extraction module 204 may include conventional, proprietary, known, and/or later developed facial feature identification codes (or instruction sets), which are usually well-defined and operable to receive standard format images from the camera 104 ( For example, but not limited to RGB color images, and at least to some extent extract one or more facial features in the images. Such known facial feature systems include, but are not limited to, Colorado State University's CSU facial recognition evaluation system.

The feature extraction module 204 can also be configured to recognize the expressions associated with the detected features (eg, whether the previously detected face is happy, sad, smiling, frowning, surprised, excited, etc.). Therefore, the feature extraction module 204 may further include conventional, proprietary, known and/or later developed facial expression detection and/or identification codes (or instruction sets), which are usually Definitely defined and operable to detect and/or recognize facial expressions. For example, the feature extraction module 204 can determine the size and/or position of facial features (such as eyes, mouth, cheeks, teeth, etc.), and can compare these facial features with a facial feature database, which includes Multiple sample facial features with corresponding facial feature categories (eg, smile, frown, excitement, sadness, etc.).

The audio conversion module 206 is configured to convert the user's voice into an avatar voice, that is, the converted user's voice. The conversion includes adjusting the rhythm (eg, extended time), pitch (eg, pitch shift), and playback rate. For example, the audio conversion module 206 may include conventional, proprietary, known, and/or later developed audio conversion codes (or instruction sets), which are usually well-defined and operable to receive sounds representing the user's voice Data, and convert the sound data into converted sound data. The sound data may be related to the audio signal based on the sound captured by the microphone 106 and processed by the camera, audio, and touch screen frame module 200. Such known sound conversion systems include, but are not limited to, voice-controlled open source audio processing libraries, which are configured to adjust the rhythm, pitch, and playback rate of audio streams or audio files.

The audio conversion module 206 may include a plurality of predefined sound styles, which correspond to conversion parameters associated with the conversion of the user's voice. For example, the conversion parameters can be configured to maintain the converted sound output heard by humans in different tones and/or rhythms. For the voice of human females or children, the pitch can be shifted to high frequencies; for the voice of human males, the pitch can be shifted to lower frequencies, and the rhythm can be adjusted up or down to increase or decrease the voice Speed, etc. In another example, the conversion parameters can be combined to produce The converted sound output corresponding to sounds such as animal sounds (eg cats) and/or cartoon characters. This can be achieved by adjusting the pitch, other frequency components and/or sampling parameters of the user's speech.

The user can select the desired audio conversion output before starting the communication, and/or can select the desired audio conversion during the communication. The audio conversion module 206 can be configured to provide a sample audio conversion output in response to a request from a user. In one embodiment, the audio conversion module 206 may include facilities that allow the user to select audio conversion parameters to generate a customized audio conversion output. The facility can be configured to provide sampled converted audio output based on the user's voice input. The user can then adjust the audio conversion parameters (eg, trial and error) until a suitable conversion output is achieved. The audio conversion parameters associated with the appropriate output to the user can then be stored and/or utilized for avatar communication, as described herein.

The touch detection module 208 is configured to receive touch data from the camera, audio, and touch screen frame module 200 and recognize touch events based on the received touch data. The touch event identifier may include touch type and/or touch location. Touch types may include single touch, double touch, touch and hold, touch and move, press and extend, swipe, and so on. The touch position may include a touch start position, a touch end position, and/or an intermediate moving touch position, and so on. The touch position may correspond to the coordinates of the touch-sensitive display 108. The touch detection module 208 may include conventional, proprietary, known, and/or later developed touch detection codes (or instruction sets), which are generally well-defined and operable to receive touch data and recognize touch events.

The gesture detection module 210 is configured to receive depth and/or image data from the camera, audio and touch screen frame module 200, recognize corresponding gestures based on the received depth and/or image data, and based on the recognized Gesture determination gesture identifier. The depth corresponds to the distance from the camera to the object. The gesture identifier is related to the recognized gesture. The gesture detection module 210 may include conventional, proprietary, known, and/or later developed gesture detection codes (or instruction sets), which are generally well-defined and operable based on the received depth and/or Image data recognizes gestures.

For example, the gesture detection module 210 may include a database of predefined gestures. The predefined gestures may include at least some relatively ordinary and relatively simple gestures, including open hand, close hand (ie, fist), wave hand, circular motion with hand, hand movement from right to left, hand movement from left to Move right and so on. Accordingly, gestures may include static non-moving hand gestures, active moving hand gestures, and/or combinations thereof. In one embodiment, the gesture detection module 210 may include a training facility that is configured to allow the user to change predefined gestures and/or train new gestures. The customized gesture and/or the new gesture can then be associated with a gesture identifier, and the gesture identifier can be associated with an animation command, as described herein. For example, the user can select an animation command to associate with a gesture from a predefined list of animation commands.

Therefore, animation commands are related to the desired response to user input. The animation commands may be associated with recognized user input such as touch event identifiers and/or gesture identifiers. In this way, the user can interact with the displayed avatar and/or can set gestures to modify the animation of the displayed avatar.

The avatar selection module 212 is configured to allow the user of the device 102 to select an avatar for display on the remote device. The avatar selection module 212 may include idiomatic, proprietary, known, and/or later developed user interface building codes (or instruction sets), which are usually clearly defined and operable to present different avatars to the user, so that The user can select one of these avatars. In an embodiment, one or more avatars may be predefined in the device 102. Predefined avatars allow all devices to have the same avatar, and only the selection of avatars (such as the identification of predefined avatars) needs to communicate with remote devices or virtual spaces during interaction, thereby reducing the amount of information that needs to be exchanged. The avatar is selected before establishing communication, but it can also be changed during active communication. Therefore, the avatar selection may be sent or received at any point during the communication, and the receiving device may change the displayed avatar according to the received avatar selection.

The avatar control module 214 is configured to receive a user input identifier based on user input of the device 102. The user input identifier may include a touch event identifier determined by the touch detection module 208 based on the touch event data or a gesture identifier determined by the gesture detection module 210. The touch event data includes touch type and/or touch location. The touch location may correspond to the coordinates associated with touch-sensitive display 108. The touch position can be mapped to one or more points on the displayed body, for example, to a feature, such as the tip of the nose, mouth, lips, ears, eyes, and so on. The points on the displayed avatar can be related to the desired response of the avatar animation (that is, the animation command).

The avatar control module 214 is configured to determine the animation command based on the user input identifier (that is, the recognized user input). Animation commands are grouped to identify the desired avatar animation. For example, the desired animation includes changing the color of the face of the displayed avatar, changing the size of one of the features of the displayed avatar (for example, making the nose larger), making winks, blinking, smiling, removing a feature (for example, ears), and so on. Therefore, the avatar control module 214 is configured to receive the user input identifier and determine the animation command based on the user input identifier.

The avatar control module 214 is configured to perform avatar animation based on animation commands. In one embodiment, for interactive animations displayed on a remote device such as device 112, animation commands can be transmitted and the remote avatar control module can then execute the animation. In another embodiment, the avatar parameters can be configured for immediate implementation of the avatar animation via transmission.

The implemented interactive animation based on the animation command may have a limited duration, after which the avatar animation may return to a passive animation based on, for example, face detection and tracking as described herein. The implemented interactive animations that affect the size of the feature can be combined to gradually change the size and gradually return to the original size. Additionally or alternatively, animations that affect the size of features can be assembled to have an effect gradient. In other words, the relative magnitude of the size change may depend on the position relative to, for example, key vertices. The points on the displayed chemistry that are closer to the key vertices may undergo greater changes than the points on the displayed chemistry that are relatively further away.

Therefore, the avatar control module 214 may receive a user input identifier based on the user input, may determine an animation command based on the user input identifier, and may perform animation based on the animation command. Interactive animations based on animation commands may be time limited to a time period (duration) and/or may include effect gradients. The animation can return to the passive avatar animation based on face detection and tracking after this time period.

The avatar control module 214 is configured to generate parameters for animating the avatar. As mentioned in this article, animation can be defined as changing the appearance of an image/model. Animations include passive animations based on, for example, facial expressions and/or head movements, and interactive animations based on user input. A single animation (which can include passive animation and interactive animation) can change the appearance of two-dimensional still images, or multiple animations can exist in sequence to simulate the motion of the image (such as turning head, nodding, blinking, talking, frowning, smiling, laughing) , Wink, blink, etc.). Examples of animations used for 3D models include deforming 3D wireframe models, applying texture mapping and recalculating model vertices for normal visualization. Changes in the position of the detected face and/or the extracted facial features can be converted into parameters that make the features of the avatar similar to the features of the user's face. In one embodiment, the general expression of the detected face may be converted into one or more parameters that cause the avatar to display the same expression. The avatar's expression can also be exaggerated to emphasize the expression. While avatar parameters can be generally applied to all predefined avatars, knowledge of the selected avatar may not be necessary. However, in an embodiment, the avatar parameters may be specific to the selected avatar, and thus may be changed if another avatar is selected. For example, human avatars may require different parameter settings than animal avatars, cartoon avatars, etc. (eg, different avatar characteristics may be changed) to demonstrate emotions such as happiness, sadness, anger, surprise, etc.

The avatar control module 214 may include conventional, proprietary, known, and/or later developed graphics processing codes (or instruction sets), which are usually well-defined and operable to make the face based on the face by the avatar selection module 212 The face/head position detected by the detection and tracking module 202, the facial features detected by the feature extraction module 204 and/or determined by the touch detection module 208 and/or gesture detection module 210 The avatar selected by the user inputting the identifier becomes an animation. For animation methods based on facial features, 2D avatar animations can be accomplished, for example, by image warping or image gradation, while 3D avatar animations can be free-form deformed (FFD) or by using animation structures defined in the head 3D model To be done. Oddcast is an example of software resources that can be used for two-dimensional avatar animation, and FaceGen is an example of software resources that can be used for three-dimensional avatar animation.

For example, for an interactive animation that includes the nose of the elongated three-dimensional avatar display, a key vertex v _k related to the tip of the nose can be defined (eg, selected). The associated three-dimensional motion vector d _k (dx, dy, dz) and the effect radius R can be defined for the key vertex v _k . The other vertices within the effect radius R can be changed (that is, moved) in the interactive animation, while the vertices outside the effect radius R can remain unchanged due to the interactive animation. The interactive animation can have an associated duration, namely the animation time T, which can extend up to multiple frames. The temporary effect parameter η _t can be defined based on time t and animation time T, such as:

Vertices within the effect radius R that are relatively closer to v _k may change relatively more than vertices that are relatively further away from the key vertex v _k . A vertex of the spatial effect parameter η _i v _i can be defined as:

則可定義為

=η _t ．η _i ．

。互動動畫化身之新坐標則為

=

+

，其中

相應於頂點v _i 之坐標，其基於面部偵測及追蹤，亦即， 被動動畫。 And the motion vector of vertex v _i at time t

Can be defined as

= η _t . η _i .

. The new coordinates of the interactive animation avatar are

=

+

,among them

Corresponding to the coordinates of the vertex v _i , it is based on face detection and tracking, that is, passive animation.

Therefore, animation can be performed on the displayed avatars including passive animations that modify interactive animations. Interactive animations can be limited to the overall duration, and the magnitude of the animation's effects can vary within that duration. Interactive animations can be assembled to affect only a part of the avatar, and these effects can be larger for points closer to the key vertices. After the interactive animation is completed, the animation can continue based on face detection and tracking as described herein.

In addition, in the system 100, the avatar control module 214 can receive remote avatar selections and can be used to display and animate remote avatar parameters corresponding to the user's avatar at the remote device. The animation may include passive animation and interactive animation. The avatar control module enables the display module 216 to display the avatar 110 on the display 108. The display module 216 may include conventional, proprietary, known, and/or later developed graphics processing codes (or instruction sets), which are generally well-defined and operable to be on the display 108 according to example device-to-device embodiments Show avatars and animate them. For example, the avatar control module 214 can receive a remote avatar selection and can interpret the remote avatar selection to correspond to a predetermined avatar. The display module 216 can then display the avatar 110 on the display 108. In addition, the remote avatar parameters received in the avatar control module 214 can be interpreted, and commands can be provided to the display module 216 to animate the avatar 110. In one embodiment, more than two users can participate in the video call. When more than two users interact in a video call, the display 108 can be split or segmented to allow more than one avatar corresponding to the remote user to be displayed simultaneously. Or, in the system 126, the avatar control module 214 can Receiving information that causes the display module 216 to display content that the user's avatar corresponding to the device 102 "sees" in the virtual space 128 (eg, from the virtual perspective of the avatar). For example, the display 108 may display buildings, objects, animals, other avatars represented in the virtual space 128, and so on.

In one embodiment, the avatar control module 214 can be configured to cause the display module 216 to display the "feedback" avatar 218. The feedback avatar 218 represents how the selected avatar is presented on the remote device, in a virtual location, etc. In detail, the feedback avatar 218 is presented as an avatar selected by the user and can be animated using the same parameters generated by the avatar control module 214. In this way, the user can confirm what the remote user saw during their interaction. The feedback avatar 218 can also be used to display interactive animations caused by remote user input of the device 112. Therefore, local users can interact with their feedback avatars (such as the avatar 218 and the user of the device 102) so that interactive animations of their associated avatars are displayed to remote users on the device 112. The local user can similarly interact with the displayed avatar of the remote user (for example, the avatar 110), so that the interactive animation of the remote user's feedback avatar is displayed on the device 112.

The communication module 220 is configured to transmit and receive information for selecting avatars, displaying avatars, making avatars animated, displaying perspective views of virtual positions, and so on. The communication module 220 may include conventional, proprietary, known, and/or later developed communication processing codes (or instruction sets), which are usually clearly defined and operable to transmit avatar selections, avatar parameters, animation commands, interactions Avatar parameters and receive remote avatar selection, remote avatar parameters, remote animation commands and remote interactive avatar parameters. The communication module 220 can also transmit and receive audio information corresponding to avatar-based interactions. The communication module 220 can transmit and receive the above information via the network 122, as previously described.

The processor 222 is configured to perform operations associated with the device 102 and one or more of the modules included therein.

FIG. 3 illustrates an example system implementation scheme according to at least one embodiment. The device 102' is configured to communicate wirelessly via the WiFi connection 300 (eg, at work), the server 124' is configured to negotiate the connection between the devices 102' and 112' via the Internet 302, and the device 112 It is configured to communicate wirelessly via another WiFi connection 304 (eg at home). In one embodiment, the video calling application based on the device-to-device avatar is launched in the device 102'. After the avatar selection, the application may allow selection of at least one remote device (eg, device 112'). The application can then cause device 102' to initiate communication with device 112'. The communication may be initiated by the device 102' transmitting an establishment request via the enterprise access point (AP) 306 to the device 112'. The enterprise AP 306 may be an AP that can be used in a commercial setting, and thus can support higher data throughput and more parallel wireless clients than the home AP 314. The enterprise AP 306 can receive the wireless signal from the device 102', and can transmit the connection establishment request through the gateway 308 through various commercial networks. The connection establishment request can then pass through the firewall 310, which can be configured to control information flowing into and out of the WiFi network 300.

The connection establishment request of the device 102' can then be processed by the server 124'. The server 124' can be configured to register an IP address, authenticate the destination address, and NAT traversal so that the connection establishment request can be directed to the correct destination on the Internet 302. For example, the server 124' can parse the desired destination (e.g., remote device 112') from the information received in the connection establishment request from the device 102, and can route the signal through the correct NAT, port, and thus arrive Destination IP address. These operations can only be performed during connection establishment, depending on the network configuration. In some cases, operations can be repeated during a video call to provide notification to NAT to keep the connection valid. The media and signal path 312 can carry video (eg, avatar selection and/or avatar parameters) and audio information guidance to the home AP 314 after the connection has been established. The device 112' may then receive the connection establishment request and may be assembled to determine whether to accept the request. Determining whether to accept the request may include, for example, presenting a visual narrative to a user of device 112' who inquires about whether to receive a connection request from device 102'. The user of the device 112' receives the connection (for example, receives the video call) to establish the connection. The cameras 104' and 114' can be configured to subsequently start capturing the images of the respective users of the devices 102' and 112', respectively, for use in the avatar animation selected by each user. The microphones 106' and 116' can be combined to start capturing audio from each user. When information exchange starts between the devices 102' and 112', the displays 108' and 118' can display the avatars of the users corresponding to the devices 102' and 112' and animate those avatars.

FIG. 4 illustrates a flowchart 400 of exemplary operations consistent with one embodiment of the present disclosure. Such operations may be performed by the device 102 and/or 112, for example. In detail, the flowchart 400 depicts an operation to implement avatar animation (including passive animation and/or interactive animation) and/or audio conversion for communication between devices via a network. It is assumed that face detection and tracking, feature extraction, and passive avatar animation are implemented and operated as described herein.

The avatar model may be selected in operation 402. The avatar model may include video avatar selection and audio conversion selection. Multiple video avatar models can be displayed, and the user can select a desired avatar from these video avatar models. In one embodiment, selecting the video avatar model may include associated audio conversion. For example, a cat-like avatar may be associated with a cat-like audio conversion. In another embodiment, audio conversion can be selected independently of the video avatar selection.

The avatar model including the audio conversion can be selected before starting the communication, but it can also be changed during the active communication. Therefore, the avatar selection may be sent or received and/or the audio conversion selection may be changed at any point during the communication, and the receiving device may change the displayed avatar according to the received avatar selection.

The avatar communication can be initiated in operation 404. For example, a user can run an application that is configured to use the avatar to communicate audio and video as described herein. Operation 404 may include grouping communications and establishing connections. The communication configuration includes identifying at least one remote device or virtual space participating in the video call. For example, a user can store it in an application, a device associated with another system (such as a list of contacts in a smart phone, mobile phone, etc.), or a remote storage such as the Internet (for example, such as Choose from a list of remote users/devices on social media sites such as Facebook, LinkedIn, Yahoo, Google+, MSN, etc.). Alternatively, users can choose to perform online operations in a virtual space such as Second Life.

In operation 406, the camera in the device may then start capturing images and/or depth, and the microphone in the device may start capturing sound. The image may be a still image or a moving image (for example, multiple images captured in sequence). The depth can be captured together with the image or can be captured independently. The depth corresponds to the distance from the camera to the object (and the point on the object) in the camera's field of view. It can be determined in operation 408 whether user input is detected. User input includes gestures captured by the image and/or depth camera and touch input detected on the touch-sensitive display. If a user input is detected, the user input can be identified in operation 410. The user input identifier includes a touch identifier or a gesture identifier. The touch identifier may be determined based on the touch on the touch-sensitive display and may include the touch type and the touch position. The gesture identifier may be determined based on the captured image and/or depth data and may include recognizing gestures.

The animation command may be recognized at operation 412. The animation commands can be configured to animate the selected avatar of the user displayed on the remote device, or animate the feedback of the remote user also displayed on the remote user's device. The animation commands correspond to the desired responses associated with user input. For example, touching the face of the displayed avatar (user input) can produce a color change of the face of the displayed avatar (the desired response identified by the animation command). The animation commands can be recognized based on the recognized user input. For example, each user input may be related to (eg, be associated with) animation commands in a database with user input identifiers and animation commands.

Operation 414 includes generating avatar parameters. The avatar parameters include passive components and may include interactive components. If no user input is detected, the avatar parameters may include passive components. If user input is detected, whether the avatar parameters can include interactive components depends on the animation command and therefore on the user input. For user input corresponding to an animation command configured to animate the selected avatar of the user, the animation command may be transmitted together with the avatar parameters including only passive components or may be applied to the avatar parameters before transmission, so that all The transmitted avatar parameters include passive components and interactive components. For the input of the animation command corresponding to the configuration to make the feedback of the remote user displayed on the remote user's device into an animation, only the animation command may be transmitted.

Operation 416 includes converting and encoding the captured audio. The captured audio can be converted into audio signals (such as user voice signals). The user's voice signal can be converted according to the audio conversion portion selected by the avatar of operation 402. The converted user voice signal corresponds to the avatar voice signal. The avatar voice signal can be encoded using techniques known for transmission to remote devices and/or virtual spaces via the network. The converted and encoded audio may be transmitted at operation 418. Operation 418 may further include transmitting at least one of animation commands and avatar parameters. The transmission animation command is configured to allow the remote device to animate the locally displayed avatar by modifying the avatar parameters according to the animation command. The transmitted avatar parameters that have been modified according to the animation commands before transmission can be directly used to animate the avatar displayed on the remote device. In other words, the modification of the avatar parameters represented by the animation command can be performed locally or remotely.

Operation 420 includes receiving remotely encoded audio that may be converted audio. Operation 420 further includes receiving at least one of a remote animation command and remote avatar parameters. The remote animation command can be used to modify the avatar parameters corresponding to the displayed avatar of the remote user or the displayed feedback avatar of the local user. Animation commands and avatar parameters are combined to generate avatar animations that are modified based on user input. At operation 422, the received audio can be decoded and played, and at operation 424, the avatar can be displayed and animated.

The animation of the displayed avatar may be based on detected and recognized user input, as described herein. In an example of device-to-device communication (eg, system 100), at least one of remote avatar selection or remote avatar parameters may be received from the remote device. The avatar corresponding to the remote user can then be displayed based on the received remote avatar selection, and can be animated based on the received remote avatar parameters. In an example of virtual location interaction (eg, system 126), information may be received that allows the device to display content corresponding to what the device user's avatar sees.

It can be determined whether the communication is completed at operation 426. If the communication is completed, the program flow can be ended at operation 428. If the communication is not completed, the program flow can proceed to operation 406 to capture images, depth and/or audio.

Although FIG. 4 illustrates various operations according to an embodiment, it is to be understood that not all operations depicted in FIG. 4 are necessary for other embodiments. In fact, this article fully covers that other embodiments of the present disclosure, the operations depicted in FIG. 4 and/or other operations described herein can be combined in one way, which is not explicitly shown in the accompanying drawings In any scheme, but still completely consistent with this disclosure. Therefore, requests for features and/or operations that are not exactly shown in a drawing are considered to be within the scope and content of this disclosure.

As used in any of the embodiments herein, the term "app" can be embodied in code or instructions that can be executed on a programmable circuit such as a host processor or other programmable circuit.

As used in any of the embodiments herein, the term "module" may represent an app, software, firmware, and/or circuit, which is configured to perform any of the above operations. The software may be embodied as a set of software, codes, instructions, instruction sets and/or materials recorded on at least one non-transitory computer-readable storage medium. The firmware may be embodied as codes, instructions or instruction sets and/or data hard-coded (eg, non-electrically dependent) in the memory device.

As used in any of the embodiments herein, "circuits" may include, for example, hardwired circuits alone or in any combination; circuits may be planned, such as computer processors containing one or more separate instruction processing cores; state machine circuits and And/or firmware that stores commands executed by programmable circuits. Modules can be collectively or individually embodied as circuits that form part of a large system, such as integrated circuits (ICs), system on a chip (SoC), desktop computers, laptop computers, tablet computers, servers, smarts Phone and so on.

Any of the operations described in this way can be implemented in a system that includes one or more storage media on which separate or combined instructions are stored. When the instructions are executed by one or more processors, These instructions perform these methods. Here, the processor may include, for example, a server CPU, a mobile device CPU, and/or other programmable circuits. In addition, the operations described herein are intended to be distributed across multiple physical devices, such as processing structures at more than one different physical location. Storage media can include any type of tangible media, for example, any type of disc, including hard disk, floppy disk, optical disc, optical disc-read only memory (CD-ROM), rewritable optical disc (CD-RW) ) And magneto-optical disks; semiconductor devices, such as read-only memory (ROM), random-access memory (RAM) (such as dynamic and static RAM), erasable and programmable read-only memory (EPROM), and electrically erasable Programmable read-only memory (EEPROM), flash memory, solid state disk (SSD), magnetic or optical cards; or any type of media suitable for storing electronic commands. Other embodiments may be implemented as software modules executed by programmable control devices. The storage medium may be non-transitory.

Therefore, the present disclosure provides a method and system for interactively animating avatars instead of moving images for video communication. Compared with the sending of moving images, the use of avatars reduces the amount of information to be exchanged. The system and method are further configured to convert user speech into avatar speech by, for example, pitch shifting and/or extending the time of the captured audio signal. Interactive animations of avatars can be based on detected user input, including touch and gestures. Interactive animation is configured to modify animation based on face detection and tracking judgment.

According to one aspect, a system is provided. The system may include: a user input device configured to capture user input; a communication module configured to transmit and receive information; and one or more storage media. In addition, individual or combined instructions are stored on the one or more storage media, and when the instructions are executed by one or more processors, the following operations are generated, including: selecting an avatar; initiating communication; detecting usage Input; recognize user input; recognize animation commands based on user input; generate avatar parameters; and transmit at least one of animation commands and avatar parameters.

Another example system includes the aforementioned components and further includes: a microphone configured to capture sound and convert the captured sound into a corresponding audio signal; and instructions when the instructions are executed by one or more processors The following additional operations are generated: capturing user voice and converting user voice into corresponding user voice signals; converting user voice signals into avatar voice signals; and transmitting avatar voice signals.

Another example system includes the aforementioned components and further includes: a camera configured to capture images; and instructions that, when executed by one or more processors, generate the following additional operations: capture images; detect images Face in; extracting features from the face; and transforming features into avatar parameters.

Another example system includes the aforementioned components and further includes: a display; and instructions that, when executed by one or more processors, generate the following additional operations: displaying at least one avatar; receiving remote animation commands and remote avatar parameters At least one; and animate a displayed avatar based on at least one of remote animation commands and remote avatar parameters.

Another example system includes the aforementioned components and further includes: a speaker configured to convert the audio signal into sound; and instructions that generate the following additional operations when executed by one or more processors: receiving remote avatar voice Signal; and convert the remote avatar voice signal into avatar voice.

Another example system includes the aforementioned components, and the user input device is a camera configured to capture a distance and the user input is a gesture.

Another example system includes the aforementioned components, and the user input device is a touch-sensitive display and the user input is a touch event.

Another example system includes the aforementioned components, and the conversion includes at least one of pitch offset and time extension.

According to another aspect, a method is provided. The method may include selecting an avatar; initiating communication; detecting user input; identifying user input; identifying animation commands based on user input; generating avatar parameters based on the animation commands; and transmitting at least one of animation commands and avatar parameters.

Another example method includes the foregoing operations and further includes: capturing user voice and converting the user voice into a corresponding user voice signal; converting the user voice signal into an avatar voice signal; and transmitting the avatar voice signal.

Another example method includes the aforementioned operations and further includes: capturing an image; detecting a face in the image; extracting features from the face; and converting the features into avatar parameters.

Another example method includes the foregoing operations and further includes: displaying at least one avatar; receiving at least one of a remote animation command and a remote avatar parameter; and making a displayed avatar based on at least one of the remote animation command and the remote avatar parameter Animation.

Another example method includes the foregoing operations and further includes: receiving a remote avatar voice signal; and converting the remote avatar voice signal into avatar voice.

Another example method includes the aforementioned operation and the user input is a gesture.

Another example method includes the aforementioned operation and the user input is a touch event.

Another example method includes the foregoing operation and the conversion includes at least one of pitch offset and time extension. According to another aspect, a system is provided. The system may include one or more storage media with individual or combined instructions stored on them, which when executed by one or more processors produce the following operations, including the selection of avatars ; Initiating communication; Detecting user input; Recognizing user input; Recognizing animation commands based on user input; Generating avatar parameters; and transmitting at least one of animation commands and avatar parameters.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and also includes: capturing user voice and converting the user voice into a corresponding user voice signal; converting the user Convert voice signals into avatar voice signals; and transmit avatar voice signals.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and also includes: capturing an image; detecting a face in the image; extracting features from the face; and integrating features Transform into avatar parameters.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and also includes: displaying at least one avatar; receiving at least one of remote animation commands and remote avatar parameters; and based on At least one of the remote animation command and the remote avatar parameters animate a displayed avatar.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and also includes: receiving a remote avatar voice signal; and converting the remote avatar voice signal into avatar voice.

Another example system includes instructions that are generated when the instructions are executed by one or more processors, and the user input is a gesture.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and the user input is a touch event.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and the conversion includes at least one of pitch offset and time extension.

The terms and expressions used herein are used for description and are not limiting, and in using these terms and expressions, it is not intended to exclude any equivalents (or parts thereof) of the features shown and described And, it should be recognized that within the scope of patent application, there may be various modifications. Therefore, the scope of patent application is intended to cover all such equivalents.

100‧‧‧device-to-device system/system

102、112‧‧‧device/remote device

104、114‧‧‧Camera

106, 116‧‧‧ microphone

107, 117‧‧‧ speaker

108、118‧‧‧Touch-sensitive display/display

110、120‧‧‧avatar

122‧‧‧ Internet

124‧‧‧Server

Claims (27)

One or more non-transitory computer-readable storage devices having instructions stored thereon, when the instructions are executed by at least one processor of a first computing device, results in operations including the following: enabling a first avatar The selection of one of the first computing device; recognize one or more facial features of a user of the first computing device; generate information to be transmitted to a second computing device to cause the selected first avatar to be animatedly displayed on On a display of the second computing device, wherein the information is based on the identified facial feature(s) of the user of the first computing device; based on a user input command to enable the selected The animation of the first avatar, wherein the user input command is different from the one or more facial features, and when controlled by the user of the first computing device, the user input command is to be used by a user Input device to generate; and convert the voice information of the user of the first computing device into target voice information to be transmitted to the second computing device; wherein the conversion uses one or more sound effects to distort the The voice information of the user of the first computing device. One or more storage devices according to claim 1, wherein the one or more facial features are to be recognized from one or more video images of the user of the first computing device. One or more storage devices according to claim 1, wherein when the instructions are executed by the at least one processor of the first computing device, further operations including the following are performed: processing is to be transferred to the second computing device The audio information of the user of the first computing device. One or more storage devices according to claim 1, wherein when the instructions are executed by the at least one processor of the first computing device, further operations including the following are performed: enabling selection of a second avatar; generating The second information to be transmitted to the second computing device to cause the selected second avatar to be animatedly displayed on the display of the second computing device, wherein the second information is based on the recognition The one or more facial features of the user of the first computing device; and causing the display of the selected second avatar on the first computing device to enable the user of the first computing device to Observe the appearance of one of the selected second avatars on the second computing device. One or more storage devices according to claim 1, wherein the one or more sound effects include a pitch shift sound effect. One or more storage devices according to claim 1, wherein when the instructions are executed by the at least one processor of the first computing device, further operations including the following are performed: causing selection on the first computing device The display of the first avatar enables the user of the first computing device to observe the The appearance of one of the selected first avatars on the second computing device. A first computing device includes: a memory circuit for storing instructions and data; a display for displaying an avatar; and a processor circuit for processing one or more instructions to perform operations including: enabling one Selection of the first avatar; identifying one or more facial features of a user of one of the first computing devices; generating information to be transmitted to a second computing device to cause the selected first avatar to animate Is displayed on a display of the first computing device, wherein the information is based on the identified facial feature(s) of the user of the first computing device; enabled based on a user input command The selected animation of the first avatar; wherein the user input command is different from the one or more facial features, and when controlled by the user of the first computing device, the user input command Generated by a user input device; and converting the user's voice information of the first computing device into target voice information to be transmitted to the second computing device; wherein, the conversion uses one or more sound effects to Distorting the voice information of the user of the first computing device. The first computing device of claim 7 further includes: one or more views of the user of the first computing device A video recording device for video images; wherein the one or more facial features are recognized from the one or more video images of the user of the first computing device that was captured. The first computing device of claim 7 further includes an audio capturing device that can capture audio information of the user of the first computing device to be transmitted to the second computing device. The first computing device of claim 7, wherein the processor is configured to process one or more instructions to perform further operations including: enabling selection of a second avatar; generating a to be transmitted to the second computing device Second information for causing the selected second avatar to be animatedly displayed on the display of the first computing device, wherein the second information is based on the identified user of the first computing device The one or more facial features; and causes the display of the selected second avatar on the first computing device to enable the user of the first computing device to observe the experience on the first computing device The appearance of one of the selected second avatars. The first computing device of claim 7, wherein the one or more sound effects include a pitch-shifting sound effect. One or more storage devices as in claim 7, wherein the processor is to process one or more instructions to perform further operations including the following: causing the display of the selected first avatar on the first computing device to cause The user of the first computing device can observe the appearance of one of the selected first avatars on the first computing device. A method for communication includes: using a first computing device to enable selection of a first avatar; using the first computing device to identify one or more facial features of a user of the first computing device ; Using the first computing device to generate information to be transmitted to a second computing device to cause the selected first avatar to be animatedly displayed on a display of the second computing device, wherein the information Based on the one or more facial features of the user of the identified first computing device; using the first computing device to enable animation of the selected first avatar based on a user input command, Wherein the user input command is different from the one or more facial features, and when controlled by the user of the first computing device, the user input command is generated by a user input device; and The first computing device converts the user's voice information of the first computing device into target voice information to be transmitted to the second computing device; wherein, the conversion uses one or more sound effects to distort The voice information of the user of the first computing device. The method of claim 13, wherein the one or more facial features are recognized from one or more video images of the user of the first computing device. The method of claim 13, further comprising: processing, by the first computing device, audio information of the user of the first computing device to be transmitted to the second computing device. The method of claim 13, further comprising: enabling selection of a second avatar by the first computing device; generating a second to be sent to the second computing device by the first computing device Information for causing the selected second avatar to be animatedly displayed on the display of the second computing device, wherein the second information is based on the identified user of the user of the first computing device One or more facial features; and the selected second avatar is displayed on the first computing device by the first computing device to enable the user of the first computing device to observe on the second computing device The appearance of one of the selected second avatars. The sound is the method of claim 13, wherein the one or more sound effects include a pitch shift sound effect. The method of claim 13, further comprising: displaying, by the first computing device, the selected first avatar on the first computing device to enable the user of the first computing device to observe the second The appearance of one of the selected first avatars on the computing device. A first computing device includes: an avatar selection module for enabling selection of a first avatar; a feature extraction module for identifying one or more users of a user of the first computing device Facial features; an avatar control module used to: generate information to be sent to a second computing device for Causing the selected first avatar to be animatedly displayed on a display of the first computing device, wherein the information is based on the identified one or more facial features of the user of the first computing device ; And based on a user input command to enable the selected animation of the first avatar, wherein the user input command is different from the one or more facial features, and when the first computing device by the During user control, the user input command is generated by a user input device, and an audio conversion module is used to: convert the user's voice information of the first computing device into target voice information to be Sent to the second computing device; wherein, the conversion uses one or more sound effects to distort the voice information of the user of the first computing device. The first computing device of claim 19 further includes a face detection and tracking module for detecting and tracking a face of the user of the first computing device. The first computing device of claim 19 further includes an audio extraction module for capturing audio information of the user of the first computing device to be transmitted to the second computing device. The first computing device of claim 19, wherein the avatar selection module can be further used to enable selection of a second avatar. The first computing device of claim 22, wherein the avatar control module is further used to: Generating second information to be transmitted to the second computing device for causing the selected second avatar to be animatedly displayed on the display of the first computing device, wherein the second information is based on being recognized The one or more facial features of the user of the first computing device. The first computing device of claim 23 further includes a display module for displaying the selected second avatar on the first computing device, so that the user of the first computing device can observe An appearance of the selected second avatar on the first computing device. The first computing device of claim 19, wherein the one or more sound effects include a pitch shift sound effect. The first computing device of claim 19 further includes a display module to display the selected first avatar on the first computing device to enable the user of the first computing device to observe the first computing device The appearance of one of the first avatars selected on the device. The first computing device of claim 19 further includes a video recording device for capturing one or more video images of the user of the first computing device, wherein the one or more facial features are Recognized from the one or more video images of the user of the first computing device that was captured.

Images