TW202107250A - Communication using interactive avatars - Google Patents

Abstract

Generally this disclosure describes a video communication system that replaces actual live images of the participating users with animated avatars. A method may include selecting an avatar; initiating communication; detecting a user input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting at least one of the animation command and the avatar parameters.

Description

Communication technology using interactive avatars (7) Invention field

The following disclosures are related to video communications, and more specifically, to video communications using interactive avatars.

Background of the invention

The increase in the types of functionalities available in mobile devices makes users want to communicate via video in addition to simple calls. For example, the user can initiate a "video call", "video conference", etc., where the camera and microphone in the device capture the user's audio and video, and the audio and video are transmitted to one or more other receivers in real time, such as Other mobile devices, desktop computers, video conferencing systems, etc. Video communication may involve the transmission of real-quality data (for example, depending on the technology of the camera, the specific video codec used to process the captured image data, etc.). Taking into account the bandwidth limitations of existing 2G/3G wireless technologies and the still limited bandwidth of emerging 4G wireless technologies, the parallel video calls made by many device users can exceed the bandwidth available in the existing wireless communication infrastructure, which can negatively affect The quality of the video call.

According to an embodiment of the present invention, a system is specifically proposed, which includes: a user input device configured to capture a user input; a communication module configured to transmit and receive information; and a Or a plurality of storage media on which instructions are stored individually or in combination, when executed by one or more processors, the instructions include the following operations: select an avatar; initiate communication; detect one User input; identifying the user input; identifying an animation command based on the user input; generating avatar parameters; and transmitting the animation command and at least one of the avatar parameters.

100: device-to-device system/system

102, 112, 102’: Device/Remote Device

104, 114, 104’/114’: camera

106, 116, 106’, 116’: microphone

107, 117: Speaker

108, 118: Touch-sensitive display/display

108’, 118’: Display

110, 120: Avatar

112’: Device/Remote Device

122: Network

124, 124’: server

126: System

128: virtual space

200: Camera, audio and touch screen frame module

202: Face detection and tracking module/face detection/tracking module/face detection module

204: Feature Extraction Module

206: Audio Conversion Module

208: Touch detection module

210: Gesture Detection Module

212: Avatar Selection Module

214: Avatar Control Module

216: System

218: Giving Back Incarnation

220: Communication module

222: processor

300, 304: WiFi connection

302: Internet

306: Enterprise AP

308: Gateway

310: firewall

312: Media and signal path

314: Home AP

400: flow chart

402~428: Operation

The features and advantages of the various embodiments of the requested subject matter will become apparent in the following detailed description and after referring to the drawings. The same numbers in the figures designate the same parts, and among them:

Figure 1A illustrates an example device-to-device system according to various embodiments of the present disclosure;

FIG. 1B illustrates an example virtual space system according to various embodiments of the present disclosure;

Figure 2 illustrates an example device according to various embodiments of the present disclosure;

Figure 3 illustrates an exemplary system implementation scheme according to at least one embodiment of the present disclosure; and

FIG. 4 is a flowchart of example operations according to at least one embodiment of the present disclosure.

Although the following detailed description is made with reference to illustrative embodiments, those skilled in the art will understand many alternatives, modifications, and variations of these embodiments.

Detailed description of the preferred embodiment

Generally, this disclosure describes a system and method for video communication using interactive avatars. Compared with moving images, the use of avatars generally reduces the amount of data to be transmitted, and therefore avatar communication requires less bandwidth. The interactive avatar is configured to enhance the user experience by changing the display of the selected avatar based on user input. In addition, the user's voice can be captured and converted to generate an avatar voice. The avatar voice can then be related to the user's voice, but can obscure the user's identity. Audio conversion may include, for example, pitch shift and/or time extension.

In one embodiment, the application program in the device coupled to the camera, microphone, and speaker is activated. The application can be configured to allow the user to choose an avatar for display on a remote device, in a virtual space, etc. The device can then be configured to initiate communication with at least one other device, virtual space, etc. For example, communication can be established via 2G, 3G, 4G cellular connections. Alternatively or additionally, communication can be established via the Internet, via a WiFi connection. After the communication is established, the camera can be configured to start capturing images and/or the distance from the object, and the microphone can be configured to start capturing sounds, such as user voice, and convert the user voice into a user voice signal.

It can then be determined whether user input is detected. User input can be captured by the user input device. User input includes touch events captured by a touch-sensitive display and gestures captured by a camera, such as a depth camera configured to capture the distance from an object, and/or a web camera. Therefore, the user input device includes a touch sensitive display And/or camera. If the user input is detected, the user input can be recognized. For touch events, the user input identifier may be related to the touch type and one or more touch positions. For gestures (such as an open hand), the user input identifier may be related to the gesture identifier. Animation commands can then be identified based on user input. The animation command corresponds to the desired response associated with the user input, such as changing the color of the displayed avatar's appearance in response to a single tap on the displayed avatar's appearance.

The avatar parameters can then be generated. Avatar parameters can be generated based on face detection, head movement, and/or animation commands. Avatar parameters can therefore include passive components based on, for example, face detection and head movement, and interactive components based on animation commands. The avatar parameters can be used to animate the avatar on at least one other device, in a virtual space, and so on. In one embodiment, the avatar parameters can be generated based on face detection, head movement, and animation commands. In this embodiment, the resulting animation includes a passive animation based on face detection and head movement, which is modified by an interactive animation based on animation commands. Therefore, the avatar animation may include passive animation based on, for example, face detection and head movement, and interactive animation based on user input.

At least one of the animation command and the avatar parameter can be transmitted subsequently. In an embodiment, at least one of a remote animation command and a remote avatar parameter is received. The remote animation command enables the device to determine the avatar parameters based on the remote animation command so as to animate the displayed avatar. The remote avatar parameters enable the device to animate the displayed avatar based on the received remote avatar parameters.

Audio communication can be accompanied by avatar animation. After the communication was established, Mike The wind can be configured to capture audio input (sound), such as user voice, and convert the captured voice into a corresponding audio signal (such as user voice signal). In one embodiment, the user's voice signal can be converted into an avatar voice signal, which can then be encoded and transmitted. The received avatar voice signal can then be converted back to sound (such as avatar voice) by the speaker. The avatar voice can therefore be based on the user's voice and the content can be saved but the spectrum data associated with the captured voice can be changed. For example, conversion includes, but is not limited to, pitch shift time extension and/or conversion playback rate.

A user input device (such as a touch-sensitive display and/or a camera) can be configured to capture user input, and the user input is configured to animate the avatar based on user input on at least one other device. User-driven animations (based on animation commands) can additionally be animations based on facial expressions and/or head movement. Animation commands may include, but are not limited to, changes in the positioning of the avatar display, distortion of facial features, changing features to convey emotions, and so on. The animation command can therefore modify the avatar animation similar to the animation based on face detection/tracking and/or modify the avatar animation in addition to the animation based on face detection/tracking. The animation command can generate a time-limited animation and can instantiate the resulting animation on the displayed avatar of the local user based on the input from the remote user.

Therefore, limited bandwidth video communication systems can be implemented using avatars. The audio can be converted and the video can be animated based on the detected user input and the recognized animation command, thereby using avatar communication to enhance the user experience. In addition, avatars can be used to preserve anonymity, including audio conversion as described in this article.

FIG. 1A illustrates a device-to-device system 100 consistent with various embodiments of the present disclosure. The system 100 may generally include devices 102 and 112 that communicate via a network 122. The device 102 at least includes a camera 104, a microphone 106, a speaker 107, and a touch-sensitive display 108. The device 112 at least includes a camera 114, a microphone 116, a speaker 117, and a touch-sensitive display 118. The network 122 at least includes a server 124.

The devices 102 and 112 may include various hardware platforms capable of wired communication and/or wireless communication. For example, the devices 102 and 112 may include, but are not limited to, video conferencing systems, desktop computers, laptops, tablets, smart phones (eg, iPhones®, Android-based phones, Blackberries®, Symbian®-based phones , Palm®-based phones, etc.), cellular phones, etc. The cameras 104 and 114 include any device for capturing digital images of the environment where the replacement table includes one or more people, and may have sufficient resolution for appearance analysis and/or gesture recognition as described herein. For example, the cameras 104 and 114 may include still cameras (for example, cameras assembled to capture still photos) or video cameras (for example, cameras assembled to capture moving images composed of multiple frames). The cameras 104 and 114 can be configured to use light in the visible spectrum or operate with light in other parts of the electromagnetic spectrum that are not limited to the infrared spectrum, the ultraviolet spectrum, and the like. In one embodiment, the cameras 104 and 114 can be configured to detect depth, that is, the distance between the camera and the object and/or the point on the object. The cameras 104 and 114 may be incorporated into the devices 102 and 112, respectively, or may be configured as independent devices that communicate with the devices 102 and 112 via wired communication or wireless communication. Specific examples of cameras 104 and 114 may include wired (e.g., universal serial Streaming (USB), Ethernet, FireWire, etc.) or wireless (e.g. WiFi, Bluetooth, etc.) web camera, such as it can be associated with a computer, video monitor, etc.; depth camera; mobile device camera (e.g. , Mobile phone or smart phone camera, which is integrated into the example device discussed previously); integrated laptop camera; integrated tablet camera (for example, iPad®, Galaxy Tab® and similar cameras), etc.

The devices 102 and 112 may further include microphones 106 and 116 and speakers 107 and 117. The microphones 106 and 116 include any devices that are configured to sense (ie, capture) sounds and convert the sensed sounds into corresponding audio signals. The microphones 106 and 116 can be integrated inside the devices 102 and 112, respectively, or can interact with these devices via wired communication or wireless communication, such as described in the example of the cameras 104 and 114 above. The speakers 107 and 117 include any device that is configured to convert audio signals into corresponding sounds. The speakers 107 and 117 can be integrated inside the devices 102 and 112, respectively, or can interact with these devices via wired communication or wireless communication, such as described above in the example of the cameras 104 and 114. The touch-sensitive displays 108 and 118 include any devices that are configured to display text, still images, moving images (such as video), user interfaces, graphics, etc., and are configured to sense touches, swipes, etc. And so on touch events. Touch events can include touch type and touch location. The touch-sensitive displays 108 and 118 can be integrated within the devices 102 and 112, respectively, or can interact with these devices via wired communication or wireless communication, such as described above in the example of the cameras 104 and 114. In one embodiment, the displays 108 and 118 are configured to display the avatars 110 and 120, respectively. As in this article It is mentioned that the avatar is defined as the graphical representation of the user in two-dimensional (2D) or three-dimensional (3D). The avatar does not have to resemble the face of the user, and therefore although the avatar can be a realistic representation, the avatar can also take the form of drawings, cartoons, sketches, etc. In the system 100, the device 102 can display an avatar 110 representing the user of the device 112 (for example, a remote user), and similarly, the device 112 can display an avatar 120 representing the user of the device 102. In this way, users can see other users' representations without having to exchange large amounts of information related to device-to-device communication using moving images. In addition, the avatar can be animated based on user input. In this way, the user can interact with the display of the local and/or remote avatar, thereby enhancing the user experience. The resulting animation can provide a wider range of animation than when only face detection and tracking might be used. In addition, the user can actively select these animations.

As mentioned in this article, avatar audio (ie, voice) is defined as converted user audio (voice). For example, the voice input may include the user's voice, that is, the user's voice, and the corresponding avatar audio may include the converted user's voice. The avatar audio may be related to the user audio. For example, the avatar voice may correspond to pitch shift, time extension, and/or other transformations of the user's voice. The avatar voice may be similar to a human voice or may correspond to a cartoon character and so on. In the system 100, the device 102 can emit an avatar audio that represents the remote user of the device 112, and similarly, the device 112 can emit an avatar audio that represents the audio captured by the device 102 (for example, the local user of the device 102 voice). In this way, the user can hear the representation of the voice of other users that can be converted.

The network 122 may include various second-generation (2G), third-generation (3G), The fourth generation (4G) is based on cellular data communication technology, Wi-Fi wireless data communication technology and so on. The network 122 includes at least one server 124 configured to establish and maintain a communication connection when using these technologies. For example, the server 124 can be configured to support Internet-related communication protocols, such as the dialog initiation protocol (SIP), which is used to establish, modify, and terminate two-party (unicast) and multi-party (multicast) dialogs; interactive connections Established Protocol (ICE), which is used to present the framework that allows the protocol to be established on top of the bit stream connection; Network Access Translator (NAT), the conversation traversal practical mechanism protocol (STUN), which allows applications operating via NAT Programs to discover the existence of other NATs; IP addresses and ports, which are allocated for the application's User Datagram Protocol (UDP) connection to connect to remote hosts; use relay traversal NAT (TURN), which allows NAT Or the components behind the firewall receive data via TCP or UDP connections, and so on.

FIG. 1B illustrates a virtual space system 126 consistent with various embodiments of the present disclosure. The system 126 can use the device 102, the device 112, and the server 124. The device 102, the device 112, and the server 124 can continue to communicate in a manner similar to that illustrated in FIG. 1A, but user interaction can occur in the virtual space 128 instead of the device-to-device format. As mentioned herein, virtual space can be defined as a digital simulation of physical location. For example, the virtual space 128 may be similar to outdoor locations, such as cities, roads, sidewalks, fields, forests, islands, etc., or indoor locations, such as offices, houses, schools, shopping malls, shops, and so on. The user represented by the avatar can appear to interact with the virtual space 128 as in the real world. The virtual space 128 may exist in one or more servers coupled to the Internet. Server, and can be maintained by a third party. Examples of virtual spaces include virtual offices, virtual meeting rooms, virtual worlds like Second Life®, massive multiplayer online role-playing games (MMORPG) like World of Warcraft®, and massive multiplayer online reality like The Sims Online® Game of Life (MMORLG) and so on. In the system 126, the virtual space 128 may contain multiple avatars corresponding to different users. Instead of the displayed avatar, the displays 108 and 118 may display a virtual space (VS) 128 in an enclosed (eg, smaller) form. For example, the display 108 may display a perspective view of what the avatar of the user of the device 102 "sees" in the virtual space 128. Similarly, the display 118 may display a perspective view of what the avatar of the user of the device 112 "sees" in the virtual space 128. Examples of content that the avatar may see in the virtual space 128 include, but are not limited to, virtual structures (such as buildings), virtual vehicles, virtual objects, virtual animals, other avatars, and so on.

Figure 2 illustrates an example device 102 according to various embodiments of the present disclosure. Although only the device 102 is described, the device 112 (such as a remote device) may include resources configured to provide the same or similar functions. As previously discussed, the device 102 is shown as including a camera 104, a microphone 106, a speaker 107, and a touch-sensitive display 108. The camera 104, the microphone 106, and the touch-sensitive display 108 can provide input to the camera, audio, and touch screen frame module 200, and the camera, audio, and touch screen frame module 200 can provide output to the speaker 107 (such as audio signals) . The camera, audio, and touch screen frame modules 200 may include customary, proprietary, known and/or later developed audio and video processing codes (or command sets). The audio and video processing codes are usually clearly defined and Operable to control at least the camera 104, a microphone 106, a speaker 107, and a touch-sensitive display 108. For example, the camera, audio and touch screen frame module 200 enables the camera 104, microphone 106, speaker 107 and touch sensitive display 108 to record images, distance from objects, sound and/or touch, and can process images, sounds, and audio signals. And/or touch, the image and/or sound can be copied, the audio signal can be provided to the speaker 107, and so on. The camera, audio, and touch screen frame module 200 may vary depending on the device 102, and more particularly, depending on the operating system (OS) running in the device 102. Example operating systems include iOS®, Android®, Blackberry® OS, Symbian®, Palm® OS, etc. The speaker 107 can receive audio information from the camera, audio, and touch screen frame module 200, and can be configured to replicate local sounds (for example, to provide audio feedback of the user's voice, the audio feedback is converted or not) And remote sound (for example, the voice of another party or parties (converted or unconverted) participating in a phone call, video call, or interaction in a virtual location).

The face detection and tracking module 202 can be configured to identify and track the head, face, and/or facial area in the image provided by the camera 104. For example, the face detection module 204 may include customary, proprietary, known and/or later developed face detection codes (or command sets), hardware and/or firmware, which are usually clearly defined and It is operable to receive standard format images (such as but not limited to RGB color images) and at least to some extent recognize faces in the images. The face detection and tracking module 202 can also be configured to track the detected face through a series of images (such as a video frame under 24 frames per second) and determine the head based on the detected face position. Known tracking systems that can be used by the face detection/tracking module 202 can include particle filtering, averaging Motion, Kalman filtering, etc., each of which can use edge analysis, square sum variance analysis, feature point analysis, histogram analysis, skin color analysis, and so on.

The feature extraction module 204 can be configured to recognize the location and/or shape of features (for example, facial indicators (such as eyes, eyebrows, nose, mouth, etc.) in the face detected by the face detection module 202) ). In one embodiment, the avatar animation can be directly based on the sensed facial movements (such as changes in facial features) without facial expression recognition. The corresponding feature points on the face of the avatar can follow or imitate the movement of a real person's face. This is called "emoji imitation" or "performance-driven facial animation". The feature extraction module 204 may include conventional, exclusive, known and/or later developed facial feature identification codes (or instruction sets), which are usually clearly defined and operable to receive standard format images from the camera 104 ( For example, but not limited to RGB color images) and at least to some extent extract one or more facial features in the image. Such known facial feature systems include, but are not limited to, the CSU facial recognition evaluation system of Colorado State University.

The feature extraction module 204 can also be configured to recognize expressions associated with the detected features (for example, recognize whether the previously detected face is happy, sad, smiling, frowning, surprised, excited, etc.)). Therefore, the feature extraction module 204 may further include conventional, proprietary, known and/or later developed facial expression detection and/or identification codes (or instruction sets), which are usually clearly defined and operable to Detect and/or recognize expressions in faces. For example, the feature extraction module 204 can determine the size and/or position of facial features (such as eyes, mouth, cheeks, teeth, etc.), and can compare these facial features with a facial feature database. Include categories with corresponding facial features (E.g., smile, frown, excitement, sadness, etc.) multiple sample facial features.

The audio conversion module 206 is configured to convert the user's voice into an avatar voice, that is, the converted user's voice. Conversion includes adjusting the tempo (for example, extending the time), pitch (for example, pitch shift), and playback rate. For example, the audio conversion module 206 may include customary, proprietary, known, and/or later-developed audio conversion codes (or command sets), which are usually clearly defined and operable to receive a voice representing the user’s voice. Data, and convert the voice data into converted voice data. The voice data may be related to audio signals based on sounds captured by the microphone 106 and processed by the camera, audio, and touch screen frame module 200. Such known voice conversion systems include, but are not limited to, a voice-activated open resource audio processing library, which is configured to adjust the rhythm, pitch, and playback rate of audio streams or audio files.

The audio conversion module 206 may include a plurality of predefined voice styles, which correspond to conversion parameters associated with the voice of the converted user. For example, the conversion parameters can be combined to maintain the converted voice output heard by humans in different tones and/or rhythms. For the voice of a human female or child, the pitch can be shifted to a high frequency; for a human male’s voice, the pitch can be shifted to a lower frequency, and the rhythm can be adjusted up or down to increase or decrease the voice. Speed, etc. In another example, the conversion parameters can be combined to generate a converted voice output corresponding to, for example, animal voices (for example, cats) and/or cartoon character voices. This can be achieved by adjusting the pitch, other frequency components and/or sampling parameters of the user's voice.

The user can select the desired audio conversion input before initiating communication. And/or you can select the desired audio conversion during the communication. The audio conversion module 206 can be configured to provide sample audio conversion output in response to a request from the user. In one embodiment, the audio conversion module 206 may include a facility that allows the user to select audio conversion parameters to generate customized audio conversion output. The facility can be configured to provide sample converted audio output based on the user's voice input. The user can then adjust the audio conversion parameters (for example, trial and error method) until a suitable conversion output is achieved. The audio conversion parameters associated with suitable output to the user can be subsequently stored and/or utilized for avatar communication, as described herein.

The touch detection module 208 is configured to receive touch data from the camera, audio, and touch screen frame module 200 and recognize touch events based on the received touch data. The touch event identifier may include touch type and/or touch location. Touch types can include single tap, double tap, tap and hold, tap and move, press and stretch, heavy tap, and so on. The touch position may include a touch start position, a touch end position, and/or an intermediate moving touch position, and so on. The touch position may correspond to the coordinates of the touch-sensitive display 108. The touch detection module 208 may include conventional, proprietary, known, and/or later-developed touch detection codes (or instruction sets), which are usually clearly defined and operable to receive touch data and recognize touch events.

The gesture detection module 210 is configured to receive the depth and/or image data from the camera, audio, and touch screen frame module 200, and recognize the corresponding gesture based on the received depth and/or image data, and based on the recognized Gesture determination gesture identifier. The depth corresponds to the distance from the camera to the object. The gesture identifier is related to the recognized gesture. Gesture detection module 210 can be packaged These include customary, proprietary, known, and/or later developed gesture detection codes (or instruction sets), which are usually clearly defined and operable to recognize gestures based on received depth and/or image data.

For example, the gesture detection module 210 may include a database of predefined gestures. The predefined gestures may include at least some relatively common and relatively simple gestures, including opening hands, closing hands (ie, making a fist), waving hands, making circular movements with hands, moving hands from right to left, and hands from left to left. Move right and so on. Therefore, gestures may include static non-moving hand gestures, active moving hand gestures, and/or a combination thereof. In one embodiment, the gesture detection module 210 may include training facilities configured to allow the user to change predefined gestures and/or train new gestures. The custom gesture and/or the new gesture can then be associated with a gesture identifier, and the gesture identifier can be associated with an animation command, as described herein. For example, the user can select animation commands to associate with gestures from a predefined list of animation commands.

Therefore, animation commands are related to the desired response to user input. Animation commands may be associated with recognized user input such as touch event identifiers and/or gesture identifiers. In this way, the user can interact with the displayed avatar and/or can set gestures to modify the animation of the displayed avatar.

The avatar selection module 212 is configured to allow the user of the device 102 to select an avatar for display on the remote device. The avatar selection module 212 may include customary, proprietary, known, and/or later developed user interface building codes (or command sets), which are usually clearly defined and operable to present different avatars to the user for the purpose of The user can select one of these avatars. in In an embodiment, one or more avatars may be predefined in the device 102. The predefined avatar allows all devices to have the same avatar, and only the selection of the avatar (such as the identification of the predefined avatar) needs to communicate with the remote device or virtual space during the interaction, thereby reducing the amount of information that needs to be exchanged. The avatar is selected before establishing communication, but it can also be changed during active communication. Therefore, the avatar selection may be sent or received at any point during the communication, and the receiving device may change the displayed avatar according to the received avatar selection.

The avatar control module 214 is configured to receive a user input identifier based on the user input of the device 102. The user input identifier may include a touch event identifier determined by the touch detection module 208 based on touch event data or a gesture identifier determined by the gesture detection module 210. The touch event data includes touch type and/or touch position. The touch location may correspond to the coordinates associated with the touch-sensitive display 108. The touch position can be mapped to one or more points on the displayed avatar, for example, to a feature, such as nose tip, mouth, lips, ears, eyes, and so on. The displayed point on the avatar may be related to the desired response (that is, the animation command) of the avatar animation.

The avatar control module 214 is configured to determine the animation command based on the user input identifier (that is, the recognized user input). The animation commands are combined to identify the desired avatar animation. For example, the desired animation includes changing the color of the face of the displayed avatar, changing the size of a feature of the displayed avatar (such as making the nose larger), making winks, blinking, smiling, removing a feature (such as ears), and so on. Therefore, the avatar control module 214 is configured to receive the user input identifier and determine the animation command based on the user input identifier.

The avatar control module 214 is configured to execute based on animation commands Avatar animation. In one embodiment, for an interactive animation displayed on a remote device such as device 112, an animation command can be transmitted and the remote avatar control module can then execute the animation. In another embodiment, the avatar parameters can be transmitted and combined for the immediate implementation of the avatar animation.

The interactive animation performed based on the animation command may have a finite duration, after which the avatar animation may return to a passive animation based on, for example, face detection and tracking as described herein. The implemented interactive animation that affects the size of the feature can be configured to gradually change the size and gradually return to the original size. Additionally or alternatively, the animation that affects the size of the feature can be combined to have an effect gradient. In other words, the relative magnitude of the size change may depend on the position relative to, for example, key vertices. Points on the displayed avatar that are closer to the key apex may experience greater changes than points that are relatively farther away on the displayed avatar.

Therefore, the avatar control module 214 can receive a user input identifier based on the user input, can determine an animation command based on the user input identifier, and can execute an animation based on the animation command. The interactive animation based on the animation command may be limited in time to a time period (duration) and/or may include an effect gradient. The animation can return to the passive avatar animation based on face detection and tracking after the time period.

The avatar control module 214 is configured to generate parameters for animate the avatar. As mentioned in this article, animation can be defined as changing the appearance of an image/model. Animation includes passive animation based on, for example, facial expressions and/or head movement, and interactive animation based on user input. A single animation (including passive animation and interactive animation) can change the appearance of a two-dimensional still image, or multiple animations There can be in sequence to simulate the motion of the image (such as turning head, nodding, blinking, talking, frowning, smiling, laughing, winking, blinking, etc.). Examples of animations used for 3D models include deforming 3D wireframe models, applying texture mapping, and recalculating model vertices for normal display. The changes in the position of the detected face and/or the extracted facial features can be transformed into parameters that make the features of the avatar resemble the features of the user's face. In one embodiment, the general expression of the detected face can be converted into one or more parameters that make the avatar display the same expression. The expression of the avatar can also be exaggerated to emphasize the expression. When avatar parameters can generally be applied to all predefined avatars, knowledge of the selected avatar is not necessary. However, in an embodiment, the avatar parameters may be specific to the selected avatar, and therefore may be changed in the case of selecting another avatar. For example, human avatars may require different parameter settings (for example, different avatar characteristics can be changed) from animal avatars, cartoon avatars, etc., to demonstrate emotions such as happiness, sadness, anger, surprise, and so on.

The avatar control module 214 may include customary, proprietary, known, and/or later developed graphics processing codes (or instruction sets), which are usually clearly defined and operable to make the avatar selection module 212 based on the face Face/head position detected by the detection and tracking module 202, facial features detected by the feature extraction module 204, and/or determined by the touch detection module 208 and/or gesture detection module 210 The avatar selected by the user inputting the identifier becomes an animation. For the animation method based on facial features, the two-dimensional avatar animation can be completed by image warping or image gradation, and the three-dimensional avatar animation can be free-form deformation (FFD) or by using the animation structure defined in the head 3D model To be done. Oddcast is a software resource that can be used for 2D avatar animation An example, and FaceGen is an example of software resources that can be used for 3D avatar animation.

For example, for an interactive animation including the nose displayed by an elongated three-dimensional avatar, the key vertex v _k related to the tip of the nose can be defined (for example, selected). The associated three-dimensional motion vector d _k (dx, dy, dz) and the effect radius R can be defined for the key vertex v _k . The other vertices within the effect radius R can be changed (that is, moved) in the interactive animation, and the vertices outside the effect radius R can remain unchanged due to the interactive animation. The interactive animation may have an associated duration, that is, the animation time T, which may extend to multiple frames. The temporary effect parameter η _t can be defined based on the time t and the animation time T, such as:

The vertices within the effect radius R that are relatively closer to v _k can change relatively more than the vertices that are relatively farther away from the key vertex v _k . A vertex of the spatial effect parameter η _i v _i can be defined as:

則可定義為

。互動動畫化身之新坐標則為

，其中

相應於頂點v _i 之坐標，其基於面部偵測及追蹤，亦即，被動動畫。 Vertices v _i and motion vector at time t

Can be defined as

. The new coordinates of the interactive animation avatar are

,among them

Corresponding to the coordinates of the vertex v _i, which is based on the face detection and tracking, i.e., passive animation.

Therefore, it is possible to animate the displayed avatar including the passive animation that modifies the interactive animation. The interactive animation can be limited by the overall duration, and the magnitude of the effect of the animation can vary within the duration. Interactive animation can be grouped Matching only affects a part of the avatar, and these effects can be greater for points closer to the key apex. After the interactive animation is completed, the animation can continue based on face detection and tracking as described herein.

In addition, in the system 100, the avatar control module 214 can receive remote avatar selection and remote avatar parameters that can be used to display and animate the avatar corresponding to the user at the remote device. Animation may include passive animation and interactive animation. The avatar control module can enable the display module 216 to display the avatar 110 on the display 108. The display module 216 may include customary, proprietary, known, and/or later developed graphics processing codes (or instruction sets), which are generally well-defined and operable to display on the display 108 according to example device-to-device embodiments Show the avatar and animate it. For example, the avatar control module 214 can receive a remote avatar selection and can enable the remote avatar selection to be interpreted to correspond to a predetermined avatar. The display module 216 can then display the avatar 110 on the display 108. In addition, the remote avatar parameters received in the avatar control module 214 can be interpreted, and commands can be provided to the display module 216 to animate the avatar 110. In one embodiment, more than two users can participate in a video call. When two or more users are interacting in a video call, the display 108 can be divided or segmented to allow more than one avatar corresponding to the remote user to be displayed at the same time. Or, in the system 126, the avatar control module 214 can Receiving information, the information causes the display module 216 to display what the avatar of the user corresponding to the device 102 "sees" in the virtual space 128 (for example, from the virtual perspective of the avatar). For example, the display 108 may display buildings, objects, animals, other avatars, etc., represented in the virtual space 128.

In one embodiment, the avatar control module 214 can be configured to display The display module 216 displays the "feedback" avatar 218. The feedback avatar 218 indicates how the selected avatar appears on the remote device, in a virtual location, and so on. In detail, the feedback avatar 218 is presented as an avatar selected by the user and can be animated using the same parameters generated by the avatar control module 214. In this way, the user can confirm what the remote user sees during his interaction. The feedback avatar 218 can also be used to display interactive animations caused by remote user input of the device 112. Therefore, the local user can interact with its feedback avatar (such as the avatar 218 and the user of the device 102) so that the interactive animation of the associated avatar is displayed to the remote user on the device 112. The local user can similarly interact with the displayed avatar (such as the avatar 110) of the remote user, so that the interactive animation of the remote user's feedback avatar is displayed on the device 112.

The communication module 220 is configured to transmit and receive information for selecting an avatar, displaying an avatar, animating the avatar, displaying a perspective view of a virtual location, and so on. The communication module 220 may include customary, exclusive, known and/or later-developed communication processing codes (or instruction sets), which are usually clearly defined and operable to transmit avatar selections, avatar parameters, animation commands, and interactions. Avatar parameters and receive remote avatar selection, remote avatar parameters, remote animation commands and remote interactive avatar parameters. The communication module 220 can also transmit and receive audio information corresponding to the interaction based on the avatar. The communication module 220 can transmit and receive the above information via the network 122, as described previously.

The processor 222 is configured to perform operations associated with the device 102 and one or more of the modules included therein.

Figure 3 illustrates an example system implementation method according to at least one embodiment case. The device 102' is configured to communicate wirelessly via the WiFi connection 300 (for example, at work), the server 124' is configured to negotiate the connection between the devices 102' and 112' via the Internet 302, and the device 112 It is configured to communicate wirelessly via another WiFi connection 304 (for example, at home). In one embodiment, a video calling application based on a device-to-device avatar is activated in the device 102'. After the avatar is selected, the application may allow the selection of at least one remote device (for example, the device 112'). The application can then cause the device 102' to initiate communication with the device 112'. The communication can be initiated by the device 102' transmitting a connection establishment request to the device 112' via an enterprise access point (AP) 306. The enterprise AP 306 can be an AP that can be used in commercial settings, and therefore can support higher data throughput and more parallel wireless clients than the home AP 314. The enterprise AP 306 can receive the wireless signal from the device 102', and can transmit the connection establishment request via the gateway 308 via various commercial networks. The connection establishment request can then pass through the firewall 310, which can be configured to control the flow of information into and out of the WiFi network 300.

The connection establishment request of the device 102' can then be processed by the server 124'. The server 124' can be configured to register the IP address, identify the destination address, and NAT traversal, so that the connection establishment request can be directed to the correct destination on the Internet 302. For example, the server 124' can resolve the desired destination (such as the remote device 112') from the information in the connection establishment request received from the device 102, and can route the signal through the correct NAT, port, and so on Destination IP address. These operations may only have to be performed during the connection establishment period, depending on the network configuration. In some cases, the operation can be repeated during the video call to provide notification to the NAT to keep the connection alive. Media The body and signal path 312 can carry video (such as avatar selection and/or avatar parameters) and audio information guidance to the home AP 314 after the connection has been established. The device 112' can then receive a connection establishment request and can be configured to determine whether to accept the request. Determining whether to accept the request may include, for example, presenting a visual narrative to the user of the device 112' who inquires about whether to receive the connection request from the device 102'. When the user of the device 112' receives the connection (for example, receives the video call), the connection can be established. The cameras 104' and 114' can be configured to then start to capture images of the respective users of the devices 102' and 112', respectively, for the purpose of forming an animation by the avatar selected by each user. The microphones 106' and 116' can be configured to then start capturing audio from each user. When the information exchange between the devices 102' and 112' starts, the displays 108' and 118' can display the avatars of the users corresponding to the devices 102' and 112' and animate the avatars.

FIG. 4 illustrates a flowchart 400 of exemplary operations consistent with an embodiment of the present disclosure. These operations may be performed by the device 102 and/or 112, for example. In detail, the flowchart 400 depicts the operation of configuring to implement avatar animation (including passive animation and/or interactive animation) and/or audio conversion for communication between devices via a network. It is assumed that face detection and tracking, feature extraction, and passive avatar animation are implemented and operated as described herein.

The avatar model can be selected in operation 402. The avatar model may include video avatar selection and audio conversion selection. Multiple video avatar models can be displayed, and the user can select a desired avatar from the video avatar models. In one embodiment, selecting a video avatar model may include an associated audio conversion. For example, an avatar like a cat can be associated with an audio conversion like a cat. In another embodiment, the audio conversion can be selected independently of the video avatar selection.

The avatar model including audio conversion can be selected before starting the communication, but it can also be changed during the active communication. Therefore, it is possible to send or receive the avatar selection and/or change the audio conversion selection at any point during the communication, and the receiving device may change the displayed avatar according to the received avatar selection.

The avatar communication can be initiated at operation 404. For example, users can run applications that are configured to communicate audio and video using the avatar described herein. Operation 404 may include assembling communication and establishing a connection. The communication configuration includes identifying at least one remote device or virtual space participating in the video call. For example, the user can be stored in an application, stored in a device associated with another system (such as a contact list in a smart phone, mobile phone, etc.), or stored remotely on the Internet (such as Choose from a list of remote users/devices on social media sites such as Facebook, LinkedIn, Yahoo, Google+, MSN, etc.). Alternatively, users can choose to perform online operations in a virtual space such as Second Life.

In operation 406, the camera in the device can then start capturing images and/or depth, and the microphone in the device can start capturing sound. The image can be a still image or a moving image (for example, multiple images captured in sequence). The depth can be captured with the image or independently. The depth corresponds to the distance from the camera to the object (and the point on the object) in the camera's field of view. It can be determined in operation 408 whether a user input is detected. User input includes gestures captured by an image and/or depth camera and touch input detected on a touch-sensitive display. If the user input is detected, the user input can be recognized in operation 410. User input identifier including touch identifier or gesture recognition symbol. The touch identifier may be determined based on the touch to the touch-sensitive display and may include the touch type and the touch position. The gesture identifier can be determined based on the captured image and/or depth data and can include recognizing gestures.

The animation command can be recognized at operation 412. The animation command can be configured to animate the selected avatar of the user displayed on the remote device, or animate the feedback avatar of the remote user also displayed on the remote user's device. The animation command corresponds to the desired response associated with the user input. For example, touching the face of the displayed avatar (user input) can produce a change in the color of the face of the displayed avatar (a desired response recognized by an animation command). Animation commands can be recognized based on recognized user input. For example, each user input may be related to (e.g., associated with) animation commands in a database with user input identifiers and animation commands.

Operation 414 includes generating avatar parameters. The avatar parameters include passive components and may include interactive components. If no user input is detected, the avatar parameters can include passive components. If user input is detected, whether the avatar parameters can include interactive components depends on the animation command and therefore on the user input. For the user input corresponding to the animation command assembled to animate the selected avatar of the user, the animation command can be transmitted together with the avatar parameters including only passive components or can be applied to the avatar parameters before transmission, so that all The transmitted avatar parameters include passive components and interactive components. For the input corresponding to the animation command configured to animate the remote user's feedback displayed on the remote user's device, only the animation command may be transmitted.

Operation 416 includes converting and encoding the retrieved audio. Captured sound The signal can be converted into an audio signal (such as a user's voice signal). The user's voice signal can be converted according to the audio conversion part selected by the avatar in operation 402. The converted user voice signal corresponds to the avatar voice signal. The avatar voice signal can be encoded using a technology known for transmission to a remote device and/or virtual space via a network. The converted and encoded audio may be transmitted at operation 418. Operation 418 may further include transmitting at least one of animation commands and avatar parameters. The transmission animation command is configured to allow the remote device to animate the locally displayed avatar by modifying the avatar parameters according to the animation command. The transmitted avatar parameters that have been modified according to the animation command before transmission can be directly used to animate the avatar displayed on the remote device. In other words, the modification of the avatar parameters indicated by the animation command can be performed locally or remotely.

Operation 420 includes receiving remotely encoded audio, which may be converted audio. Operation 420 further includes receiving at least one of a remote animation command and remote avatar parameters. The remote animation command can be used to modify the avatar parameters corresponding to the remote user's displayed avatar or the local user's displayed feedback avatar. Animation commands and avatar parameters are combined to generate an avatar animation modified based on user input. At operation 422, the received audio can be decoded and played, and at operation 424, the avatar can be displayed and animated.

The animation of the displayed avatar can be based on the detected and recognized user input, as described herein. In an example of device-to-device communication (such as the system 100), at least one of remote avatar selection or remote avatar parameters can be received from a remote device. The avatar corresponding to the remote user can then be displayed based on the received remote avatar selection, and can be animated based on the received remote avatar parameters. Example of interaction in a virtual location (e.g. system 126) , Can receive information that allows the device to display content that corresponds to what the device user’s avatar sees.

It can be determined at operation 426 whether the communication is complete. If the communication is completed, the program flow can be ended at operation 428. If the communication is not completed, the program flow can continue to operation 406 to capture images, depth, and/or audio.

Although FIG. 4 illustrates various operations according to one embodiment, it should be understood that not all operations depicted in FIG. 4 are necessary for other embodiments. In fact, what this article completely covers is that other embodiments of the present disclosure, the operations depicted in FIG. 4, and/or other operations described herein can all be combined in one way, which is not explicitly shown in the accompanying drawings. In any scheme, it is still completely consistent with this disclosure. Therefore, the requirements for features and/or operations that are not exactly displayed in a drawing are deemed to belong to the scope and content of this disclosure.

As used in any of the embodiments herein, the term "application (app)" can be embodied by codes or instructions, which can be executed on a programmable circuit such as a host processor or other programmable circuits.

As used in any of the embodiments herein, the term "module" can refer to apps, software, firmware, and/or circuits, which are configured to perform any of the above operations. The software may be embodied as packaged software, codes, instructions, instruction sets and/or data recorded on at least one non-transitory computer-readable storage medium. The firmware can be embodied as hard-coded (for example, non-electricity) codes, commands or command sets and/or data in the memory device.

As used in any of the embodiments herein, "circuits" can include, for example, hard-wired circuits alone or in any combination; circuits can be programmed, such as A computer processor that includes one or more separate instruction processing cores; a state machine circuit and/or a firmware that stores instructions executed by a programmable circuit. Modules can be collectively or individually embodied as circuits that form part of a large system, such as integrated circuits (IC), system-on-chip (SoC), desktop computers, laptop computers, tablet computers, servers, and intelligence Type phone and so on.

Any of the operations described in this way can be implemented in a system that includes one or more storage media on which separate or combined instructions are stored. When the instructions are executed by one or more processors, These instructions execute these methods. Here, the processor may include, for example, a server CPU, a mobile device CPU, and/or other programmable circuits. In addition, the operations described herein are intended to be distributed across multiple physical devices, such as processing structures at more than one different physical location. Storage media can include any type of tangible media, for example, any type of discs, including hard disks, floppy discs, optical discs, optical discs-read only memory (CD-ROM), and rewritable discs (CD-RW) ) And magneto-optical disks; semiconductor devices, such as read-only memory (ROM), random access memory (RAM) (such as dynamic and static RAM), erasable programmable read-only memory (EPROM), electrical erasable It can be designed as read-only memory (EEPROM), flash memory, solid state disk (SSD), magnetic or optical card; or any type of media suitable for storing electronic commands. Other embodiments can be implemented as a software module executed by a programmable control device. The storage medium may be non-transitory.

Therefore, the present disclosure provides a method and system for interactively animate an avatar instead of moving images for video communication. Compared with the sending of moving images, the use of avatars reduces the amount of information to be exchanged. The The system and method are further configured to convert the user's voice into an avatar voice by, for example, pitch shifting and/or prolonging the time of the captured audio signal. The interactive animation of the avatar can be based on the detected user input, including touch and gesture. Interactive animation is combined to modify the animation based on face detection and tracking judgment.

According to one aspect, a system is provided. The system may include: a user input device configured to capture user input; a communication module configured to transmit and receive information; and one or more storage media. In addition, the one or more storage media stores individual or combined instructions, and the following operations are generated when the instructions are executed by one or more processors, including: selecting an avatar; initiating communication; detecting use User input; identifying user input; identifying animation commands based on user input; generating avatar parameters; and transmitting at least one of animation commands and avatar parameters.

Another example system includes the aforementioned components and further includes: a microphone configured to capture sound and convert the captured sound into a corresponding audio signal; and instructions, when the instructions are executed by one or more processors The following additional operations are generated: capturing the user's voice and converting the user's voice into a corresponding user's voice signal; converting the user's voice signal into an avatar voice signal; and transmitting the avatar voice signal.

Another example system includes the aforementioned components and further includes: a camera configured to capture an image; and instructions, when the instructions are executed by one or more processors, the following additional operations are generated: capture an image; detect an image The face in, extract features from the face, and convert the features into avatar parameters.

Another example system includes the aforementioned components and further includes: a display; and instructions. When the instructions are executed by one or more processors, the following additional operations are generated: displaying at least one avatar; receiving one of remote animation commands and remote avatar parameters At least one; and animating a displayed avatar based on at least one of a remote animation command and a remote avatar parameter.

Another example system includes the aforementioned components and further includes: a speaker configured to convert an audio signal into sound; and instructions, when the instructions are executed by one or more processors, the following additional operations are generated: receiving remote avatar voice Signal; and convert the remote avatar voice signal into avatar voice.

Another example system includes the aforementioned components, and the user input device is a camera configured to capture a distance, and the user input is a gesture.

Another example system includes the aforementioned components, and the user input device is a touch sensitive display and the user input is a touch event.

Another example system includes the aforementioned components, and the conversion includes at least one of pitch shift and time extension.

According to another aspect, a method is provided. The method may include selecting an avatar; initiating communication; detecting user input; recognizing user input; recognizing animation commands based on user input; generating avatar parameters based on the animation commands; and transmitting at least one of the animation commands and avatar parameters.

Another example method includes the foregoing operations and further includes: capturing user voice and converting the user voice into a corresponding user voice signal; converting the user voice signal into an avatar voice signal; and transmitting the avatar voice signal.

Another example method includes the foregoing operations and further includes: capturing an image; detecting a face in the image; extracting features from the face; and converting the features into avatar parameters.

Another example method includes the foregoing operations and further includes: displaying at least one avatar; receiving at least one of a remote animation command and a remote avatar parameter; and making a displayed avatar into an avatar based on at least one of the remote animation command and the remote avatar parameter Animation.

Another example method includes the foregoing operations and further includes: receiving a remote avatar voice signal; and converting the remote avatar voice signal into an avatar voice.

Another example method includes the aforementioned operations and the user input is a gesture.

Another example method includes the aforementioned operations and the user input is a touch event.

Another example method includes the foregoing operations and the conversion includes at least one of pitch shift and time extension.

According to another aspect, a system is provided. The system may include one or more storage media storing individual or combined instructions on the one or more storage media, and the following operations are generated when the instructions are executed by one or more processors, including selecting an avatar Initiate communication; detect user input; recognize user input; recognize animation commands based on user input; generate avatar parameters; and transmit at least one of animation commands and avatar parameters.

Another example system includes instructions that generate the aforementioned operations when executed by one or more processors, and also includes: fetching users Voice and convert the user's voice into a corresponding user's voice signal; convert the user's voice signal into an avatar voice signal; and transmit the avatar voice signal.

Another example system includes instructions that generate the foregoing operations when executed by one or more processors, and also includes: capturing an image; detecting a face in the image; extracting features from the face; and combining the features Converted into avatar parameters.

Another example system includes instructions that generate the foregoing operations when the instructions are executed by one or more processors, and also includes: displaying at least one avatar; receiving at least one of remote animation commands and remote avatar parameters; and based on At least one of the remote animation command and the remote avatar parameter animates a displayed avatar.

Another example system includes instructions that generate the aforementioned operations when the instructions are executed by one or more processors, and also includes: receiving a remote avatar voice signal; and converting the remote avatar voice signal into an avatar voice.

Another example system includes instructions that when executed by one or more processors generate the aforementioned operations, and the user input is a gesture.

Another example system includes instructions that generate the aforementioned operations when the instructions are executed by one or more processors, and the user input is a touch event.

Another example system includes instructions that when executed by one or more processors produce the aforementioned operations, and the conversion includes at least one of pitch shift and time extension.

The terms and expressions used in this article are used as descriptive terms and are not limiting, and when using these terms and expressions, it is not intended to exclude the display And any equivalents (or parts thereof) of the described features, and it should be recognized that there may be various modifications within the scope of the patent application. Therefore, the scope of patent application is intended to cover all such equivalents.

100: device-to-device system/system

102, 112: device/remote device

104, 114: Camera

106, 116: Microphone

107, 117: Speaker

108, 118: Touch-sensitive display/display

110, 120: Avatar

122: Network

124: Server

Claims (28)

A system including: A user input device configured to capture user input; A communication module configured to transmit and receive information; and One or more storage media stored thereon, individually or in combination, instructions that when executed by one or more processors cause the first device to perform the following operations, which include: Choose an avatar Receive at least one image of the user; Passively cause the avatar to be animatedly presented based at least in part on the at least one image, so as to generate a passively animated avatar for display on a remote device, wherein the passively animated avatar simulates the movement of a user's body part ； Using the user input device to detect user input; Determining the interactive animation of the avatar presented by the interactive animation based at least in part on the user input; Using the interactive animation to modify the avatar presented by the passive animation, so that the avatar presented by the interactive animation is generated for display as a feedback avatar on the first device; and Transmitting a signal to the remote device, the signal being configured to cause the avatar presented in the interactive animation to be displayed on the remote device; Wherein the user input device includes a touch screen, and the user input includes touch events on the touch screen Pieces. The system according to claim 1, which further includes: A microphone configured to capture sound and convert the captured sound into a corresponding audio signal, wherein the instructions, when executed by one or more processors, cause the following additional operations: Capturing the user's voice and converting the user's voice into a corresponding user's voice signal; Transforming the user voice signal into an avatar voice signal; and Transmitting the avatar voice signal to the remote device. The system according to claim 1, comprising a camera configured to capture an image, wherein the instructions, when executed by one or more processors, cause the following additional operations: Capturing the at least one image of the user; Detecting the face in the at least one image; Extract features from the face; Convert the features into avatar parameters, and The passively animated avatar is generated based at least in part on the avatar parameter. The system of claim 1, further comprising a display, wherein the instructions, when executed by one or more processors, cause the following additional operations: Displaying at least one avatar on the display; Receiving at least one of a remote animation command and a remote avatar parameter; and Passively causing the at least one avatar to be animatedly presented based on at least one of the remote animation command and the remote avatar parameter. The system of claim 1, further comprising a speaker configured to convert an audio signal into sound, wherein the instructions, when executed by one or more processors, cause the following additional operations: Receive remote avatar voice signals; and Convert the remote avatar voice signal into avatar voice. The system according to any one of claims 1 to 5, wherein the user input is one or more points on the touch screen mapped to one or more points of the avatar presented by the passive animation Touch events at touch locations. The system according to claim 2, wherein the transformation includes at least one of transposition and time extension. A method to be executed by a first device, which includes: Choose an avatar Receive at least one image of the user; Passively cause the avatar to be animatedly presented based at least in part on the at least one image, so as to generate a passively animated avatar for display on a remote device, wherein the passively animated avatar simulates the movement of a user's body part ； Use a user input device to detect user input; Determining the interactive animation of the avatar presented by the passive animation based at least in part on the user input; Use the interactive animation to modify the rendering of the passive animation An avatar, such that an avatar presented in an interactive animation is generated to be displayed as a feedback avatar on the first device; and Transmitting a signal to the remote device, the signal being configured to cause the avatar presented in the interactive animation to be displayed on the remote device; The user input device includes a touch screen, and the user input includes a touch event on the touch screen. The method according to claim 8, which further includes: Capturing the user's voice and converting the user's voice into a corresponding user's voice signal; Transforming the user voice signal into an avatar voice signal; and Transmitting the avatar voice signal to the remote device. The method according to claim 8, which further includes: Capturing the at least one image of the user; Detecting the face in the at least one image; Extracting features from the face; converting the features into avatar parameters; and The passively animated avatar is generated based at least in part on the avatar parameter. The method according to claim 8, which further includes: Display at least one avatar on the display; Receiving at least one of a remote animation command and a remote avatar parameter; and Passively causing the at least one avatar to be animatedly presented based on at least one of the remote animation command and the remote avatar parameter. The method according to claim 8, which further includes: Receive remote avatar voice signals; and Convert the remote avatar voice signal into avatar voice. The method according to any one of claims 8 to 12, wherein the user input is one or more points on the touch screen mapped to one or more points of the avatar presented by the passive animation Touch events at touch locations. The method according to claim 9, wherein the transformation includes at least one of transposition and time extension. A system, which includes one or more storage media, the one or more storage media separately or in combination stores instructions thereon, the instructions when executed by one or more processors cause a first The device performs the following operations, which include: Choose an avatar Receive at least one image of the user; Passively cause the avatar to be animatedly presented based at least in part on the at least one image, so as to generate a passively animated avatar for display on a remote device, wherein the passively animated avatar simulates the movement of a user's body part ； Use a user input device to detect user input; Determining the interactive animation of the avatar presented by the passive animation based at least in part on the user input; The interactive animation is used to modify the avatar presented by the passive animation, so that the avatar presented by the interactive animation is generated for the first device To show up as an avatar of feedback; and Transmitting a signal to the remote device, the signal being configured to cause the avatar presented in the interactive animation to be displayed on the remote device; The user input device includes a touch screen, and the user input includes a touch event on the touch screen. The system of claim 15, wherein the instructions, when executed by one or more processors, cause the following additional operations: Capturing the user's voice and converting the user's voice into a corresponding user's voice signal; Transforming the user voice signal into an avatar voice signal; and Transmitting the avatar voice signal to the remote device. The system of claim 15, wherein the instructions, when executed by one or more processors, cause the following additional operations: Capturing the at least one image of the user; Detecting the face in the at least one image; Extract features from the face; Convert the features into avatar parameters, and The passively animated avatar is generated based at least in part on the avatar parameter. The system of claim 15, wherein the instructions, when executed by one or more processors, cause the following additional operations: Display at least one avatar on the display; Receiving at least one of a remote animation command and a remote avatar parameter; and Passively causing the at least one displayed avatar to be animatedly presented based on at least one of the remote animation command and the remote avatar parameter. The system of claim 15, wherein the instructions, when executed by one or more processors, cause the following additional operations: Receive remote avatar voice signals; and Convert the remote avatar voice signal into avatar voice. The system according to any one of claims 15 to 19, wherein the user input is one or more points on the touch screen mapped to one or more points of the avatar presented by the passive animation Touch events at touch locations. The system according to claim 16, wherein the transformation includes at least one of transposition and time extension. A device to be used by a first device, which includes: Parts used to select the avatar; A component for receiving at least one image of the user; A component for passively causing the avatar to be animated at least partially based on the at least one image so as to generate a passively animated avatar for display on a remote device, wherein the passively animated avatar simulates the body of the user Movement of parts; A component used to detect user input using a user input device; Means for determining the interactive animation of the avatar presented by the passive animation based at least in part on the user input; For using the interactive animation to modify the avatar presented by the passive animation so that the avatar presented by the interactive animation is generated so as to be displayed on the first device The components displayed as the avatar of the feedback; and A component for transmitting a signal to the remote device, the signal being configured to cause the avatar presented in the interactive animation to be displayed on the remote device; The user input device includes a touch screen, and the user input includes a touch event on the touch screen. The device according to claim 22, which further includes: A component for capturing the user's voice and converting the user's voice into a corresponding user's voice signal; Means for transforming the user's voice signal into an avatar voice signal; and A component used to transmit the avatar voice signal to the remote device. The device according to claim 22, which further includes: Means for capturing the at least one image of the user; A component for detecting a face in the at least one image; Components used to extract features from the face; Means for converting the features into avatar parameters; and A component for generating the passively animated avatar based at least in part on the avatar parameter. The device according to claim 22, which further includes: A component for displaying at least one avatar on the display; A component for receiving at least one of a remote animation command and a remote avatar parameter; and A component for passively causing the at least one avatar to be animatedly presented based on at least one of the remote animation command and the remote avatar parameter. The device according to claim 22, which further includes: Components for receiving voice signals of the remote avatar; and A component used to convert the remote avatar voice signal into an avatar voice. The device according to any one of claims 22 to 26, wherein the user input is one or more points on the touch screen mapped to one or more points of the avatar presented by the passive animation Touch events at touch locations. The device according to claim 23, wherein the transformation includes at least one of transposition and time extension.

Images