TWM652806U - Interactive virtual portrait system - Google Patents

Abstract

The new interactive virtual portrait system of the present invention includes a facial recognition module, an artificial intelligence facial modeling module, a deep fake module and an animation module. The facial recognition module is installed in a photographing device, and the facial recognition module obtains and processes one or more clear frontal face photos through the photographing device. The artificial intelligence facial modeling module is installed on at least one server. The artificial intelligence facial modeling module is designed to form a training facial model based on the frontal facial photos. The deep fake module is installed on the server and connected to the artificial intelligence facial modeling module. The deep fake module synthesizes the facial features of a virtual character based on the training facial model. The animation module is installed on the server and connected to the deep fake module. The animation module is used to generate specified actions and expressions of the virtual character. Interactive avatar systems can produce a realistic and more attractive virtual character that can be used in a wide range of applications from customer service to interactive education and entertainment.

Description

Interactive virtual portrait system

The invention relates to the field of interactive digital media, and is particularly related to systems for creating and interacting with virtual characters. More specifically, the new model involves using artificial intelligence (AI) and deep learning techniques to generate, animate and interact with realistic avatar characters.

The development of virtual characters has always been an important focus in the field of digital media. Traditional approaches rely primarily on manual animation and scripting, which limits the dynamics and interactivity of these characters. With the advent of AI and deep learning, significant progress has been made in creating more realistic and responsive virtual characters. However, many challenges remain. Existing systems often struggle with creating realistic facial expressions and lip synchronization, especially in real-time interaction settings. The use of pre-recorded animations and responses limits the scope of interaction and makes virtual characters less adaptable to variable and improvised user input. Furthermore, integrating speech recognition and natural language processing to achieve seamless interaction between users and virtual characters has been a complex task, which often results in poor user experience. Deepfake technology has been used in various fields to generate realistic images and videos, but its application in real-time interactive systems is still limited. Therefore, there is a need to design an improved interactive virtual portrait system to address these challenges.

This new interactive virtual portrait system combines advanced AI facial modeling, real-time facial synthesis deepfakes technology, sophisticated lip synchronization algorithms, and an intuitive interface for user interaction. This will result in a more realistic, responsive and engaging virtual character that can be used in a wide range of applications from customer service to interactive education and entertainment. The new interactive virtual portrait system of the present invention includes a facial recognition module, an artificial intelligence facial modeling module, a deep fake module and an animation module. The facial recognition module is installed in a photographing device, and the facial recognition module obtains and processes one or more clear frontal face photos through the photographing device. The artificial intelligence facial modeling module is installed on at least one server. The artificial intelligence facial modeling module is designed to form a training facial model based on the frontal facial photos. The deep fake module is installed on the server and connected to the artificial intelligence facial modeling module. The deep fake module synthesizes the facial features of a virtual character based on the training facial model. The animation module is installed on the server and connected to the deep fake module. The animation module is used to generate specified actions and expressions of the virtual character. Wherein, the photographing device is communicatively connected to the server. This new model presents an advanced interactive avatar system that marks a significant advancement in the art of creating, animating, and interacting with virtual characters. This system leverages the latest developments in artificial intelligence (AI) and deep learning technology and is designed to overcome the limitations of current virtual character technology and provide new levels of augmented reality, responsiveness and interactivity. This system uses AI algorithms to process high-quality face photos to create accurate and detailed facial models. This technology can handle a variety of facial expressions and features, providing a basis for realistic facial synthesis of virtual characters. By integrating deep fake technology, this new model has achieved a major breakthrough, allowing the facial expressions of virtual characters to be synthesized in real time. The application of deepfake not only enhances the realism of facial movements and expressions, but also captures subtle details that reflect human interaction. A key aspect of this system is its AI-driven lip-sync module. The mod ensures that the avatar's lip movements are perfectly synchronized with spoken words, whether they are pre-recorded or generated on the fly. This synchronization plays a vital role in enhancing the believability and naturalness of a character's voice. The system is also equipped with a dynamic animation module that enables virtual characters to create specific movements and expressions. This functionality enables characters to perform a wide range of movements and gestures, making interactions more engaging and realistic. Complementing these capabilities is a flexible user interface that handles various forms of input, including text, voice commands, and touch commands. This flexibility enables the system to adapt to a wide range of interaction scenarios and meet diverse user needs and situations. An innovative aspect of this system is its ability to interact and generate responses on the fly. By integrating speech-to-text (STT) technology, semantic analysis, and keyword matching, the system can instantly process and understand user input. This advanced processing enables virtual characters to respond appropriately by outputting interactive speech via text-to-speech (TTS) or triggering relevant pre-recorded videos from an extensive database. The design of this system is multifunctional in nature, making it suitable for numerous applications such as customer service, virtual assistants, educational tools, entertainment, and interactive marketing. Its ability to deliver authentic, reactive interactions makes it an invaluable tool in any field that benefits from enhanced user engagement. Advantages of the new model include enhanced realism, thanks to the application of advanced AI and deepfake technology, and increased interactivity, thanks to the system's ability to understand and respond instantly to a variety of user inputs. In addition, the system's broad application scope and user-friendly interface expand its potential markets and use cases, allowing it to reach a broad audience. In summary, our interactive avatar system represents a transformative step in the field of virtual characters, providing an unprecedented combination of realism, responsiveness and versatility. It not only solves the current limitations of avatar technology, but also opens up new avenues for user participation and interaction, setting new standards for digital interactive experiences.

Please refer to FIGS. 1 and 2 . FIG. 1 shows a schematic diagram of the interactive virtual portrait system 10 of this embodiment, and FIG. 2 shows a three-dimensional schematic diagram of the communication connection between the shooting device 13 and the server 12 . The interactive virtual portrait system 10 of this embodiment includes a facial recognition module 131, an artificial intelligence facial modeling module 121, a deep fake module 122, an animation module 123, and an AI-based lip synchronization Module 124, a user interface 132, a video recording module 125, a database 126, a speech-to-text (STT) module 127, a semantic analysis module 128 and a text-to-speech (TTS) module 129. The facial recognition module 131 is installed in a camera device 13. The camera device 13 is, for example, a smartphone or a tablet computer. In this embodiment, a smartphone is used as an example (see Figure 2). Among them, the facial recognition module 131 obtains and processes one or more clear frontal face photos 131P through the shooting device 13 . In addition, the artificial intelligence facial modeling module 121 is installed on a server 12, and the server 12 is generally in the shape of a rectangular body. It is worth noting that the photographing device 13 is connected to the server 12 through communication. Therefore, the artificial intelligence facial modeling module 121 can be connected to the facial recognition module 131 through the server 12 and the shooting device 13 . Because of this, the artificial intelligence facial modeling module 121 can form a training facial model 121M based on the frontal facial photo 131P. In addition, the deep fake module 122 is installed on the server 12 and connected to the artificial intelligence face modeling module 121. The deep fake module 122 synthesizes the face of a virtual character based on the training face model 121M. Features 122F. In addition, the animation module 123 is also installed on the server 12 and connected to the deep fake module 122. The animation module 123 is used to generate designated actions and expressions 123E of the virtual character. Please refer to Figure 1. The AI-based lip synchronization module 124 is also installed on the server 12 and connected to the animation module 123. The AI-based lip synchronization module 124 is used to combine the lip movements of the virtual character with Spoken discourse synchronization. In addition, the user interface 132 is provided on the camera device 13. The user interface 132 allows the user of the camera device 13 to input a specific content 132C. The specific content 132C is from the following groups: text, voice commands and touch commands. The virtual character speaks or acts. The video recording module 125 is also provided on the server 12, and the video recording module 125 is connected to the AI-based lip synchronization module 124. In this embodiment, the video recording module 125 is used to capture and record the lip movements and spoken words of the virtual character to form a recorded video 125V. In addition, the database 126 is also installed on the server 12 and connected to the video recording module 125. The database 126 is mainly used to store the recorded video 125V and a keyword 126K corresponding to the recorded video 125V. The speech-to-text (STT) module 127 is also installed on the server 12 . The speech-to-text (STT) module 127 also obtains the specific content 132C through the server 12 and the camera 13 . When the specific content 132C is the voice command, the speech-to-text (STT) module 127 is used to convert the voice command into a voice text 127W. Please refer to Figure 1. The semantic analysis module 128 is connected to the database 126 and the speech-to-text (STT) module 127. The semantic analysis module 128 is used to process the converted speech text 127W to understand the specific content input by the user. 132C. Moreover, in this embodiment, the semantic analysis module 128 can also match the specific content 132C input by the user with the keywords 126K stored in the database 126, and trigger a corresponding response 128R. The response 128R is usually the recorded video 125V stored in the database 126 . In addition, the text-to-speech (TTS) module 129 is also connected to the database 126 and the semantic analysis module 128. The text-to-speech (TTS) module 129 is used to convert the assigned keywords 126K into an interactive voice 129V. . In this embodiment, the response 128R triggered by the semantic analysis module 128 is changed to the interactive voice 129V. In this embodiment, when the semantic analysis module 128 triggers the response 128R, the triggered response 128R will be sent to the shooting device 13 by the server 12. In addition to displaying the virtual character, the user interface 132 of the shooting device 13 will also Instant rendering response 128R. In this way, the interactive virtual character system 10 of this embodiment will generate a more realistic, responsive and attractive virtual character. The following paragraphs will describe the technical features of the interactive virtual portrait system 10 in more detail: The interactive virtual portrait system 10 of this embodiment is a comprehensive system that integrates various artificial intelligence (AI) technologies to create realistic and responsive virtual images. Role. The way this system works is to obtain and process clear frontal facial photos 131P, which are used as basic data for creating the appearance of the virtual character. The quality and clarity of these images are extremely important because they directly affect the accuracy of the facial features of the virtual character. Once clear frontal facial photos 131P are obtained, the system will use AI technology to train a facial model 121M based on these images. This process may involve deep learning technology to understand and replicate the facial features and expressions of individuals in photos. This system will then use the trained face model 121M to synthesize the facial features 122F of the virtual character, and utilize advanced algorithms and deep fake technology to create a dynamic and convincing replica of the human face. This system also has the ability to generate specific animations or actions for virtual characters. This is achieved by combining predefined animation templates and AI-generated actions consistent with the trained face model 121M. In addition, this system also integrates AI lip synchronization technology to align the virtual character's lip movements with the spoken words, thereby enhancing the realism of the virtual character's interaction. The user can interact with the system by inputting specific content 132C for the virtual character to speak or perform. The system is designed to accept various forms of specific content 132C such as text, voice commands, and touch commands. This system can record video of a virtual character speaking or performing actions, and can store it in a database for future use or streaming. The system's user interface 132 presents the virtual character's responses in real time, creating an interactive and engaging experience. Overall, the Interactive Avatar System 10 represents a sophisticated integration of AI technology for creating realistic and responsive virtual characters. The operation of this system involves a series of interconnected steps and modules, each contributing to the creation of a realistic virtual character capable of instant interaction with the user. The facial recognition module 131 plays a key role in the initial stage of operation of the interactive virtual portrait system 10 . The facial recognition module 131 is configured to obtain one or more clear frontal facial photos 131P. These photos serve as base data for creating the appearance of the virtual character. The facial recognition module 131 processes these images to prepare for subsequent steps of the system operation. The quality and clarity of the photos obtained are crucial to the operation of the system. High-resolution, clear images with distinct facial features are preferred as they provide a detailed and accurate representation of the human face. The facial recognition module 131 processes these images, focusing on the person's facial features such as eyes, nose, mouth and other distinguishing features. The mod may use various image processing techniques to improve the quality of the image and isolate facial features from the rest of the image. The processed images were then used as the basis for creating facial features of the virtual character. The accuracy and realism of the virtual character's face is directly affected by the quality and clarity of the original photo. For example, clear and high-quality images can more accurately replicate people's facial features, resulting in virtual characters that closely resemble people. On the other hand, images with lower quality or poor definition may result in less accurate representation of human faces, which may affect the realism of virtual characters. Therefore, the facial recognition module 131 plays a fundamental role in the process of obtaining and processing the clear frontal facial photo 131P. The facial recognition module 131 paves the way for subsequent steps in the operation of this system, affecting the accuracy and realism of the virtual character's appearance. After the facial recognition module 131 acquires and processes the facial photos, the interactive virtual portrait system 10 uses the artificial intelligence facial modeling module 121 . The artificial intelligence face modeling module 121 is designed to generate a training face model 121M based on the provided photos. The artificial intelligence facial modeling module 121 uses deep learning technology to understand and replicate the facial features and expressions of individuals in photos. The AI Face Modeling Module 121 uses these deep learning techniques to train a model that accurately represents an individual's face. The training process involves feeding processed photos into the system, which then analyzes the images and learns the patterns and characteristics of individual facial features. The system uses this learned information to create detailed models of individual faces, capturing the nuances of their facial features and expressions. The functions of the artificial intelligence facial modeling module 121 are not limited to static facial features. It also extends to dynamic aspects such as facial expressions. By analyzing a series of photos showing different facial expressions of individuals, the module can learn and replicate these expressions in virtual characters. This ability enhances the realism of virtual characters, allowing them to display a range of emotions and reactions similar to those of an individual. Therefore, the artificial intelligence facial modeling module 121 plays a key role in the operation of the interactive virtual portrait system 10 . By utilizing deep learning technology to generate a trained face model 121M from a clear frontal face photo 131P, the module contributes to the creation of a virtual character that not only resembles an individual, but also behaves like an individual. This sense of realism and responsiveness is a defining characteristic of the interactive avatar system 10 . After the artificial intelligence facial modeling module 121 completes facial model training, the interactive virtual portrait system 10 uses the deep fake module 122 . The deep fake module 122 is responsible for synthesizing the facial features 122F of the virtual character based on the training facial model 121M. Deepfake Mod 122 utilizes a specific type of AI technology called generative adversarial networks (GANs) to generate convincing facial movements and expressions. Generative adversarial networks (GANs) are a type of AI algorithm used for unsupervised machine learning, implemented by two neural networks competing against each other in a zero-sum game framework. This technique was proposed by Ian Goodfellow and his colleagues in 2014. GANs are used to generate synthetic data that is nearly indistinguishable from real data. In the context of deepfakes module 122, GANs are used to create realistic and dynamic replicas of individual faces based on trained face models 121M. The operation of GANs involves two neural networks: a generative network and a discriminative network. The generative network creates synthetic data, i.e. facial features and expressions, from the trained face model 121M. The discriminative network, on the other hand, evaluates the authenticity of the generated data compared to the original photo. The two networks continuously learn from each other, with the generative network striving to produce more realistic data, and the discriminative network striving to better distinguish between synthetic and real data. Through this iterative process, the deepfake module 122 can generate facial features and expressions that are very similar to the individuals in the photos. Deepfake module 122 can create a range of facial movements and expressions, from subtle facial muscle changes to more obvious expressions such as smiles or frowns. This capability enables virtual characters to exhibit dynamics and realism that enhance interactive experiences. In addition, the deepfake module 122 can real-time adjust the facial features 122F of the virtual character in response to user input or changes in the virtual environment. This dynamic reactivity contributes to the interactivity of this system, allowing virtual characters to interact with users in a more realistic and believable way. Therefore, the deep fake module 122 plays a key role in the operation of the interactive virtual portrait system 10 . By utilizing GANs to synthesize the facial features 122F of the virtual character based on the trained face model 121M, the module contributes to the creation of a virtual character that not only looks like an individual, but also moves and expresses like an individual. After the deep fake module 122 synthesizes the facial features 122F of the virtual character, the animation module 123 of the interactive virtual portrait system 10 is responsible for generating specified actions and expressions for the virtual character. The animation module 123 operates by using a set of predefined animation templates and AI-generated movements consistent with the trained facial model 121M. Predefined animation templates serve as the basis for the avatar's movements and expressions. These templates may include a variety of common actions and expressions such as walking, running, waving, smiling, frowning, etc. These templates provide a starting point for animation modules 123, providing a set of standard actions and expressions that can be applied to virtual characters. However, animation module 123 does not rely solely on these templates. It also uses AI-generated movements to create more specific and detailed movements and expressions for virtual characters. These AI-generated actions are also based on the trained facial model 121M and are designed to be consistent with the individual’s facial features and expressions. These AI-generated actions are created using machine learning techniques similar to those used in the AI Face Modeling Module 121. The animation module 123 inputs the trained facial model into the machine learning system, and then generates actions and expressions consistent with the individual's facial features and expressions. These movements and expressions are then applied to the virtual character, enhancing its realism and responsiveness. The function of the animation module 123 is not limited to generating static movements and expressions. It also includes the ability to adjust these movements and expressions on the fly in response to user input or changes in the virtual environment. This dynamic reactivity contributes to the interactivity of this system, allowing virtual characters to interact with users in a more realistic and believable way. Therefore, the animation module 123 plays a key role in the operation of the interactive virtual portrait system 10 . By utilizing predefined animation templates and AI-generated movements to generate specified movements and expressions 123E of a virtual character, the animation module 123 creates a virtual character that not only looks and moves like an individual, but also behaves consistent with the individual's movements and expressions. The character contributed. After the animation module 123 generates specified actions and expressions for the virtual character, the AI-based lip synchronization module 124 of the interactive virtual portrait system 10 is responsible for synchronizing the lip movements of the virtual character with spoken words. The AI-based lip sync module 124 operates by analyzing audio input and adjusting the avatar's mouth movements accordingly. The AI-based lip sync module 124 utilizes advanced AI technology to analyze audio input, which may be pre-recorded speech or real-time speech input from the user. This module processes audio input, breaking it down into phonemes, which are the smallest units of sound that distinguish one word from another in a particular language. The module then maps these phonemes to corresponding embouchures, which are visual representations of the phonemes. The AI-based lip sync module 124 then adjusts the avatar's lip movements to match these mouth shapes, thereby creating the illusion that the avatar is speaking. This process involves a complex interaction of various facial muscles, as the shape and position of the lips, chin, and tongue all contribute to the different mouth shapes. The AI-based lip sync module 124 uses a trained facial model to accurately replicate the movements of these muscles, thereby producing realistic lip sync and enhancing the credibility of the virtual character's voice. The function of the AI-based lip sync module 124 is not limited to static lip sync. It also includes the ability to adjust lip movements on the fly in response to changes in audio input. This dynamic reactivity contributes to the interactivity of virtual human systems, allowing virtual characters to interact with users in a more realistic and believable way. For example, if a user enters a new piece of speech, the module can analyze the audio input and adjust the avatar's lip movements accordingly, creating the illusion that the avatar is saying something new. Therefore, the AI-based lip sync module 124 plays a key role in the operation of the interactive virtual portrait system 10 . By leveraging advanced AI technology to synchronize the avatar's lip movements with spoken utterances, the mod contributes to the creation of an avatar that not only looks, moves and behaves like an individual, but also speaks in a manner consistent with the individual's voice . After the AI-based lip synchronization module 124 synchronizes the lip movements of the virtual character with the spoken words, the user interface 132 of the interactive virtual portrait system 10 is used to allow the user of the shooting device 13 to input specific content 132C for the virtual character. speak or act. The user interface 132 serves as a channel for user interaction, enabling the user to communicate with the virtual character and guide its movements and voices. User interface 132 is designed to accept various forms of input, providing flexibility and convenience to the user. These input forms may include text, voice commands, and touch commands. Each input form provides different interaction modes to meet different user preferences and usage scenarios. Input of text for virtual characters to speak or perform. This form of input is particularly useful for scripted interactions, where the content of speech or action is predetermined. The system processes the text input and then generates corresponding speech or actions for the virtual character. Voice commands, on the other hand, offer a more dynamic and interactive input mode. Users can speak directly to the system, and their voice commands are converted into spoken text 127 by the system's speech-to-text (STT) module 127. Then, the system processes the converted voice text 127W to generate corresponding voices or actions for the virtual character. This form of input allows for instant interaction, allowing users to interact with virtual characters in a conversational manner. This touch command provides a tactile input mode that allows users to interact with the system through touch gestures. These gestures can be used to guide the actions of a virtual character, such as pointing at an object or moving in a specific direction. The system processes touch commands and then generates corresponding actions for the virtual character. Therefore, the user interface 132 plays a key role in the operation of the interactive avatar system 10 . The interface facilitates user interaction and enhances the interactivity of the system by allowing the user to input specific content 132C for the virtual character to speak or perform. The various input forms accepted by this interface satisfy different user preferences and usage scenarios, providing users with flexibility and convenience. After the virtual character completes its actions and voice, the video recording module 125 of the interactive virtual portrait system 10 is responsible for capturing and recording the actions and voice of the virtual character. The video recording module 125 operates by encoding the movements and voices of the virtual character into a digital video format that can be stored and retrieved for future use. The video recording module 125 captures the movements and voices of virtual characters in real time, ensuring that every movement and word is accurately recorded to form a recorded video 125V. The video recording module 125 may use various video encoding technologies to compress recorded data and reduce storage space requirements while maintaining video quality. The video recording module 125 also adds a timestamp to each recorded video 125V so that the movements and voices of the virtual characters can be accurately tracked. Then, the recorded video 125V is stored in a database 126, which serves as a storage database for the virtual character's movements and voices. Database 126 is used to store and retrieve these recorded videos 125V, providing a comprehensive record of virtual character interactions. The database 126 may be organized in various ways, such as by date, by user, or by interaction type, in order to efficiently retrieve recorded video 125V. Database 126 is not a passive storage system. It is designed to be triggered by specific content 132C received by user interface 132. The specific content 132C may include a request to retrieve a specific recorded video 125V, delete the recorded video 125V, or play the recorded video 125V. After receiving the specific content 132C, the database 126 retrieves the corresponding recorded video 125V and transmits it to the user interface 132 for processing. For example, if a user requests to view a particular recorded video 125V, the database 126 retrieves the video and passes it to the user interface 132 for playback. If a user requests deletion of recorded video 125V, database 126 will remove the video from its storage. If a user requests to play a recorded video 125V, the database 126 retrieves the video and sends it to the playback module for processing. Therefore, the video recording module 125 and the database 126 play a key role in the operation of the interactive virtual portrait system 10 . By capturing and recording the movements and speech of virtual characters, and by storing and retrieving these recorded videos 125V, these components contribute to the system providing a comprehensive and interactive virtual human experience. After receiving user input through the user interface 132, the speech-to-text (STT) module 127 is responsible for interpreting the speech input and converting the spoken language into text that can be processed by the system. The speech-to-text (STT) module 127 serves as a bridge between the user and the system, enabling the system to understand and respond to voice commands. The speech-to-text (STT) module 127 operates by analyzing audio input, which may be pre-recorded speech or real-time speech input from the user. The module processes audio input, breaking it down into phonemes, which are the smallest units of sound that distinguish one word from another in a particular language. The module then converts these phonemes into their corresponding text, creating a textual representation of the spoken language. The speech-to-text (STT) module 127 utilizes advanced AI technology to perform this conversion process. The mod uses machine learning technology to learn the patterns and characteristics of spoken language, allowing it to accurately convert spoken language into text. The module may also combine natural language processing technology to understand the context and semantics of spoken language to improve the accuracy of the conversion process. The functionality of the speech-to-text (STT) module 127 is not limited to static speech-to-text conversion. It also includes the ability to adjust the conversion process on the fly in response to changes in audio input. This dynamic reactivity contributes to the interactivity of the virtual human system, allowing the system to understand and respond to user input in a timely manner. For example, if a user enters a new voice command, the module can analyze the audio input and convert spoken words into text, allowing the system to process the new command. Therefore, the speech-to-text (STT) module 127 plays a key role in the operation of the interactive avatar system 10 . By utilizing advanced AI technology to convert spoken words into text, the module facilitates user interaction and enhances the interactivity of the system. After the speech-to-text (STT) module 127 converts the spoken language into text, the interactive avatar system 10 uses a semantic analysis module 128 to process the converted text to understand the user's input. The semantic analysis module 128 serves as a bridge between user input and system response, enabling the system to understand and respond to user commands. The semantic analysis module 128 operates by analyzing the converted text, which represents the user's spoken language. This mod processes text, breaks it down into words and phrases, and then analyzes the semantics of those words and phrases. Semantics refers to the meaning of words and phrases in a specific context. By analyzing the semantics of the converted text, the module can understand the user's intent and the context of their commands. The semantic analysis module 128 uses advanced AI technology to perform this semantic analysis. The semantic analysis module 128 uses natural language processing technology to understand the context and semantics of the converted text to improve the accuracy of the analysis process. The module may also incorporate machine learning technology to learn patterns and characteristics of language, allowing it to accurately understand the user's intent. The function of the semantic analysis module 128 is not limited to static semantic analysis. It also includes the ability to match user input to keywords 126K stored in the database 126 . Keyword 126K represents a specific action or response that this system can trigger. By matching the user's input to these keywords 126K, the semantic analysis module 128 can determine the appropriate response to the user's command. For example, if a user enters a command to make the avatar smile, the module can match the word "smile" with keywords 126K in the database 126 that trigger the avatar's smiling action. Likewise, if a user enters a command to have the avatar speak a specific word, the module can match the word with the keyword 126K in the database 126 that triggers the avatar's voice action. Therefore, the semantic analysis module 128 plays a key role in the operation of the interactive virtual portrait system 10 . This module facilitates user interaction and enhances the interactivity of the system by processing the converted text using advanced AI technology to understand user input, and by matching user input with keywords 126K stored in the database 126 . After the semantic analysis module 128 performs semantic analysis on the user input, the interactive virtual portrait system 10 uses a response mechanism 128R. The mechanism is to provide specific content 132C for user input in the form of text-to-speech (TTS) output and video content streamed from the database 126 . The mechanism of response 128R is to serve as a bridge between the system's understanding of user input and rendering appropriate responses, allowing the system to interact with users in a dynamic and engaging way. The response 128R mechanism operates by analyzing the results of the semantic analysis module 128, which represent the user's intent and the context of their command. The mechanism processes these results to determine an appropriate response based on the user's input. The response 128R may be a specific action or voice of the virtual character, which is generated by the system and presented to the user. The mechanism for responding to 128R utilizes advanced AI technology to generate these responses. The mechanism uses text-to-speech (TTS) technology to convert text input into spoken language, creating the illusion that the virtual character is speaking the user's input. Text-to-speech (TTS) technology works by analyzing text input, breaking it down into phonemes, and then generating corresponding speech sounds to create a spoken representation of the text. The capabilities of the mechanisms responding to 128R are not limited to static text-to-speech (TTS) output and video streaming. It also includes the ability to instantaneously render the avatar's responses on the user interface 132 . This dynamic presentation contributes to the interactivity of the virtual human system, allowing virtual characters to interact with users in a more realistic and believable way. For example, if a user enters a command to have the virtual character speak a specific word, the mechanism can generate corresponding text-to-speech (TTS) output and present it in real time on the user interface 132, creating the illusion that the virtual character is speaking the word. Therefore, the mechanism responding to 128R plays a key role in the operation of the interactive avatar system 10 . This mechanism facilitates user interaction and enhances the system by leveraging advanced AI technology to provide text-to-speech (TTS) output and video content streamed from the database 126 and by instantly rendering the virtual character's responses on the user interface 132 Interactivity. The interactive avatar system 10 can be used in a variety of applications. One such application might be in the field of virtual customer service. In this case, this system can be used to create virtual customer service representatives who can interact with customers instantly. These virtual representatives can answer customer inquiries, provide information about products or services, and guide customers through various processes. The system's ability to instantly understand and respond to user input, combined with the realism and dynamics of virtual characters, can enhance the customer service experience, making it more engaging and efficient. This system can also be used in the educational field. It can be used to create virtual tutors or teachers who can interact with students instantly. These virtual tutors can provide explanations, answer questions, and guide students through study material. This system can instantly understand and respond to the information input by the user, and instantly present the virtual character's response 128R on the user interface 132, which can make the learning experience more interactive and attractive. To sum up, this new type of interactive virtual portrait system 10 can produce a more realistic, responsive and attractive image through advanced AI facial modeling, real-time facial synthesis deep fake technology, and sophisticated lip synchronization algorithm. A powerful virtual character. Although the present invention has been disclosed above in terms of preferred embodiments, they are not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make slight changes and modifications without departing from the spirit and scope of the present invention. , therefore, the scope of protection of the present invention shall be subject to the scope of the patent application attached.

10:Interactive virtual portrait system 12:Server 121:Artificial intelligence facial modeling module 121M: Training face model 122:Deep fake module 122F: Facial features of virtual characters 123:Animation module 123E: Specified actions and expressions of virtual characters 124: AI-based lip sync module 125:Video recording module 125V: record video 126:Database 126K:Keywords 127: Speech-to-text (STT) module 127W: Voice text 128:Semantic analysis module 128R:Response 129: Text-to-speech (TTS) module 129V: Interactive voice 13: Shooting device 131: Facial recognition module 131P: Frontal face photo 132: User interface 132C:Specific content

FIG. 1 shows a schematic diagram of the interactive virtual portrait system 10 of this embodiment. FIG. 2 shows a three-dimensional schematic diagram of the communication connection between the photographing device 13 and the server 12 .

10:Interactive virtual portrait system

12:Server

121:Artificial intelligence facial modeling module

121M: Training face model

122:Deep fake module

122F: Facial features of virtual characters

123:Animation module

123E: Specified actions and expressions of virtual characters

124: AI-based lip sync module

125:Video recording module

125V: record video

126:Database

126K:Keywords

127: Speech to text (STT) module

127W: Voice text

128:Semantic analysis module

128R:Response

129: Text-to-speech (TTS) module

129V: Interactive voice

13: Shooting device

131: Facial recognition module

131P: Frontal face photo

132: User interface

132C:Specific content

Claims (9)

An interactive virtual portrait system, including: A facial recognition module is provided in a photographing device, and the facial recognition module obtains and processes one or more clear frontal facial photos through the photographing device; An artificial intelligence facial modeling module, installed on at least one server, the artificial intelligence facial modeling module is designed to form a training facial model based on the frontal facial photos; A deep fake module, installed on the server and connected to the artificial intelligence facial modeling module, the deep fake module synthesizes the facial features of a virtual character based on the training facial model; and An animation module, installed on the server and connected to the deep pseudo module, the animation module is used to generate specified actions and expressions of the virtual character; Wherein, the photographing device is communicatively connected to the server. The interactive virtual portrait system as described in request item 1 also includes: An AI-based lip synchronization module, installed on the server and connected to the animation module, the AI-based lip synchronization module is used to synchronize the lip movements of the virtual character with spoken words; A user interface is provided on the shooting device. The user interface allows the user to input a specific content for the virtual character to speak or perform. The interactive virtual portrait system as described in claim 2, wherein the specific content is from the following groups: text, voice commands and touch commands. The interactive virtual portrait system as described in request item 3 also includes: A video recording module, installed on the server, the video recording module is used to capture and record the lip movements and spoken words of the virtual character to form a recorded video; and A database is provided on the server and connected to the video recording module. The database is used to store the recorded video and a keyword corresponding to the recorded video. The interactive virtual portrait system as described in request item 4 also includes: A speech-to-text (STT) module, installed on the server, the speech-to-text (STT) module is used to convert the voice command into a voice text; and A semantic analysis module is connected to the speech-to-text (STT) module. The semantic analysis module processes the converted speech text to understand the specific content input by the user. The interactive virtual portrait system of claim 5, wherein the semantic analysis module is configured to match the specific content input by the user with the keyword stored in the database, and trigger a corresponding response. The interactive virtual portrait system as described in claim 6, wherein the response is the recorded video of the database. The interactive virtual portrait system as described in claim 6 further includes a text-to-speech (TTS) module connected to the database and the semantic analysis module, and the text-to-speech (TTS) module is connected to the database and the semantic analysis module. The module is used to convert the keyword into an interactive voice, wherein the response is the interactive voice. The interactive virtual portrait system as claimed in claim 7 or claim 8, wherein the user interface is also used to display the virtual character and present the response in real time.

Images