TWM594767U - Virtual character live streaming system - Google Patents

Abstract

A virtual character live broadcast system includes a live broadcast management platform and a first terminal device. The live broadcast management platform is used to transmit an interactive signal and a live broadcast animation, wherein the live broadcast animation is generated based on a control signal and a virtual character. The first terminal device is used to detect a motion of a user to generate a detection signal corresponding to the motion, and generate interactive information according to the interactive signal, and display a setting interface and a live interface, wherein the setting interface includes a virtual character setting Area, and the live interface includes virtual characters and interactive information, and the virtual characters are set through the virtual character setting area.

Description

Virtual character live broadcast system

This creation is about a virtual character live broadcast system, especially a virtual character live broadcast system based on real images.

Virtual reality (Virtual Reality, VR) is to put real-world users in a virtual space and shape a three-dimensional virtual environment close to the real through the computer, for example: through visual aids to hearing, simulate real dynamic binocular parallax, let users experience the experience Its situation. Augmented Reality (Augmented Reality, AR) is a technology that uses the actuarial calculation of the position and angle of the camera and the image analysis technology to allow the virtual characters on the screen to interact with the real world. In other words, the user can be in the real world. I saw an imaginary object appear in the reality screen.

The above virtual reality and augmented reality image service providers/game manufacturers usually provide a virtual image based on the existing database for users to watch, for example, display an expansion through an augmented reality system in the medical field Augmented reality images, such as three-dimensional virtual organ images generated based on ultrasound images, allow medical personnel to operate and display the augmented reality images for patients to view and explain. However, the above-mentioned image technology is only a virtual object image stored through a database, and there is no function for users to modify or customize virtual images, and it is often limited to specific fields and devices to display, and is still limited in application. .

In view of this, some embodiments of the present creation provide a virtual character live broadcast system.

The virtual character live broadcast system according to an embodiment of the present invention includes a live broadcast management platform and a first terminal device. The live broadcast management platform includes a management communication module and a management computing unit. The management communication module is used to transmit an interactive signal and a live animation corresponding to a control signal, wherein the live animation is generated based on the control signal and a virtual character. The management computing unit is electrically connected to the management communication module. The management computing unit is used to identify a detection signal to generate a control signal corresponding to the detection signal. The first terminal device includes a terminal communication module, a terminal display unit, a motion detection unit, and a terminal arithmetic unit. The terminal computing unit is electrically connected to the terminal communication module, the terminal display unit and the motion detection unit. The terminal communication module is communicatively connected to the management communication module. The terminal communication module is used to receive interactive signals and send detection signals. The terminal display unit is used to display a setting interface and a live interface. The setting interface includes a virtual character setting area, and the live interface includes virtual characters and interactive information, and the virtual character is set through the virtual character setting area. The motion detection unit is used to detect a motion of a user. The terminal computing unit is used to generate a detection signal corresponding to the motion and generate interactive information according to the interactive signal.

In this way, the user can connect to the live broadcast management platform through the first terminal device, first customize the virtual character on the setting interface through customization, and then the user can control the virtual character's expressions and actions in real time on the live broadcast platform as Your own stand-in video, according to which you can share the live broadcast and real-time interaction with the audience through the live broadcast platform, can break through the traditional limitations of pre-recorded virtual images and viewing locations, thereby increasing the fun of live broadcasting and maintaining user privacy in specific applications .

The following is a detailed description with specific examples and accompanying drawings, so that it is easier to understand the purpose, technical content, characteristics and effects of the creation.

In the following, each embodiment of the creation will be described in detail, and the drawings will be used as examples. In the description of the specification, in order to make the reader have a more complete understanding of this creation, many specific details are provided; however, this creation may still be implemented on the premise that some or all of the specific details are omitted. The same or similar elements in the drawings will be represented by the same or similar symbols. It is important to note that the drawings are for illustrative purposes only, and do not represent the actual size or number of components. Some details may not be fully drawn for simplicity.

FIG. 1 is a schematic flowchart of a method for creating a virtual character live broadcast according to an embodiment. FIG. 2 is a schematic structural diagram of creating a virtual character live broadcast system according to an embodiment. FIG. 3 is a block diagram of a first terminal device according to an embodiment of the invention. FIG. 4 is a block diagram of a live broadcast management platform according to an embodiment. Please refer to FIG. 1 to FIG. 4 together. The virtual character live streaming method according to an embodiment of the present invention can be implemented by a live broadcast management platform 2 communicating with the first terminal device 1 to provide one or more second The terminal devices 3, 3'receive live video or interactive tasks from users of the first terminal device 1 in real time. For example, users can be live hosts of online media and live broadcast platforms, also known as VTuber. Their live web content includes but is not limited to chat, singing, dance, food, e-sports, anime, beauty, entertainment For other types of live broadcasters, these live broadcasters can share live broadcasts and real-time interactions with viewers through the live broadcast platform, which can be rewarded by the digital transfer and the live broadcast platform sharing.

In some embodiments, the virtual character live broadcast method can be implemented by a computer program, so that it becomes a computer (ie, any electronic device with a terminal communication module 10, a terminal display unit 12, a motion detection unit 14, and a terminal arithmetic unit 16, For example, the first terminal device 1) is connected to a server via a network (that is, any electronic device with a management communication module 20 and a management computing unit 22, such as a live broadcast management platform 2). After the program is loaded and executed, it can be Complete the live streaming method for virtual characters of any embodiment. In this embodiment, a user such as a live broadcaster can use the first terminal device 1 such as, but not limited to, a mobile phone, a tablet computer, or a notebook computer to establish a communication connection with the live broadcast management platform 2 through the Internet to perform virtual Character live broadcast method for the viewer to operate the second terminal device 3, 3'to establish a communication connection with the live broadcast management platform 2 via the Internet to watch live broadcast content, interact or participate in game solving tasks, etc.

The creator recognizes that existing virtual reality and augmented reality image service providers usually provide existing virtual images based on the image database for viewers to watch on specific occasions, and their applications are still limited. However, the virtual character live streaming method of any embodiment of the present invention can break through the limitations of virtual images and viewing locations to bring a wider range of applications and fun. For example, using virtual characters for live broadcasting, the audience can instantly use the mobile device to Watch and interact with the virtual images created by the live host at any location. The following examples illustrate related embodiments and detailed steps.

In this embodiment, first, the user uses the first terminal device 1 to connect to the live broadcast management platform 2 through a web browser (Browser) or application (APP) built in the first terminal device 1 to read Take and operate a setting interface from the live broadcast management platform 2. The setting interface can display a virtual character setting area, where the virtual character setting area provides one or more image editing elements such as face, expression, body shape, clothing or character, for users to select and set the characteristics of virtual characters for live broadcast . In one embodiment, the user can choose to add image editing elements such as virtual clothing, decorations, props, and dynamic expressions based on his own real image, through image modification, integration, or through artificial intelligence algorithms, to generate a virtual reality different from the real world. Characters to replace their own real images to increase the fun of live broadcasting or to maintain the user's personal privacy in specific applications.

For example, the live broadcaster converts to high-slim type virtual characters wearing comic costumes based on his real image modification, or sweeps the face to build a 3D virtual character (3D Avatar), and based on this virtual image, live broadcast, whether it is virtual reality Virtual reality (VR), augmented reality (AR) or mixed reality (MR) technologies are all feasible solutions, but the original creation does not limit these implementation details; or, users can simply choose a virtual character as their own avatar image , Where the virtual character can be a two-dimensional or three-dimensional image, represented by pictures, cartoons, computer animations, etc., and is not necessarily the same or similar to the user's image, that is, the user can choose a virtual different from his own real image The character acts as a stand-in for live streaming. In short, through step S1, the terminal display unit 12 of the first terminal device 1 displays a setting interface, where the setting interface includes a virtual character setting area, and a virtual character is set through the virtual character setting area for live broadcast.

In at least one embodiment, the configuration interface includes an identity verification mechanism. For example, a login verification method such as binding a user’s real data or a social site account can be used. For example, the user must fill in personal information to register an account and log in After that, you can customize the virtual character or the image elements you want to add based on the real image modification and editing as the virtual character for subsequent live broadcast. In this way, it can be ensured that the same virtual character is set and used by the same user, so as to prevent the third person from embezzling the virtual character, and maintain the management order of the live broadcast platform and transaction security.

Next, through step S2, the first terminal device 1 activates and displays a live interface, which includes virtual characters. For example, the terminal computing unit 16 activates a live interface, and displays the virtual character on the live interface through the terminal display unit 12 according to the virtual character set by the user. Overall, the terminal display unit 12 can display a setting interface and a live interface, where the setting interface includes a virtual character setting area, and the live interface includes a virtual character set by the user and interactive information from the audience, wherein the user passes the virtual character The setting area sets the image editing elements such as the face shape, expression, body shape, clothing or character of the virtual character for live broadcast.

Furthermore, in step S3, the first terminal device 1 detects an action of a user to generate a detection signal of the corresponding action. In an embodiment, the first terminal device 1 detects a user's motion through the motion detection unit 14 to generate a corresponding detection signal. Among them, the user's actions can be the live host's facial features, expressions, gestures, displacement, dance, and combinations thereof. For example, the motion detection unit 14 is used to capture the user's visible light image, invisible light (infrared light, far infrared light) image, thermal image, or capture the user's gestures, displacement, limb movements, etc. to generate Corresponding to one of the detection signals, for example: digital image signal, audio signal, etc.

Next, in step S4, the first terminal device 1 transmits the detection signal to the live broadcast management platform 2 via the network, and the live broadcast management platform 2 recognizes the detection signal to generate a control signal corresponding to the detection signal, and broadcasts the corresponding control One of the live broadcast animations of the signal, where the live broadcast animation is generated based on the control signal and the virtual character. In an embodiment, the terminal computing unit 16 transmits the detection signal to the live broadcast management platform 2 through the network through the terminal communication module 10, wherein the management computing unit 22 receives the detection signal through the management communication module 20, and then, manages the operation The unit 22 recognizes the detection signal to generate a control signal corresponding to the detection signal. For example, the motion detection unit 14 captures the user's motion image, so the detection signal is a digital image signal for the management computing unit 22 to recognize The digital image signal. In detail, the terminal arithmetic unit 16 calculates the coordinate change of the action in the action image based on the action image and recognizes the action to output the corresponding control signal. Then, the management computing unit 22 generates a live animation associated with the virtual character and corresponding to the control signal through the motion capture algorithm based on the control signal and the virtual character image, and broadcasts the live animation to one or more third parties through the management communication module 20 Two terminal devices 3, 3'. In another embodiment, the first terminal device 1 selects favorite image editing elements such as face shape, expression, body shape, clothing, or character in the virtual character setting area through step S1 as the characteristics of the virtual character for live broadcast to create The initial animation model, and the live broadcast management platform 2 in step S4, through the motion capture algorithm to modify the animation model associated with the virtual character according to the control signal, generate live animation, and broadcast live animation to one or more through the management communication module 20 Second terminal devices 3, 3'.

It should be noted that the control signal is suitable for controlling the movement of the virtual character in the live animation according to the user's action, and the control signal is based on the detection signal captured by the first terminal device 1 (such as but limited to reading: sound, video, etc.) Signal), generated by identification of the live broadcast management platform 2 such as a cloud server. The live broadcast management platform 2, based on the control signals and virtual characters, is processed by the image calculation technology to generate the corresponding live broadcast animation, which can effectively save the computing resources and hardware and software costs of the first terminal device 1, but not limited to this. However, in some embodiments, the terminal computing unit 16 of the first terminal device 1 can also be used to identify the detection signal to generate a control signal corresponding to the user's action. For example, the motion detection unit 14 captures the user's motion image as a detection signal, and the terminal computing unit 16 calculates the coordinate change of the motion in the motion image based on the motion image and recognizes the motion to output the corresponding control Signal. In addition, the motion detection unit 14 and the terminal computing unit 16 can be integrated into one, but not limited to this.

In this embodiment, the live broadcast management platform 2 includes a management communication module 20, a management computing unit 22, and a management database 24, as shown in FIG. In one embodiment, the management database 24 may be implemented by one or more memories. The management database 24 stores a plurality of image editing elements such as multiple faces, expressions, body shapes, clothing, or characters, which can be used for query and maintenance of the setting interface or live interface. Although not specifically depicted, in some embodiments, the live broadcast management platform 2 can be implemented by a web server, a management server, and a database. In at least one embodiment, the live broadcast management platform 2 may be a public cloud service (Public Cloud Service) platform provided by Microsoft, such as Microsoft Azure, but not limited thereto. The management computing unit 22 is a service group composed of a large number of virtual machines (Virtual Machine) formed by virtualization of the server group in Azure, and its main function is to provide CPU, memory and other resources with computing capabilities. The management database 24 can be provided by Azure with basic storage (Azure Storage) and relational database (SQL Azure), including but not limited to Blob, Table and Queue to manage unstructured data, structured data and message communication respectively, and Supports fast data sharing between virtual machines in the cloud.

Finally, in step S5, the first terminal device 1 receives an interactive signal transmitted by the live broadcast management platform 2, and displays an interactive information on the live broadcast interface according to the interactive signal. In an embodiment, the viewer uses the second terminal device 3 to establish a communication connection with the live broadcast management platform 2 through the Internet, thereby reading and operating through the web browser or application program built in the second terminal device 3 A live broadcast interface derived from the live broadcast management platform 2 to watch live broadcast animations of the above virtual characters in real time. At the same time, during the live broadcast process, the audience can use the second terminal device 3 to communicate relevant requests or responses to the live broadcast host through text, voice, video and combinations thereof and other media for interaction. For example, the second terminal device 3 transmits an interactive signal to the live broadcast management platform 2, where the interactive signal may be, but not limited to, a text signal, a voice signal, and an image signal, whether it is realized by a digital signal or an analog signal, and the live broadcast management platform 2 Receive the interactive signal and send it to the first terminal device 1, then, the terminal computing unit 16 of the first terminal device 1 receives the interactive signal transmitted by the live broadcast management platform 2 through the terminal communication module 10, and the terminal computing unit 16 according to the interaction After the signals are processed correspondingly, the terminal display unit 12 displays the live interface including interactive information, such as but not limited to: client requests, gift notifications, rewards, text chats, instant voice communications, viewer real-time images, second terminal devices 3 The live animation of the guest virtual character generated by executing the above virtual character live broadcasting method, but not limited to this.

The live streaming method of virtual characters in some embodiments of this creation can realize interactive game task-removing activities to increase the fun. It can also be generated through computer technology and wearable devices and provide all real and virtual combined environments and human-computer interactions. To implement Extended Reality (XR) applications. In one embodiment, the first terminal device 1 transmits an interactive task to the live broadcast management platform 2, and then the live broadcast management platform 2 can be transmitted to the viewers who log in to the live broadcast interface for interaction, such as but not limited to guessing activities, making money at work, and treasure hunting Interactive tasks such as tasks and subrogation marketing. In short, the first terminal device 1 transmits the interactive task to the live broadcast management platform 2 for the plurality of second terminal devices 3, 3'to generate the interactive signal according to the interactive task, whereby the first terminal device 1 can display through step S5 Interactive information is given back to users.

In other embodiments, the virtual character live broadcast method may allow the first terminal device 1 to preset an active task and a corresponding task location in an electronic map, and when the viewer moves to carry the second terminal device 3 and is located at the task location, the first The two terminal devices 3 can display active tasks for the audience to perform interactive tasks. In other words, the second terminal device 3 can transmit the location information to the live broadcast management platform 2, and the live broadcast management platform 2 determines whether the location information of the second terminal device 3 matches the task location in the electronic map. When the location information matches the task location, the live broadcast The management platform 2 allows the second terminal device 3 to perform interactive tasks. In some embodiments, the first terminal device 1 may pre-record at least one virtual character image as a task image and transmit it to the live broadcast management platform 2, thereby, when the live broadcast management platform 2 determines whether the location information of the second terminal device 3 matches At the task location in the electronic map, the live broadcast management platform 2 can transmit the task image to the holographic projection device 4 set at the task location, so as to project the task image at the task location. In short, the holographic projection device 4 receives and delivers the task image to the task location, increasing the visual effect and interestingness of the interactive task.

For example, the virtual character live broadcast method can be combined with augmented reality technology. The above-mentioned task locations can be physical stores and shops, and the first terminal device 1/second terminal device 3 also generates their own virtual characters according to the user/viewer settings , And the user/viewer can go to the physical store to transfer the virtual characters they own from the mobile device such as the first terminal device 1 and the second terminal device 3 to the display device of the physical store such as the holographic projection device 4 for display, Take this to integrate offline and real physical stores. When the avatar is displayed on the display device of the physical store, the avatar is not displayed on the mobile device. In this way, it is possible to simulate the situation of "working" from a virtual character to a physical store, which is an example of an interactive task, and through the augmented reality display device installed in the physical store, it seems that there are more virtual shop assistants in the store. For physical stores, the opportunity for consumers (including users and viewers) to physical stores can be increased. For consumers, it can also increase the practicality and fun of virtual characters. Let this avatar not only be executed in the mobile device, but also be transferred to another device to perform the task, and get the corresponding rewards during the execution of the task, for example, let the avatar also go to "work to make money ", "Treasure Hunting Task", "Subrogation Marketing" and other interactive tasks. Further, the operation process will be stored in cloud servers such as live broadcast management platform 2, to save consumers' consumption behavior in physical stores, and can further cooperate with various cloud data providers to finally obtain a complete consumer O2O marketing model (online-to -offline) behavior data, used as a variety of business.

Please continue to refer to FIGS. 3 and 4. The virtual character live broadcast system according to another embodiment of the present invention includes a first terminal device 1 and a live broadcast management platform 2. The live broadcast management platform 2 includes a management communication module 20 and a management computing unit 22. The management computing unit 22 is electrically connected to the management communication module 20. In an embodiment, the management communication module 20 may be a wireless communication interface, such as but not limited to: wireless local area network (WIFI), cellular network (3G, 4G, 5G), Zigbee, etc. For transmitting interactive signals and live animations corresponding to control signals, please refer to the foregoing for the signal transmission mechanism, virtual characters, live animation calculation mechanism and other related embodiments; the management computing unit 22 can be one or more such as microprocessors, micro-controllers Devices, digital signal processors, microcomputers, central processors, field programming gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, and/or any operating signals (analog and/or digital) based on operating instructions ) Is implemented by a processing element for identifying the detection signal to generate a corresponding control signal, and generating live animation based on the control signal and the virtual character. For the image calculation mechanism and related embodiments, please refer to the foregoing.

The first terminal device 1 includes a terminal communication module 10, a terminal display unit 12, a motion detection unit 14, and a terminal arithmetic unit 16. The terminal arithmetic unit 16 is electrically connected to the terminal communication module 10, the terminal display unit 12 and the motion detection unit 14. The terminal communication module 10 is communicatively connected to the management communication module 20. In one embodiment, the terminal communication module 10 may be a wireless communication interface, such as but not limited to: wireless local area network (WIFI), cellular network (3G, 4G, 5G), Zigbee, etc. For receiving interactive signals and transmitting control signals, please refer to the foregoing for details of the signal transmission and reception mechanism and related embodiments.

In an embodiment, the terminal display unit 12 may be a touch screen for displaying a setting interface and a live broadcast interface. The related technical content, functions, and advantages of the setting interface and the live broadcast interface are described in detail above. The motion detection unit 14 may be an image capture unit, such as but not limited to a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), or a depth camera, used to detect the user's motion. For the mechanism and related embodiments, please refer to the foregoing. The terminal arithmetic unit 16 may be one or more such as a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processor, a field programming gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital The circuit and/or any processing element based on the operation command operation signal (analog and/or digital) is used to generate a detection signal corresponding to the action and generate interactive information based on the interactive signal. Regarding the identification action mechanism and related embodiments , Please refer to the aforementioned.

In some embodiments, the computer program product for live broadcasting of virtual characters is composed of a set of instructions. When a computer such as the first terminal device 1 loads and executes the set of instructions, the virtual character of any of the above embodiments can be completed Live streaming method.

Based on the above, some embodiments of this creation provide a virtual character live broadcast system, which is mainly connected to the live broadcast management platform through the first terminal device, first through the customized virtual preferences of the setting interface, and then on the live broadcast platform Real-time control of the expressions and actions of virtual characters as user stand-in images, based on which live broadcast and real-time interaction can be shared with viewers through the live broadcast platform, which can break through the traditional limitations of pre-recorded virtual images and viewing locations, thereby increasing the fun of live broadcast And keep the user's personal privacy in specific applications. In addition, all real and virtual combined environments and human-computer interactions can be generated and provided through computer technology and wearable devices. For example, virtual characters combined with augmented reality technology can set physical stores as task locations, users/ The audience can go to the physical store to transfer the virtual characters they own from the mobile device, such as the first/second terminal device, to the display device of the physical store, so that the virtual characters can not only be executed in the mobile device, but also can be transferred to In another device, the task is performed, thereby integrating the virtual and real offline interactive physical stores to increase the practicality and fun of the virtual character.

The above-mentioned embodiments are only to illustrate the technical ideas and characteristics of this creation, and its purpose is to enable those who are familiar with this skill to understand the content of this creation and implement it accordingly. When the patent scope of this creation cannot be limited by this, That is to say, the equal changes or modifications made by Dafan in accordance with the spirit of this creation should still be covered by the patent scope of this creation.

S1~S5: Step 1: the first terminal device 10: Terminal communication module 12: Terminal display unit 14: Motion detection unit 16: Terminal arithmetic unit 2: Live broadcast management platform 20: Management communication module 22: Management computing unit 24: Manage the database 3. 3’: Second terminal device 4: Holographic projection device

FIG. 1 is a schematic flowchart of a method for creating a virtual character live broadcast according to an embodiment. FIG. 2 is a schematic structural diagram of creating a virtual character live broadcast system according to an embodiment. FIG. 3 is a block diagram of a first terminal device according to an embodiment of the invention. FIG. 4 is a block diagram of a live broadcast management platform according to an embodiment.

1: the first terminal device

2: Live broadcast management platform

3. 3’: Second terminal device

4: Holographic projection device

Claims (7)

A virtual character live broadcast system, including: A live broadcast management platform, including: A management communication module for transmitting an interactive signal and a live animation corresponding to a control signal, wherein the live animation is generated based on the control signal and a virtual character; and A management computing unit, electrically connected to the management communication module, for identifying a detection signal to generate the control signal corresponding to the detection signal; and A first terminal device, including: A terminal communication module, which is connected to the management communication module for receiving the interactive signal and transmitting the detection signal; A terminal display unit is used for displaying a setting interface and a live interface, wherein the setting interface includes a virtual character setting area, and the live interface includes the virtual character and an interactive information, and the virtual character setting area is used to set the Virtual characters; A motion detection unit for detecting a motion of a user; and A terminal computing unit, electrically connected to the terminal communication module, the terminal display unit and the motion detection unit, is used to generate the detection signal corresponding to the motion and generate the interactive information according to the interactive signal. The virtual character live broadcast system according to claim 1, wherein the motion detection unit of the first terminal device captures a motion image of the user; and the management computing unit of the live broadcast management platform calculates based on the motion image The coordinate of the action changes and the action is recognized to output the corresponding control signal. The virtual character live broadcast system according to claim 1, wherein the terminal computing unit of the first terminal device recognizes the detection signal to generate the control signal corresponding to the detection signal. The virtual character live broadcast system according to claim 1, wherein the management communication module of the live broadcast management platform receives the detection signal transmitted by the terminal communication module of the first terminal device, and the management operation unit is based on The control signal generated by the detection signal is modified and associated with an animation model of the virtual character to generate the live animation. The virtual character live broadcast system as described in claim 1, wherein the first terminal device transmits an interactive task to the live broadcast management platform for a plurality of second terminal devices in communication with the live broadcast management platform to generate the Interactive signal. The virtual character live broadcast system according to claim 5, wherein at least one of the second terminal devices transmits a location information to the live broadcast management platform, and the live broadcast management platform determines whether the location information meets a task in an electronic map Location, when the location information matches the task location, the live broadcast management platform allows the second terminal device to perform the interactive task. The virtual character live broadcast system as described in claim 6 further includes: A holographic projection device is connected to the management communication module for receiving and sending a task image to the task location.

Images