CN113658213A - Image presentation method, related device and computer program product - Google Patents
Image presentation method, related device and computer program product Download PDFInfo
- Publication number
- CN113658213A CN113658213A CN202110934906.0A CN202110934906A CN113658213A CN 113658213 A CN113658213 A CN 113658213A CN 202110934906 A CN202110934906 A CN 202110934906A CN 113658213 A CN113658213 A CN 113658213A
- Authority
- CN
- China
- Prior art keywords
- information
- video stream
- target user
- action
- role model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present disclosure provides an image presentation method, an image presentation apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which are applied to an intelligent mirror and relate to the technical field of artificial intelligence, such as image processing, virtual broadcasting, and intelligent recommendation. One embodiment of the method comprises: after the role model of the target user is obtained, the role model is synchronously driven according to the motion information of the target user, so that the action video stream of the role model is obtained, and finally the action video stream is presented to the target user. The method and the system can synchronously drive the character model based on the motion information of the user so as to restore the motion condition of the user through the character model, and protect the personal privacy of the user while increasing the interest of user interaction.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to the field of artificial intelligence technologies such as image processing, virtual broadcasting, and intelligent recommendation, and in particular, to an image presentation method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.
Background
With the increasingly vigorous pursuit of health, in order to better understand the motion information such as body posture of the user in the process of body building and exercise, the user needs to use the mirror for assistance so as to understand the motion information of the user through the picture reflected by the mirror.
In the prior art, in order to provide more auxiliary and reference information for a user in the process, a conventional Mirror is improved and replaced by using a Smart Mirror (Smart Mirror), so that on the basis of keeping the functions of the conventional Mirror, more auxiliary and reference information is provided by using the Smart Mirror.
Disclosure of Invention
The embodiment of the disclosure provides an image presenting method, an image presenting device, an electronic device, a computer readable storage medium and a computer program product.
In a first aspect, an embodiment of the present disclosure provides an image presenting method, including: acquiring a role model of a target user; synchronously driving the role model according to the motion information of the target user to obtain the action video stream of the role model; the action video stream is presented to the target user.
In a second aspect, an embodiment of the present disclosure provides an image presenting apparatus, including: a role model acquisition unit configured to acquire a role model of a target user; the action video stream generating unit is configured to synchronously drive the role model according to the motion information of the target user to obtain an action video stream of the role model; and a video stream presenting unit configured to present the action video stream to the target user.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor, when executed, to implement the avatar rendering method as described in any of the implementations of the first aspect.
In a fourth aspect, the disclosed embodiments provide a non-transitory computer-readable storage medium storing computer instructions for enabling a computer to implement the character rendering method as described in any implementation manner of the first aspect when executed.
In a fifth aspect, the embodiments of the present disclosure provide a computer program product comprising a computer program, which when executed by a processor is capable of implementing the character presentation method as described in any implementation manner of the first aspect.
The image presenting method, device, electronic device, computer-readable storage medium and computer program product provided by the embodiments of the present disclosure obtain a role model of a target user, synchronously drive the role model according to motion information of the target user, obtain an action video stream of the role model, and finally present the action video stream to the target user.
The character model can be synchronously driven based on the motion information of the user, so that the motion condition of the user is restored through the character model, the image of the original user is replaced by the virtual image while the interaction interestingness of the user is increased, and the personal privacy safety of the user is protected.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture to which the present disclosure may be applied;
FIG. 2 is a flowchart of an image rendering method provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart of another image rendering method provided by the embodiments of the present disclosure;
FIG. 4 is an effect intention of the image rendering method in an application scene according to the embodiment of the present disclosure;
fig. 5 is a block diagram of an image presenting apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device suitable for executing an avatar rendering method according to an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness. It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict.
In addition, in the technical scheme related to the disclosure, the acquisition, storage, application and the like of the personal information of the related user (for example, the image including the human face object, which is related to the following of the disclosure) all conform to the regulations of related laws and regulations, and do not violate the customs of the public order.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of the display optimization methods, apparatus, electronic devices, and computer-readable storage media of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include a terminal device 101 and a user 102 using the terminal device.
The terminal device 101 may interact with other terminal devices and servers through a network or the like, so as to provide more functions for the terminal device 101 locally by using the other terminal devices and servers.
The terminal apparatus 101 may be hardware or software. When the terminal device 101 is hardware, it can be an intelligent mirror or other electronic devices with display screen and functions equivalent to the intelligent mirror; when the terminal device 101 is software, it may be installed in the electronic device, and it may be implemented as multiple software or software modules, or may be implemented as a single software or software module, and is not limited in this respect.
The terminal device 101 may provide various services through various built-in applications, taking a motion-assisted application that can provide an avatar action video stream for a user as an example, the terminal device 101 may implement the following effects when running the motion-assisted application: firstly, the terminal device 101 acquires a role model of a target user; then, the terminal device 101 synchronously drives the role model according to the motion information of the target user to obtain the action video stream of the role model; finally, the terminal device 101 presents the working video stream to the target user.
The motion information is usually collected by an image collecting component of the terminal device 101 to obtain real-time motion information of the target user, and the character model is driven according to the motion information to realize real-time presentation of an image of the character model that is synchronous with the motion information of the target user and moves.
In consideration of the fact that in the actual usage scenario, in the process of presenting the image and action video stream for the user, the requirement of the user on the timeliness of presenting the action video stream is high, the image presentation method provided in the subsequent embodiments of the present disclosure is generally executed by the terminal device 101 which is directly used by the user and has high timeliness of presenting the action video stream to the user, and accordingly, the image presentation apparatus is generally also disposed in the terminal device 101. However, it should be noted that when the performance of the terminal device 101 itself is reduced, part or all of the calculation task may be forwarded to a server at the back end, and the terminal device 101 only needs to receive the calculation result returned by the server.
It should be understood that the number and size of the terminal devices in fig. 1 are merely illustrative. The adaptation can be performed according to implementation requirements.
Referring to fig. 2, fig. 2 is a flowchart of an image presenting method according to an embodiment of the disclosure, in which the process 200 includes the following steps:
In this embodiment, an executive body of the character presentation method (for example, the terminal device 101 shown in fig. 1) obtains a character model of the target user, where the character model is used to replace the character of the target user, and may be a character model generated based on the contents of a face image, a human body image, and the like of the target user, or a character model generated in advance based on an animal character, a cartoon character, and the like, and on this basis, the character model should have a body shape similar to a human body, so that when the character model is subsequently driven, the motion information of the target user can be fed back.
It should be noted that the role model may be directly obtained from a local storage device by the execution main body, or may be obtained from a non-local storage device (for example, from other non-local terminal devices or a server). The local storage device may be a data storage module, such as a storage hard disk, disposed in the execution main body, in which case the character model can be quickly read locally; the non-local storage device may also be any other electronic device configured to store data, such as some terminal device, a server, etc., in which case the executing entity may obtain the required character model by sending a obtaining command to the electronic device.
In practice, a plurality of character models can be generated in advance, so that the character model of the target user can be directly determined from the plurality of character models generated in advance, and the efficiency of the image presenting method is improved.
Furthermore, a role model generation plug-in can be configured in the execution main body, a plurality of preset role models exist in the plug-in, and a corresponding set target user can adjust the preset role models through model parameters in the plug-in, so that the actual requirements of the target user can be obtained in a man-machine interaction mode, the corresponding role models are generated, and the user experience is improved.
In this embodiment, after the role model of the target user is obtained in step 201, the motion information of the target user is obtained, where the motion information is obtained by continuously extracting the motion posture of the target user in the video image after the video image including the target user is usually collected by the image capture device of the execution subject, and after the motion information of the target user is obtained, the role model is driven by using the target information to restore the motion information of the target user.
The motion information of the target object can be acquired in a form of extracting the motion key points, for example, each joint part of the target user is determined as the motion key point, the motion information of the target object is determined and acquired according to the change of the motion key point, correspondingly, the target driving point can be determined at each corresponding motion key point in the character model, and then after the motion information of the target object is acquired, the target driving point corresponding to each motion key point is driven, so that the purposes of driving the character model and acquiring the motion video stream of the character model are achieved.
In this embodiment, the action video stream obtained in step 202 is presented to the target user by using the display component of the execution main body, and it should be understood that, when the execution main body is not provided with a display component or is only used as the execution main body for generating the action video stream, the step is that the execution main body sends the generated action video stream to the terminal device used by the target user and provided with a display component, and controls the terminal device to display the action video stream.
The image presenting method provided by the embodiment of the disclosure can synchronously drive the role model based on the motion information of the user so as to restore the motion condition of the user through the role model, and the virtual image replaces the image of the original user while increasing the interaction interest of the user, thereby protecting the personal privacy safety of the user.
In order to further improve the value of the action video stream, a corresponding action item may be determined according to the motion information of the target user, and after the standard motion information of the action item is obtained, prompt information is generated for the target user based on the difference between the standard motion information and the motion information of the target user, and the promotion information is added to the action video stream, so that the target user can know the motion state of the target user and perform corresponding adjustment according to the action video stream to which the promotion information is added, in this case, in some optional implementations of this embodiment, the image presentation method further includes: determining the sport item of the target user based on the sport information, and acquiring standard sport information of the sport item; in response to the motion information, adding first prompt information to an image frame corresponding to the target motion in the motion video stream, wherein the difference between the target motion information and the standard motion information is larger than a preset threshold value; and in response to the target motion not being present in the motion information, adding second cue information in a last image frame of the motion video stream.
Specifically, after acquiring motion information of a target user, analyzing a motion item related to the target user in the motion information to determine a current motion item performed by the target user, and acquiring standard motion information of the motion item, where the standard motion information includes a standard motion action, a form, and the like under the corresponding motion item, after acquiring the standard motion information, comparing a difference between the motion information and a corresponding action in the standard information, and when determining that the difference is greater than a preset threshold, determining the action as the target action, and adding first prompt information in an image frame corresponding to the target action to feed back the target user through the first prompt information, so that the target user knows that the current action is different from the standard action greatly and needs to adjust.
Further, when the target action does not exist in the motion information, that is, the difference between the action in the motion information and each corresponding action in the standard motion information meets the requirement of a preset threshold, second prompt information is added in a last image frame of the generated action video stream to prompt the target user that the execution of the action meets the requirement of the standard action, and correspondingly, encouragement information can be added in the second prompt information to encourage the target user and improve the user experience.
In practice, the standard exercise information may be pre-stored in a local memory of the execution main body, or after the execution main body determines the exercise item, a standard exercise information acquisition request may be sent to another server or terminal device, so as to acquire the standard exercise information from another server or terminal device.
Referring to fig. 3, fig. 3 is a flowchart of another image presenting method according to an embodiment of the disclosure, in which the process 300 includes the following steps:
In this embodiment, when the exercise information includes a preset action, a response is made to obtain incentive information corresponding to the preset action, where the incentive information may be set according to a requirement and corresponding to a property of the preset action, for example, when the incentive information is used to encourage the target user to continue exercising and to persist the current action, the preset action may be an action posture such as "crouch horse walking" or "flat plate support" that requires endurance for the target user, and when the incentive information is used to invoke an incentive of the target user and make the target user more excited, the preset action may be set to be a "punch" or "high leg kicking" action, so as to promote the action explosive force of the target user through the incentive information.
The specific form of the motivation information can be interactive with the target user, and dynamic images, text prompt information, voice information and the like can be presented.
And step 304, adding the excitation information to the image frames corresponding to the preset action in the action video stream to obtain an optimized video stream.
In this embodiment, after obtaining the incentive information based on the step 303, the incentive information is correspondingly added to the image frames corresponding to the preset action in the action video stream to obtain an optimized video stream, and the optimized video stream may present the incentive information when playing the content of the image frames corresponding to the preset action.
It should be understood that, when the motivational information is content in the form of images, characters, etc., the motivational information may be correspondingly added to the image frames to achieve interaction with the target user, and when the motivational information is voice information, the motivational information is added corresponding to the image frames on the playing time axis of the action video stream, so that when an optimized video stream is subsequently obtained, the optimized video stream is played to the image frames to send out corresponding voice information.
The above steps 301, 302, 305 are similar to the step 201-203 shown in fig. 2, and the contents of the same portions refer to the corresponding portions of the previous embodiment, which are not repeated herein.
On the basis of the embodiment shown in fig. 2, the present embodiment further performs corresponding stimulation when the target user performs the preset action, so as to perform interaction and feedback in cooperation with the actual motion situation of the target user, thereby improving the user experience of the target user.
In some optional implementations of this embodiment, the method further includes: acquiring first voiceprint information of the target user, and acquiring first voice excitation information based on the text information of the excitation information and the voiceprint information; and adding the excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream, including: and adding the first voice excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream.
Specifically, after first voiceprint information of a target user is collected, first excitation voice information is generated based on the first voiceprint information and text information of the excitation information, the first excitation voice information is determined to be excitation information corresponding to a preset action and added to a corresponding image frame in an action video stream, a corresponding optimized video stream is obtained, when interaction between the excitation information and the target user is achieved in a voice information mode, corresponding voice information is generated based on sound of the target user, discomfort of the target user due to strange playing sound of the excitation information is avoided, and user experience is improved.
In some optional implementations of this embodiment, the method further includes: acquiring second voiceprint information appointed by the target user, and acquiring second voice excitation information based on the text information of the excitation information and the voiceprint information; and adding the excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream, including: and adding the second voice excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream.
Specifically, when a target user specifies second voiceprint information, second voice excitation information is obtained based on the second voiceprint information specified by the target user and text information of the excitation information, wherein the second voiceprint information can be interacted with the target user after being configured in advance, is determined based on selection information of the target user, or is obtained by analyzing an audio file uploaded by the target user, and then the second excitation voiceprint information is determined as excitation information corresponding to a preset action and is added to a corresponding image frame in an action video stream to obtain a corresponding optimized video stream, so that when interaction between the excitation information and the target user is realized in a voice information mode, voice playing excitation information which is expected to be listened by the target user and is used for exciting the target user is used in combination with expectation of the target user, and user experience is improved.
On the basis of any one of the above embodiments, the character presentation method further includes: presenting a set of role models to a target user; determining a target role model according to the selection information returned by the target user aiming at the role model set; correspondingly, the obtaining of the role model of the target user includes: and acquiring the target role model, and taking the target role model as the role model of the target user.
Specifically, a role model set is presented to a target user, a plurality of role models are preset in the role model set, the target user can select the role model according to the own requirements, corresponding selection information is returned to the execution main body after the selection is completed, so that the execution main body can determine the corresponding target role model according to the selection information, the target role model is used as a determined target role model of a subsequently generated action video stream, the role model closer to the intention of the target user is selected in an interactive mode, and the user experience is improved.
In order to deepen understanding, the disclosure further provides a specific implementation scheme in combination with a specific application scenario, specifically as follows:
acquiring a role model a of a target user A;
at the moment, the target user A carries out opening and closing jumping movement, the role model is driven according to the movement information of the target user A, and the action video stream of the role model is obtained;
responding to the action video stream with a preset action of ' shooting two hands over the head ', and adding excitation information ' oil!in the form of characters and voice information into the image frame corresponding to the preset action in the action video stream! ", obtaining an optimized video stream;
and playing the optimized video stream to a target user A, wherein when the optimized video stream is played to an image frame corresponding to a preset action, the optimized video stream can be as shown in FIG. 4.
With further reference to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an image presenting apparatus, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 5, the character presentation apparatus 500 of the present embodiment may include: a character model acquisition unit 501, a motion video stream generation unit 502, and a video stream presentation unit 503. The role model obtaining unit 501 is configured to obtain a role model of a target user; an action video stream generating unit 502 configured to synchronously drive the role model according to the motion information of the target user, so as to obtain an action video stream of the role model; a video stream presenting unit 503 configured to present the action video stream to the target user.
In the present embodiment, in the character presentation apparatus 500: the specific processing and the technical effects thereof of the role model obtaining unit 501, the action video stream generating unit 502, and the video stream presenting unit 503 can refer to the related descriptions of step 201 and step 203 in the corresponding embodiment of fig. 2, which are not described herein again.
In some optional implementations of the present embodiment, the character presentation apparatus 500 further includes: an excitation information acquisition unit configured to acquire excitation information corresponding to a preset action in response to the motion information including the preset action; an optimized video stream generating unit configured to add the excitation information to the image frames corresponding to the preset action in the action video stream to obtain an optimized video stream; accordingly, the video stream presentation unit is further configured to present the optimized video stream to the target user.
In some optional implementations of the present embodiment, the character presentation apparatus 500 further includes: the first voice excitation information generating unit is configured to acquire first voiceprint information of the target user and obtain first voice excitation information based on character information of the excitation information and the voiceprint information; and the optimized video stream generating unit is further configured to add the first voice excitation information to the image frames corresponding to the preset action in the action video stream to obtain an optimized video stream.
In some optional implementations of the present embodiment, the character presentation apparatus 500 further includes: a second voice excitation information generation unit configured to acquire second voiceprint information specified by the target user and obtain second voice excitation information based on text information of the excitation information and the voiceprint information; and the optimized video stream generating unit is further configured to add the second voice excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream.
In some optional implementations of the present embodiment, the character presentation apparatus 500 further includes: a standard exercise information acquisition unit configured to determine an exercise item of the target user based on the exercise information and acquire standard exercise information of the exercise item; a first cue information adding unit configured to add first cue information in an image frame corresponding to the target motion in the motion video stream in response to the target motion having a difference greater than a preset threshold value from the standard motion information; and a second cue information adding unit configured to add second cue information in a last image frame of the motion video stream in response to the absence of the target motion in the motion information.
In some optional implementations of the present embodiment, the character presentation apparatus 500 further includes: a role model set presenting unit configured to present a role model set to a target user; a target role model determining unit configured to determine a target role model according to selection information returned by the target user for the role model set; accordingly, the character model obtaining unit is further configured to obtain the target character model and use the target character model as the character model of the target user.
The embodiment exists as an apparatus embodiment corresponding to the above method embodiment, and the image presenting apparatus provided in this embodiment may synchronously drive the character model based on the motion information of the user, so as to restore the motion condition of the user through the character model, and replace the image of the original user with the virtual image while increasing the user interaction interest, thereby protecting the personal privacy security of the user.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, such as the avatar rendering method. For example, in some embodiments, the avatar rendering method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the character rendering method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the avatar rendering method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in the conventional physical host and Virtual Private Server (VPS) service. The server may also be divided into servers of a distributed system, or servers that incorporate a blockchain.
According to the technical scheme of the embodiment of the disclosure, the character model can be synchronously driven based on the motion information of the user, so that the motion condition of the user can be restored through the character model, the image of the original user is replaced by the virtual image while the interaction interestingness of the user is increased, and the personal privacy safety of the user is protected.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure may be performed in parallel or sequentially or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved, and are not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (15)
1. An image presentation method applied to an intelligent mirror comprises the following steps:
acquiring a role model of a target user;
synchronously driving the role model according to the motion information of the target user to obtain the action video stream of the role model;
presenting the action video stream to the target user.
2. The method of claim 1, further comprising:
responding to the preset action contained in the motion information, and acquiring excitation information corresponding to the preset action;
adding the excitation information to an image frame corresponding to the preset action in the action video stream to obtain an optimized video stream;
correspondingly, the presenting the action video stream to the target user includes:
presenting the optimized video stream to the target user.
3. The method of claim 2, further comprising:
acquiring first voiceprint information of the target user, and obtaining first voice excitation information based on the text information of the excitation information and the voiceprint information; and
adding the excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream, including:
and adding the first voice excitation information to an image frame corresponding to the preset action in the action video stream to obtain an optimized video stream.
4. The method of claim 2, further comprising:
acquiring second voiceprint information appointed by the target user, and acquiring second voice excitation information based on the text information of the excitation information and the voiceprint information; and
adding the excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream, including:
and adding the second voice excitation information to the image frame corresponding to the preset action in the action video stream to obtain an optimized video stream.
5. The method of claim 1, further comprising:
determining the sport item of the target user based on the sport information, and acquiring standard sport information of the sport item;
in response to the motion information, adding first prompt information to an image frame corresponding to the target motion in the motion video stream, wherein the difference between the target motion information and the standard motion information is larger than a preset threshold value; and
in response to the target action not being present in the motion information, adding second cue information in a last image frame of the action video stream.
6. The method of any of claims 1-5, further comprising:
presenting a set of role models to a target user;
determining a target role model according to selection information returned by the target user aiming at the role model set;
correspondingly, the obtaining of the role model of the target user includes:
and acquiring the target role model, and taking the target role model as the role model of the target user.
7. An image presentation device applied to an intelligent mirror, comprising:
a role model acquisition unit configured to acquire a role model of a target user;
the action video stream generating unit is configured to synchronously drive the role model according to the motion information of the target user to obtain an action video stream of the role model;
a video stream presentation unit configured to present the action video stream to the target user.
8. The apparatus of claim 7, further comprising:
the excitation information acquisition unit is configured to respond to the fact that the motion information contains a preset action, and acquire excitation information corresponding to the preset action;
an optimized video stream generating unit configured to add the excitation information to an image frame corresponding to the preset action in the action video stream to obtain an optimized video stream;
accordingly, the video stream presentation unit is further configured to present the optimized video stream to the target user.
9. The apparatus of claim 8, further comprising:
the first voice excitation information generating unit is configured to acquire first voiceprint information of the target user and obtain first voice excitation information based on character information of the excitation information and the voiceprint information; and
the optimized video stream generating unit is further configured to add the first voice excitation information to an image frame corresponding to the preset action in the action video stream, resulting in an optimized video stream.
10. The apparatus of claim 8, further comprising:
a second voice excitation information generation unit configured to acquire second voiceprint information specified by the target user and obtain second voice excitation information based on text information of the excitation information and the voiceprint information; and
the optimized video stream generating unit is further configured to add the second voice excitation information to an image frame corresponding to the preset action in the action video stream, resulting in an optimized video stream.
11. The apparatus of claim 7, further comprising:
a standard exercise information acquisition unit configured to determine an exercise item of the target user based on the exercise information and acquire standard exercise information of the exercise item;
a first cue information adding unit configured to add first cue information in an image frame corresponding to the target motion in the motion video stream in response to the target motion having a difference greater than a preset threshold from the standard motion information; and
a second cue information adding unit configured to add second cue information in a last image frame of the motion video stream in response to the absence of the target motion in the motion information.
12. The apparatus of any of claims 7-11, further comprising:
a role model set presenting unit configured to present a role model set to a target user;
a target role model determining unit configured to determine a target role model according to selection information returned by the target user for the role model set;
correspondingly, the role model obtaining unit is further configured to obtain the target role model and use the target role model as the role model of the target user.
13. An electronic device applied to a smart mirror, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the character presentation method of any one of claims 1-6.
14. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the character presentation method of any one of claims 1-6, applied to a smart mirror.
15. A computer program product, applied to a smart mirror, comprising a computer program which, when executed by a processor, implements the avatar rendering method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934906.0A CN113658213B (en) | 2021-08-16 | 2021-08-16 | Image presentation method, related device and computer program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934906.0A CN113658213B (en) | 2021-08-16 | 2021-08-16 | Image presentation method, related device and computer program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113658213A true CN113658213A (en) | 2021-11-16 |
| CN113658213B CN113658213B (en) | 2023-08-18 |
Family
ID=78480393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110934906.0A Active CN113658213B (en) | 2021-08-16 | 2021-08-16 | Image presentation method, related device and computer program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113658213B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010142305A (en) * | 2008-12-16 | 2010-07-01 | Square Enix Co Ltd | Game apparatus, game replay displaying method, game program, and recording medium |
| CN101816831A (en) * | 2010-04-20 | 2010-09-01 | 陆怡然 | Game role property development system and method based on body building appliances |
| US20180345128A1 (en) * | 2017-06-06 | 2018-12-06 | Invensense, Inc. | Inciting user action for motion sensor calibration |
| CN110719533A (en) * | 2019-10-18 | 2020-01-21 | 广州虎牙科技有限公司 | Live virtual image broadcasting method and device, server and storage medium |
| CN111935491A (en) * | 2020-06-28 | 2020-11-13 | 百度在线网络技术（北京）有限公司 | Live broadcast special effect processing method and device and server |
| CN111970535A (en) * | 2020-09-25 | 2020-11-20 | 魔珐(上海)信息科技有限公司 | Virtual live broadcast method, device, system and storage medium |
| CN112199002A (en) * | 2020-09-30 | 2021-01-08 | 完美鲲鹏(北京)动漫科技有限公司 | Interaction method and device based on virtual role, storage medium and computer equipment |
-
2021
- 2021-08-16 CN CN202110934906.0A patent/CN113658213B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010142305A (en) * | 2008-12-16 | 2010-07-01 | Square Enix Co Ltd | Game apparatus, game replay displaying method, game program, and recording medium |
| CN101816831A (en) * | 2010-04-20 | 2010-09-01 | 陆怡然 | Game role property development system and method based on body building appliances |
| US20180345128A1 (en) * | 2017-06-06 | 2018-12-06 | Invensense, Inc. | Inciting user action for motion sensor calibration |
| CN110719533A (en) * | 2019-10-18 | 2020-01-21 | 广州虎牙科技有限公司 | Live virtual image broadcasting method and device, server and storage medium |
| CN111935491A (en) * | 2020-06-28 | 2020-11-13 | 百度在线网络技术（北京）有限公司 | Live broadcast special effect processing method and device and server |
| CN111970535A (en) * | 2020-09-25 | 2020-11-20 | 魔珐(上海)信息科技有限公司 | Virtual live broadcast method, device, system and storage medium |
| CN112199002A (en) * | 2020-09-30 | 2021-01-08 | 完美鲲鹏(北京)动漫科技有限公司 | Interaction method and device based on virtual role, storage medium and computer equipment |
Non-Patent Citations (1)
| Title |
|---|
| 董伟: "基于微表情识别的虚拟环境交互系统", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 2, pages 138 - 1394 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113658213B (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10313746B2 (en) | Server, client and video processing method | |
| CN112527115B (en) | User image generation method, related device and computer program product | |
| US11631408B2 (en) | Method for controlling data, device, electronic equipment and computer storage medium | |
| CN112102448B (en) | Virtual object image display method, device, electronic equipment and storage medium | |
| CN112653898B (en) | User image generation method, related device and computer program product | |
| CN113325954B (en) | Method, apparatus, device and medium for processing virtual object | |
| CN113365146B (en) | Method, apparatus, device, medium and article of manufacture for processing video | |
| CN114187392A (en) | Virtual even image generation method and device and electronic equipment | |
| CN113724398A (en) | Augmented reality method, apparatus, device and storage medium | |
| CN113221846A (en) | Image recognition method, apparatus, device, storage medium, and program product | |
| CN112714337A (en) | Video processing method and device, electronic equipment and storage medium | |
| CN114092608B (en) | Expression processing method and device, computer readable storage medium and electronic equipment | |
| CN113327311B (en) | Virtual character-based display method, device, equipment and storage medium | |
| CN114554110B (en) | Video generation method, device, electronic equipment and storage medium | |
| CN113658213A (en) | Image presentation method, related device and computer program product | |
| CN113742581B (en) | Method and device for generating list, electronic equipment and readable storage medium | |
| CN113327309B (en) | Video playing method and device | |
| CN114140560A (en) | Animation generation method, device, equipment and storage medium | |
| CN113127058A (en) | Data annotation method, related device and computer program product | |
| CN113760431B (en) | Application control method and device, electronic equipment and readable storage medium | |
| CN113438428B (en) | Method, apparatus, device and computer-readable storage medium for automated video generation | |
| JP7346638B2 (en) | Image data modification method, modification device, electronic equipment, storage medium, computer program and self-driving vehicle | |
| CN114398131B (en) | Information display method, device, equipment, medium and program product | |
| EP4086905B1 (en) | Voice activity detection method and apparatus, electronic device and storage medium | |
| CN116385829B (en) | Gesture description information generation method, model training method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |