WO2023011356A1 - 视频生成方法及电子设备 - Google Patents

视频生成方法及电子设备 Download PDF

Info

Publication number
WO2023011356A1
WO2023011356A1 PCT/CN2022/109033 CN2022109033W WO2023011356A1 WO 2023011356 A1 WO2023011356 A1 WO 2023011356A1 CN 2022109033 W CN2022109033 W CN 2022109033W WO 2023011356 A1 WO2023011356 A1 WO 2023011356A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
data
motion
user
users
Prior art date
Application number
PCT/CN2022/109033
Other languages
English (en)
French (fr)
Inventor
刘晓鹏
周波
王伟
尹明伟
Original Assignee
花瓣云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 花瓣云科技有限公司 filed Critical 花瓣云科技有限公司
Publication of WO2023011356A1 publication Critical patent/WO2023011356A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content

Definitions

  • the present application relates to the field of electronic technology, in particular to a video generation method and electronic equipment.
  • the system or terminal software can collect the user's video image in real time and capture the action data synchronized with the video image to generate the user's simulation video.
  • Fig. 1 shows a scheme for synthesizing sports video in the current game scene.
  • the software such as dance center, just dance
  • the smart screen can collect the user's image in real time based on the camera of the smart screen, and then generate a dance video corresponding to the user based on the collected image to simulate the user's action. Users can watch their own real-time dance videos on the smart screen online.
  • the video generation method and electronic equipment provided in the present application can generate motion videos in scenes without professional data collection equipment, and reduce the complexity of the video generation process.
  • the present application provides a method for generating a video, the method comprising: acquiring data of a video template, collecting motion data of one or more users, and generating motion data of one or more users based on the motion data and data of the video template.
  • Motion data includes any one or more of the following data: speed, amplitude, frequency, trajectory.
  • the data of the video template is the data of the character motion video or the data of the scene video.
  • the video template can be preset or input by the user.
  • the character motion video may be, but not limited to, a motion video of characters such as celebrities and friends.
  • the character motion video may be a motion video synthesized by using the video generation method of the embodiment of the present application, or an ordinary motion video, that is, a motion video not synthesized by the video generation method of the embodiment of the present application.
  • the exercise data has a corresponding first exercise type, and the exercise data corresponding to the first exercise type includes a trajectory;
  • one or more user motion videos including:
  • the trajectory in the motion data corresponding to the first motion type, and the data of the video template generate motion videos of one or more users.
  • the motion data has a corresponding second motion type;
  • the data of the video template includes route data;
  • one or more user motion videos including:
  • an exercise video of one or more users exercising along the route indicated by the route data is generated.
  • one or more can be generated according to the route data of the video template.
  • the motion data of each user moving along the route indicated by the route data can increase the interest of the motion video.
  • the exercise video is the exercise video of multiple users, a certain user can learn about the exercise situation of himself and others from the exercise video, so that he can guide his own exercise and fitness.
  • the route data may be selected by the user or set by default by the system.
  • collecting motion data of one or more users includes: collecting motion data of one or more users from a target software module or a target hardware module.
  • different manufacturers and different types of hardware modules such as auxiliary exercise equipment
  • software modules such as auxiliary exercise software
  • collecting motion data of one or more users includes: receiving motion data input by one or more users.
  • the method further includes: acquiring image information of one or more users; wherein, the image information is preset image information, or the image information of one or more users is image information input by one or more users.
  • the user's image information may be preset by the system or uploaded by the user. There is no need to acquire the image information of the user synchronously when collecting motion data. Therefore, it is suitable for more sports scenes, for example, it is suitable for sports scenes where it is inconvenient to shoot fitness images while exercising.
  • image information of the user and motion data of the user are acquired asynchronously.
  • the image information of one or more users includes any one or a combination of the following items: images of one or more users, text describing features of one or more users, a or multiple users' videos.
  • the information of the user's image is no longer limited to the real-time video of the user, but can be used to describe the characteristics of the user, or any information of the user portrait. Because the scope of image information is widened, correspondingly, there are many ways to collect image information, which are no longer limited to synchronous collection with motion data, and are no longer limited to real-time shooting and collection of image information by cameras.
  • the motion video of one or more users includes a plurality of image frames, and the Xth image frame of the plurality of image frames is inserted into the Xth image of the video template by the image of one or more users at the target moment The image frame is obtained; the target time is the time corresponding to the Xth image frame; X is a positive integer.
  • motion data of multiple users are acquired synchronously.
  • motion data of multiple users is acquired asynchronously.
  • multiple users can complete the exercise separately in an asynchronous manner, that is, complete the exercise at different times and different places, that is to say, users do not have to exercise at the same time, for any user, it can be at a convenient time, Location to complete the campaign.
  • the system can synthesize multiple sports videos of multiple users who are moving at different times and in different places into a sports video of a team moving together, which provides convenience for users to use the function of synthesizing sports videos.
  • the exercise data also includes any one or more of the following data: heart rate, blood pressure, blood oxygen, body temperature.
  • the present application provides a video generation system, including:
  • the video template processing module is used to obtain the data of the video template
  • a motion data collection adaptation module configured to collect motion data of one or more users
  • the sports video generating module is used to generate sports videos of one or more users according to the sports data and the data of the video template.
  • Motion data includes any one or more of the following data: speed, amplitude, frequency, trajectory.
  • the data of the video template is the data of the character motion video or the data of the scene video.
  • the exercise data has a corresponding first exercise type, and the exercise data corresponding to the first exercise type includes a trajectory;
  • the motion video generation module is used to generate motion videos of one or more users according to motion data and video template data, including:
  • the motion video generation module is used to generate motion videos of one or more users according to the track in the motion data corresponding to the first motion type and the data of the video template.
  • the motion data has a corresponding second motion type;
  • the data of the video template includes route data;
  • the motion video generation module is used to generate motion videos of one or more users according to motion data and video template data, including:
  • an exercise video of one or more users exercising along the route indicated by the route data is generated.
  • the motion data collection adaptation module is configured to collect motion data of one or more users, including: collecting motion data of one or more users from a target software module or a target hardware module.
  • the motion data collection adaptation module configured to collect motion data of one or more users, includes: receiving motion data input by one or more users.
  • the system includes: an image data processing module, which is used to acquire image information of one or more users; wherein, the information of the image is information of a preset image, or the information of the image of one or more users is information of one or more users Information about the input image.
  • the image information of the user and the motion data of the user are acquired asynchronously.
  • the image information of one or more users includes any one or a combination of the following items: images of one or more users, text describing features of one or more users, a or multiple users' videos.
  • the motion video of one or more users includes a plurality of image frames, and the Xth image frame of the plurality of image frames is inserted into the Xth image of the video template by the image of one or more users at the target moment The image frame is obtained; the target time is the time corresponding to the Xth image frame; X is a positive integer.
  • motion data of multiple users are acquired synchronously.
  • motion data of multiple users is acquired asynchronously.
  • the exercise data also includes any one or more of the following data: heart rate, blood pressure, blood oxygen, body temperature.
  • the present application provides an electronic device, which has a function of implementing the video generation method in the above first aspect and any possible implementation manner.
  • This function may be implemented by hardware, or may be implemented by executing corresponding software on the hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the present application provides a computer-readable storage medium, including computer instructions.
  • the computer instructions When the computer instructions are run on the electronic device, the electronic device executes any one of the first aspect and any possible implementation manner. video generation method.
  • the present application provides a computer program product that, when the computer program product is run on an electronic device, causes the electronic device to execute the video generation method according to any one of the first aspect and any possible implementation manners thereof.
  • a circuit system includes a processing circuit, and the processing circuit is configured to execute the video generation method in the above first aspect and any possible implementation manner thereof.
  • the embodiment of the present application provides a chip system, including at least one processor and at least one interface circuit.
  • at least one processor executes the video generation method described in the first aspect and any possible implementation manner thereof.
  • Fig. 1 is the flowchart of generating motion video in the prior art
  • FIG. 2 is a schematic structural diagram of a motion video synthesis system provided by an embodiment of the present application
  • FIG. 3A is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 3B is a schematic diagram of the software architecture of the electronic device provided by the embodiment of the present application.
  • FIG. 4 is another schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • Fig. 5 is a schematic diagram of a group of interfaces provided by the embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a video generation method provided in an embodiment of the present application.
  • FIGS 7-9B are schematic diagrams of the interface provided by the embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a video generation method provided in an embodiment of the present application.
  • Fig. 11 is a schematic diagram of the motion data model provided by the embodiment of the present application.
  • FIGS 18A-18B are schematic diagrams of the interface provided by the embodiment of the present application.
  • FIG. 19 is a schematic flowchart of a video generation method provided in an embodiment of the present application.
  • Figure 20A, Figure 20B, Figure 21A, and Figure 21B are schematic diagrams of the interface provided by the embodiment of the present application.
  • Figure 22 is a schematic diagram of the interface provided by the embodiment of the present application.
  • FIG. 23 is a schematic flowchart of a video generation method provided by an embodiment of the present application.
  • Figure 24 is a schematic diagram of the interface provided by the embodiment of this application.
  • FIG. 25 and Figure 26 are schematic flowcharts of the video generation method provided by the embodiment of the present application.
  • Figure 27 is a schematic diagram of the device provided by the embodiment of the present application.
  • FIG. 28 is a schematic diagram of a chip system provided by an embodiment of the present application.
  • the embodiment of the present application provides a video generation method, which can acquire the data of the video template, collect the motion data of one or more users, and generate the video of the one or more users based on the motion data and the data of the video template. exercise video.
  • the motion data includes any one or more of the following data: speed, amplitude, frequency, trajectory.
  • motion data and image information are decoupled, that is, the collection process of image information and motion data can be is separate. In this way, image information and motion data can be collected asynchronously, thereby reducing the implementation complexity of the data collection device.
  • Sports videos can still be generated based on motion data and video templates without the need for proprietary auxiliary equipment (such as cameras) or special personnel to assist in data collection (such as special personnel to assist in shooting and collecting images).
  • FIG. 2 shows an exemplary architecture of the motion video synthesis system.
  • the system includes a data acquisition subsystem and a video synthesis subsystem.
  • the data acquisition subsystem includes a motion data acquisition adaptation module.
  • the video synthesis subsystem includes a video template processing module, an information library (such as a video template library, a template scene library, a character model library, and a motion database), a virtual character (also called a character model) motion image generation module, and a motion video generation module. .
  • Sports data collection and adaptation module it can be used to collect the user's sports data.
  • the sports data collection and adaptation module can build corresponding models for sports data of different sports types according to sports types (such as running, swimming, rowing, etc.) .
  • the motion data model may include one or more of the following information: motion data summarized under this type of motion, and time-sliced data obtained by dividing the motion data.
  • FIG. 11 shows a motion data model under the running motion type.
  • the exercise data in the running mode includes time slice data 1-slice data n.
  • the running durations corresponding to different time slices can be the same or different. Taking the exercise data as one hour of running data as an example, the one hour of running data can be divided into 5 time slice data, and each time slice data corresponds to 12 minutes of running time.
  • the motion data acquisition and adaptation module can be realized by software, by hardware, or by a combination of software and hardware.
  • the motion data acquisition adaptation module may include a software module and a sensor, and the software module may collect the user's motion data by driving the sensor, and then the software module may Do things like format the motion data, build models for the motion data.
  • the athletic data collection adaptation module may receive user input athletic data.
  • exercise software can be used to detect exercise data uploaded by users.
  • the motion video synthesis system in the embodiment of the present application may further include third-party motion software and/or third-party auxiliary motion equipment.
  • the sports data acquisition and adaptation module can obtain the user's sports data from third-party sports software and third-party auxiliary sports equipment.
  • Motion software can be, but not limited to, a system-level application or a user-level application.
  • System-level applications may refer to applications pre-installed on the terminal, for example, sports and health applications.
  • a user-level application may refer to an application installed by a subsequent user, such as a sports application downloaded and installed through an application market.
  • Auxiliary exercise equipment includes, but is not limited to, exercise bracelets, watches, and exercise equipment (such as treadmills, rowing machines, elliptical machines, and spinning bikes).
  • the motion data acquisition and adaptation module can convert the motion data based on the motion data obtained from the above-mentioned third-party motion software or third-party motion assistance equipment, such as format conversion, and obtain the motion data model under the corresponding motion type, so as to make different
  • the motion data in the format can be adapted to the subsequent processing flow.
  • the motion data acquisition adaptation module by expanding the function of the motion data acquisition adaptation module, it can adapt to the interface of various software modules or hardware modules (exercise auxiliary equipment), and collect motion data from various software modules or hardware modules, so as to be able to expand Wide source range of motion data.
  • various software modules or hardware modules exercise auxiliary equipment
  • collect motion data from various software modules or hardware modules so as to be able to expand Wide source range of motion data.
  • the motion data of the user may be collected in an indoor scene, but also the motion data of the user may be collected in an outdoor scene.
  • the user's exercise scene includes but not limited to running, walking, cycling, swimming, football, rowing, basketball, yoga, etc.
  • the user's motion data collected by the motion data collection and adaptation module includes but is not limited to motion type, motion distance, steps, motion duration, motion intensity, motion trajectory, motion posture, and physiological parameters.
  • Physiological parameters include, but are not limited to, heart rate, blood pressure, blood oxygen, body temperature, respiration rate, and bioelectrical impedance.
  • Sports types include, but are not limited to, running, walking, cycling, and fitness.
  • the exercise data acquisition and adaptation module collects the user's exercise data, which may be the exercise data acquired by the user authorized by the exercise data acquisition and adaptation module.
  • the motion data acquisition and adaptation module formats the motion data. After the motion data model is constructed, the relevant information of the motion data model can be written into the information database, and the relevant information of the motion data model can be used as a source for generating motion images of virtual characters.
  • Video template processing module used to obtain the video template, and process the video template to obtain the data of the video template.
  • the video template may be in a video format, or may be in other formats, such as an image format.
  • the embodiment of the present application mainly takes a video format template as an example, but this does not constitute a format restriction on the video template.
  • the video template is a sequence composed of a series of template image frames.
  • the video template may be, but not limited to, a video of a sports game or a video of a background venue. Video templates are available.
  • the video template is used as a template or background, and the image frame of the xth frame in the video template is inserted at the same moment (ie, the target moment) and the image of the user under the same scene model to form a user motion video, x is a positive integer.
  • the processing of the video template may be acquiring scene model information and character model information.
  • motion data (such as running stride, stride frequency, etc.) of the character model in the video template can also be obtained.
  • the video template processing module can store the data of the above-mentioned video template extracted from the video template into an information base for use in video synthesis.
  • the information of the scene model is information obtained by digitally processing the scene of each frame of the video template, including but not limited to scene angle, scene size, track slope and other information.
  • the information base may include one or more sub-bases. Different sub-libraries store different information.
  • the information library includes a video template library, a template scene library, a character model library, and a sports database.
  • the video template library is used to store video templates.
  • the scene library is used to store the scene information in the template.
  • the motion database is used to store the motion data of characters.
  • the character model library is used to store information related to the character model, such as storing the portrait information of the character and the information of the character model.
  • the character model information includes the character model information in the video template, and the character model information of the user whose motion video needs to be synthesized.
  • the sources of video templates include but are not limited to the following sources:
  • System preset the system can collect authorized sports videos from the network (such as sports videos of stars), and process the sports videos to obtain relevant information of sports videos (such as scene model information, sports data information, character model information) written into the repository.
  • the system can also obtain authorized motion data from mobile phones in the network, and generate corresponding video templates based on the motion data.
  • Video template uploaded by the user the user uploads his own sports video (optional, can also upload sports data), and the system writes various information into the information database after processing.
  • the user uploads his own motion data, and the system automatically generates a video template based on the motion data, and can process the video template and write various information into the information database.
  • Image data processing module it can be used to obtain the user's portrait data (or called image data), process the user's portrait data, and obtain a character model used to represent the user's appearance.
  • the character model can be, but is not limited to, a three-dimensional model.
  • the user's portrait data includes, but is not limited to, data that can characterize the user's physical appearance and other characteristics.
  • the format of the portrait data may be, but not limited to, text, images from various angles of the user, video, etc.
  • the image data (or user portrait data) processed by the image data processing module may be image data uploaded/selected/input by the user.
  • the image data processed by the image data processing module may be image data preset by the system.
  • Virtual character moving image generation module AI (artificial intelligence, AI) technology can be used to generate virtual characters (or character models) based on the user's portrait information (which can be input by the user or defaulted by the system), motion data, and the 3D model of the scene At the corresponding time point and the moving image under the corresponding scene model, the moving video of the virtual character with the video template as the background is finally synthesized.
  • AI artificial intelligence, AI
  • Motion video generation module the avatar motion image generated by the avatar motion image generation module can be inserted into the corresponding template image frame of the video template, in other words, the video template is used as the background of the avatar movement. In this way, a synthesized motion video can be obtained.
  • the motion video synthesis system may further include a video service module and a video playback and download module.
  • the video service module can be located on the cloud side (such as the server side), and is used to store the synthesized sports video and the fragments of the synthesized sports video, and provide functions such as online playback, download, and sharing.
  • Video playback and download module it can be located on the terminal side (such as the mobile phone side), and is used to download videos, play videos or play videos online.
  • End-side devices can be, but are not limited to, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) devices, notebook computers, ultra-mobile personal computers (ultra-mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA), etc., the embodiments of the present application do not impose any restrictions on the specific types of end-side devices.
  • FIG. 3A shows a schematic structural diagram of the end-side device 100 .
  • the end-side device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, And a subscriber identification module (subscriber identification module, SIM) card interface 195, etc.
  • SIM subscriber identification module
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit, NPU
  • the controller may be the nerve center and command center of the end-side device 100 .
  • the controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is a cache memory.
  • the memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • the processor 110 may be configured to obtain motion data of the user, and obtain motion videos of the user according to the motion data.
  • the charging management module 140 is configured to receive a charging input from a charger.
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives the input of the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the display screen 194 , the camera 193 and so on.
  • the wireless communication function of the end-side device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.
  • the mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the end-side device 100 .
  • the wireless communication module 160 can provide wireless communication including wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT) and so on applied on the terminal device 100. s solution.
  • wireless local area networks wireless local area networks, WLAN
  • Wi-Fi wireless fidelity
  • BT bluetooth
  • the end-side device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the display screen 194 includes a display panel.
  • the end-side device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • the end-side device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 , and the application processor.
  • the ISP is used for processing the data fed back by the camera 193 .
  • the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin color.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be located in the camera 193 .
  • the ISP can control the photosensitive element to expose and take pictures according to the shooting parameters.
  • Camera 193 is used to capture still images or video.
  • the object generates an optical image through the lens and projects it to the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other image signals.
  • the end-side device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • the camera 193 may be located in the edge area of the electronic device, and may be an under-screen camera, a liftable camera, or a hole-digging camera.
  • the camera 193 may include a rear camera, and may also include a front camera. The embodiment of the present application does not limit the specific position and shape of the camera 193 .
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the end-side device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the end-side device 100 may support one or more video codecs.
  • the end-side device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the end-side device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the end-side device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.
  • the internal memory 121 may be used to store computer-executable program codes including instructions.
  • the processor 110 executes various functional applications and data processing of the end-side device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the end-side device 100 may implement an audio function through an audio module 170 , a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio data into an analog audio electrical signal output, and is also used to convert an analog audio electrical signal input into digital audio data.
  • the audio module 170 may include an analog/digital converter and a digital/analog converter.
  • the audio module 170 is used to convert the analog audio electrical signal output by the microphone 170C into digital audio data.
  • the audio module 170 may also be used to encode and decode audio data.
  • the audio module 170 can be set in the processor 110, or some functional modules of the audio module 170 can be set in the processor 110.
  • Speaker 170A also called “horn" is used to convert analog audio electrical signals into sound signals.
  • the end-side device 100 can listen to music through the speaker 170A, or listen to hands-free calls.
  • Receiver 170B also called “earpiece” is used to convert analog audio electrical signals into sound signals.
  • the end-side device 100 answers a phone call or voice message, it can listen to the voice by bringing the receiver 170B close to the human ear.
  • the microphone 170C also called “microphone” or “microphone”, is used to convert sound signals into analog audio electrical signals.
  • the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C.
  • the microphone 170C may be a built-in component of the end-side device 100 , or may be an external accessory of the end-side device 100 .
  • the end-side device 100 may include one or more microphones 170C, where each microphone or multiple microphones cooperate to collect sound signals from various directions, and convert the collected sound signals into analog audio electrical signals It can also achieve noise reduction, identify the source of sound, or directional recording functions.
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.
  • the gyro sensor 180B may be used to determine the motion posture of the end-side device 100 .
  • the angular velocity of the end-side device 100 about three axes ie, x, y, and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the gyro sensor 180B detects the shake angle of the end-side device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shake of the end-side device 100 through reverse movement, realizing anti- shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • the distance sensor 180F is used to measure the distance.
  • the end-side device 100 can measure the distance by infrared or laser. In some embodiments, in a shooting scene, the end-side device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.
  • Touch sensor 180K also known as "touch panel”.
  • the touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the end-side device 100 .
  • the end-side device 100 may include more or fewer components than those shown in the illustration, or combine some components, or separate some components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.
  • the Android system with a layered architecture is taken as an example to illustrate the software structure of the device 100 on the terminal side.
  • FIG. 3B is a block diagram of the software structure of the end-side device 100 according to the embodiment of the present invention.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the operating system such as the Android system
  • the operating system of the electronic device is divided into four layers, which are respectively the kernel layer, the hardware abstract layer (hardware abstract layer, HAL), the application program framework layer, and the application program layer from bottom to top. .
  • the kernel layer is the layer between hardware and software.
  • the kernel layer includes at least camera driver, audio driver, display driver, and sensor driver.
  • Sensor drivers include but are not limited to image sensor drivers and acoustic sensors.
  • a kernel layer such as a sensor driver is invoked to enable a corresponding sensor to complete motion data collection.
  • the hardware abstract layer (hardware abstract layer, HAL) is located between the kernel layer and the application framework layer, and is used to define the interface that drives the hardware implementation of the application program, and converts the value implemented by the driving hardware into the software implementation program language. For example, identify the value driven by the camera, convert it into a software program language and upload it to the application framework layer, and then call the corresponding function.
  • the HAL can upload the motion data collected by the sensor to the application framework layer for further processing.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer obtains the original input event from the kernel layer via the HAL, and identifies the control corresponding to the input event.
  • the application framework layer includes some predefined functions.
  • the application framework layer may include a view system, a phone manager, a resource manager, a notification manager, a window manager, and the like.
  • the application framework layer includes a first module.
  • the first module can be used to obtain the user's motion data, and obtain the user's motion video according to the motion data.
  • the first module can also be arranged in other layers, and the first module can also be divided into more sub-modules. Each sub-module is used to perform a corresponding function.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
  • the view system can be used to build applications.
  • a display interface can consist of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide the communication function of the end-side device 100 .
  • the management of call status including connected, hung up, etc.).
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify the download completion, message reminder, etc.
  • the notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window.
  • prompting text information in the status bar issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • the application layer can consist of a series of application packages.
  • the application package can include applications such as camera, video, call, WLAN, music, short message, bluetooth, map, calendar, gallery, and navigation.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application program layer and the application program framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the structure of the cloud-side device 300 can also refer to the structure of the device-side device 100 in FIG. 3A , and the cloud-side device 300 can have more or fewer components than the structure shown in FIG. , or split some parts, or arrange different parts.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the cloud-side device 300 may include a processor 401 (optionally, a processor 408), a memory 403, a transceiver 404, and the like.
  • a path may be included between the above components, for transmitting information between the above components.
  • the transceiver 404 is used for communicating with other devices or communication networks through protocols such as Ethernet and WLAN.
  • the technical solutions of the embodiments of the present application can be applied to sports scenes, including but not limited to running, walking, rowing, swimming, and the like.
  • the user's movement trajectory at different moments can be obtained, and the image of the moving user can be inserted into the video template.
  • the video template includes some scenes (such as buildings, trees, etc.). In this way, a motion video of the user moving in the scene of the video template can be obtained.
  • the video template can also include sporting figures, then inserting a sporting user into the video template can obtain a video of the sporting user exercising with other sporting figures.
  • the user can collect the user's motion data through the sports bracelet and other equipment carried with him. These motion data can be obtained by the target application.
  • the target application can be set by the user.
  • the user can authorize sports applications such as sports health to obtain the user's exercise data.
  • the target application as an example including a sports application, the user can then use the sports application to synthesize sports data into a sports video.
  • the exercise application can invoke exercise data collected by sensors in devices such as wristbands, and record the collected exercise data in a storage space corresponding to the exercise application.
  • the exercise data records include: exercise data for running on May 11, exercise data for running on April 21, and the like.
  • interface 601 the user selects the exercise data such as by clicking on the exercise data on May 11, and in response to the user's operation of selecting the exercise data, the mobile phone presents an image such as that shown in Figure 5.
  • Interface 602 shown.
  • the user can intuitively observe his own running route.
  • the interface 602 may include a control 603 .
  • the mobile phone can generate a motion video corresponding to the motion data according to a motion video generation algorithm.
  • the mobile phone sends a request or notification to the server to generate a motion video, and the server generates a motion video corresponding to the motion data according to a motion video generation algorithm.
  • the mobile phone itself performs some steps of generating the motion video, and the server performs another part of the operation of generating the motion video. That is to say, motion video synthesis can be completed on terminals such as mobile phones on the device side, or can be completed on a server on the cloud side, or the motion video synthesis process can be completed through collaboration between the device and the cloud.
  • the mobile phone Take the motion video synthesized by the mobile phone as an example. For each frame of the motion video, the mobile phone calculates the movement, position and other information of the person according to the motion data, scene model information and image information of the corresponding time, generates the user's motion image, and The user's moving image is inserted into the corresponding frame in the video template. Taking the calculation of the user's position by the mobile phone based on the motion data as an example, the mobile phone can initialize the user's position in the first frame, and calculate the user's position in the subsequent frames based on the motion data in the motion data, such as stride, stride frequency, running speed, etc. Location.
  • the image information may not be collected synchronously with the motion data.
  • the image information may be uploaded/selected by the user, or the image information may be preset by the system.
  • there are other ways of collecting image information which are not limited in this embodiment of the present application.
  • the mobile phone initializes the initial position of the character in the first frame of the video template, and subsequently, the mobile phone can calculate according to parameters such as the initial position, the running speed in the motion data, and the scene model information in the video template.
  • the position of the person in the Xth frame of the video template and insert the person into the Xth frame of the video template to obtain the Xth frame of the motion synthesis video.
  • the mobile phone can calculate the position of the person in the Yth frame of the video template according to the parameters such as the initialization position and the running speed in the motion data, and insert the person into the Yth frame of the video template to obtain the Yth frame of the synthetic motion synthesis video .
  • the user can view the generated exercise video through the exercise application.
  • the user can view the generated motion video 605 through the interface 604 shown in (c) of FIG. 5 .
  • the interface 604 includes a video play control control 606 through which the user can control the play, pause, fast forward, fast rewind, etc. of the motion video 605 .
  • the user's motion video can be synthesized based on the motion data.
  • the implementation complexity of the embodiment of the present application is lower, and the user can use the function of synthesizing motion video more conveniently.
  • the solution of the embodiment of the present application can be applied when there is no cameraman to assist in taking pictures outdoors, and the user's motion video can be synthesized after the motion data is acquired.
  • the interface 602 shown in FIG. 5 may further include other controls.
  • a track animation generation control that generates a "track animation” could be included.
  • the mobile phone can generate a track animation (or track video) corresponding to the motion data based on the user's motion data.
  • the 3D character model can move along a certain trajectory.
  • the three-dimensional character model may be a character model constructed based on the user's image data (which may be system default or user uploaded/selected).
  • Figure 7 shows a trajectory animation generated from motion data.
  • the track video synthesized in Figure 7 does not need to use a video template.
  • the mobile phone can recommend places suitable for different sports along the trajectory such as shown in Figure 7 for the user. For example, recommendations are made to the user through an interface such as that shown in (a) or (b) of FIG. 7 .
  • the mobile phone can provide the user with an entry to edit the trajectory animation. For example, as shown in FIG. 7 , the user can move the position of the three-dimensional character model in the track by dragging or other operations.
  • the user's motion software and/or wearable device corresponds to the motion data collection and adaptation module shown in FIG. 2 .
  • the mobile phone may include modules such as a video template processing module and a motion video generation module shown in FIG. 2 , and the mobile phone completes motion video synthesis according to the collected motion data.
  • the server may include modules such as a video template processing module and a motion video generation module shown in FIG. 2 , and the server completes motion video synthesis.
  • the server may include some modules of the video synthesis subsystem
  • the mobile phone may include other modules of the video synthesis subsystem
  • the motion video synthesis is completed jointly by the terminal side (such as a mobile phone) and the cloud side (such as a server).
  • the motion video generation method of the embodiment of the present application does not need to collect motion data and image data at the same time. Therefore, when the user moves, it is not required to carry special equipment such as a camera, or it does not require a special person to assist in real-time image shooting, which reduces the usage.
  • the threshold of the function of synthesizing motion video has greatly expanded the applicable scenarios of generating motion video technical solutions.
  • a library of video templates may include one or more video templates.
  • Video templates can be different types of sports videos, including but not limited to running, swimming, rowing and other sports videos.
  • the video templates in the video template library can be added and updated automatically by the system, or can be added and updated by users.
  • FIG. 8 shows a process for a user to add a video template to the video template library.
  • the sports application may include a video template library interface
  • the video template library interface may include a control 801 .
  • the mobile phone may present an interface 802 shown in (b) of FIG. 8 .
  • the interface 802 may be a preview interface of a local photo album, that is, through the interface 802, some videos stored locally may be selected as video templates.
  • some videos on the network can be selected as video templates through the interface 802 .
  • the user can select or input the website www.xxx.com of the star's running video. Then, the mobile phone adds the running video of the URL into the video template library.
  • Interface 803 includes a video selected by the user.
  • the interface 803 also includes a clipping control 804 corresponding to the video selected by the user. The user can edit the video selected by the user by dragging the editing control 804, so as to capture a certain duration of the video as a video template.
  • the interface 803 may also include other controls, such as a control 805 for editing soundtrack shown in (c) of FIG. 8 .
  • the user can select or upload background music (background music, BGM) by operating the control 805 .
  • BGM background music
  • the mobile phone can also provide various setting options for BGM. For example, provide playback mode options so that users can choose the BGM playback mode (single-song cycle or multi-song traversal), provide sound effect options, and so on.
  • the mobile phone when the user's operation such as clicking the "OK to add" control is detected, the mobile phone will add the video selected by the user to the video template library, and can present a template such as that shown in (d) of Figure 8. display interface 804.
  • the interface 804 includes the video template 4 newly added by the user.
  • the user when uploading or adding a video template, can enter the motion data of the video template into the video template library.
  • the user can input the running stride, stride frequency, duration, etc. of the characters in the video template.
  • the system can analyze and calculate the motion data of the characters in the video template based on a motion video analysis algorithm. Subsequently, the motion data of the characters in the video template can be compared with the motion data of other users. The system can also display the motion data of each character to the user in the synthesized motion video.
  • the user can select a video template from the video template library, so that when the user instructs to generate a sports composite video, the system can insert moving figures into the video template selected by the user.
  • the movement of the character is simulated according to the movement data.
  • the mobile phone may image an interface as shown in (b) of FIG. 9A .
  • the interface includes controls 903 .
  • the mobile phone may present an interface 901 as shown in (c) of FIG. 9A .
  • Interface 901 may include controls 902 .
  • the mobile phone can determine that the user selects the video template 4 as the video template to be used for this composite video.
  • the mobile phone can automatically jump back to the interface shown in (b) of FIG. 9A from the interface 901 shown in (c) of FIG. 9A 908.
  • a control X that jumps back to the interface 908 may be set in the interface 901 , and the mobile phone jumps back to the interface 908 when an operation such as clicking on the control X by the user is detected.
  • the mobile phone inserts moving characters into the video template 4 to synthesize a moving video as shown in FIG. 9B.
  • the motion of the inserted character can be obtained from the motion data.
  • the running stride and stride frequency of the inserted character can be obtained based on the collected sports data.
  • the mobile phone presents the interface 908 shown in (b) of FIG.
  • the video template required for synthesizing the sports video or the mobile phone uses the video template used for the last synthetic sports video, or the mobile phone uses the recommended video template based on the authorized user portrait.
  • the mobile phone may also automatically select a video template required for synthesizing the motion video for the user based on other policies.
  • the interface 901 may further include a control 905 for the user to add a new video template.
  • the added new video template can be a mobile phone local video or a non-local network video.
  • some editing may be performed on the video template selected by the user according to the user's operation.
  • the video template 4 selected by the user as shown in (c) of FIG. 9A can be edited according to the user's operation instruction, and moving characters can be inserted into the edited video template 4 .
  • the default video template can be set by the system or the user. In this way, when the user instructs to generate a sports composite video, the system can insert the moving figure into the default video template.
  • the method for synthesizing motion video mainly includes collecting motion data of the user, and inserting the motion user into the video template based on the motion data, the video template, and the image information of the user.
  • collecting the user's motion data may be that the motion data collection and adaptation module collects motion data from the user, or the motion data collection and adaptation module obtains the user's motion data from third-party software and/or third-party sports auxiliary equipment.
  • collecting the motion data of the user may be by means of a device, module or device that has an interactive relationship with the user.
  • the user's motion data can be collected through sensors set on the user's wearable device, the user's mobile phone, and the fitness equipment used by the user. It is also possible to collect the user's motion data through the application installed on the user's mobile phone and other devices.
  • the embodiment of the present application does not limit the way of collecting motion data, and the user's motion data may be collected in various possible ways.
  • the motion data collected by different modules may have different formats.
  • the motion data of different types of motion can also have different formats.
  • Sports types include but are not limited to running, rowing, cycling, mountain climbing, etc.
  • a motion data collection and adaptation module can be set in the data collection subsystem of the embodiment of the present application.
  • the motion data collection and adaptation module can process different types of motion data and provide various types of Sports data to build a unified sports data model.
  • Fig. 11 shows an example of a motion data model.
  • a user's character model can also be constructed.
  • the character model is used to represent the user, and the character model is inserted into the video template to obtain the motion video of the user under the motion data.
  • the character model may be a three-dimensional model or other types of models.
  • the user can upload portrait information (or image information) that can indicate his own characteristics
  • the motion video synthesis system builds a character model based on the portrait information uploaded by the user.
  • the user's portrait information may be information in various forms. For example, various forms such as text, pictures, and videos can characterize user characteristics.
  • the text that characterizes user features includes, but is not limited to, the user's facial features, height, weight, age, clothing, etc.
  • the system builds a character model based on portrait information, and any character modeling algorithm can be used, which is not limited in this embodiment of the present application.
  • the profile information of the user is optional. Users can also not upload portrait information. In this case, the system specifies a default portrait for the user and builds a default character model.
  • the mobile phone when a user's preset operation (such as clicking) on the control 603 is detected, the mobile phone presents an interface 908 such as that shown in (b) of FIG. 9A , and the interface 908 includes the control 904 .
  • the mobile phone may present an interface such as that shown in (d) of FIG. 9A , and the interface may include one or more image data (such as image 1, image 2). Taking the user's selection of image 1 as the image data as an example, when the user's click operation on image 1 is detected, and the user clicks on the control 906, the mobile phone determines that the user has selected image 1 as the image data required for this composite video .
  • the mobile phone may automatically jump back to the interface 908 or, after detecting the user's operation on the interface 909 to jump to the interface 908, the mobile phone jumps to the interface 908.
  • the mobile phone can insert the character model corresponding to image 1 into the video template used in this synthetic video, so as to obtain the user's motion video such as shown in FIG. 9B.
  • the interface 909 shown in (d) of 9A may further include a control 907 for the user to add new image data.
  • the system can automatically collect the user's portrait information, and build the user's avatar model according to the portrait information.
  • the system may also use a default character model of the system to represent the user.
  • the default personas of the system are the same for different users.
  • the system's default character models can also be different.
  • the system automatically recommends different personas for different users according to their authorized user portraits. For example, for male users, the system recommends male character models for them, and for female users, the system recommends female character models for them.
  • the system may store information related to the character model in an information base.
  • Information related to the character model includes, but is not limited to, image data and information about the constructed character model.
  • FIG. 12 shows the interaction between the user (or the system administrator) and each module and each module during the process of adding character modeling information to the video template library by the user or the system administrator.
  • the video templates used for synthesizing motion videos may be automatically added to the video template library by the system, in other words, the video templates may be preset by the system.
  • the system can collect user portraits that can be authorized for use by the user, and determine the video templates that the user may use based on the user portraits.
  • the system can obtain network information, and recommend or add some video templates frequently used by other users to the user in the video template library. The system recommends video templates to users, which can also be combined with user portraits that users can authorize to use.
  • the video template may also be added into the video template library by the user or the system administrator.
  • FIG. 8 shows a set of interfaces in the process of adding a video template to the video template library by the user.
  • FIG. 15 shows the interaction between the user (or the system administrator) and each module and each module during the process of adding a video template to the video template library by the user or the system administrator.
  • Users or system administrators
  • Users can add video templates to the video template library through operations such as the interface shown in Figure 8.
  • the video template processing module After the video template processing module detects the operation of adding a video template by the user, it can process the video template to obtain the relevant information of the video template. information.
  • the video template processing module can extract the relevant information of any one or more video templates in the scene, characters, and motion data in the video template.
  • the video template processing module can also store the related information of the video template into the information base.
  • FIG. 13 shows the interaction between various modules during the process of generating a motion video of a character model (or virtual character).
  • it may be an exercise data acquisition adaptation module (such as an auxiliary device such as an exercise application and a sensor) that collects exercise data from the user.
  • the exercise data collection and adaptation module collects the user's exercise data from third-party exercise software or third-party exercise assistance equipment.
  • the sports data acquisition and adaptation module can perform formatting processing on the sports data, and build a unified sports data model for the sports data.
  • the virtual character motion image generation module can obtain the processed motion data from the motion data acquisition and adaptation module, obtain the information about the character model and the information related to the video template (such as the relevant information of the scene model) from the information base, and according to the motion data, the character
  • the information of the model and the relevant information of the video template generate a moving image of the avatar.
  • the motion data is mainly used to calculate the actions and positions of characters in each frame of the motion video.
  • the information of the character model is mainly used to simulate the characters in the motion video.
  • the relevant information of the video template can be used to calculate the viewing angle of each frame of the motion video, the position of the characters in each frame, the size of the characters in each frame, and so on.
  • the viewing angle of each frame of the motion video can be calculated by the relevant information of the video template.
  • the scene models are different, the angle of view of the frame picture, and the change of the size and position of the characters are shown.
  • the size of the model of the scene such as trees and roads
  • the size of the characters is also different.
  • the perspective of the scene changes, the perspective of the frame picture also changes, and the display perspective of the character model also changes accordingly.
  • FIG. 6 shows the moving images of the first frame, the Xth frame, and the Yth frame of the avatar generated by the system, and the background of each moving image is a transparent background.
  • the system can insert the image of the avatar into the corresponding image frame of the video template.
  • the video template can be used as the background of the avatar's movement.
  • Fig. 16 shows the interaction between various modules in the process of synthesizing motion video.
  • the motion video synthesis method of the embodiment of the present application is described above by taking the synthesized motion video of a single person as an example.
  • motion videos including multiple people can also be synthesized.
  • the moving user may be synthesized into motion videos (video templates) of other characters to form a video of the motion user moving with other characters.
  • the exercise type of the exercising user is the same as the exercise type of the characters in the video template.
  • the exercise type of the exercising user is running, and the exercise type of the characters in the video template is also running.
  • the system can collect motion data of the user, and generate frames of moving images of the avatar according to the motion data (only one frame of which is shown in FIG. 17 exemplarily).
  • the frame motion image is inserted into the corresponding template image frame of the video template A to obtain a motion video of the user moving with other characters.
  • the video template A may be a sports video of a star, or a sports video of other characters (such as a friend).
  • a sports video of other characters such as a friend.
  • the following describes an example of synthesizing a sports video including multiple people (which may be called a multi-person mode sports video) in several scenarios.
  • the synthesized sports video may be a video of the user exercising with a celebrity.
  • the video template can be a celebrity's sports video, and the user can select a certain celebrity's sports video, and the system can insert the user's sports video into the celebrity's sports video, and synthesize the user and the celebrity to exercise together (such as game) sports video.
  • FIG. 18A and FIG. 18B show some interfaces in the process of synthesizing video.
  • Fig. 19 is an example of editing and synthesizing a sports video in an editing application program. For the specific implementation of each interface and the corresponding user operations and mobile phone responses, please refer to the above-mentioned embodiments, which will not be repeated here.
  • FIG. 19 shows the interaction between various modules in the process of synthesizing a video in which the user and the star move together.
  • the star's sports video can be uploaded to the system by the star himself, for example, the star uploads to the server through an editing application. It may also be an authorized sports video obtained by the server in the embodiment of the present application from other servers.
  • the synthesized sports video may be a video of the user exercising with friends. Taking friend B whose video template is the user as an example, user B can complete motion data entry and image information entry in the system according to the method of the above-mentioned embodiment. Wherein, the entry of image information is an optional step.
  • User A can select user B's sports video as a video template, and the system can insert user A's sports video into user B's sports video to synthesize the sports video of user A and user B's competition.
  • FIG. 20A and FIG. 20B show some interfaces in the process of the system synthesizing the game video between user A and user B.
  • Scene three the synthetic motion video is the motion video of the team (team)
  • the embodiment of the present application can synthesize asynchronous team sports video.
  • the asynchronous line means that the user is not required to exercise while synthesizing the video, or the sports video is not synthesized while collecting sports data, or the sports data of multiple users is acquired asynchronously. of. Users can exercise first, and then upload their own exercise data, so that the system can synthesize exercise videos based on the exercise data.
  • the mobile phone presents an editing application interface as shown in (a) of FIG. 21A , and upon detection of a user's click operation on the control 1901, the mobile phone may present an interface such as that shown in (b) of FIG. 21A .
  • the interface includes control 2101 and control 2102 .
  • control 2101 can be used to set the relevant information of synthesizing the team sports video
  • control 2102 can be used to set the relevant information of synthesizing the individual sports video (such as the single sports video shown in (c) of FIG. 5 ).
  • the interface shown in (b) of FIG. 21A may also include other controls or delete certain controls.
  • the mobile phone When detecting the user's click operation on the control 2101, the mobile phone may present the interface 2108 shown in (c) of FIG. 21A.
  • the interface 2108 includes controls 2104-2107. Wherein, for the control 2104 and the control 2105, reference may be made to the relevant descriptions of the control 903 and the control 904 in the foregoing embodiment.
  • Controls 2106 may be used to set parameters such as the size of the team.
  • Control 2107 may be used to set the course of motion for the team sport.
  • the interface 2108 may also include other controls for setting other parameters of the team sports video.
  • the mobile phone When detecting the user's click operation on the control 2106, the mobile phone may present the interface shown in (d) of FIG. 21A.
  • a team can be selected through the interface shown in (d) of FIG. 21A.
  • you can also set the number of people in the team, set the upper limit of the number of people in the team, and so on. If the upper limit is set for the number of people in the team, when the number of people who choose to join the team reaches the upper limit, the system will no longer allow other users to join the team.
  • the type of team sports can also be set through the interface shown in (d) of FIG. 21A , for example, the type is set to running competition.
  • friends can also be invited to participate in recording the sports video of the team.
  • other team-related parameters can also be set through this interface.
  • the user can select a team and view the authorized personal information of the team members.
  • users can choose their preferred team according to the team member information of different teams, and use the system to synthesize the sports video of the team.
  • users can choose a team composed of some users with top sports rankings, join the team, and use the system to synthesize the team's sports videos to enhance the user's sports fun and play a role in urging users to exercise.
  • user A can also choose the video template used to synthesize the team sports video (or use the system default video template). For example, user A can select a video template through the control 2104 shown in (c) of FIG. 21A.
  • User A can also set his own image data in the team sports video. For example, user A can upload his own image data (or use system default image data) through the control 2105 shown in (c) of FIG. 21A .
  • User A can also set his own exercise data. For example, the user uploads his exercise data through the control 2109 shown in (c) of FIG. 21A , or authorizes the acquisition of the exercise data stored in the mobile phone through the control 2109 .
  • user B and user C can also select their desired video templates (or use a default video template).
  • User B and user C can also set their respective image data in the team sports video. Users B and C can also set their own exercise data.
  • team members upload their own exercise data individually, and there are no restrictions on the upload time and exercise location between each other. They can complete exercise at different times and locations, or complete exercises at the same time and location, and upload them offline motion data.
  • the system When synthesizing the sports video of the team, the system generates their own sports videos according to the respective sports data and image data of users A, B, and C, and inserts the three sports videos into the video template to obtain the sports video of the team.
  • FIG. 22 shows the process of synthesizing team sports videos.
  • the moving images of the multiple avatars may be synthesized into corresponding template image frames of the video template in a certain order.
  • FIG. 21B shows team sports videos displayed by mobile phones A and B of users A and B respectively.
  • the backgrounds of the team sports videos displayed by mobile phones A and B are different.
  • the background of the team sports videos displayed by mobile phone A is video template 1
  • the background of the team sports videos displayed by mobile phone B is video template 2.
  • the team organizer can create tasks and specify certain parameters of the synthesized sports videos, such as specifying video templates, motion types, duration and other requirements.
  • the motion videos synthesized by the system for different users are usually consistent.
  • sports videos composed for different team members all have the same background.
  • part of the parameters may be specified by the team organizer, and another part of the parameters may be specified by the team members.
  • the team organizer specifies the type of exercise as running, and the duration is required to be one hour. Team members specify their own video templates and upload their own exercise data. Then, the background of the motion video synthesized by the system for different members may be different.
  • FIG. 23 shows the interaction between modules in the process of synthesizing team sports videos. It should be noted that, limited to the size of the drawings, Figure 23 only exemplarily shows the technical details of the motion data collection and adaptation module collecting motion data from the user through third-party software or sports assistance equipment, but does not show the motion data collection The adaptation module directly collects the technical details of motion data from the user.
  • multiple users can complete the exercise separately in an asynchronous manner, that is, complete the exercise at different times and different places, that is to say, users do not have to exercise at the same time, for any user, it can be at a convenient time, Location to complete the campaign.
  • the system can synthesize multiple sports videos of multiple users who are moving at different times and in different places into a sports video of a team moving together, which provides convenience for users to use the function of synthesizing sports videos.
  • Scene 4 The synthetic motion video is the motion video of the team (team)
  • Scenario 4 team members choose a unified movement route. For example, based on the map, select the unified map track. For example, choose the outer edge of XX Park in Beijing as the unified exercise route.
  • the system collects the sports data of each user along the route, and can generate their own sports videos according to the sports data, and then insert their respective sports videos into the video template to obtain the team's sports videos.
  • the synthesized various sports videos can be uploaded, shared, etc. under the authorization of the user.
  • they can be watched within the team, or can be uploaded or shared on the Internet with the authorization of team members.
  • the user may choose to synthesize a sports video in the interface shown in (a) of FIG. 24 .
  • the mobile phone may display an interface such as that shown in (c) of FIG. 24 .
  • the interface includes an online mode control 2401 and an offline mode control 2402 .
  • the mobile phone may display an interface 2404 such as that shown in (d) of FIG. 24 .
  • the user can set options related to synthesizing motion video through the controls in the interface 2404 .
  • image information can be selected through the control 2105
  • a video template can be selected through the control 2104 .
  • the user can also turn on the switch 2403 for collecting exercise data, so that the mobile phone can report the user's exercise data in real time during the process of collecting exercise data.
  • the system can generate the moving image of the avatar in real time, and combine the moving image of the avatar into the corresponding template image frame of the video template in real time.
  • the server executes the step of synthesizing sports videos as an example, and the server may deliver the synthesized team sports videos to the terminal through streaming media.
  • the user pulls the stream in real time to obtain the latest video motion stream through the video playback and download module on the terminal (such as a mobile phone). Users can watch the synthesized sports video on the terminal side.
  • the server may store the segments of the motion video generated in real time.
  • the system can synchronously acquire the motion data of multiple users and synthesize the motion videos of multiple users in the scheme of synthesizing motion video online.
  • the system synthesizes the user's exercise video based on the exercise data during the user's exercise.
  • the system synthesizes the user's motion video at other possible timings, and this embodiment of the present application does not limit the timing of the system generating the motion video.
  • the community organizer can create a task, specify a video template, and one or more parameters in the sports type, duration, and start time.
  • Members of each club can open the sports application according to the specified time, and perform the same mode of sports in different places.
  • the exercise application of each member will automatically collect the exercise data of each member, and the system can complete the synthesis of exercise videos based on the exercise data of each member.
  • the technical solution of the embodiment of the present application can support the collection of the user's motion data from various software and/or hardware through the motion data collection and adaptation module, and can complete the video without proprietary equipment. Synthesis, which can lower the technical threshold of synthesizing motion video.
  • the user can set related options for synthesizing motion video on a terminal such as a mobile phone. For example, select a video template, upload portrait data, etc. through a set of interfaces such as shown in FIG. 24 . Users can also choose to display the synthesized exercise video in real time on the display of a device with a screen such as a treadmill. Exemplarily, as shown in Figure 25, the user is running on a treadmill, and the user can choose to display the synthesized exercise video on the screen of the treadmill through a terminal such as a mobile phone. In this way, the user can watch teammates running while exercising Situation, increase the fun of sports.
  • fitness equipment such as treadmills can also simulate road conditions in sports videos. For example, when there is a steep slope in the sports video, the treadmill can create a sense of bumps through vibration and other methods. Treadmills can also warn users of upcoming steep inclines before vibrating to remind users to get ready.
  • the user's motion data may be marked in the synthesized motion video.
  • the synthesized team sports video is marked with the sports data of each team member. These exercise data can represent the user's exercise state.
  • the motion data of the template character can also be marked in the synthesized motion video.
  • the template character refers to the character in the video template.
  • template characters ie, star motion data
  • the motion data of the user that is, the black character model
  • the synthesized motion video can also be marked in the synthesized motion video.
  • the motion video synthesis system such as that shown in FIG. 2 may also include more or fewer modules, or combine some modules, or split some modules. Alternatively, certain modules can be replaced with other modules of similar functionality.
  • the embodiment of the present application does not limit the specific architecture of the system.
  • video templates, user portrait data, etc. may be set by the system by default, or may be selected or uploaded by the user.
  • the embodiment of the present application does not limit the timing of uploading or selecting video templates and user portrait data.
  • the user can select/upload a video template when synthesizing a motion video, or pre-select/upload a video template.
  • a corresponding exercise type may be determined for the exercise data.
  • the exercise data corresponding to the first exercise type includes a trajectory.
  • the exercise data corresponding to the second exercise type does not include a trajectory.
  • the first exercise type includes but is not limited to outdoor running, swimming, walking, cycling, and football. Movements of the first movement type may generally form a movement trajectory.
  • the second type of exercise includes but is not limited to running on a treadmill. The second type of exercise usually has no change in physical position, or the change in physical position is small, and it is considered that no movement track has been formed.
  • generating the exercise video of the one or more users according to the exercise data and the data of the video template may be implemented as: according to the trajectory in the exercise data corresponding to the first exercise type, and the data of the video template to generate motion videos of the one or more users.
  • the motion data shown in (b) of FIG. 5 is the motion data of the user running along xx park
  • the system can generate a motion video according to the motion track in the motion data and the data of the video template.
  • generating the exercise video of the one or more users according to the exercise data and the data of the video template may be implemented as: according to the exercise data corresponding to the second exercise type and the route data, and generate a motion video of the one or more users moving along the route indicated by the route data.
  • the user is running on a treadmill, and the system can generate a motion video of the user moving along the route indicated by the route data (such as the road of the video template in FIG. 25 ) according to the route data of the video template.
  • Figure 26 shows an exemplary process of the video generation method of the embodiment of the present application, the method includes:
  • the data of the person motion video or the data of the scene video may be, but not limited to, a motion video of a character such as a celebrity.
  • the character movement video may be the video shown in FIG. 21B.
  • the scene video may be the video template shown in FIG. 6 .
  • the character motion video may be a motion video synthesized by using the video generation method of the embodiment of the present application, or an ordinary motion video, that is, a motion video not synthesized by the video generation method of the embodiment of the present application.
  • the mobile phone selects the video template 4 shown in (c) of FIG. 9A as a template from the video template library.
  • steps S2601-S2603 may be performed by the motion video synthesis system shown in FIG. 2 .
  • the modules in the system may consist of end-side devices (such as terminals such as mobile phones), or modules in the end-side devices. It can also be a cloud side (such as a server, etc.) or a cloud side module.
  • step S2601 is executed by the video template processing module shown in FIG. 2 .
  • S2602. Collect motion data of one or more users.
  • the motion data includes any one or more of the following data: speed, amplitude, frequency, trajectory.
  • the exercise data may also include any one or more of the following data: heart rate, blood pressure, blood oxygen, body temperature.
  • the amplitude can refer to the stride length
  • the frequency can refer to the stride frequency.
  • the mobile phone collects motion data as shown in (a) of FIG. 9A .
  • step S2602 is executed by one or more modules shown in FIG. 2 : exercise data collection and adaptation module, exercise software module, and auxiliary exercise equipment (hardware module).
  • the system generates the motion video of the user shown in FIG. 9B according to the motion data shown in (a) of FIG. 9A and the video template 4 shown in (c) of FIG. 9A .
  • step S2603 is executed by the motion video generation module of the video synthesis subsystem shown in FIG. 2 .
  • the motion video generation module can be implemented as: generating the user's motion video according to the data of the video template and the character motion image generated by the avatar motion image generation module.
  • the apparatus may be the above-mentioned electronic device (such as a folding screen mobile phone).
  • the apparatus may include: a display screen, memory and one or more processors.
  • the display screen, memory and processor are coupled.
  • the memory is used to store computer program code comprising computer instructions.
  • the processor executes the computer instructions, the electronic device can execute various functions or steps performed by the mobile phone in the foregoing method embodiments.
  • FIG. 3A or FIG. 4 For the structure of the electronic device, reference may be made to the electronic device shown in FIG. 3A or FIG. 4 .
  • the core structure of the electronic device may be represented as the structure shown in FIG. 27 , and the core structure may include: a processing module 1301 , an input module 1302 , a storage module 1303 , and a display module 1304 .
  • the processing module 1301 may include at least one of a central processing unit (CPU), an application processor (Application Processor, AP) or a communication processor (Communication Processor, CP).
  • the processing module 1301 may perform operations or data processing related to control and/or communication of at least one of other elements of the user electronic device.
  • the processing module 1301 can be configured to control the content displayed on the main screen according to a certain trigger condition. Or determine what is displayed on the screen according to preset rules.
  • the processing module 1301 is also used to process the input instruction or data, and determine the display style according to the processed data.
  • the processing module 1301 may have the functions of one or more modules in the system shown in FIG. 2 , such as the motion video generation module, video template processing module, image data processing module, and motion data collection video module.
  • the input module 1302 is configured to obtain instructions or data input by the user, and transmit the obtained instructions or data to other modules of the electronic device.
  • the input mode of the input module 1302 may include touch, gesture, approaching the screen, etc., and may also be voice input.
  • the input module may be a screen of an electronic device, acquires user input operations, generates input signals according to the acquired input operations, and transmits the input signals to the processing module 1301 .
  • the input module may be used to receive image data, motion data, etc. input by the user, and/or perform other steps.
  • the storage module 1303 may include a volatile memory and/or a nonvolatile memory.
  • the storage module is used to store at least one related instruction or data in other modules of the user terminal device. Specifically, the storage module can record the position of the interface where the terminal interface element UI is located.
  • the display module 1304 may include, for example, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic Light Emitting Diode (OLED) display, a Micro Electro Mechanical System (MEMS) display or an electronic paper display. Used to display user-viewable content (eg, text, images, videos, icons, symbols, etc.).
  • the display module may be implemented as a display screen, such as a first screen and/or a second screen.
  • the structure shown in FIG. 27 can also have a communication module 1305, which is used to support the electronic device to communicate with other electronic devices.
  • the communication module can be connected to a network via wireless communication or wired communication to communicate with other personal terminals or a network server.
  • the wireless communication may employ at least one of cellular communication protocols, such as Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Universal Mobile Communications System (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM).
  • Wireless communications may include, for example, short-range communications.
  • the short-range communication may include at least one of wireless fidelity (Wi-Fi), Bluetooth, near field communication (NFC), magnetic stripe transmission (MST), or GNSS.
  • the chip system includes at least one processor 1401 and at least one interface circuit 1402 .
  • the processor 1401 and the interface circuit 1402 may be interconnected through wires.
  • interface circuit 1402 may be used to receive signals from other devices, such as memory of an electronic device.
  • the interface circuit 1402 may be used to send signals to other devices (such as the processor 1401).
  • the interface circuit 1402 can read instructions stored in the memory, and send the instructions to the processor 1401 .
  • the electronic device may be made to execute various steps in the foregoing embodiments.
  • the chip system may also include other discrete devices, which is not specifically limited in this embodiment of the present application.
  • the embodiment of the present application also provides a computer storage medium, the computer storage medium includes computer instructions, and when the computer instructions are run on the above-mentioned electronic device, the electronic device is made to perform the various functions or steps performed by the mobile phone in the above-mentioned method embodiment .
  • the embodiment of the present application also provides a computer program product, which, when the computer program product is run on a computer, causes the computer to execute each function or step performed by the mobile phone in the method embodiment above.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation.
  • multiple units or components can be Incorporation or may be integrated into another device, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请提供视频生成方法及电子设备；涉及通信技术领域，在不借助专业数据采集设备的场景中能够生成运动视频，降低了生成视频过程的复杂度。该方法包括：获取视频模板的数据，采集一个或多个用户的运动数据，并根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频。所述运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹。

Description

视频生成方法及电子设备
本申请要求于2021年07月31日提交国家知识产权局、申请号为202110877302.7、发明名称为“视频生成方法及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及电子技术领域，尤其涉及视频生成方法及电子设备。
背景技术
目前，在一些诸如健身锻炼，游戏等运动场景中，系统或者终端软件可以实时采集用户的视频图像以及捕捉与视频图像同步的动作数据，生成用户的模拟仿真类视频。图1示出了目前游戏场景中合成运动类视频的一种方案。该方案中，安装在诸如智慧屏上的软件(比如dance center、just dance)可以基于智慧屏的摄像头，实时采集用户的图像，然后根据采集的图像生成用户对应的舞蹈视频，模拟用户的动作，用户可以在线观看智慧屏中自己的实时舞蹈视频。
该方案中，需要采集大量图像，因此，通常需要摄像头、专用的动作捕捉设备等采集设备，对采集设备的种类要求较高。
发明内容
本申请提供的视频生成方法及电子设备，在不借助专业数据采集设备的场景中能够生成运动视频，降低了生成视频过程的复杂度。
为达到上述目的，本申请采用如下技术方案：
第一方面，本申请提供一种视频生成方法，该方法包括：获取视频模板的数据，采集一个或多个用户的运动数据，并根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频。运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹。通过本申请实施例的技术方案，运动数据与图像信息解耦，即图像信息和运动数据的采集过程可以是分离的。这样一来，可以异步采集图像信息和运动数据，进而能够降低数据采集设备的实现复杂度。在无需专有辅助设备(比如摄像头)或专人辅助采集数据(比如专人辅助拍摄采集图像)的情况下，仍可以根据运动数据和视频模板生成运动类视频。
在一种可能的设计中，视频模板的数据为人物运动视频的数据或场景视频的数据。
在一种可能的设计中，视频模板可以是预设的，或用户输入的。
可选的，人物运动视频比如可以但不限于为明星、好友等人物的运动视频。可选的，人物运动视频可以是采用本申请实施例的视频生成方法合成的运动视频，或者，普通的运动视频，即没有采用本申请实施例的视频生成方法合成的运动视频。
在一种可能的设计中，运动数据具有对应的第一运动类型，第一运动类型对应的运动数据包括轨迹；
根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频，包括：
根据第一运动类型对应的运动数据中的轨迹，以及视频模板的数据，生成一个或 多个用户的运动视频。
如此一来，能够生成一个或多个用户的沿相应轨迹运动的运动视频，增加运动视频的趣味性。
在一种可能的设计中，运动数据具有对应的第二运动类型；视频模板的数据包括路线数据；
根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频，包括：
根据第二运动类型对应的运动数据以及路线数据，生成一个或多个用户沿路线数据指示的路线运动的运动视频。
通过该方案，在一些没有运动轨迹的运动类型(比如在跑步机上跑步)中，或不容易获取运动轨迹的运动类型中，或一些其他场景中，能够根据视频模板的路线数据，生成一个或多个用户沿路线数据所指示的路线运动的运动数据，能够增加运动视频的趣味性。并且，在运动视频为多个用户的运动视频的情况下，某个用户可以从运动视频了解自己与他人的运动情况，如此，能够对自身运动健身进行指导。
可选的，路线数据可以是用户设置选择的，或系统默认设置的。
在一种可能的设计中，采集一个或多个用户的运动数据，包括：从目标软件模块或目标硬件模块采集一个或多个用户的运动数据。通过该方案，能够对接不同厂家，不同型号的硬件模块(比如辅助运动设备)或者软件模块(比如辅助运动软件)，采集各种格式的运动数据。可见，由于能够适配不同厂家、型号的软件或硬件，则无需额外的专业设备，即可完成运动数据采集。
在一种可能的设计中，采集一个或多个用户的运动数据，包括：接收一个或多个用户输入的运动数据。
方法还包括：获取一个或多个用户的图像的信息；其中，图像的信息是预设图像的信息，或者，一个或多个用户的图像的信息是一个或多个用户输入的图像的信息。本申请实施例中，用户的图像的信息可以是系统预置的，或用户上传的。无需在采集运动数据时同步获取用户的图像的信息。因此，适用于更多的运动场景，比如，适用于不方便一边健身一边拍摄健身图像的运动场景。
在一种可能的设计中，对于一个或多个用户中的任一个用户，异步获取用户的图像的信息与用户的运动数据。
在一种可能的设计中，一个或多个用户的图像的信息包括如下任一项或多项的组合：一个或多个用户的图像、用于描述一个或多个用户的特征的文本、一个或多个用户的视频。本申请实施例中，用户的图像的信息不再局限于用户的实时视频，而是可以用来描述用户特征，或者用户画像的任何信息。由于扩宽了图像信息的范围，相应的，采集图像信息的方式也可以有多种，不再局限于与运动数据同步采集，也不再局限于通过摄像头实时拍摄采集图像信息。
在一种可能的设计中，一个或多个用户的运动视频包括多个图像帧，多个图像帧的第X个图像帧由一个或多个用户在目标时刻的图像插入视频模板的第X个图像帧得到；目标时刻是第X个图像帧对应的时刻；X为正整数。
在一种可能的设计中，多个用户的运动数据是同步获取的。
在一种可能的设计中，多个用户的运动数据是异步获取的。通过该方法，多个用 户可以通过异步方式分别完成运动，即在不同时间和不同地点完成运动，也就是说，用户不必在同一时间运动，对于任一用户来说，其可以在便利的时间、地点完成运动。系统可以将在不同时间、不同地点运动的多个用户的多个运动视频合成团队一起运动的运动视频，为用户使用合成运动视频功能提供了便利。
在一种可能的设计中，运动数据还包括如下任一项或多项数据：心率、血压、血氧、体温。
第二方面，本申请提供一种视频生成系统，包括：
视频模板处理模块，用于获取视频模板的数据；
运动数据采集适配模块，用于采集一个或多个用户的运动数据；
运动视频生成模块，用于根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频。运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹。
在一种可能的设计中，视频模板的数据为人物运动视频的数据或场景视频的数据。
在一种可能的设计中，运动数据具有对应的第一运动类型，第一运动类型对应的运动数据包括轨迹；
运动视频生成模块，用于根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频，包括：
运动视频生成模块，用于根据第一运动类型对应的运动数据中的轨迹，以及视频模板的数据，生成一个或多个用户的运动视频。
在一种可能的设计中，运动数据具有对应的第二运动类型；视频模板的数据包括路线数据；
运动视频生成模块，用于根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频，包括：
根据第二运动类型对应的运动数据以及路线数据，生成一个或多个用户沿路线数据指示的路线运动的运动视频。
在一种可能的设计中，运动数据采集适配模块，用于采集一个或多个用户的运动数据，包括：从目标软件模块或目标硬件模块采集一个或多个用户的运动数据。
在一种可能的设计中，运动数据采集适配模块，用于采集一个或多个用户的运动数据，包括：接收一个或多个用户输入的运动数据。
系统包括：图像数据处理模块，用于获取一个或多个用户的图像的信息；其中，图像的信息是预设图像的信息，或者，一个或多个用户的图像的信息是一个或多个用户输入的图像的信息。
在一种可能的设计中，对于一个或多个用户中的任一个用户，用户的图像的信息与用户的运动数据是异步获取的。
在一种可能的设计中，一个或多个用户的图像的信息包括如下任一项或多项的组合：一个或多个用户的图像、用于描述一个或多个用户的特征的文本、一个或多个用户的视频。
在一种可能的设计中，一个或多个用户的运动视频包括多个图像帧，多个图像帧的第X个图像帧由一个或多个用户在目标时刻的图像插入视频模板的第X个图像帧得到；目标时刻是第X个图像帧对应的时刻；X为正整数。
在一种可能的设计中，多个用户的运动数据是同步获取的。在一种可能的设计中，多个用户的运动数据是异步获取的。
在一种可能的设计中，运动数据还包括如下任一项或多项数据：心率、血压、血氧、体温。
第三方面，本申请提供一种电子设备，该电子设备具有实现如上述第一方面及其中任一种可能的实现方式中的视频生成方法的功能。该功能可以通过硬件实现，也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的模块。
第四方面，本申请提供一种计算机可读存储介质，包括计算机指令，当计算机指令在电子设备上运行时，使得电子设备执行如第一方面及其中任一种可能的实现方式中任一项的视频生成方法。
第五方面，本申请提供一种计算机程序产品，当计算机程序产品在电子设备上运行时，使得电子设备执行如第一方面及其中任一种可能的实现方式中任一项的视频生成方法。
第六方面，提供一种电路系统，电路系统包括处理电路，处理电路被配置为执行如上述第一方面及其中任一种可能的实现方式中的视频生成方法。
第七方面，本申请实施例提供一种芯片系统，包括至少一个处理器和至少一个接口电路，至少一个接口电路用于执行收发功能，并将指令发送给至少一个处理器，当至少一个处理器执行指令时，至少一个处理器执行如上述第一方面及其中任一种可能的实现方式中所述的视频生成方法。
附图说明
图1为现有技术中生成运动视频的流程图；
图2为本申请实施例提供的运动视频合成系统的架构示意图；
图3A为本申请实施例提供的电子设备的结构示意图；
图3B为本申请实施例提供的电子设备的软件架构示意图；
图4为本申请实施例提供的电子设备的另一结构示意图；
图5为本申请实施例提供的一组界面示意图；
图6为本申请实施例提供的视频生成方法的流程示意图；
图7-图9B为本申请实施例提供的界面示意图；
图10为本申请实施例提供的视频生成方法的流程示意图；
图11为本申请实施例提供的运动数据模型的示意图；
图12-图14B为本申请实施例提供的视频生成方法的流程示意图；
图15-图17为本申请实施例提供的视频生成方法的流程示意图；
图18A-图18B为本申请实施例提供的界面示意图；
图19为本申请实施例提供的视频生成方法的流程示意图；
图20A、图20B、图21A、图21B为本申请实施例提供的界面示意图；
图22为本申请实施例提供的界面示意图；
图23为本申请实施例提供的视频生成方法的流程示意图；
图24为本申请实施例提供的界面示意图；
图25、图26为本申请实施例提供的视频生成方法的流程示意图；
图27为本申请实施例提供的装置的示意图；
图28为本申请实施例提供的芯片系统的示意图。
具体实施方式
本申请实施例提供一种视频生成方法，能够获取视频模板的数据，采集一个或多个用户的运动数据，并根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频。其中，所述运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹。与现有技术中，人物的图像信息和运动数据是同步采集，采集难度大相比，本申请实施例的视频生成方法，运动数据与图像信息解耦，即图像信息和运动数据的采集过程可以是分离的。这样一来，可以异步采集图像信息和运动数据，进而能够降低数据采集设备的实现复杂度。在无需专有辅助设备(比如摄像头)或专人辅助采集数据(比如专人辅助拍摄采集图像)的情况下，仍可以根据运动数据和视频模板生成运动类视频。
下面结合附图对本申请实施例提供的视频生成方法及电子设备进行详细地描述。
本申请实施例提供的视频生成方法可以应用于运动视频合成系统中，图2示出了运动视频合成系统的示例性架构。该系统包括数据采集子系统、视频合成子系统。其中，数据采集子系统包括运动数据采集适配模块。视频合成子系统包括视频模板处理模块、信息库(比如可包括视频模板库、模板场景库、人物模型库、运动数据库)、虚拟人物(又可称人物模型)运动图像生成模块以及运动视频生成模块。
运动数据采集适配模块：可用来采集用户的运动数据，针对采集的运动数据，运动数据采集适配模块可以按照运动类型(比如跑步，游泳，划船等)为不同运动类型的运动数据构建相应模型。对于某类运动类型的运动数据来说，运动数据模型中可包括如下一项或多项信息：该运动类型下汇总的运动数据，运动数据划分得到的时间分片数据。示例性的，图11示出了跑步运动类型下的运动数据模型。其中，跑步模式下的运动数据包括时间分片数据1-分片数据n。不同时间分片对应的跑步时长可以相同或不同。以运动数据为一个小时的跑步数据为例，一个小时的跑步数据可划分为5个时间分片数据，每个时间分片数据对应12分钟的跑步时长。
运动数据采集适配模块可以以软件实现，可以以硬件实现，或者采用软件和硬件结合方式实现。示例性的，以运动数据采集适配模块闻手机中的模块为例，运动数据采集适配模块可以包括软件模块以及传感器，软件模块可以通过驱动调用传感器采集用户的运动数据，之后，软件模块可以对运动数据进行诸如格式化处理，为运动数据构建模型。
在一些实施例中，运动数据采集适配模块可接收用户输入的运动数据。比如，可以通过运动软件检测用户上传的运动数据。
在另一些实施例中，可选的，本申请实施例的运动视频合成系统还可包括第三方的运动软件和/或第三方的辅助运动设备。运动数据采集适配模块可以从第三 方的运动软件、第三方辅助运动设备获取用户的运动数据。运动软件可以但不限于是系统级应用或者用户级应用。系统级别应用可以指终端预装的应用，比如，运动健康应用。用户级应用可以指后续用户自行安装的应用，比如通过应用市场下载安装的运动应用。辅助运动设备包括但不限于运动手环、手表、运动器材(比如跑步机、划船机、椭圆机、动感单车)。运动数据采集适配模块可以基于从上述第三方运动软件或第三方运动辅助设备获取的运动数据，对这些运动数据进行诸如格式的转换处理，并得到相应运动类型下的运动数据模型，以便使得不同格式的运动数据能够适配后续的处理流程。
本申请实施例中，通过扩展运动数据采集适配模块的功能，可以适配各类软件模块或者硬件模块(运动辅助设备)的接口，从各类软件模块或者硬件模块采集运动数据，从而能够扩宽运动数据的来源范围。在一些方案中，不仅可以在室内场景采集用户的运动数据，也可以是室外场景采集用户的运动数据。
本申请实施例中，用户的运动场景包括但不限于跑步，行走，骑行，游泳，足球，划船，篮球，瑜伽等。在运动场景中，运动数据采集适配模块采集的用户运动数据包括但不限于运动类型、运动距离、步数、运动时长、运动强度、运动轨迹、运动姿态、生理参数。生理参数包括但不限于心率、血压、血氧、体温、呼吸率、生物电阻抗。运动类型包括但不限于跑步、步行、骑行、健身。
需要说明的是，运动数据采集适配模块采集用户的运动数据，可以是运动数据采集适配模块采集用户授权获取的运动数据。运动数据采集适配模块对运动数据进行格式化处理，构建运动数据模型后，可以将运动数据模型的相关信息写入信息库，运动数据模型的相关信息可以作为虚拟人物运动图像的生成来源。
视频模板处理模块:用于获取视频模板，并对视频模板进行加工处理，得到视频模板的数据。可选的，视频模板可以是视频格式，或者也可以是其他格式，比如可以为图像格式。本申请实施例主要以视频格式模板为例，但这并不构成对视频模板的格式限制。示例性的，视频模板是一系列模板图像帧构成的序列。其中，视频模板可以但不限于是运动比赛类视频或者背景场地类视频。视频模板可用来。作为一种可能的示例，最终合成运动视频时，以视频模板作为模板或背景，在视频模板中的第x帧图像帧插入相同时刻(即目标时刻)，相同场景模型下用户的图像，形成用户的运动视频,x为正整数。
对视频模板进行加工处理可以是获取场景模型的信息、人物模型的信息。可选的，还可以获取视频模板中人物模型的运动数据(比如跑步的步幅、步频等)。视频模板处理模块可以将从视频模板中提取的上述视频模板的数据存储到信息库中，以供视频合成使用。
其中，场景模型的信息，是对视频模板的每帧图像的场景进行数字化处理得到的信息，包括但不限于场景视角，场景大小，赛道坡度等信息。
作为一种可能的实现方式，信息库可以包括一个或多个子库。不同子库存储不同信息。比如，信息库包括视频模板库、模板场景库、人物模型库、运动数据库。视频模板库用于存储视频模板。场景库用于存储模板中的场景信息。运动数据库，用于存储人物的运动数据。人物模型库用于存储人物模型相关的信息，比 如存储人物的画像信息，人物模型的信息。人物模型的信息包括视频模板中的人物模型信息，以及需合成运动视频的用户的人物模型信息。
本申请实施例中，视频模板的来源包括但不限于如下来源：
1、系统预置：系统可以从网络中收集已获得授权的运动视频(比如明星的运动视频)，并处理运动视频得到运动视频的相关信息(比如场景模型信息、运动数据信息、人物模型信息)写入信息库中。系统还可以从网络中手机已获得授权的运动数据，并根据运动数据生成相应视频模板。
2、由用户上传视频模板：用户上传自己的运动视频(可选的，还可以上传运动数据)，系统进行加工处理后将各类信息写入信息库中。
3、由用户上传的运动数据计算得到：用户上传自己的运动数据，系统基于运动数据自动生成视频模板，并可以对视频模板进行加工处理后将各类信息写入信息库中。
图像数据处理模块：可用于获取用户的画像数据(或称图像数据)，对用户的画像数据进行处理，得到用于表征用户外观的人物模型。人物模型可以但不限于是三维模型。用户的画像数据包括但不限于能够表征用户体貌等特征的数据。画像数据的格式可以但不限于是文本、用户各角度的图像、视频等。在一些实施例中，图像数据处理模块处理的图像数据(或称为用户的画像数据)可以是用户上传/选择/输入的图像数据。或者，图像数据处理模块处理的图像数据可以是系统预置的图像数据。
虚拟人物运动图像生成模块：可使用人工智能(artificial intelligence，AI)技术，基于用户的画像信息(可以由用户输入或系统默认)，运动数据，场景3D模型，生成虚拟人物(或称人物模型)在对应时间点和对应场景模型下的运动图像，最终合成虚拟人物的以视频模板为背景的运动视频。
运动视频生成模块：可以将虚拟人物运动图像生成模块产生的虚拟人物运动图像插入视频模板相应的模板图像帧中，换句话说，视频模板作为虚拟人物运动的背景。如此，能够得到合成的运动视频。
在一些实施例帧，如图2所示，可选的，运动视频合成系统还可以包括视频服务模块、视频播放下载模块。其中，视频服务模块：可位于云侧(比如服务器侧)，用来存储合成的运动视频以及合成的运动视频的分片，提供在线播放、下载、分享等功能。视频播放下载模块:可位于端侧(比如手机侧)，用于下载视频，播放视频或者在线播放视频。
本申请实施例的运动视频合成系统中部分模块可位于端侧设备中，部分模块可位于云侧设备中。端侧设备可以但不限于是手机、平板电脑、可穿戴设备、车载设备、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)等，本申请实施例对端侧设备的具体类型不作任何限制。
以端侧设备为手机为例，示例性的，图3A示出了端侧设备100的一种结构示意图。端侧设备100可以包括处理器110，外部存储器接口120，内部存储器121， 通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是端侧设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在本申请的一些实施例中，处理器110可用于获取用户的运动数据，并根据运动数据获取用户的运动视频。
充电管理模块140用于从充电器接收充电输入。
电源管理模块141用于连接电池142，充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110，显示屏194，摄像头193等供电。
端侧设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
移动通信模块150可以提供应用在端侧设备100上的包括2G/3G/4G/5G等无线通信的解决方案。无线通信模块160可以提供应用在端侧设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)等无线通信的解决方案。
端侧设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。在一些实施例中，端侧设备100可以包括1个或N个显示屏194，N为大于1的正整数。
端侧设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过 镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进行算法优化。ISP还可以对拍摄场景的曝光，色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。例如，ISP可以根据拍摄参数控制感光元件进行曝光和拍照。
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device，CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor，CMOS)光电晶体管。感光元件把光信号转换成电信号，之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB，YUV等格式的图像信号。
在一些实施例中，端侧设备100可以包括1个或N个摄像头193，N为大于1的正整数。其中，摄像头193可以位于电子设备的边缘区域，可以为屏下摄像头，也可以是可升降的摄像头，或者挖孔摄像头。摄像头193可以包括后置摄像头，还可以包括前置摄像头。本申请实施例对摄像头193的具体位置和形态不予限定。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当端侧设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。端侧设备100可以支持一种或多种视频编解码器。这样，端侧设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。通过NPU可以实现端侧设备100的智能认知等应用，例如：图像识别，人脸识别，语音识别，文本理解等。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展端侧设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行端侧设备100的各种功能应用以及数据处理。
端侧设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块170用于将数字音频数据转换成模拟音频电信号输出，也用于将模拟音频电信号输入转换为数字音频数据，音频模块170可以包括模/数转换器和数/模转换器。例如，音频模块170用于将麦克风170C输出的模拟音频电信号转换为数字音频数据。音频模块170还可以用于对音频数据进行编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170的部分功能 模块设置于处理器110中。
扬声器170A，也称“喇叭”，用于将模拟音频电信号转换为声音信号。端侧设备100可以通过扬声器170A收听音乐，或收听免提通话。
受话器170B，也称“听筒”，用于将模拟音频电信号转换成声音信号。当端侧设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为模拟音频电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风170C发声，将声音信号输入到麦克风170C。其中，该麦克风170C可以是端侧设备100的内置部件，也可以是端侧设备100的外接配件。
在一些实施例中，端侧设备100可以包括一个或多个麦克风170C，其中每一麦克风或多个麦克风合作可以实现采集各个方向的声音信号，并将采集到的声音信号转换为模拟音频电信号的功能，还可以实现降噪，识别声音来源，或定向录音功能等。
其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
其中，陀螺仪传感器180B，可以用于确定端侧设备100的运动姿态。在一些实施例中，可以通过陀螺仪传感器180B确定端侧设备100围绕三个轴(即，x，y和z轴)的角速度。陀螺仪传感器180B可以用于拍摄防抖。示例性的，当按下快门，陀螺仪传感器180B检测端侧设备100抖动的角度，根据角度计算出镜头模组需要补偿的距离，让镜头通过反向运动抵消端侧设备100的抖动，实现防抖。陀螺仪传感器180B还可以用于导航，体感游戏场景。
距离传感器180F，用于测量距离。端侧设备100可以通过红外或激光测量距离。在一些实施例中，在拍摄场景下，端侧设备100可以利用距离传感器180F测距以实现快速对焦。
触摸传感器180K，也称“触控面板”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。
可以理解的是，本申请实施例示意的结构并不构成对端侧设备100的具体限定。在本申请另一些实施例中，端侧设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
端侧设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本发明实施例以分层架构的Android系统为例，示例性说明端侧设备100的软件结构。
图3B是本发明实施例的端侧设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将电子设备的操作系统(例如Android系统) 分为四层，从下至上分别为内核层，硬件抽象层(hardware abstract layer，HAL)，应用程序框架层，以及应用程序层。
内核层是硬件和软件之间的层。内核层至少包含摄像头驱动，音频驱动，显示驱动，传感器驱动。传感器驱动包括但不限于图像传感器驱动、声敏传感器。
在一些实施例中，比如在运动场景中，内核层的诸如传感器驱动被调用以开启相应传感器完成运动数据采集。
硬件抽象层(hardware abstract layer，HAL)位于内核层和应用程序框架层之间，用于定义驱动应用程序硬件实现的接口，将驱动硬件实现的值转化为软件实现程序语言。例如识别摄像头驱动的值，将其转化为软件程序语言上传至应用程序框架层，进而实现调用相应功能。
在一些实施例中，HAL可以将传感器采集到的运动数据上传至应用程序框架层进行进一步的处理。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层从内核层经由HAL获取原始输入事件，识别该输入事件所对应的控件。应用程序框架层包括一些预先定义的函数。
如图3B所示，应用程序框架层可以包括视图系统，电话管理器，资源管理器，通知管理器，窗口管理器等。
在一些实施例中，应用框架层包括第一模块。第一模块可用于获取用户的运动数据，并根据运动数据获取用户的运动视频。
可选的，第一模块还可以是设置在其他层中，并且，第一模块还可以划分为更多子模块。每个子模块用于执行相应功能。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供端侧设备100的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
应用程序层可以包括一系列应用程序包。
如图3B所示，应用程序包可以包括相机，视频，通话，WLAN，音乐，短信 息，蓝牙，地图，日历，图库，导航等应用程序。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
需要说明的是，云侧设备300的结构也可以参考图3A的端侧设备100的结构，云侧设备300可以具有比图3A所示的结构更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
例如，以云侧设备为服务器为例，如图4所示，云侧设备300可以包括处理器401(可选的，包括处理器408)、存储器403、收发器404等。
其中，上述各组件之间可包括一通路，用于在上述组件之间传送信息。
收发器404，用于诸如以太网，WLAN等协议与其他设备或通信网络通信。
处理器、存储器的详细内容请参考图3A的手机中相关结构的描述，这里不再赘述。
本申请实施例的技术方案可应用在运动场景，运动场景包括但不限于跑步、步行、划船、游泳等。用户发生运动后，基于用户的运动数据，可以得到用户在不同时刻的运动轨迹，并将运动的用户的图像插入到视频模板中。其中，视频模板包括一些场景(比如建筑物、树木等)。如此，能够得到用户在视频模板的场景下运动的运动视频。在一些实施例中，若视频模板还可包括运动人物，那么，将运动的用户插入到视频模板，则可以得到该运动用户与其他运动人物一起运动的视频。
以跑步场景为例，用户在跑步过程中，可以通过随身携带的运动手环等设备采集用户的运动数据。这些运动数据可以被目标应用获取。作为一种可能的实现方式，目标应用可以由用户设置。比如，用户可以授权运动健康等运动应用能够获取用户的运动数据。以目标应用包括运动应用为例，后续，用户可以通过运动应用将运动数据合成运动视频。
如图5所示，运动应用可以调用诸如手环等设备中的传感器采集到的运动数据，并将采集的运动数据记录在运动应用对应的存储空间中。如图5的(a)所示，运动数据记录包括：5月11日跑步的运动数据，4月21日跑步的运动数据等。
接下来，如图5的(a)所示，在界面601中，用户通过诸如点击5月11日的运动数据选择该运动数据，响应于用户的选择运动数据的操作，手机呈现诸如图5的(b)所示界面602。可选的，通过该界面602，用户可以直观的观察自己的跑步线路。
在本申请实施例中，界面602可包括控件603。当检测到用户对控件603的预设操作，手机可以根据运动视频生成算法生成该运动数据对应的运动视频。或者，手机向服务器发送生成运动视频的请求或通知，由服务器根据运动视频生成算法生成该运动数据对应的运动视频。或者，手机自身执行生成运动视频的部分步骤，服务器执行生成运动视频的另外部分操作。也就是说，可以在端侧的手机等终端完成运动视频合成，也可以在云侧的服务器完成运动视频合成，或者，由端云协 作完成运动视频合成流程。
以手机合成运动视频为例，对于运动视频的每一帧画面来说，手机根据相应时间的运动数据，场景模型信息以及图像信息计算人物的动作、位置等信息，生成用户的运动图像，并将用户的运动图像插入视频模板中相应的帧中。以手机根据运动数据计算用户位置为例，手机可初始化用户在第一帧画面中的位置，并根据运动数据中的诸如步幅、步频、跑步速度等运动数据计算用户在后续帧画面中的位置。
其中，图像信息可以不是与运动数据同步采集的。比如，图像信息可以是用户自己上传/选择的，或者，图像信息可以是系统预置的。或者，还有其他采集图像信息的方式，本申请实施例对此不进行限制。
示例性的，如图6所示，手机初始化人物在视频模板的第一帧中的初始化位置，后续，手机可根据初始化位置、运动数据中跑步速度以及视频模板中的场景模型信息等参数，计算人物在视频模板的第X帧中的位置，并将人物插入视频模板的第X帧，得到运动合成视频的第X帧。同理，手机可根据初始化位置以及运动数据中跑步速度等参数，计算人物在视频模板的第Y帧中的位置，并将人物插入视频模板的第Y帧，得到合成运动合成视频的第Y帧。
在生成运动视频后，用户可通过运动应用查看生成的运动视频。比如，用户可通过如图5的(c)所示界面604查看已生成的运动视频605。界面604包括视频播放控制控件606，用户可通过控制控件606控制运动视频605的播放、暂停、快进、快退等。
通过本申请实施例的技术方案，不必同时采集用户的图像信息和运动数据，而是在获取运动数据后就可以基于运动数据合成用户的运动视频。与现有技术中需同时采集图像信息和运动数据相比，本申请实施例的实现复杂度更低，用户可以更便捷的使用合成运动视频的功能。并且，由于不再局限于用摄像头等设备获取实时的图像信息，从而可以应用在各种运动场景中。比如，可以在室外无摄像人员辅助摄像时应用本申请实施例的方案，在获取运动数据后就可以合成用户的运动视频。
再示例性的，在如图1所示场景中，如果使用本申请实施例的视频合成方案，则无需通过智慧屏的摄像头实时采集用户的视频图像信息，也无需智慧屏处理多帧视频图像，不仅降低了数据采集难度，也降低了数据处理复杂度。
可选的，图5所示界面602还可以包括其他控件。比如，可以包括生成“轨迹动画”的轨迹动画生成控件。当检测到用户对该轨迹动画生成控件的预设操作，手机可基于用户的运动数据生成运动数据对应的轨迹动画(或者轨迹视频)。在轨迹动画中，三维人物模型可沿着一定运动轨迹运动。三维人物模型可以是基于用户的图像数据(可以是系统默认或用户上传/选择)构建的人物模型。如图7示出了由运动数据生成的一种轨迹动画。
与图5中合成的运动视频不同，图7中合成的轨迹视频可以无需使用视频模板。
作为一种可能的实现方式，手机可以为用户推荐诸如图7所示轨迹沿途适合 不同运动的地点。比如，通过诸如图7的(a)或(b)所示界面向用户推荐。
作为一种可能的实现方式，手机可以为用户提供编辑轨迹动画的入口。比如，如图7所示，用户可以通过拖动等操作移动三维人物模型在轨迹中的位置。
在图5所示场景中，用户的运动软件和/或可穿戴设备(比如运动手环等)对应图2所示的运动数据采集适配模块。在一些示例中，手机可包括图2所示视频模板处理模块、运动视频生成模块等模块，并由手机根据采集的运动数据完成运动视频合成。在另一些示例中，服务器可包括图2所示视频模板处理模块，运动视频生成模块等模块，并由服务器完成运动视频合成。在另一些示例中，服务器可包括视频合成子系统的部分模块，手机可包括视频合成子系统的另外部分模块，并由端侧(比如手机)和云侧(比如服务器)共同完成运动视频合成。
本申请实施例的运动视频生成方法，无需同时采集运动数据和图像数据，因此用户发生运动时，可以不必要求携带相机等专有设备，或者说不要求专人辅助配合进行实时图像拍摄,降低了使用合成运动视频功能的门槛，极大扩宽了生成运动视频技术方案的适用场景。
在一些实施例中，视频模板库(是信息库中的部分)可包括一个或多个视频模板。视频模板可以是不同类型的运动视频，包括但不限于跑步、游泳、划船等运动类型的视频。视频模板库中的视频模板可以由系统自动添加、更新，也可以由用户添加、更新。
示例性的，图8示出了用户向视频模板库中添加视频模板的过程。如图8的(a)所示，运动应用可包括视频模板库界面，视频模板库界面可包括控件801。当检测到用户对控件801的预设操作(比如点击操作)，手机可呈现图8的(b)所示界面802。在一些设计中，界面802可以是本地相册的预览界面，也就是说，通过界面802可以选择本地存储的一些视频作为视频模板。在另一些设计中，通过界面802可以选择网络上(非本地存储)的一些视频作为视频模板。比如通过界面802，用户可以选择或输入明星的跑步视频的网址www.xxx.com。那么，手机将该网址的跑步视频添加进视频模板库中。
假定如图8的(b)所示，用户选择了本地存储的第一个视频作为视频模板。那么，在检测到用户选择该视频的操作(比如点击该视频)后，可选的，手机可呈现如图8的(c)所示界面803。界面803包括用户选择的视频。可选的，界面803还包括用户所选视频对应的剪辑控件804。用户可以通过拖动剪辑控件804，对用户所选视频进行剪辑，以便截取其中一定时长的视频作为视频模板。
可选的，界面803还可包括其他控件，比如包括图8的(c)所示编辑配乐的控件805。用户可以通过操作控件805自己选择或上传背景音乐(background music，BGM)。手机还可以提供BGM的多种设置选项。比如，提供播放方式选项，以便用户选择BGM的播放方式(单曲循环或者多曲遍历)，提供音效选项等。
如图8的(c)所示，当检测到用户的诸如点击“确定添加”控件的操作后，手机将用户所选视频添加到视频模板库中，并可呈现诸如图8的(d)所示界面804。界面804包括用户新添加的视频模板4。
在一些示例中，可选的，用户在上传或添加视频模板时，可以向视频模板库 录入该视频模板的运动数据。以跑步视频模板为例，用户可以输入视频模板中人物的跑步步幅、步频、时长等。
在另一些示例中，系统可以基于运动视频分析算法分析计算视频模板中人物的运动数据。后续，视频模板中人物的运动数据可以与其他用户的运动数据进行对比。系统还可以在合成的运动视频中向用户显示各人物的运动数据。
在一些实施例中，用户可以从视频模板库中选择视频模板，如此，当用户指示生成运动类合成视频时，系统可以将运动的人物插入用户选择的视频模板中。其中，人物的运动是根据运动数据模拟出来的。示例性的，如图9A的(a)所示，在检测到用户的诸如点击控件603的操作时，手机可成像如图9A的(b)所示的界面。该界面包括控件903。当检测到用户针对控件903的预设操作(比如点击操作)，手机可呈现如图9A的(c)所示界面901。界面901可包括控件902。当检测到用户的诸如点击视频模板4的操作，且检测到用户的诸如点击控件902的操作时，手机可确定用户选择视频模板4作为本次合成视频所需使用的视频模板。
在一些示例中，可选的，在用户选择本次合成视频所需使用的视频模板后，手机可以由图9A的(c)所示界面901自动跳转回图9A的(b)所示界面908。或者，界面901中可以设置跳转回界面908的控件X，在检测到用户对控件X的诸如点击操作时，手机跳转回界面908。当检测到用户对界面908的“确定”按钮的诸如点击操作，手机将运动的人物插入视频模板4中，合成如图9B所示的运动视频。其中，运动视频中，所插入人物的运动可由运动数据得到。比如，运动视频中，所插入人物的跑步步幅、步频可以根据采集的运动数据得到。
在一些示例中，可选的，手机呈现图9A的(b)所示界面908后，若未检测到用户点击控件903，而是直接点击“确定”按钮，则手机使用默认视频模板作为本次合成运动视频所需的视频模板，或者，手机使用上一次合成运动视频所使用的视频模板，或者手机基于可授权使用的用户画像，使用推荐的视频模板。或者，手机还可以基于其他策略自动为用户选择本次合成运动视频所需的视频模板。
在一些示例中，可选的，界面901还可包括控件905，控件905用于用户添加新的视频模板。添加的新视频模板可以是手机本地视频，也可以是非本地的网络视频。
在一些实施例中，可以在用户选择视频模板后，根据用户的操作对用户所选择视频模板进行一些编辑。比如，可以根据用户的操作指令对用户选择的如图9A的(c)所示视频模板4进行剪辑操作，并将运动的人物插入剪辑后的视频模板4中。
在另一些实施例中，可由系统或用户设置默认的视频模板。这样，当用户指示生成运动类合成视频时，系统可以将运动的人物插入默认视频模板中。
如下，对合成运动视频的技术实现细节进行介绍。如图10所示，本申请实施例提供的合成运动视频的方法主要包括采集用户的运动数据，以及基于运动数据以及视频模板、用户的图像信息，将运动的用户插入视频模板中。其中，采集用户的运动数据，可以是运动数据采集适配模块从用户那里采集运动数据，也可以是运动数据采集适配模块从第三方软件和/或第三方运动辅助设备获取用户的运动 数据。
其中，采集用户的运动数据，可以借助于与用户之间具有交互关系的装置、模块或设备。比如，可以通过用户的可穿戴设备，用户的手机、用户使用的健身器材等设置的传感器采集用户的运动数据。还可以通过用户手机等设备上安装的应用来采集用户的运动数据。本申请实施例不限制采集运动数据的方式，可以通过各种可能的方式采集用户的运动数据。
在一些实施例中，不同模块采集的运动数据可以有不同格式。同一模块采集运动数据时，由于诸如运动类型不同，不同运动类型的运动数据也可以有不同格式。运动类型包括但不限于跑步、划船、骑行、登山等。为了适配各种类型的运动数据，本申请实施例的数据采集子系统中可以设置运动数据采集适配模块，该运动数据采集适配模块可以处理不同类型的运动数据，并为各种类型的运动数据构建统一的运动数据模型。示例性的，图11示出了运动数据模型的一种示例。
可选的，在获取用户的运动数据后，还可以构建用户的人物模型。该人物模型用于代表用户，将该人物模型插入视频模板中，能够得到用户在该运动数据情况下的运动视频。可选的，人物模型可以是三维模型或其他类型模型。
作为一种可能的实现方式，用户可以上传能够表明自己特征的画像信息(或称图像信息)，运动视频合成系统基于用户上传的画像信息构建人物模型。用户的画像信息可以是各种形式的信息。比如，可以表征用户特征的是文本、图片、视频等各种形式。表征用户特征的文本包括但不限于用户的人脸五官特征、身高、体重、年龄、着装等。系统基于画像信息构建人物模型，可以采用任意人物建模算法，本申请实施例对此不做限制。
需要说明的是，用户的画像信息是可选的。用户也可以不上传画像信息，这种情况下，系统为该用户指定默认画像，并构建默认的人物模型。
示例性的，如图9A的(a)所示，当检测到用户对控件603的预设操作(比如点击)，手机呈现诸如图9A的(b)所示界面908，界面908包括控件904。当检测到用户对控件904的诸如点击操作，手机可呈现诸如图9A的(d)所示界面，该界面可包括一个或多个图像数据(比如图像1、图像2)。以用户选择图像1作为图像数据为例，当检测到用户对图像1的诸如点击操作，且用户对控件906进行诸如点击后，手机确定用户选择了图像1作为本次合成视频所需的图像数据。手机可自动跳转回界面908或者，在检测到用户在界面909的指示跳转到界面908的操作后，手机跳转到界面908。当检测到用户在界面908对“确定”按钮的诸如点击操作，手机可以将图像1对应的人物模型插入本次合成视频使用的视频模板中，从而得到诸如图9B所示的用户的运动视频。
在一些示例中，可选的，9A的(d)所示界面909中还可包括控件907，用于用户添加新的图像数据。
作为另一种可能的实现方式，在获得用户的授权后，系统可以自动采集用户的画像信息，并根据画像信息构建用户的虚拟人物模型。
作为另一种可能的实现方式，系统还可以是使用系统默认的人物模型来表示用户。在一些示例中，对于不同用户，系统默认的人物模型是相同的。在另一些 示例中，对于不同用户，系统默认的人物模型还可以不同。可选的，系统根据不同用户可授权的用户画像，自动为不同用户推荐不同的人物模型。比如，针对男士用户，系统为其推荐男士人物模型，针对女士用户，系统为其推荐女士人物模型。
可选的，系统可将人物模型相关的信息存储到信息库中。人物模型相关的信息包括但不限于图像数据、构建的人物模型的信息。
示例性的，图12示出了用户或系统管理员向视频模板库中添加人物建模信息的过程中，用户(或系统管理员)与各模块，各模块之间的交互。
如上述实施例所描述，用于合成运动视频的视频模板可以是系统自动添加进视频模板库中，换言之，视频模板可以是系统预置的。比如，系统可以采集用户的可授权使用的用户画像，并基于用户画像确定用户可能使用的视频模板。再比如，系统可以获取网络信息，并向用户推荐或在视频模板库中添加一些其他用户使用较频繁的视频模板。系统向用户推荐视频模板，还可以结合用户可授权使用的用户画像。
在另一些实施例中，可选的，视频模板也可以是用户或系统管理员添加进视频模板库中。示例性的，图8示出了用户向视频模板库中添加视频模板过程中的一组界面。示例性的，图15示出了用户或系统管理员向视频模板库中添加视频模板的过程中，用户(或系统管理员)与各模块，各模块之间的交互。用户(或系统管理员)可以通过诸如图8所示界面等操作向视频模板库添加视频模板，视频模板处理模块检测到用户添加视频模板的操作后，可以处理该视频模板，得到视频模板的相关信息。比如，视频模板处理模块可以提取视频模板中的场景、人物、运动数据中的任一个或多个视频模板的相关信息。视频模板处理模块还可以将视频模板的相关信息存储进信息库。
在获取用户的图像数据(比如可以是用户上传或系统默认)、运动数据以及视频模板之后，可以基于图像数据、运动数据以及视频模板中的场景模型信息生成人物模型的运动图像。该人物模型的运动图像的背景可以是透明背景。示例性的，图13示出了生成人物模型(或称虚拟人物)的运动视频的过程中，各模块之间的交互。其中，在采集运动数据时，可以是运动数据采集适配模块(比如运动应用、传感器等辅助装置)从用户采集运动数据。或者，运动数据采集适配模块从第三方运动软件或第三方运动辅助设备采集用户的运动数据。运动数据采集适配模块可以对运动数据进行诸如格式化处理，并为运动数据构建统一的运动数据模型。虚拟人物运动图像生成模块可以从运动数据采集适配模块获取处理后的运动数据，从信息库获取人物模型的信息以及视频模板相关的信息(比如场景模型的相关信息)，并根据运动数据，人物模型的信息以及视频模板的相关信息生成虚拟人物的运动图像。其中，如图14A所示，运动数据主要用来计算运动视频的每帧画面中人物的动作、位置等。人物模型的信息主要用来模拟运动视频中的人物。视频模板的相关信息(比如场景模型的信息)可用来计算运动视频的每帧画面的视角，每帧画面中人物位置，每帧画面中人物大小等。示例性的，如图14B示出了场景模型不同的情况下，帧画面的视角，人物大小、位置的变化。可以看 出，当场景(比如树木、道路)模型的大小不同，人物大小也不同。当场景变换视角，帧画面的视角也发生改变，人物模型的展示视角也相应改变。
示例性的，图6示出了系统生成的虚拟人物的第1帧、第X帧、第Y帧运动图像，各运动图像的背景为透明背景。
在获取虚拟人物的运动图像(透明背景)后，系统可以将虚拟人物的图像插入视频模板的相应图像帧中，话句话说，可以将视频模板作为虚拟人物运动的背景。示例性的，图16示出了合成运动视频过程中各模块之间的交互。
上述以合成的运动视频是单人的运动视频为例对本申请实施例的运动视频合成方法进行说明，本申请实施例中，还可以合成包括多人的运动视频。具体的，可以根据用户的运动数据，将该运动的用户合成到其他人物的运动视频(视频模板)，形成该运动用户与其他人物一起运动的视频。可选的，该运动用户的运动类型与视频模板中人物的运动类型相同。比如，该运动用户的运动类型为跑步，视频模板中人物的运动类型同样为跑步。插入视频模板中的运动用户可以是一个或多个。当插入视频模板中的运动用户为多个，可以按照一定顺序依次将多个运动用户合成到一个视频模板中，形成多用户一起运动的视频。
示例性的，如图17所示，系统可以采集用户的运动数据，并根据运动数据生成虚拟人物的各帧运动图像(图17仅示例性示出了其中的一帧)，将虚拟人物的各帧运动图像插入到视频模板A的相应模板图像帧中，得到用户与其他人物一起运动的运动视频。
其中，视频模板A可以是明星的运动视频，也可以是其他人物(比如好友)的运动视频。视频模板A添加到视频模板库的方法可参见前述实施例，这里不再赘述。
如下分几种场景来介绍合成包括多人的运动视频(可称为多人模式的运动视频)的示例。
场景一：合成的运动视频可以是用户与明星一起运动的视频。在该实现方式中，视频模板可以是明星的运动视频，用户可以选择某位明星的运动视频，系统可以将用户的运动视频插入到该明星的运动视频中，合成用户和该明星一起运动(比如比赛)的运动视频。示例性的，图18A的(a)-(d)、图18B示出了合成视频过程中的一些界面。图19是以在剪辑类应用程序中编辑合成运动视频为例。各界面的具体实现以及相应的用户操作，手机响应可参见上述实施例，这里不再赘述。示例性的，如图19示出了合成用户与明星一起运动的视频过程中，各模块之间的交互。
需要说明的是，明星的运动视频可以是明星自己上传到系统中，比如明星通过剪辑应用上传到服务器。也可以是本申请实施例的服务器从其他服务器中获取的已经授权的运动视频。
场景二：合成的运动视频可以是用户与好友一起运动的视频。以视频模板为用户的好友B为例，用户B可以按照上述实施例的方法在系统中完成运动数据录入和图像信息的录入。其中，图像信息的录入为可选步骤。用户A可以将用户B的运动视频选择为视频模板，系统能够将用户A的运动视频插入到用户B的运动 视频中，合成用户A、用户B比赛的运动视频。示例性的，如图20A、图20B示出了系统合成用户A与用户B比赛视频过程中的一些界面。
场景三:合成的运动视频为团队(team)的运动视频
本申请实施例可合成异步的团队运动视频异步线的含义是不要求用户在合成视频的同时进行运动，或者并非一边采集运动数据，一边合成运动视频，或者，多个用户的运动数据是异步获取的。用户可以先进行运动，运动后再上传自己的运动数据，以便系统根据运动数据合成运动视频。示例性的，手机呈现如图21A的(a)所示的剪辑应用界面，并且，检测到用户对控件1901的诸如点击操作，那么，手机可呈现诸如图21A的(b)所示界面。该界面包括控件2101以及控件2102。其中，控件2101可用来设置合成团队运动视频的相关信息，控件2102可用来设置合成单人运动视频(比如图5的(c)所示单人运动视频)的相关信息。图21A的(b)所示界面还可包括其他控件或删除某些控件。
当检测到用户对控件2101的诸如点击操作，手机可呈现图21A的(c)所示界面2108。该界面2108包括控件2104-2107。其中，控件2104、控件2105可参见的上述实施例的控件903、控件904的相关描述。控件2106可用来设置诸如团队的人数这些参数。控件2107可用来设置团队运动的运动路线。可选的，界面2108还可包括其他控件，用来设置团队运动视频的其他参数。当检测到用户对控件2106的诸如点击操作，手机可呈现图21A的(d)所示界面。
通过图21A的(d)所示界面可选择团队。可选的，通过该界面还可以设置团队的人数，设置团队的人数上限等。如果团队人数被设置了上限，则当选择加入该团队的人数达到上限后，系统不再允许其他用户加入该团队。
可选的，通过图21A的(d)所示界面还可设置团队运动的类型，比如设置类型为跑步比赛。可选的，通过该界面，还可以邀请好友参加录制该团队的运动视频。可选的，通过该界面还可以设置其他的团队相关参数。
在一些示例中，可选的，用户可以选择团队，并可以查看团队成员的已经授权的个人信息。如此，用户可以根据不同团队的团队成员信息选择自己偏好的团队，并借助系统合成该团队的运动视频。比如，用户可以选择运动排名靠前的一些用户组成的团队，加入该团队，并借助系统合成该团队的运动视频，以增强用户的运动趣味性，起到督促用户运动的作用。
以用户A为例，假设用户A选择加入的团队包括用户B、用户C，用户A还可以选择合成团队运动视频所使用的视频模板(或者使用系统默认的视频模板)。比如，用户A可以通过图21A的(c)所示控件2104选择视频模板。用户A还可以设置团队运动视频中自己的图像数据。比如，用户A可以通过图21A的(c)所示控件2105上传自己的图像数据(或者可以使用系统默认的图像数据)。用户A还可以设置自己的运动数据。比如，用户通过图21A的(c)所示控件2109上传自己的运动数据，或者，通过控件2109授权获取手机中存储的运动数据。
同理，用户B、用户C也可以选择各自所需使用的视频模板(或使用默认视频模板)。用户B、用户C也可以设置团队运动视频中各自的图像数据。用户B、C也可以设置各自的运动数据。
需要说明的是，团队成员单独上传自己的运动数据，相互之间的上传时间，运动地点等不做限制，可以在不同的时间和地点完成运动，或者在相同时间、地点完成运动，并离线上传运动数据。
在合成团队的运动视频时，系统根据用户A、B、C各自的运动数据以及图像数据，生成各自的运动视频，并将三者的运动视频插入到视频模板中，得到团队的运动视频。示例性的，图22示出了合成团队运动视频的过程。可选的，针对多个用户分别对应的虚拟人物运动图像，可以按照一定顺序将多个虚拟人物的运动图像分别合成到视频模板相应的模板图像帧中。
需要说明的是，用户A、B、C选择的视频模板可以是同一视频模板，也可以是不同视频模板。示例性的，假设用户A选择了视频模板1，用户B、C选择了视频模板2，图21B示出了用户A、B的手机A、B分别所展示的团队运动视频。其中，手机A、B展示的团队运动视频的背景不同，手机A展示的团队运动视频的背景是视频模板1，手机B展示的团队运动视频的背景是视频模板2。
需要说明的是，在合成多人的运动视频时，可以由团队组织人创建任务，指定合成运动视频的某些参数，比如指定视频模板，运动类型，时长等要求。这样一来，系统为不同用户合成的运动视频通常是一致的。比如，为不同团队成员合成的运动视频的背景均相同。或者，也可以由团队组织人指定部分参数，团队成员指定另一部分参数。比如，团队组织人指定运动类型为跑步，时长要求为一小时，团队成员分别指定各自的视频模板，分别上传各自的运动数据。那么，系统为不同成员合成的运动视频的背景可能不同。
示例性的，如图23示出了合成团队运动视频过程中各模块之间的交互。需要说明的是，限于附图尺寸，图23中仅示例性示出了运动数据采集适配模块通过第三方软件或运动辅助设备从用户采集运动数据的技术细节，而并未示出运动数据采集适配模块从用户直接采集运动数据的技术细节。
通过该方法，多个用户可以通过异步方式分别完成运动，即在不同时间和不同地点完成运动，也就是说，用户不必在同一时间运动，对于任一用户来说，其可以在便利的时间、地点完成运动。系统可以将在不同时间、不同地点运动的多个用户的多个运动视频合成团队一起运动的运动视频，为用户使用合成运动视频功能提供了便利。
场景四:合成的运动视频为团队(team)的运动视频
与场景三不同的是，场景四中，团队成员选择统一的运动路线。比如，基于地图，选择统一地图轨迹。比如选择北京市XX公园的外沿作为统一运动路线。系统采集各用户沿该路线的运动数据，并可以根据运动数据生成各自的运动视频，进而将各自的运动视频插入到视频模板中，得到团队的运动视频。
本申请实施中，合成的各种运动视频可以在用户授权的情况下被上传，分享等。其中，对于团队运动视频，可以在团队内部观看，也可以在获得团队成员授权的情况下，被上传或分享到网络。
场景五:合成在线的团队运动视频
与场景三、四不同，场景五中，团队成员需在同一时段内进行运动，或者团 队成员同步运动，团队成员的运动数据是同步获取的。系统可获取各用户的运动数据，并生成各用户的运动视频，以及将各用户的运动数据插入到视频模板中，合成团队的运动比赛视频。
示例性的，用户可以在图24的(a)所示界面中选择合成运动视频。示例性的，如图24的(b)所示，当检测到用户指示选择合成团队运动视频的操作，比如检测到用户点击控件2101后，手机可显示诸如图24的(c)所示界面。该界面包括在线模式控件2401和离线模式控件2402。检测到用户选择在线模式的操作后，手机可显示诸如图24的(d)所示界面2404。用户可以通过界面2404中的控件设置合成运动视频相关的选项。比如，可以通过控件2105选择图像信息，通过控件2104选择视频模板。用户还可以打开采集运动数据的开关2403，如此一来，手机可在采集运动数据的过程中实时上报用户的运动数据。系统可基于用户的运动数据，实时生成虚拟人物的运动图像，并且将虚拟人物的运动图像实时合入到视频模板相应的模板图像帧中。
该在线生成运动视频的方案中，示例性的，以服务器执行合成运动视频的步骤为例，服务器可以通过流媒体将合成的团队运动视频下发到终端。用户通过终端(比如手机)上的视频播放下载模块，实时拉流获取最新的视运动频流。用户在终端侧可以观看合成的运动视频。可选的，服务器可以对实时生成的运动视频的分片进行存储。
可见，该在线合成运动视频的方案，系统可以同步获取多个用户的运动数据，并能够合成多个用户的运动视频。可选的，系统在用户运动过程中基于运动数据合成用户的运动视频。或者，系统在其他可能的时机合成用户的运动视频，本申请实施例并不限制系统生成运动视频的时机。
作为一种可能的实现方式，可选的，场景五的在线模式下，可以由社团组织人创建任务，指定视频模板以及运动类型，时长，开始时间中的一个或多个参数。各社团成员可以按照规定时间打开运动应用，并在不同的地点进行相同模式的运动。各成员的运动应用会自动采集各成员的运动数据，系统可以根据各成员的运动数据完成运动视频的合成。
相对于现有的视频合成方案，本申请实施例的技术方案，可以通过运动数据采集适配模块支持从各种软件和/或硬件采集用户的运动数据，无需专有的设备，既可以完成视频合成，能够降低合成运动视频的技术门槛。
需要说明的是，该场景五中，多个用户可以在不同地点运动。也就是说，可以将不同地点运动的人物合成到一个运动视频中，增加了运动视频的趣味性。
通过前述实施例，用户可以在诸如手机等终端上设置合成运动视频的相关选项。比如，通过诸如图24所示一组界面选择视频模板、上传画像数据等。用户还可以选择将合成的运动视频实时显示在跑步机等带屏幕设备的显示屏上。示例性的，如图25所示，用户在跑步机上跑步，用户可以通过诸如手机等终端选择将合成的运动视频显示在跑步机的屏幕上，这样一来，用户可以在健身同时观看队友的跑步情况，增加运动的趣味性。
可选的，跑步机等健身器材还可以模拟运动视频中的路况。比如，当运动视 频中出现陡坡，跑步机可以通过振动等方式制造颠簸感。跑步机还可以在振动之前提示用户即将出现陡坡，以便提醒用户做好准备。
本申请的上述实施例中，主要以运动应用、剪辑应用为例对生成运动视频的技术方案进行说明，可以理解，其他可以编辑生成运动视频的应用也可以适用本申请实施例的生成运动视频的技术方案。
此外，除了运动健身等场景，本申请实施例的技术方案还可以应用在综艺，唱歌等场景中。
在一些实施例中，可选的，可以在合成的运动视频中标记用户的运动数据。示例性的，如图21B所示，合成的团队运动视频中标记有各团队成员的运动数据。这些运动数据可以表征用户的运动状态。
可选的，还可以在合成的运动视频中标记模板人物的运动数据。模板人物即视频模板中的人物。示例性的，如图18B所示，可以在合成的运动视频中标记模板人物，即明星的运动数据。可选的，还可以在该合成的运动视频中标记用户(即黑色人物模型)的运动数据。
可选的，诸如图2等运动视频合成系统还可以包括更多或更少模块，或组合一些模块，或拆分一些模块。或者，某些模块可以替换为其他类似功能的模块。本申请实施例对系统的具体架构不做限制。
如上文所介绍，视频模板、用户画像数据等可以是系统默认设置的，也可以是用户选择或上传的。并且，本申请实施例对视频模板、用户画像数据的上传或选择时机不做限定。比如，用户可以在需要合成运动视频时选择/上传视频模板，或预先选择/上传视频模板。
在本申请的一些实施例中，可选的，可以为运动数据确定对应的运动类型。对于对应第一运动类型的运动数据，所述第一运动类型对应的运动数据包括轨迹。对于对应第二运动类型的运动数据，所述第二运动类型对应的运动数据不包括轨迹。示例性的，第一运动类型包括但不限于室外跑步、游泳、行走、骑行、足球。第一运动类型的运动通常可以形成运动轨迹。第二运动类型包括但不限于在跑步机上跑步，第二运动类型通常没有物理位置的变化，或物理位置变化较小，视为未形成运动轨迹。
对于第一运动类型，根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频，可以实现为：根据所述第一运动类型对应的运动数据中的轨迹，以及所述视频模板的数据，生成所述一个或多个用户的运动视频。示例性的，假设图5的(b)所示运动数据为用户沿xx公园跑步的运动数据，系统可以根据该运动数据中的运动轨迹，以及视频模板的数据，生成运动视频。
对于第二运动类型，根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频，可以实现为：根据所述第二运动类型对应的运动数据以及所述路线数据，生成所述一个或多个用户沿所述路线数据指示的路线运动的运动视频。示例性的，如图25所示，用户在跑步机上跑步，系统可以根据视频模板的路线数据，生成用户沿路线数据指示的路线(比如图25中视频模板的道路)运动的运动视频。
如图26示出了本申请实施例的视频生成方法的一种示例性流程，该方法包括：
S2601、获取视频模板的数据。
可选的，人物运动视频的数据或场景视频的数据。人物运动视频比如可以但不限于为明星等人物的运动视频。示例性的，人物运动视频可以是图21B所示的视频。示例性的，场景视频可以是图6所示视频模板。
可选的，人物运动视频可以是采用本申请实施例的视频生成方法合成的运动视频，或者，普通的运动视频，即没有采用本申请实施例的视频生成方法合成的运动视频。
示例性的，手机从视频模板库中选择图9A的(c)所示视频模板4作为模板。
本申请实施例中，步骤S2601-S2603可由图2所示运动视频合成系统执行。该系统中的模块可以由端侧装置(比如手机等终端)，或端侧装置中的模块。也可以是云侧(比如服务器等)或云侧模块。示例性的，步骤S2601由图2所示视频模板处理模块执行。
S2602、采集一个或多个用户的运动数据。
其中，运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹。可选的，运动数据还可以包括如下任一项或多项数据：心率、血压、血氧、体温。以跑步为例，幅度可以指步幅，频率可以指步频。
示例性的，手机采集到如图9A的(a)所示运动数据。
示例性的，步骤S2602由图2所示一种或多种模块配合执行：运动数据采集适配模块执行、运动软件模块、辅助运动设备(硬件模块)。
S2603、根据运动数据以及视频模板的数据，生成一个或多个用户的运动视频。
示例性的，系统根据图9A的(a)所示运动数据以及图9A的(c)所示视频模板4，生成图9B所示用户的运动视频。
示例性的，步骤S2603由图2所示视频合成子系统的运动视频生成模块执行。可以理解，运动视频生成模块根据运动数据以及视频模板的数据，可以实现为：根据视频模板的数据以及虚拟人物运动图像生成模块生成的人物运动图像，生成用户的运动视频。
本申请另一些实施例提供了一种装置，该装置可以是上述电子设备(比如折叠屏手机)。该装置可以包括：显示屏、存储器和一个或多个处理器。该显示屏、存储器和处理器耦合。该存储器用于存储计算机程序代码，该计算机程序代码包括计算机指令。当处理器执行计算机指令时，电子设备可执行上述方法实施例中手机执行的各个功能或者步骤。该电子设备的结构可以参考图3A或图4所示的电子设备。
其中，该电子设备的核心结构可以表示为图27所示的结构，该核心结构可包括：处理模块1301、输入模块1302、存储模块1303、显示模块1304。
处理模块1301，可包括中央处理器(CPU)、应用处理器(Application Processor，AP)或通信处理器(Communication Processor，CP)中的至少一个。处理模块1301可执行与用户电子设备的其他元件中的至少一个的控制和/或通信相关的操作或数据处理。具体地，处理模块1301可用于根据一定的触发条件，控制主屏上显示的内容。或者根据预设规则确定屏幕上显示的内容。处理模块1301还用于将输入的指令或数据进行处理，并根据处理后的数据确定显示样式。作为一种可能的实现方式，处理模块1301可具有上述图2所示系统中诸如运动视频生成模块、视频模板处理模块、图像数据处理模块、运动数据采集视 频模块等一个或多个模块的功能。
输入模块1302，用于获取用户输入的指令或数据，并将获取到的指令或数据传输到电子设备的其他模块。具体地说，输入模块1302的输入方式可以包括触摸、手势、接近屏幕等，也可以是语音输入。例如，输入模块可以是电子设备的屏幕，获取用户的输入操作并根据获取到的输入操作生成输入信号，将输入信号传输至处理模块1301。在本申请实施例中，输入模块可用于接收用户输入的图像数据、运动数据等，和/或执行其他步骤。
存储模块1303，可包括易失性存储器和/或非易失性存储器。存储模块用于存储用户终端设备的其他模块中的至少一个相关的指令或数据，具体地说，存储模块可记录终端界面元素UI所在界面的位置。
显示模块1304，可包括例如液晶显示器(LCD)、发光二极管(LED)显示器、有机发光二极管(OLED)显示器、微机电系统(MEMS)显示器或电子纸显示器。用于显示用户可观看的内容(例如，文本、图像、视频、图标、符号等)。在本申请实施例中，显示模块可实现为显示屏幕，比如第一屏幕和/或第二屏幕。
可选的，图27所示结构还可通信模块1305，用于支持电子设备与其他电子设备通信。例如，通信模块可经由无线通信或有线通信连接到网络，以与其他个人终端或网络服务器进行通信。无线通信可采用蜂窝通信协议中的至少一个，诸如，长期演进(LTE)、高级长期演进(LTE-A)、码分多址(CDMA)、宽带码分多址(WCDMA)、通用移动通信系统(UMTS)、无线宽带(WiBro)或全球移动通信系统(GSM)。无线通信可包括例如短距通信。短距通信可包括无线保真(Wi-Fi)、蓝牙、近场通信(NFC)、磁条传输(MST)或GNSS中的至少一个。
本申请实施例还提供一种芯片系统，如图28所示，该芯片系统包括至少一个处理器1401和至少一个接口电路1402。处理器1401和接口电路1402可通过线路互联。例如，接口电路1402可用于从其它装置(例如电子设备的存储器)接收信号。又例如，接口电路1402可用于向其它装置(例如处理器1401)发送信号。示例性的，接口电路1402可读取存储器中存储的指令，并将该指令发送给处理器1401。当所述指令被处理器1401执行时，可使得电子设备执行上述实施例中的各个步骤。当然，该芯片系统还可以包含其他分立器件，本申请实施例对此不作具体限定。
本申请实施例还提供一种计算机存储介质，该计算机存储介质包括计算机指令，当所述计算机指令在上述电子设备上运行时，使得该电子设备执行上述方法实施例中手机执行的各个功能或者步骤。
本申请实施例还提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述方法实施例中手机执行的各个功能或者步骤。
通过以上实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示 或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上内容，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (14)

  1. 一种视频生成方法，其特征在于，所述方法包括：
    获取视频模板的数据；
    采集一个或多个用户的运动数据；所述运动数据包括如下任一项或多项数据：速度、幅度、频率、轨迹；
    根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频。
  2. 根据权利要求1所述的方法，其特征在于，所述视频模板的数据为人物运动视频的数据或场景视频的数据。
  3. 根据权利要求1或2所述的方法，其特征在于，所述运动数据具有对应的第一运动类型，所述第一运动类型对应的运动数据包括轨迹；
    根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频，包括：
    根据所述第一运动类型对应的运动数据中的轨迹，以及所述视频模板的数据，生成所述一个或多个用户的运动视频。
  4. 根据权利要求1或2所述的方法，其特征在于，所述运动数据具有对应的第二运动类型；所述视频模板的数据包括路线数据；
    根据所述运动数据以及所述视频模板的数据，生成所述一个或多个用户的运动视频，包括：
    根据所述第二运动类型对应的运动数据以及所述路线数据，生成所述一个或多个用户沿所述路线数据指示的路线运动的运动视频。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，采集一个或多个用户的运动数据，包括：从目标软件模块或目标硬件模块采集所述一个或多个用户的运动数据；
    或，接收所述一个或多个用户输入的运动数据。
  6. 根据权利要求1-5中任一项所述的方法，其特征在于，所述方法还包括：获取所述一个或多个用户的图像的信息；其中，所述图像的信息是预设图像的信息，或者，所述一个或多个用户的图像的信息是所述一个或多个用户输入的图像的信息。
  7. 根据权利要求6所述的方法，其特征在于，对于所述一个或多个用户中的任一个用户，异步获取所述用户的图像的信息与所述用户的运动数据。
  8. 根据权利要求6或7所述的方法，其特征在于，所述一个或多个用户的图像的信息包括如下任一项或多项的组合：所述一个或多个用户的图像、用于描述所述一个或多个用户的特征的文本、所述一个或多个用户的视频。
  9. 根据权利要求6-8中任一项所述的方法，其特征在于，所述一个或多个用户的运动视频包括多个图像帧，所述多个图像帧的第X个图像帧由所述一个或多个用户在目标时刻的图像插入所述视频模板的第X个图像帧得到；所述目标时刻是第X个图像帧对应的时刻；X为正整数。
  10. 根据权利要求1-9中任一项所述的方法，其特征在于，所述多个用户的运动数据是同步获取的，或者，所述多个用户的运动数据是异步获取的。
  11. 根据权利要求1-10中任一项所述的方法，其特征在于，所述运动数据还包括如 下任一项或多项数据：心率、血压、血氧、体温。
  12. 一种电子设备，其特征在于，包括：处理器，存储器，所述存储器与所述处理器耦合，所述存储器用于存储计算机程序代码，所述计算机程序代码包括计算机指令，当所述处理器从所述存储器中读取所述计算机指令，使得所述电子设备执行如权利要求1-11任一项所述的视频生成方法。
  13. 一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，其特征在于，当所述指令在电子设备上运行时，使得所述电子设备执行如权利要求1-11中任一项所述的视频生成方法。
  14. 一种包含指令的计算机程序产品，其特征在于，当所述计算机程序产品在电子设备上运行时，使得所述电子设备执行如权利要求1-11中任一项所述的视频生成方法。
PCT/CN2022/109033 2021-07-31 2022-07-29 视频生成方法及电子设备 WO2023011356A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110877302.7A CN115690272A (zh) 2021-07-31 2021-07-31 视频生成方法及电子设备
CN202110877302.7 2021-07-31

Publications (1)

Publication Number Publication Date
WO2023011356A1 true WO2023011356A1 (zh) 2023-02-09

Family

ID=85059802

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/109033 WO2023011356A1 (zh) 2021-07-31 2022-07-29 视频生成方法及电子设备

Country Status (2)

Country Link
CN (1) CN115690272A (zh)
WO (1) WO2023011356A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130225369A1 (en) * 2010-10-21 2013-08-29 Bensy, Llc Systems and methods for exercise in an interactive virtual environment
CN105797349A (zh) * 2016-03-17 2016-07-27 深圳市智游人科技有限公司 实景跑步装置、方法及系统
CN106310643A (zh) * 2016-09-27 2017-01-11 深圳市智游人科技有限公司 一种基于真实环境的实景3d运动系统
CN108187301A (zh) * 2017-12-28 2018-06-22 必革发明（深圳）科技有限公司 跑步机人机交互方法、装置及跑步机
CN110141829A (zh) * 2019-05-18 2019-08-20 厦门致诺实业有限公司 智能联网跑步机
CN111862348A (zh) * 2020-07-30 2020-10-30 腾讯科技（深圳）有限公司 视频显示方法、视频生成方法、装置、设备及存储介质
CN112911363A (zh) * 2021-01-15 2021-06-04 深圳市爱都科技有限公司 轨迹视频生成方法、终端设备以及计算机可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130225369A1 (en) * 2010-10-21 2013-08-29 Bensy, Llc Systems and methods for exercise in an interactive virtual environment
CN105797349A (zh) * 2016-03-17 2016-07-27 深圳市智游人科技有限公司 实景跑步装置、方法及系统
CN106310643A (zh) * 2016-09-27 2017-01-11 深圳市智游人科技有限公司 一种基于真实环境的实景3d运动系统
CN108187301A (zh) * 2017-12-28 2018-06-22 必革发明（深圳）科技有限公司 跑步机人机交互方法、装置及跑步机
CN110141829A (zh) * 2019-05-18 2019-08-20 厦门致诺实业有限公司 智能联网跑步机
CN111862348A (zh) * 2020-07-30 2020-10-30 腾讯科技（深圳）有限公司 视频显示方法、视频生成方法、装置、设备及存储介质
CN112911363A (zh) * 2021-01-15 2021-06-04 深圳市爱都科技有限公司 轨迹视频生成方法、终端设备以及计算机可读存储介质

Also Published As

Publication number Publication date
CN115690272A (zh) 2023-02-03

Similar Documents

Publication Publication Date Title
CN111726536B (zh) 视频生成方法、装置、存储介质及计算机设备
US10609334B2 (en) Group video communication method and network device
US11122258B2 (en) Method and apparatus for generating and displaying 360-degree video based on eye tracking and physiological measurements
CN106502388B (zh) 一种互动式运动方法及头戴式智能设备
WO2021244457A1 (zh) 一种视频生成方法及相关装置
WO2022161037A1 (zh) 一种用户确定方法、电子设备和计算机可读存储介质
US20240098211A1 (en) User interface for pose driven virtual effects
WO2022073417A1 (zh) 融合场景感知机器翻译方法、存储介质及电子设备
US20220319061A1 (en) Transmitting metadata via invisible light
WO2022037479A1 (zh) 一种拍摄方法和拍摄系统
WO2021036954A1 (zh) 一种智能语音播放方法及设备
CN113996046B (zh) 热身判断方法、装置及电子设备
CN115188064A (zh) 一种运动指导信息的确定方法、电子设备和运动指导系统
WO2023011356A1 (zh) 视频生成方法及电子设备
US20220318303A1 (en) Transmitting metadata via inaudible frequencies
WO2022111704A1 (zh) 心率检测方法及电子设备
WO2021036839A1 (zh) 摄像头控制的方法、装置和终端设备
CN116440495A (zh) 场景画面的显示方法、装置、终端及存储介质
CN114697516A (zh) 三维模型重建方法、设备和存储介质
CN112447272A (zh) 用于健身训练的提示方法和电子设备
US11874960B2 (en) Pausing device operation based on facial movement
CN114915722B (zh) 处理视频的方法和装置
WO2022161027A1 (zh) 动作提示图标序列生成方法、电子设备和可读存储介质
US20240203072A1 (en) Dynamic augmented reality experience
US20240069626A1 (en) Timelapse re-experiencing system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22852069

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE