WO2021189995A1 - 视频渲染方法、装置、电子设备及存储介质 - Google Patents

视频渲染方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021189995A1
WO2021189995A1 PCT/CN2020/137398 CN2020137398W WO2021189995A1 WO 2021189995 A1 WO2021189995 A1 WO 2021189995A1 CN 2020137398 W CN2020137398 W CN 2020137398W WO 2021189995 A1 WO2021189995 A1 WO 2021189995A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
text
material block
alpha
block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/137398
Other languages
English (en)
French (fr)
Inventor
肖逸
侯沛宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to JP2022549142A priority Critical patent/JP2023515411A/ja
Publication of WO2021189995A1 publication Critical patent/WO2021189995A1/zh
Priority to US17/889,817 priority patent/US20220417591A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs

Definitions

  • the present disclosure relates to the field of video processing technology, and in particular, to a video rendering method, device, electronic equipment, and storage medium.
  • the present disclosure provides a video rendering method, device, electronic device, and storage medium to at least solve the problem that when two or more video effects need to be rendered to a video in the related art, since two material videos cannot be decoded at the same time, they still need to be used Frame-sequence-based rendering can cause the problem of device freezing caused by too much memory space.
  • a video rendering method including: acquiring a video to be rendered and a material video used to render the video to be rendered; the material video is obtained by splicing and combining at least two video special effects
  • the video special effect includes a plurality of material pictures; a plurality of material blocks are disassembled from the material video; the material block is obtained by sequentially arranging and combining a plurality of material pictures belonging to the same video special effect; determining the material block At the target position of the video frame of the video; superimpose the material block into the target position of the video frame to complete the video rendering.
  • a video rendering device including: an acquisition unit configured to perform acquisition of a to-be-rendered video and a material video for rendering the to-be-rendered video; the material video is at least Two video special effects are spliced and combined, the video special effects include multiple material pictures; the disassembly unit is configured to perform disassembly of multiple material blocks from the material video; the material blocks are composed of multiple pieces belonging to the same video The material pictures of the special effects are arranged and combined in order; the determining unit is configured to determine the target position of the material block in the video frame of the video;
  • an electronic device including a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to implement the foregoing The video rendering method described in the first aspect.
  • a storage medium is provided.
  • the electronic device can execute the video rendering method described in the first aspect. .
  • a computer program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of the device obtains data from the readable storage medium.
  • the computer program is read and executed, so that the device executes the video rendering method described in any one of the embodiments of the first aspect.
  • the technical solutions provided by the embodiments of the present disclosure can save memory space occupied by video special effects used for video rendering, avoid device jams, and improve the smoothness of video rendering.
  • Figure 1 is a schematic diagram of the animation effect of a video special effect.
  • Figure 2 is a schematic diagram of a material picture of a video special effect.
  • Figure 3 is a schematic diagram of another video special effect material picture.
  • Fig. 4 is a diagram showing the internal structure of an electronic device according to an exemplary embodiment.
  • Fig. 5 is a flowchart showing a video rendering method according to an exemplary embodiment.
  • Fig. 6 is a schematic diagram showing the placement of material pictures of a certain frame of a material video according to an exemplary embodiment.
  • Fig. 7 is a schematic diagram of calculation of a current material video memory.
  • Fig. 8 is a schematic diagram showing a calculation of a material video memory according to an exemplary embodiment.
  • Fig. 9 is a schematic diagram showing a calculation of a first position coordinate according to an exemplary embodiment.
  • Fig. 10 is a schematic diagram showing a mapping between a material picture and a target effect picture according to an exemplary embodiment.
  • Fig. 11 is a schematic diagram showing a superposition of text images according to an exemplary embodiment.
  • Fig. 12 is a schematic diagram showing a superposition of firework materials according to an exemplary embodiment.
  • Fig. 13 is a flow chart showing the processing of a material picture according to an exemplary embodiment.
  • Fig. 14 is a schematic diagram showing a video rendering effect according to an exemplary embodiment.
  • Fig. 15 is a block diagram showing a video rendering device according to an exemplary embodiment.
  • the video rendering method provided by the present disclosure can be applied to the device 100 as shown in FIG. 4.
  • the device 100 may be a mobile terminal, such as a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and so on.
  • the device 100 may include one or more of the following components: a processing component 101, a memory 102, a power supply component 103, a multimedia component 104, an audio component 105, an input/output (I/O) interface 106, a sensor component 107, and Communication component 108.
  • a processing component 101 a memory 102
  • a power supply component 103 a multimedia component 104
  • an audio component 105 an input/output (I/O) interface 106
  • sensor component 107 a sensor component 107
  • Communication component 108 Communication component
  • the processing component 101 generally controls the overall operations of the device 100, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 101 may include one or more processors 110 to execute instructions to complete all or part of the steps of the foregoing method.
  • the processing component 101 may include one or more modules to facilitate the interaction between the processing component 101 and other components.
  • the processing component 101 may include a multimedia module to facilitate the interaction between the multimedia component 104 and the processing component 101.
  • the memory 102 is configured to store various types of data to support operations in the device 100. Examples of such data include instructions for any application or method operating on the device 100, contact data, phone book data, messages, pictures, videos, and the like.
  • the memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable Programmable read only memory
  • PROM programmable read only memory
  • ROM read only memory
  • magnetic memory flash memory
  • flash memory magnetic or optical disk.
  • the power supply component 103 provides power to various components of the device 100.
  • the power supply component 103 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 100.
  • the multimedia component 104 includes a screen that provides an output interface between the device 100 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure related to the touch or slide operation.
  • the multimedia component 104 includes a front camera and/or a rear camera. When the device 100 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 105 is configured to output and/or input audio signals.
  • the audio component 105 includes a microphone (MIC).
  • the microphone When the device 100 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals.
  • the received audio signal can be further stored in the memory 102 or sent via the communication component 108.
  • the audio component 105 further includes a speaker for outputting audio signals.
  • the I/O interface 106 provides an interface between the processing component 101 and a peripheral interface module.
  • the above-mentioned peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume button, start button, and lock button.
  • the sensor component 107 includes one or more sensors for providing the device 100 with various aspects of state evaluation.
  • the sensor component 107 can detect the on/off status of the device 100 and the relative positioning of components.
  • the component is the display and keypad of the device 100.
  • the sensor component 107 can also detect the position change of the device 100 or a component of the device 100. , The presence or absence of contact between the user and the device 100, the orientation or acceleration/deceleration of the device 100, and the temperature change of the device 100.
  • the sensor assembly 107 may include a proximity sensor configured to detect the presence of nearby objects when there is no physical contact.
  • the sensor component 107 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 107 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 108 is configured to facilitate wired or wireless communication between the device 100 and other devices.
  • the device 100 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G, or 5G), or a combination thereof.
  • the communication component 108 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 108 further includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the device 100 can be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gates Array (FPGA), controller, microcontroller, microprocessor, or other electronic components are implemented to implement the above-mentioned video rendering method.
  • ASIC application-specific integrated circuits
  • DSP digital signal processors
  • DSPD digital signal processing devices
  • PLD programmable logic devices
  • FPGA field programmable gates Array
  • controller microcontroller, microprocessor, or other electronic components are implemented to implement the above-mentioned video rendering method.
  • Fig. 5 is a flowchart showing a video rendering method according to an exemplary embodiment. As shown in Fig. 5, the video rendering method is used in the device 100 of Fig. 4.
  • step S51 a to-be-rendered video and a material video for rendering the to-be-rendered video are obtained; the material video is obtained by splicing and combining at least two video special effects, and the video special effects include multiple material pictures.
  • each video effect is a combination of multiple material pictures.
  • the method may further include the following steps:
  • the video special effects is a text special effect composed of text pictures containing text information
  • the text material pictures belonging to the same text special effect and belonging to the same text are arranged and combined as text material blocks;
  • a material video is obtained by splicing and combining.
  • a material video Before rendering the to-be-rendered video, a material video can be spliced and combined for the to-be-rendered video in the early stage, where the material video is obtained by splicing two or more video special effects.
  • the material pictures of the video special effects can be neatly placed in the material video according to the special effect requirements.
  • the material pictures can be arranged according to the special effect requirements, and then according to the special effect requirements.
  • the sequence of arrangement combines the material pictures one by one into each frame of the material video. For material pictures that appear at the same time in the video frame but belong to different video effects, they are spliced in the same frame of the material video.
  • the video special effects include text special effects and animation special effects.
  • the text special effects are composed of text images containing text information, such as the text effects of two texts "Year of the Rat", and the animation special effects are made of materials other than text images.
  • Picture composition for example, a firework special effect can be composed of a firework picture containing fireworks, and a flower special effect can be composed of a flower picture containing flowers.
  • Figure 6 is a schematic diagram of the placement of the material pictures of a certain frame of the material video.
  • Each frame of the material video will be spliced with text pictures and fireworks pictures that appear at the same time.
  • text pictures for text special effects can belong to the same
  • a text picture of a text is in a grid area, the grid space is reused and the space is not left as much as possible to achieve the highest utilization of space.
  • fireworks pictures with special effects of fireworks can also be placed in a grid area to achieve the highest utilization of space.
  • the material pictures in the same grid area are sorted and combined into one material block.
  • step S52 multiple material blocks are disassembled from the material video; the material blocks are obtained by sequentially arranging and combining multiple material pictures belonging to the same video special effect.
  • the material video when using the material video to render the to-be-rendered video, it is first necessary to disassemble each material picture in the material video, and the disassembly can be performed in tools such as Opengl's vertex shader.
  • the material pictures belonging to the same video special effect are neatly placed in a square area. Therefore, when the material pictures are obtained from the material video, the material pictures belonging to the same square (in the The first position coordinates of the material video are the same) in the material block, thereby obtaining the material picture of the material block.
  • step S53 the target position of the material block in the video frame of the video is determined.
  • the target position of the material block it is possible to set the target position of the material block to be placed in the video frame of the video, so that the desired ideal effect picture can be obtained.
  • the material picture that can be set as a material block is placed in the middle of the video frame of the video to be rendered, it can also be placed in the lower left corner of the video frame of the video to be rendered, or it can be set to overlap the entire video frame of the video to be rendered .
  • step S54 the material block is superimposed on the target position of the video frame to complete the video rendering.
  • the material block After determining the target position of each material block in the video frame in the material video, the material block can be correspondingly superimposed to the target position of the video frame of the to-be-rendered video to achieve video rendering.
  • this embodiment splices at least two video special effects into the same material video, and then uses the material video to render the to-be-rendered video, the memory space occupied by the video special effect for video rendering can be saved.
  • the four sets of text pictures with text effects and fireworks pictures with fireworks effects are spliced and combined in the same material video (combined video material), and the total memory space occupied is 2150.4KB, which is equivalent Compared with the way that only one video special effect can be converted into video, it reduces the memory space occupation by 46.13%.
  • the video to be rendered and the material video used to render the video to be rendered are first obtained; the material video is obtained by splicing and combining multiple video special effects, and the video special effect includes multiple material pictures. Disassemble multiple material blocks in the video frame, determine the target position of the material block in the video frame of the video, and then superimpose the material block into the target position of the video frame to complete the video rendering. This can save the memory space occupied by the video effects used for video rendering, avoid device jams, and improve the smoothness of video rendering.
  • the method may further include the following steps:
  • the video special effects is a text special effect composed of text pictures containing text information
  • the text material pictures belonging to the same text special effect and belonging to the same text are arranged and combined as text material blocks;
  • a material video is obtained by splicing and combining.
  • text effects and other special effects can be neatly placed in the material video.
  • the text pictures of each text in the text effects can be neatly placed, and Arrange the firework pictures with firework effects neatly.
  • a corresponding alpha material block can be generated for the text material block, where the alpha picture in the alpha material block corresponds to the text picture in the text material block in a one-to-one correspondence.
  • the step S52 may specifically include the following steps: acquiring the first position coordinates of the material block in the material video; disassembling from the material video according to the first position coordinates Material block.
  • the material pictures can be divided into small squares (material blocks), and then when the material pictures are obtained, they can be obtained according to the material blocks, so that multiple material pictures belonging to the same video special effect can be obtained at one time.
  • the width and height of the material video (pixel value 960*1980) is known, and the width and height (pixel value) of each square is also known.
  • the value range of the coordinates is 0 ⁇ 1, set the horizontal axis as the x axis and the vertical axis as the y axis.
  • the step S53 may specifically include the following steps: acquiring a coordinate mapping relationship; the coordinate mapping relationship is that the coordinates of the material block at the first position of the material video and the material block at the first position The mapping relationship between the second position coordinates of the video frame of the video; according to the coordinate mapping relationship, the second position coordinates corresponding to the first position coordinates are determined, and the position where the second position coordinates are located is taken as the target Location.
  • the extracted material pictures need to be repositioned.
  • This embodiment establishes a one-to-one correspondence between the material video and the ideal rendering in advance.
  • the second position coordinate of the material picture in the video frame of the video to be rendered can be determined according to the coordinate mapping relationship, and used as the material block in the video of the video to be rendered The target position placed in the frame.
  • the coordinate mapping relationship is established according to the required target renderings, so that the four words "Happy Year of the Rat” will be moved from the four points (first position coordinates) in the material video to the video to be rendered. Four points (second position coordinates).
  • the material picture in the material video is mapped according to the established coordinate mapping relationship
  • the material picture can be superimposed on the second coordinate position of the video frame.
  • the background of the material picture is a black background. This involves the superposition and mixing of the material picture and the video frame of the video to be rendered. Note that text pictures and non-text pictures (For example, firework pictures) and video frames are superimposed in a different way.
  • step S54 that is, the superimposing the material block into the target position of the video frame includes:
  • the material block is a text material block composed of text pictures containing text information
  • the text material block and the alpha material block corresponding to the text material block are superimposed on the target position of the video frame; wherein,
  • the alpha material block includes alpha pictures corresponding to the text pictures of the text material block one-to-one.
  • the material picture of the material block and the alpha picture of the alpha material block are superimposed on the second position coordinates of the video frame of the video.
  • the text image is superimposed using the "normal" superimposition method.
  • This superimposition method needs to use the alpha channel of the text image, that is, it involves the superposition of the alpha image.
  • "Normal" superimposition The formula is:
  • color is the video rendering effect picture after overlaying text pictures
  • overlay is the RGB value of text pictures (text materials), which is reflected here as the left part of the material video in Figure 6
  • base is the RGB value of the video frame
  • alpha is the text picture
  • Corresponding alpha picture is embodied here as the right part of the material video in Figure 6.
  • the "addition" superposition method is adopted for the superposition of non-text pictures and video frames of the video to be rendered.
  • This superposition method does not require the use of fireworks materials
  • the "addition" superposition formula is:
  • color is the video rendering effect picture after superimposing text pictures and fireworks pictures
  • overlay is the RGB value of non-text pictures (fireworks materials), here it is reflected in the middle part of the material video in Figure 6, and base is after text pictures have been superimposed The RGB value of the video frame.
  • material pictures of different mixing modes such as text pictures of text special effects and fireworks pictures of firework special effects, are spliced into one material video to achieve the purpose of realizing the mixing of multiple video special effects with a low memory space occupation.
  • At least two video special effects can be spliced and combined into one material video, and video rendering can be performed on the video.
  • a material video contains four sets of text pictures with special text effects: "Gong Xi Fa Cai”, “Happy Year of the Rat”, “Lucky Money”, and “Good Fortune”, as well as a set of fireworks pictures with special effects.
  • Figure 13 repeats the steps of splicing and combination, disassembly, positioning, and superimposition, and four video rendering effect diagrams as shown in Figure 14 can be obtained respectively.
  • the spatial multiplexing technology of material video proposed in this embodiment puts the material pictures of video special effects, especially text pictures, into a small grid to optimize the effective use area of the material pictures in the material video. It is not necessary to arrange and combine the material pictures of video special effects.
  • the rendering method of this embodiment can be used to disassemble, locate and superimpose the material pictures.
  • One material video can be used to render multiple videos for the video. Special effects.
  • the device memory and performance need to be strictly controlled, for example, for some models, the device memory is greatly optimized, and the smoothness of video rendering is improved.
  • steps in the flowchart of FIG. 5 are displayed in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in FIG. 5 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times, and the order of execution of these steps or stages is also It is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
  • Fig. 15 is a block diagram showing a video rendering device according to an exemplary embodiment. 15, the device includes an acquisition unit 151, a disassembly unit 152, a determination unit 153, and a rendering unit 154.
  • the obtaining unit 151 is configured to perform obtaining a video to be rendered and a material video used to render the video; the material video is obtained by splicing and combining at least two video special effects, and the video special effects include multiple material pictures;
  • the disassembling unit 152 is configured to disassemble a plurality of material blocks from the material video; the material blocks are obtained by sequentially arranging and combining a plurality of material pictures belonging to the same video special effect;
  • the determining unit 153 is configured to determine the target position of the material block in the video frame of the video;
  • the rendering unit 154 is configured to perform superimposition of the material block into the target position of the video frame to complete video rendering.
  • the rendering unit 154 is configured to perform when the material block is a text material block composed of text pictures containing text information, the text material block is corresponding to the text material block
  • the alpha material block is superimposed on the target position of the video frame; wherein, the alpha material block includes an alpha picture corresponding to the text picture of the text material block one-to-one.
  • the disassembly unit 152 is configured to perform acquiring the first position coordinates of the material block in the material video; and disassemble the material video according to the first position coordinates. Unpack the material block.
  • the determining unit 153 is configured to perform a coordinate mapping relationship; the coordinate mapping relationship is that the coordinates of the first position of the material block in the material video and the position of the material block are The mapping relationship between the second position coordinates of the video frame of the video to be rendered; the second position coordinate corresponding to the first position coordinate is determined according to the coordinate mapping relationship; the position where the second position coordinate is As the target location.
  • the device further includes: a special effect acquisition unit configured to execute acquiring at least two video special effects; the video special effect includes multiple material pictures; and a splicing and combining unit configured to execute
  • the video special effects is a text special effect composed of text pictures containing text information
  • the text pictures belonging to the same text special effect and belonging to the same text are arranged and combined as a text material block; an alpha for the text material block is generated
  • the alpha material block includes an alpha picture one-to-one corresponding to the text picture of the text material block; according to the text material block and the alpha material block, a material video is obtained by splicing and combining.
  • an electronic device is also provided.
  • the schematic structural diagram of the electronic device may be as shown in FIG. 1.
  • the electronic device includes: a processor; a memory for storing executable instructions of the processor;
  • the processor is configured to execute the instructions to implement the video rendering method as described in the foregoing embodiments.
  • non-transitory computer-readable storage medium including instructions, such as the memory 102 including instructions, and the foregoing instructions may be executed by the processor 120 of the device 100 to complete the foregoing method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
  • a computer program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of the device reads from the readable storage medium And execute the computer program, so that the device executes the video rendering method described in the above embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Studio Circuits (AREA)

Abstract

本公开关于一种视频渲染方法、装置、电子设备及存储介质，所述方法可以包括：获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；确定所述素材块在所述视频的视频帧的目标位置；将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。

Description

视频渲染方法、装置、电子设备及存储介质
相关申请的交叉引用
本公开要求于2020年03月24日提交中国专利局、申请号为202010212800.5、发明名称为“视频渲染方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及视频处理技术领域，尤其涉及一种视频渲染方法、装置、电子设备及存储介质。
背景技术
随着诸如智能手机等设备软硬件技术的发展，短视频的设备渲染越来越流行。在视频特效对视频进行渲染，如何更好的实现对视频处理成为亟待解决的问题。
发明内容
本公开提供一种视频渲染方法、装置、电子设备及存储介质，以至少解决相关技术中在需要对视频渲染两个或者以上的视频特效时，由于无法同时解码两个素材视频，故而仍然需要使用帧序列方式渲染，所造成的占用的内存空间太多所造成的设备卡顿的问题。
根据本公开实施例的第一方面，提供一种视频渲染方法，包括：获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；确定所述素材块在所述视频的视频帧的目标位置；将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
根据本公开实施例的第二方面，提供一种视频渲染装置，包括：获取单元，被配置为执行获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；拆解单元，被配置为执行从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；确定单元，被配置为执行确定所述素材块在所述视频的视频帧的目标位置；
根据本公开实施例的第三方面，提供一种电子设备，包括处理器；用于存储所述处理器 可执行指令的存储器；其中，所述处理器被配置为执行所述指令，以实现上述第一方面所述的视频渲染方法。
根据本公开实施例的第四方面，提供一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行上述第一方面所述的视频渲染方法。
根据本公开实施例的第五方面，提供一种计算机程序产品，所述程序产品包括计算机程序，所述计算机程序存储在可读存储介质中，设备的至少一个处理器从所述可读存储介质读取并执行所述计算机程序，使得设备执行第一方面的任一项实施例中所述的视频渲染方法。
本公开的实施例提供的技术方案可以节省用于视频渲染的视频特效占用的内存空间，避免设备卡顿，提升视频渲染的流畅度。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理，并不构成对本公开的不当限定。
图1是一种视频特效的动画效果示意图。
图2是一种视频特效的素材图片的示意图。
图3是另一种视频特效的素材图片的示意图。
图4是根据一示例性实施例示出的一种电子设备的内部结构图。
图5是根据一示例性实施例示出的一种视频渲染方法的流程图。
图6是根据一示例性实施例示出的一种素材视频的某一帧的素材图片摆放示意图。
图7是目前一种素材视频内存的计算示意图。
图8是根据一示例性实施例示出的一种素材视频内存的计算示意图。
图9是根据一示例性实施例示出的一种第一位置坐标的计算示意图。
图10是根据一示例性实施例示出的一种素材图片与目标效果图的映射示意图。
图11是根据一示例性实施例示出的一种文字图片的叠加示意图。
图12是根据一示例性实施例示出的一种烟花素材的叠加示意图。
图13是根据一示例性实施例示出的一种素材图片的处理流程图。
图14是根据一示例性实施例示出的一种视频渲染效果示意图。
图15是根据一示例性实施例示出的一种视频渲染装置的框图。
具体实施方式
为了使本领域普通人员更好地理解本公开的技术方案，下面将结合附图，对本公开实施例中的技术方案进行清楚、完整地描述。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
本公开所提供的视频渲染方法，可以应用于如图4所示的设备100中。设备100可以是移动端，如：移动电话、计算机、数字广播终端、消息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等。
参照图4，设备100可以包括以下一个或多个组件：处理组件101、存储器102、电源组件103、多媒体组件104、音频组件105、输入/输出(I/O)的接口106、传感器组件107以及通信组件108。对这些组件详细说明如下：
处理组件101通常控制设备100的整体操作，诸如与显示、电话呼叫、数据通信、相机操作和记录操作相关联的操作。处理组件101可以包括一个或多个处理器110来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件101可以包括一个或多个模块，便于处理组件101和其他组件之间的交互。例如，处理组件101可以包括多媒体模块，以方便多媒体组件104和处理组件101之间的交互。
存储器102被配置为存储各种类型的数据以支持在设备100的操作。这些数据的示例包括用于在设备100上操作的任何应用程序或方法的指令、联系人数据、电话簿数据、消息、图片、视频等。存储器102可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)、电可擦除可编程只读存储器(EEPROM)、可擦除可编程只读存储器(EPROM)、可编程只读存储器(PROM)、只读存储器(ROM)、磁存储器、快闪存储器、磁盘或光盘。
电源组件103为设备100的各种组件提供电力。电源组件103可以包括电源管理系统，一个或多个电源，及其他与为设备100生成、管理和分配电力相关联的组件。
多媒体组件104包括在所述设备100和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以 感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件104包括一个前置摄像头和/或后置摄像头。当设备100处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件105被配置为输出和/或输入音频信号。例如，音频组件105包括一个麦克风(MIC)，当设备100处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器102或经由通信组件108发送。在一些实施例中，音频组件105还包括一个扬声器，用于输出音频信号。
I/O接口106为处理组件101和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件107包括一个或多个传感器，用于为设备100提供各个方面的状态评估。例如，传感器组件107可以检测到设备100的打开/关闭状态，组件的相对定位，例如所述组件为设备100的显示器和小键盘，传感器组件107还可以检测设备100或设备100一个组件的位置改变，用户与设备100接触的存在或不存在，设备100方位或加速/减速和设备100的温度变化。传感器组件107可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件107还可以包括光传感器，如CMOS或CCD图片传感器，用于在成像应用中使用。在一些实施例中，该传感器组件107还可以包括加速度传感器、陀螺仪传感器、磁传感器、压力传感器或温度传感器。
通信组件108被配置为便于设备100和其他设备之间有线或无线方式的通信。设备100可以接入基于通信标准的无线网络，如WiFi，运营商网络(如2G、3G、4G或5G)，或它们的组合。在一个示例性实施例中，通信组件108经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件108还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在一些实施例中，设备100可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述视频渲染方法。
图5是根据一示例性实施例示出的一种视频渲染方法的流程图，如图5所示，视频渲染方法用于图4设备100中。
在步骤S51中，获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视 频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片。
待渲染视频时需要为其添加视频特效的视频，在本实施例中需要为待渲染视频添加两个或者两个以上的视频特效。其中，每个视频特效是由多张素材图片组合所得。
在一些实施例中，本实施例在渲染视频之前，需要利用视频特效进行拼接组合，得到一个素材视频。在一些实施例中，所述方法还可以包括如下步骤：
获取至少两个视频特效；
当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字素材图片进行排列组合，作为文字素材块；
生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；
根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
在渲染待渲染视频之前，可以在前期针对待渲染视频进行素材视频的拼接组合，其中，素材视频为两个或者两个以上的视频特效拼接所得。
在素材视频的前期拼接组合时，可以根据特效需求将视频特效的素材图片整齐摆放在素材视频中，比如，对于属于同一视频特效的素材图片，可以将素材图片按照特效需求进行排列，然后按照排列的顺序将素材图片一一组合到素材视频的各个帧中，对于在视频帧同一时刻出现但属于不同视频特效的素材图片，则拼接在素材视频的同一帧中。
在一些实施例中，视频特效包括文字特效和动画特效，其中文字特效是由包含文字信息的文字图片组成，比如“鼠年”两个文字的文字特效，动画特效则是由非文字图片的素材图片组成，比如烟花特效可以由包含烟花的烟花图片组成，鲜花特效可以由包含鲜花的鲜花图片。
假设现有“恭喜发财”、“鼠年快乐”、“招财进宝”、“大吉大利”四套文字特效的文字图片和烟花特效的烟花图片。在拼接素材视频时，可以将利用流行的视频制作软件，比如After Effect(简称AE)进行拼接组合。在AE中，参照图6，可以将文字特效中各个文字对应的文字图片整齐摆放(如图6左边所示)，还需要将烟花特效的烟花图片整齐摆放(如图6中间所示)。
图6为素材视频的某一帧的素材图片摆放示意图，在素材视频的每一帧都会拼接有在同一时刻出现的文字图片和烟花图片，比如，针对文字特效的文字图片，可以将属于同一个文字的文字图片在一个方格的区域内，复用方格空间且尽量不留白，以实现空间最高利用率。当然，对于烟花特效的烟花图片，同样可以放在一个方格的区域内，以实现空间最高利用率。其中，在同一个方格的区域内的素材图片排序组合为一个素材块。比如，“鼠”这个文字所在的方格的区域内，所有的“鼠”的文字图片组合为一个“鼠”文字的素材块，烟花所在的 方格的区域内，所有的烟花图片组合为一个“烟花”的素材块。
在一些实施例中，在对视频渲染两个或者两个以上的视频特效时，将需要获取到待渲染视频以及针对该待渲染视频的预先拼接组合的素材视频。
在步骤S52中，从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得。
在一些实施例中，在利用素材视频渲染待渲染视频时，首先需要将素材视频中的各个素材图片拆解开来，拆解可以在Opengl的顶点着色器等工具中进行操作。
在拼接素材图片得到素材视频时，是将属于同一视频特效的素材图片整齐摆放在一个方格的区域内，因此，在从素材视频中获取素材图片时，可以获取到属于同一方格(在素材视频的第一位置坐标相同)内的素材块，从而获得素材块的素材图片。
在步骤S53中，确定所述素材块在所述视频的视频帧的目标位置。
在一些实施例中，可以针对素材块中，设置好其将要摆放在视频的视频帧的目标位置，从而可以获得所需的理想效果图。比如可以设置为素材块的素材图片摆放在待渲染视频的视频帧的中间，也可以设置摆放在待渲染视频的视频帧的左下角，也可以设置为与待渲染视频的整个视频帧重叠。
在步骤S54中，将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
在确定素材视频中每个素材块在视频帧的目标位置后，就可以将素材块相应叠加至该待渲染视频的视频帧的目标位置中，实现视频渲染。
其中，由于本实施例是将至少两个视频特效拼接在同一个素材视频中，然后利用素材视频对待渲染视频进行渲染，因此可以节省用于视频渲染的视频特效占用的内存空间。
参照图7，假设现有“恭喜发财”、“鼠年快乐”、“招财进宝”、“大吉大利”四套文字特效的文字图片和烟花特效的烟花图片，如果将烟花特效的烟花图片转换为素材视频，则烟花特效的素材视频占用的内存空间的大小为128KB，四个文字特效的文字图片则还是帧序列，那么四个文字特效的文字图片占用的内存空间的大小为3864KB，则需要占用总占用的内存空间的大小为3992KB。
参照图8，应用本实施例，将四套文字特效的文字图片和烟花特效的烟花图片均拼接组合在同一素材视频(组合视频素材)中，则总占用的内存空间的大小为2150.4KB，相比只能将一个视频特效转换为视频的方式而言，减少了46.13％的内存空间的占用。
在上述视频渲染方法中，首先获取待渲染视频以及用于渲染待渲染视频的素材视频；其中，素材视频为通过多个视频特效拼接组合所得，且该视频特效包括多张素材图片，从素材视频中拆解出多个素材块，并确定素材块在视频的视频帧的目标位置，然后将素材块叠加至视频帧的目标位置中，完成视频渲染。由此可以节省用于视频渲染的视频特效占用的内存空 间，避免设备卡顿，提升视频渲染的流畅度。
在一些实施例中，在所述获取待渲染视频以及用于渲染所述视频的素材视频之前，所述方法还可以包括如下步骤：
获取至少两个视频特效；
当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字素材图片进行排列组合，作为文字素材块；
生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；
根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。在视频的前期拼接组合时，利用视频制作软件，可以将文字特效和其他特效都整齐摆放在素材视频中，比如，参照图6，可以将文字特效中各个文字的文字图片整齐摆放，以及将烟花特效的烟花图片整齐摆放。
需要说明的是，由于素材视频没有alpha通道，而文字特效的混合需要alpha通道，所以还需要将文字图片的alpha通道单独放进去(如图6右边所示)。具体的，可以针对文字素材块生成相应的alpha素材块，其中，在alpha素材块中的alpha图片与文字素材块中的文字图片是一一对应的。
在一些实施例中，所述步骤S52，具体可以包括如下步骤：获取所述素材块在所述素材视频中的第一位置坐标；根据所述第一位置坐标从所述素材视频中拆解出素材块。
在渲染待渲染视频时，需要将素材视频中的各个素材图片拆解出来，可以是在Opengl的顶点着色器中进行操作。可以将素材图片划分成一个一个的小方块(素材块)，然后在获取素材图片时，可以按照素材块进行获取，使得能够一次性得到属于同一视频特效的多张素材图片。
参照9图所示，素材视频的宽高(像素值960*1980)是已知的，每个方格的宽高(像素值)也是已知的，假设要取出“鼠”素材块的素材图片，坐标的数值范围是0～1，设横向为x轴，纵向为y轴，“鼠”素材块的所在方格的四个点为ABCD，如A点，在“鼠”的左下角，其x坐标就为0，y坐标为480/960＝0.5；又如D点，在“鼠”的右上角，x坐标为180/1980＝0.09，y坐标为240*3/960＝0.75，按照同样方法，分别计算出“鼠”素材块的在素材视频中的四个点ABCD的位置坐标，得到四个点的位置坐标A(0,0.5)，B(0.09,0.5)，C(0,0.75)，D(0.09,0.75)后,可以单独取出“鼠”素材块，从而取出“鼠”素材块，同理也可以类似的取出素材视频中的其他素材块。
在一些实施例中，所述步骤S53，具体可以包括如下步骤：获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述视频的视频 帧的第二位置坐标之间的映射关系；根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标，并将所述第二位置坐标所在的位置作为目标位置。
由于在拼接组合得到素材视频的时候，打乱了原有素材图片的排列位置，所以需要对取出的素材图片重新定位，本实施例预先在素材视频和理想效果图两者之间建立一一对应的坐标映射关系，根据素材图片在素材视频的第一位置坐标，即可根据坐标映射关系确定该素材图片在待渲染视频的视频帧的第二位置坐标，以作为素材块在待渲染视频的视频帧中摆放的目标位置。
如图10所示，根据所需要的目标效果图建立坐标映射关系，从而将分别将“鼠年快乐”四个字从素材视频中的四个点(第一位置坐标)移动到待渲染视频的四个点(第二位置坐标)上。
在将素材视频中的素材图片根据已建立坐标映射关系，在确定素材图片在视频帧的第二位置坐标后，就可以将素材图片叠加至视频帧的第二坐标位置中。在移动到待渲染视频的视频帧后可以看到，素材图片的背景为黑底，此处涉及到素材图片与待渲染视频的视频帧的叠加混合，需要注意的是，文字图片和非文字图片(比如烟花图片)与视频帧的叠加方式不同。
在一示例性实施例中，步骤S54，即所述将所述素材块叠加至所述视频帧的目标位置中包括：
当所述素材块为包含文字信息的文字图片组成的文字素材块时，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片。
将所述素材块的素材图片和所述alpha素材块的alpha图片叠加至所述视频的视频帧的第二位置坐标中。
在本实施例中，文字图片的叠加，采取的是“正常”叠加方式，此叠加方式需要用到文字图片的alpha通道，也即是涉及到alpha图片的叠加，参照图11，“正常”叠加公式为：
color＝overlay+base*(1.0-alpha)
其中，color为叠加文字图片后的视频渲染效果图，overlay为文字图片(文字素材)的RGB值，此处体现为图6素材视频的左边部分，base为视频帧的RGB值，alpha为文字图片对应的alpha图片，此处体现为图6素材视频的右边部分。
其中，alpha值的范围是0～1，0代表完全透明，假设素材图片完全透明(overlay就是透明的)，那么base*(1.0-0.0)＝base即最终的颜色就是底部的颜色，即视频的视频帧的颜色，体现在左图中没有文字的部分；同理，1代表完全不透明，那么overlay就有东西，而base*(1.0-1.0)＝0.0即最终颜色就是素材图片的颜色，也就是说素材图片完全不透明的时候，最终就只有素材图片的颜色，而不会透过素材图片看到待渲染视频的视频帧 的颜色。
在一示例性实施例中，针对非文字图片与待渲染视频的视频帧的叠加，比如烟花图片与视频帧的叠加，采取的是“相加”叠加方式，此叠加方式不需要用到烟花素材的alpha通道，参照图12，“相加”叠加公式为：
color＝overlay+base
其中，color为叠加文字图片和烟花图片后的视频渲染效果图，overlay为非文字图片(烟花素材)的RGB值，此处体现为图6素材视频的中边部分，base为已经叠加文字图片后的视频帧的RGB值。
本实施例将不同混合方式的素材图片，比如文字特效的文字图片和烟花特效的烟花图片拼接在一个素材视频里，达到了以低的内存空间的占用实现多种视频特效混合的目的。
应用本实施例，可以将至少两个视频特效拼接组合在一个素材视频中，并对视频进行视频渲染。例如，假设一素材视频中有“恭喜发财”、“鼠年快乐”、“招财进宝”、“大吉大利”这四套文字特效的文字图片和一套烟花特效的烟花图片，可以按照如图13重复执行拼接组合、拆解、定位、叠加的步骤，可分别得到如图14所示的四种视频渲染效果图。
本实施例提出的素材视频的空间复用技术，将视频特效的素材图片，尤其是文字图片放入到一个小方格中，以优化素材图片在素材视频中的有效使用面积，在空间上对不用样式的视频特效的素材图片进行排列组合，在设备解码视频后，就可以利用本实施例的渲染方法对素材图片进行拆解、定位与叠加，利用一个素材视频就可以为视频渲染多种视频特效。本实施例在设备内存和性能都需要严格控制的情况下，比如对于部分机型，大大优化了设备内存，提升视频渲染的流畅度。
应该理解的是，虽然图5的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图5中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
图15是根据一示例性实施例示出的一种视频渲染装置框图。参照图15，该装置包括获取单元151，拆解单元152，确定单元153和渲染单元154。
获取单元151，被配置为执行获取待渲染视频以及用于渲染所述视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；
拆解单元152，被配置为执行从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；
确定单元153，被配置为执行确定所述素材块在所述视频的视频帧的目标位置；
渲染单元154，被配置为执行将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
在一示例性实施例中，所述渲染单元154，被配置为执行当所述素材块为包含文字信息的文字图片组成的文字素材块时，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片。
在一示例性实施例中，所述拆解单元152，被配置为执行获取所述素材块在所述素材视频中的第一位置坐标；根据所述第一位置坐标从所述素材视频中拆解出素材块。
在一示例性实施例中，所述确定单元153，被配置为执行获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述待渲染视频的视频帧的第二位置坐标之间的映射关系；根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标；将所述第二位置坐标所在的位置作为目标位置。
在一示例性实施例中，所述装置还包括：特效获取单元，被配置为执行获取至少两个视频特效；所述视频特效包括多张素材图片；拼接组合单元，被配置为执行
当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字图片进行排列组合，作为文字素材块；生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
在一示例性实施例中，还提供了一种电子设备，其结构示意图可以如图1所示，该电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令，以实现如前述实施例所述的视频渲染方法。
在一示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器102，上述指令可由设备100的处理器120执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
在一示例性实施例中，提供一种计算机程序产品，所述程序产品包括计算机程序，所述计算机程序存储在可读存储介质中，设备的至少一个处理器从所述可读存储介质读取并执行所述计算机程序，使得设备执行如上实施例中所述的视频渲染方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施 方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (20)

  1. 一种视频渲染方法，包括：
    获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；
    从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；
    确定所述素材块在所述待渲染视频的视频帧的目标位置；
    将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
  2. 根据权利要求1所述的视频渲染方法，所述将所述素材块叠加至所述视频帧的目标位置中包括：
    基于所述素材块为包含文字信息的文字图片组成的文字素材块，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片。
  3. 根据权利要求1所述的视频渲染方法，所述从所述素材视频中拆解出多个素材块包括：
    获取所述素材块在所述素材视频中的第一位置坐标；
    根据所述第一位置坐标从所述素材视频中拆解出素材块。
  4. 根据权利要求3所述的视频渲染方法，所述确定所述素材块在所述待渲染视频的视频帧的目标位置包括：
    获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述待渲染视频的视频帧的第二位置坐标之间的映射关系；
    根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标；
    将所述第二位置坐标所在的位置作为目标位置。
  5. 根据权利要求1所述的视频渲染方法，所述获取待渲染视频以及用于渲染所述视频的素材视频包括：
    获取至少两个视频特效；
    当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字图片进行排列组合，作为文字素材块；
    生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；
    根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
  6. 一种视频渲染装置，包括：
    获取单元，被配置为执行获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；
    拆解单元，被配置为执行从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；
    确定单元，被配置为执行确定所述素材块在所述视频的视频帧的目标位置；
    渲染单元，被配置为执行将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
  7. 根据权利要求6所述的视频渲染装置，所述渲染单元，被配置为执行当所述素材块为包含文字信息的文字图片组成的文字素材块时，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片。
  8. 根据权利要求6所述的视频渲染装置，所述拆解单元，被配置为执行获取所述素材块在所述素材视频中的第一位置坐标；根据所述第一位置坐标从所述素材视频中拆解出素材块。
  9. 根据权利要求9所述的视频渲染装置，所述确定单元，被配置为执行获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述待渲染视频的视频帧的第二位置坐标之间的映射关系；根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标；将所述第二位置坐标所在的位置作为目标位置。
  10. 根据权利要求6所述的视频渲染装置，所述装置还包括：
    特效获取单元，被配置为执行获取至少两个视频特效当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字图片进行排列组合，作为文字素材块；生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
  11. 一种电子设备，其特征在于，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令，以实现以下操作：
    获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；
    从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；
    确定所述素材块在所述待渲染视频的视频帧的目标位置；
    将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
  12. 根据权利要求11所述的电子设备，所述处理器被执行为所述指令，以实现以下操作：
    基于所述素材块为包含文字信息的文字图片组成的文字素材块，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片。
  13. 根据权利要求11所述的电子设备，所述处理器被执行为所述指令，以实现以下操作：
    获取所述素材块在所述素材视频中的第一位置坐标；
    根据所述第一位置坐标从所述素材视频中拆解出素材块。
  14. 根据权利要求13所述的电子设备，所述处理器被执行为所述指令，以实现以下操作：
    获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述待渲染视频的视频帧的第二位置坐标之间的映射关系；
    根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标；
    将所述第二位置坐标所在的位置作为目标位置。
  15. 根据权利要求11所述的电子设备，所述处理器被执行为所述指令，以实现以下操作：
    获取至少两个视频特效；
    当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字图片进行排列组合，作为文字素材块；
    生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；
    根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
  16. 一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下操作：
    获取待渲染视频以及用于渲染所述待渲染视频的素材视频；所述素材视频为通过至少两个视频特效拼接组合所得，所述视频特效包括多张素材图片；
    从所述素材视频中拆解出多个素材块；所述素材块由多张属于同一视频特效的素材图片按序排列组合所得；
    确定所述素材块在所述待渲染视频的视频帧的目标位置；
    将所述素材块叠加至所述视频帧的目标位置中，完成视频渲染。
  17. 根据权利要求16所述的存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下操作：
    基于所述素材块为包含文字信息的文字图片组成的文字素材块，将所述文字素材块和所述文字素材块对应的alpha素材块，叠加至所述视频帧的目标位置；其中，所述alpha素材 块包括与所述文字素材块的文字图片一一对应的alpha图片。
  18. 根据权利要求16所述的存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下操作：
    获取所述素材块在所述素材视频中的第一位置坐标；
    根据所述第一位置坐标从所述素材视频中拆解出素材块。
  19. 根据权利要求18所述的存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下操作：
    获取坐标映射关系；所述坐标映射关系为所述素材块在所述素材视频的第一位置坐标与所述素材块在所述待渲染视频的视频帧的第二位置坐标之间的映射关系；
    根据所述坐标映射关系，确定与所述第一位置坐标对应的第二位置坐标；
    将所述第二位置坐标所在的位置作为目标位置。
  20. 根据权利要求16所述的存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下操作：
    获取至少两个视频特效；
    当至少一个所述视频特效为包含文字信息的文字图片组成的文字特效时，将属于同一文字特效且属于同一文字的文字图片进行排列组合，作为文字素材块；
    生成针对所述文字素材块的alpha素材块，所述alpha素材块包括与所述文字素材块的文字图片一一对应的alpha图片；
    根据所述文字素材块和所述alpha素材块，拼接组合得到素材视频。
PCT/CN2020/137398 2020-03-24 2020-12-17 视频渲染方法、装置、电子设备及存储介质 Ceased WO2021189995A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022549142A JP2023515411A (ja) 2020-03-24 2020-12-17 ビデオレンダリング方法、装置、電子機器及び記憶媒体
US17/889,817 US20220417591A1 (en) 2020-03-24 2022-08-17 Video rendering method and apparatus, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010212800.5A CN113452929B (zh) 2020-03-24 2020-03-24 视频渲染方法、装置、电子设备及存储介质
CN202010212800.5 2020-03-24

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/889,817 Continuation US20220417591A1 (en) 2020-03-24 2022-08-17 Video rendering method and apparatus, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2021189995A1 true WO2021189995A1 (zh) 2021-09-30

Family

ID=77806496

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/137398 Ceased WO2021189995A1 (zh) 2020-03-24 2020-12-17 视频渲染方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US20220417591A1 (zh)
JP (1) JP2023515411A (zh)
CN (1) CN113452929B (zh)
WO (1) WO2021189995A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114157867A (zh) * 2021-11-15 2022-03-08 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN114630181A (zh) * 2022-02-24 2022-06-14 深圳亿幕信息科技有限公司 一种视频处理方法、系统、电子设备及介质
CN114782579A (zh) * 2022-04-26 2022-07-22 北京沃东天骏信息技术有限公司 一种图像渲染方法及装置、存储介质

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114187169B (zh) * 2021-12-10 2024-06-28 北京字节跳动网络技术有限公司 视频特效包的生成方法、装置、设备及存储介质
CN115049776B (zh) * 2022-04-28 2025-08-08 广州博冠信息科技有限公司 视频渲染方法及装置、存储介质、电子设备
CN117218269A (zh) * 2022-09-07 2023-12-12 腾讯科技（深圳）有限公司 视频处理方法、装置、设备及存储介质
CN115633212A (zh) * 2022-09-26 2023-01-20 深圳市大头兄弟科技有限公司 文字动画的生成方法及相关设备
CN115801978A (zh) * 2022-10-24 2023-03-14 网易（杭州）网络有限公司 特效视频制作方法、装置、电子设备及可读存储介质
CN116132733A (zh) * 2023-01-20 2023-05-16 北京字跳网络技术有限公司 一种视频展示方法及装置
CN116055700B (zh) * 2023-03-23 2023-06-20 北京清扬通信有限公司 一种降低网络流量的多路视频处理方法、设备及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120028706A1 (en) * 2010-02-24 2012-02-02 Valve Corporation Compositing multiple scene shots into a video game clip
CN103928039A (zh) * 2014-04-15 2014-07-16 北京奇艺世纪科技有限公司 一种视频合成方法及装置
CN103971713A (zh) * 2014-05-07 2014-08-06 厦门美图之家科技有限公司 一种视频文件的滤镜处理方法
CN107333176A (zh) * 2017-08-14 2017-11-07 北京百思科技有限公司 一种分布式视频渲染的方法及系统
CN107770626A (zh) * 2017-11-06 2018-03-06 腾讯科技（深圳）有限公司 视频素材的处理方法、视频合成方法、装置及存储介质
CN110070551A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 视频图像的渲染方法、装置和电子设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4203812B2 (ja) * 2003-12-29 2009-01-07 ソニー株式会社 ファイル記録装置、ファイル記録方法、ファイル記録方法のプログラム、ファイル記録方法のプログラムを記録した記録媒体、ファイル再生装置、ファイル再生方法、ファイル再生方法のプログラム及びファイル再生方法のプログラムを記録した記録媒体
JP4340973B2 (ja) * 2004-12-22 2009-10-07 ソニー株式会社 画像編集装置及びビデオ編集装置
JP2008135923A (ja) * 2006-11-28 2008-06-12 Taiwan Mukojo Kagi Kofun Yugenkoshi リアルタイムに相互作用する映像の制作方法、映像の制作装置、および映像の制作システム
CN101247482B (zh) * 2007-05-16 2010-06-02 北京思比科微电子技术有限公司 一种实现动态图像处理的方法和装置
JP2013109161A (ja) * 2011-11-21 2013-06-06 Xing Inc カラオケシステム、カラオケ端末装置、カラオケ背景編集方法、及びコンピュータプログラム
JP5836138B2 (ja) * 2012-01-17 2015-12-24 オリンパス株式会社 撮像装置、画像処理方法およびプログラム
CN105874782A (zh) * 2014-01-03 2016-08-17 汤姆逊许可公司 用于当呈现视频内容时优化至超高清分辨率的画质增强方法、装置和计算机程序产品
CN107071580A (zh) * 2017-03-20 2017-08-18 北京潘达互娱科技有限公司 数据处理方法及装置
CN108022279B (zh) * 2017-11-30 2021-07-06 广州市百果园信息技术有限公司 视频特效添加方法、装置及智能移动终端

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120028706A1 (en) * 2010-02-24 2012-02-02 Valve Corporation Compositing multiple scene shots into a video game clip
CN103928039A (zh) * 2014-04-15 2014-07-16 北京奇艺世纪科技有限公司 一种视频合成方法及装置
CN103971713A (zh) * 2014-05-07 2014-08-06 厦门美图之家科技有限公司 一种视频文件的滤镜处理方法
CN107333176A (zh) * 2017-08-14 2017-11-07 北京百思科技有限公司 一种分布式视频渲染的方法及系统
CN107770626A (zh) * 2017-11-06 2018-03-06 腾讯科技（深圳）有限公司 视频素材的处理方法、视频合成方法、装置及存储介质
CN110070551A (zh) * 2019-04-29 2019-07-30 北京字节跳动网络技术有限公司 视频图像的渲染方法、装置和电子设备

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114157867A (zh) * 2021-11-15 2022-03-08 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN114630181A (zh) * 2022-02-24 2022-06-14 深圳亿幕信息科技有限公司 一种视频处理方法、系统、电子设备及介质
CN114782579A (zh) * 2022-04-26 2022-07-22 北京沃东天骏信息技术有限公司 一种图像渲染方法及装置、存储介质

Also Published As

Publication number Publication date
CN113452929A (zh) 2021-09-28
JP2023515411A (ja) 2023-04-13
US20220417591A1 (en) 2022-12-29
CN113452929B (zh) 2022-10-04

Similar Documents

Publication Publication Date Title
WO2021189995A1 (zh) 视频渲染方法、装置、电子设备及存储介质
US11445255B2 (en) Operation method, device, apparatus and storage medium of playing video
US11315336B2 (en) Method and device for editing virtual scene, and non-transitory computer-readable storage medium
US12216881B2 (en) Method for displaying application icon and electronic device
CN111338743B (zh) 一种界面处理方法、装置及存储介质
US12170047B2 (en) Display control method, display control device and storage medium
CN110609649A (zh) 界面显示方法、装置及存储介质
US10325569B2 (en) Method and apparatus for coding image information for display
CN111754607A (zh) 图片处理方法及装置、电子设备、计算机可读存储介质
EP3032482A1 (en) Page display method and apparatus
CN107391073B (zh) 显示模组及电子设备
WO2021077621A1 (zh) 屏幕显示装置、屏幕显示控制方法、屏幕显示控制装置及存储介质
US20250054454A1 (en) Image display methods and apparatuses
CN106371714A (zh) 信息显示方法及装置
CN111246012B (zh) 应用界面显示方法、装置及存储介质
CN112445318B (zh) 一种对象显示方法、装置、电子设备及存储介质
CN107920015A (zh) 发布图片的方法和发布图片的装置
CN114463478A (zh) 图像渲染的处理方法、装置及电子设备
CN109413232B (zh) 屏幕显示方法及装置
CN111782321A (zh) 一种查看页面层级结构的方法、装置及介质
CN109389547A (zh) 图像显示方法及装置
CN115934225A (zh) 合成方法、装置、电子设备及存储介质
WO2024259955A1 (zh) 一种显示方法、第一设备以及第二设备
CN115222841A (zh) 元素处理方法、装置、电子设备及计算机可读存储介质
KR20130066911A (ko) 휴대용 단말기에서 이미지 드로우 속도를 개선하기 위한 장치 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20926655

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022549142

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06/02/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20926655

Country of ref document: EP

Kind code of ref document: A1