WO2024051394A1 - 视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品 - Google Patents

视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品 Download PDF

Info

Publication number
WO2024051394A1
WO2024051394A1 PCT/CN2023/110290 CN2023110290W WO2024051394A1 WO 2024051394 A1 WO2024051394 A1 WO 2024051394A1 CN 2023110290 W CN2023110290 W CN 2023110290W WO 2024051394 A1 WO2024051394 A1 WO 2024051394A1
Authority
WO
WIPO (PCT)
Prior art keywords
transparency
picture
video
color
video frame
Prior art date
Application number
PCT/CN2023/110290
Other languages
English (en)
French (fr)
Inventor
李棚
邓轩颖
Original Assignee
腾讯科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技(深圳)有限公司 filed Critical 腾讯科技(深圳)有限公司
Publication of WO2024051394A1 publication Critical patent/WO2024051394A1/zh

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/20Perspective computation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/50Lighting effects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras

Definitions

  • Embodiments of the present application relate to the field of Internet technology, and relate to but are not limited to a video processing method, device, electronic equipment, computer-readable storage media, and computer program products.
  • the above code implementation method is very time-consuming and labor-intensive, and requires writing a large amount of special effects code to implement; and the dynamic picture method usually only supports software decoding, and the animation special effects produced have problems such as unclear edges, etc.
  • the files of animated special effects are also relatively large, which will occupy a lot of memory overhead; the method of playing video does not support transparency, so the played special effects video cannot be completely integrated into the background image of the current screen.
  • Embodiments of the present application provide a video processing method, device, electronic equipment, computer-readable storage medium and computer program product, which can be used at least in the field of animation production and special effects video processing, can simplify the production method of special effects videos, and support the production of special effects videos. Transparency information, so that the special effects in the original special effects video can be restored when the transparency special effects video is played, and the memory usage when the transparency special effects video is produced is reduced.
  • An embodiment of the present application provides a video processing method.
  • the method includes: for each original video frame of the original special effects video, obtain the color channel information of each pixel in the original video frame and the transparency of each pixel. Channel information; based on the color channel information of each pixel in the original video frame, draw a color picture corresponding to the original video frame, and based on the transparency channel information of each pixel in the original video frame, draw Transparency pictures corresponding to the original video frames; performing splicing processing on the color pictures and transparency pictures of each original video frame according to the preset configuration information to obtain at least one spliced picture; Perform video conversion processing to obtain a transparency splicing video; perform special effects video rendering based on the preset configuration information and the transparency splicing video to obtain a transparency special effects video.
  • An embodiment of the present application provides a video processing device.
  • the device includes: an acquisition module configured to acquire, for each original video frame of the original special effects video, the color channel information and the color channel information of each pixel in the original video frame. Transparency channel information of one pixel; picture generation module configured to based on each of the original video frames Based on the color channel information of the pixel points, draw a color picture corresponding to the original video frame, and, based on the transparency channel information of each pixel point in the original video frame, draw a transparency picture corresponding to the original video frame; splicing
  • the processing module is configured to splice the color picture and transparency picture of each original video frame according to the preset configuration information to obtain at least one spliced picture; the video conversion module is configured to splice the at least one spliced picture. Perform video conversion processing to obtain a transparency splicing video; a rendering module configured to perform special effects video rendering based on the preset configuration information and the transparency splicing video to obtain a transparency
  • An embodiment of the present application provides an electronic device, including: a memory for storing executable instructions; and a processor for implementing the above video processing method when executing the executable instructions stored in the memory.
  • Embodiments of the present application provide a computer program product or computer program.
  • the computer program product or computer program includes executable instructions, and the executable instructions are stored in a computer-readable storage medium; wherein, the processor of the electronic device obtains the information from the computer-readable storage medium.
  • the executable instructions are read and executed, the above video processing method is implemented.
  • Embodiments of the present application provide a computer-readable storage medium that stores executable instructions for causing the processor to execute the executable instructions to implement the above video processing method.
  • the embodiments of the present application have the following beneficial effects: for each original video frame in the original special effects video, a color picture corresponding to the original video frame is drawn based on the color channel information of each pixel in the original video frame, and, based on the original The transparency channel information of each pixel in the video frame is used to draw a transparency picture corresponding to the original video frame, and the color picture and the transparency picture are spliced to obtain at least one spliced picture.
  • the generated transparency spliced video also carries the transparency information of the original special effects video, so that in When rendering special effects video, the transparency special effects video can be rendered based on the transparency information, so that the special effects in the original special effects video can be highly restored when the transparency special effects video is played.
  • the transparency special effects video can be rendered based on the transparency information, so that the special effects in the original special effects video can be highly restored when the transparency special effects video is played.
  • it since there is no need to write code to realize the production of special effects, it is extremely convenient. Greatly reduce the production cost of transparency special effects videos and the memory usage during production.
  • Figure 1 is a schematic diagram of the implementation process of using Lottie animation to create dynamic pictures
  • Figure 2 is a schematic diagram of different button animation effects in Lottie animation design
  • Figure 3 is a schematic diagram of animation special effects produced using APNG technology
  • Figure 4 is a schematic diagram comparing the sizes of GIF dynamic images and WebP dynamic images
  • FIG. 5 is an optional architectural schematic diagram of the video processing system provided by the embodiment of the present application.
  • Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 7 is an optional flow diagram of the video processing method provided by the embodiment of the present application.
  • FIG. 8 is another optional flow diagram of the video processing method provided by the embodiment of the present application.
  • FIG. 9 is another optional flow diagram of the video processing method provided by the embodiment of the present application.
  • Figure 10 is a schematic diagram of the light and shadow special effects provided by the embodiment of the present application.
  • Figure 11 is a schematic diagram of the special effects of game gifts provided by the embodiment of the present application.
  • Figure 12 is a schematic diagram of particle special effects provided by an embodiment of the present application.
  • Figure 13 is a schematic diagram of flame special effects provided by the embodiment of the present application.
  • Figure 14 is a schematic flow diagram of the overall video processing method provided by the embodiment of the present application.
  • Figure 15 is a schematic diagram of continuous PNG images of game special effects exported by AE software according to the embodiment of the present application.
  • Figure 16 is a schematic diagram of two pictures obtained by separating the four RGBA channel information provided by the embodiment of the present application.
  • Figure 17 is a schematic diagram of spliced pictures provided by the embodiment of the present application.
  • Figure 18 is a schematic diagram of the insertion position of alpha configuration information provided by the embodiment of the present application.
  • Figure 19 is a schematic diagram of the rendering process provided by the embodiment of the present application.
  • Alpha blending technology refers to the overall technical solution for generating transparency special effects videos in the embodiment of this application.
  • Alpha blending effect refers to some case effects produced by using the methods of the embodiments of this application.
  • Light and shadow effects refers to the light and shadow display effects in the display interface (such as the game interface).
  • Particle special effects refers to the dynamic effects of many small objects in the display interface (such as a game interface).
  • Hardware decoding is a method of video decoding that does not use central processing unit (CPU, Central Processing Unit) resources and uses dedicated hardware devices, such as mobile phone hardware devices, such as graphics processing units (GPU). , Graphics Processing Unit) way to decode.
  • CPU Central Processing Unit
  • GPU graphics processing units
  • GPU Graphics Processing Unit
  • Soft decoding Software decoding is a method of video decoding, which refers to the use of CPU decoding.
  • Lottie is an open source cross-platform complete animation effect solution.
  • Designers can use special effects production tools (AE, Adobe After Effects) to design animations, and then use the Bodymovin plug-in provided by Lottie to export the designed animations into JS.
  • the object notation (JSON, JavaScript Object Notation) format can be directly used on different systems without any additional operations.
  • JSON JavaScript Object Notation
  • FIG 1 it is the implementation process of using Lottie animation to create dynamic pictures.
  • the designed animation is exported into a JSON file 103 in JSON format through AE 101 and Bodymovin plug-in 102, and then it can be applied to iOS system, Android system, Web and React Native (an open source cross-platform mobile application development framework).
  • Lottie animation design it is usually used for animation effects of buttons, etc., as shown in Figure 2, position arrow 201, pointing arrow 202, heart shape 203, etc. can all be realized through Lottie animation design.
  • GIF animation refers to the use of specialized animation production tools or the method of photographing objects frame by frame to allow multiple GIF images to quickly and continuously play moving images according to certain rules.
  • various dynamic emoticon packages can be made of GIF animations.
  • the PNG-based bitmap animation format (APNG, Animated Portable Network Graphics) is an animation extension of the Portable Network Graphics (PNG, Portable Network Graphics) format.
  • the first frame of APNG is a standard PNG image, and the remaining animation and frame rate data other than the first frame are placed in the PNG extension data block.
  • APNG is similar to the key frames of a video. The key frames have complete image information, while only the changes between the two key frames are retained. information. Simply put, APNG supports full color and transparency without blurring problems. As shown in Figure 3, it is an animation special effect produced using APNG technology, in which the boundary of the object 301 in the animation special effect is clear and has no edges.
  • the main goal of WebP is to reduce the size of image files while maintaining the same image quality as JPEG (Joint Photographic Experts Group) format, thereby reducing the time and traffic consumption of sending images on the Internet.
  • Figure 4 is a comparison of the sizes of GIF dynamic images and WebP dynamic images. It can be seen that the image size of WebP dynamic images 402 is much smaller than the image size of GIF dynamic images 401.
  • MP4 video is a video composed of frame-by-frame animation.
  • the Lottie animation is a string file, supports soft decoding, and has no particle effects.
  • GIF animation also supports soft decoding, 8-bit color, and alpha transparent background, but the edges around it are very obvious.
  • APNG also supports soft decoding. Although the file size is larger than the alpha video file, it solves the problem of GIF animations with edges.
  • WebP also supports soft decoding, which has larger video files than alpha.
  • MP4 video supports hard decoding but does not support alpha transparent background.
  • the video processing method of the embodiment of the present application can perfectly solve these problems. At the same time, it also has advantages in decoding capabilities, ease of use, special effects restoration, and transparency support.
  • the color channel information of each pixel and the transparency channel information of each pixel in the original video frame are obtained; Draw a color picture corresponding to the original video frame based on the color channel information of each pixel in the original video frame, and draw a color picture corresponding to the original video based on the transparency channel information of each pixel in the original video frame.
  • the transparency picture corresponding to the frame then, according to the preset configuration information, the color picture and transparency picture of each original video frame are spliced to obtain at least one spliced picture; and the video conversion process is performed on at least one spliced picture to obtain Transparency splicing video; finally, special effects video rendering is performed based on the preset configuration information and transparency splicing video, a transparency special effects video is obtained, and the transparency special effects video is displayed.
  • the generated transparency spliced video can carry the transparency information of the original special effects video, thereby performing When rendering special effects video, the transparency special effects video can be rendered based on the transparency information, so that the special effects in the original special effects video can be highly restored when the transparency special effects video is played, which greatly reduces the production cost of the transparency special effects video and the memory usage during production. quantity.
  • the video processing device is an electronic device used to implement the video processing method.
  • the video processing device (ie, electronic device) provided by the embodiment of the present application can be implemented as a terminal or as a server.
  • the video processing device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable Game equipment), smart robots, smart home appliances, smart vehicle-mounted equipment, and other terminals with video rendering and display functions, video processing functions, and game functions; in another implementation, the video processing device provided by the embodiment of the present application also It can be implemented as a server, where the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, Cloud servers for basic cloud computing services such as network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms.
  • the terminal and the server can be connected directly or indirectly through wired or wireless communication methods, which are not limited in the embodiments of this application.
  • an independent physical server
  • FIG. 5 is an optional architectural schematic diagram of the video processing system provided by the embodiment of the present application.
  • the embodiment of the present application takes the video processing method applied to special effects video production as an example. That is to say, the transparency of the final generated Special effects videos can be any type of special effects videos.
  • the video processing system 10 at least includes a terminal 100, a network 200 and a server 300.
  • the terminal 100 in the embodiment of the present application is at least installed with a special effects video production application, or, A video playback application is installed on the terminal 100.
  • the video playback application has a special effects video production function module.
  • the server 300 is a server for special effects video production applications, and the server 300 may constitute the video processing device of the embodiment of the present application.
  • the terminal 100 connects to the server 300 through the network 200.
  • the network 200 may be a wide area network or a local area network, or a combination of the two.
  • the terminal 100 When running the special effects video production application, the terminal 100 obtains the original special effects video through the client of the special effects video production application, encapsulates the original special effects video into a video processing request, and sends the video processing request to the server 300 through the network 200 .
  • the server 300 parses the video processing request to obtain the original special effects video, and obtains each original video frame of the original special effects video, and obtains the color channel information and transparency channel information of each pixel in the original video frame; then, for each original Video frame, based on the color channel information of each pixel in the original video frame, draw a color picture corresponding to the original video frame, based on the transparency channel information of each pixel in the original video frame, draw a transparency picture corresponding to the original video frame ; Then, according to the preset configuration information, the color picture and transparency picture of each original video frame are spliced to obtain at least one spliced picture; video conversion processing is performed on at least one spliced picture to obtain a transparency spliced
  • the video processing method can also be implemented by the terminal 100, that is, the terminal serves as the execution subject to obtain each original video frame of the original special effects video, and the color channel information of each pixel of the original video frame. and transparency channel information; for each original video frame, the terminal generates a color picture and transparency picture corresponding to the original video frame based on the color channel information and transparency channel information of each pixel in the original video frame; and, the terminal generates a color picture and a transparency picture corresponding to the original video frame; and, the terminal Preset configuration information, perform splicing processing on the color picture and transparency picture of each original video frame to obtain at least one spliced picture; perform video conversion processing on at least one splicing picture to obtain a transparency splicing video; finally, the terminal performs a splicing process based on The preset configuration information and transparency splicing video are used for special effects video rendering to obtain a transparency special effects video. While the terminal obtains the transparency special effects video, it can also display the transparency special effects
  • the video processing method provided by the embodiment of the present application can also be based on a cloud platform and implemented through cloud technology.
  • the above-mentioned server 300 can be a cloud server. For each original video frame of the original special effects video, obtain the color channel information of each pixel and the transparency channel information of each pixel in the original video frame through the cloud server, or generate a video corresponding to the original video frame through the cloud server.
  • the cloud server performs splicing processing on the color pictures and transparency pictures of each original video frame, or the cloud server performs video conversion processing on at least one spliced picture, or the cloud server performs video conversion processing based on preset Assume configuration information and transparency splicing video for special effects video rendering to obtain transparency special effects video, etc.
  • there may also be cloud storage and the original special effects video may be stored in the cloud storage, or the color channel information and transparency channel information of each pixel in each original video frame may be stored in the cloud.
  • cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize data calculation, storage, processing, and sharing.
  • Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool and use it on demand, which is flexible and convenient. Cloud computing technology will become an important support.
  • the background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data at different levels will be processed separately, and all types of industry data need to be powerful. System backing support can only be achieved through cloud computing.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device shown in Figure 6 includes: at least one processor 310, a memory 350, at least one network interface 320 and a user interface 330.
  • the various components in the electronic device are coupled together through bus system 340 .
  • the bus system 340 is used to implement connection communication between these components.
  • the bus system 340 also includes a power bus, a control bus and a status signal bus.
  • the various buses are labeled bus system 340 in FIG. 6 .
  • the processor 310 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware Components, etc., wherein the general processor can be a microprocessor or any conventional processor, etc.
  • DSP Digital Signal Processor
  • User interface 330 includes one or more output devices 331 that enable the presentation of media content, and one or more input devices 332.
  • Memory 350 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. Memory 350 optionally includes one or more storage devices physically located remotely from processor 310 . Memory 350 includes volatile memory or non-volatile memory, and may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM, Read Only Memory), and volatile memory can be random access memory (RAM, Random Access Memory). The memory 350 described in the embodiments of this application is intended to include any suitable type of memory. In some embodiments, the memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplarily described below.
  • Operating system 351 including system programs used to process various basic system services and perform hardware-related tasks, such as framework layer, core library layer, driver layer, etc., used to implement various basic services and process hardware-based tasks; network communication Module 352 for reaching other computing devices via one or more (wired or wireless) network interfaces 320, example network interfaces 320 include: Bluetooth, Wireless Compliance (WiFi), and Universal Serial Bus (USB, Universal Serial Bus), etc.; input processing module 353 for detecting one or more user inputs or interactions from one or more input devices 332 and translating the detected inputs or interactions.
  • WiFi Wireless Compliance
  • USB Universal Serial Bus
  • the device provided by the embodiment of the present application can be implemented in software.
  • Figure 6 shows a video processing device 354 stored in the memory 350.
  • the video processing device 354 can be a video processing device in an electronic device.
  • the device which can be software in the form of programs and plug-ins, includes the following software modules: acquisition module 3541, picture generation module 3542, splicing processing module 3543, video conversion module 3544 and rendering module 3545. These modules are logical, so according to The implemented functions can be combined or further divided into any combination. The functions of each module are explained below.
  • the device provided by the embodiment of the present application can be implemented in hardware.
  • the device provided by the embodiment of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the present application.
  • the processor in the form of a hardware decoding processor may use one or more Application Specific Integrated Circuits (ASICs), DSPs, or Programmable Logic Devices (PLDs). ), complex programmable logic devices (CPLD, Complex Programmable Logic Device), Field-Programmable Gate Array (FPGA, Field-Programmable Gate Array) or other electronic components.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processing
  • PLDs Programmable Logic Devices
  • CPLD Complex Programmable Logic Device
  • FPGA Field-Programmable Gate Array
  • the video processing method provided by each embodiment of the present application can be executed by an electronic device, where the electronic device can be any terminal with video rendering and display functions, video processing functions, and game functions, or it can also be a server, that is,
  • the video processing methods in various embodiments of the present application can be executed through a terminal, a server, or the terminal interacts with a server.
  • Figure 7 is an optional flow diagram of the video processing method provided by the embodiment of the present application. The following will be described in conjunction with the steps shown in Figure 7. It should be noted that the video processing method in Figure 7 is performed by The server is used as the execution subject as an example to illustrate.
  • Step S701 For each original video frame of the original special effects video, obtain the color channel information of each pixel and the transparency channel information of each pixel in the original video frame.
  • the original special effects video can be a special effects video produced using a special effects production tool.
  • the special effects production tool can be AE software (Adobe After Effects).
  • the original special effects video can be produced and output through the AE software.
  • the original special effects video can include any type of special effects content.
  • the special effects content includes: particle special effects, light and shadow special effects, water wave effects, game gift special effects, flame special effects, etc.
  • the special effects content is content with certain transparency information. That is to say, each pixel in the original video frame includes color channel information and transparency channel information. If the terminal displays the original special effects video, the display effect of the special effects content in the original special effects video will be clear-cut and clutter-free.
  • the original video frame is an RGB image
  • the RGB channel of the original video frame stores color channel information
  • the alpha channel (ie, transparency channel) of the original video frame stores transparency channel information.
  • the original special effects video is not a video that can be directly displayed, because the video decoding and rendering process is to first obtain the video YUV data through hard decoding or soft decoding of an MP4 format video file. Before video rendering, the video YUV data needs to be converted. into RGB format data, and then render the RGB format data to display a frame of video. Since the video data format before rendering is YUV format, the YUV format does not carry alpha information (that is, transparency channel information), so the MP4 video before rendering cannot carry alpha information, resulting in the loss of transparency information of the original special effects video, resulting in rendering The final special effects video cannot be effectively integrated with the background, and the effect will be that the background is blocked by other areas other than the special effects content in the original special effects video.
  • alpha information that is, transparency channel information
  • the original video frames in the original special effects video are all rectangular, and if the special effects content of the original video frame is a circle, after rendering using the above rendering method, the background will be blocked by the entire rectangle due to the loss of transparency information. , instead of just showing a circle.
  • the method of the embodiment of the present application extracts the transparency information and effectively stores it in subsequent steps, which can ensure the accurate rendering of the special effects video based on the transparency information during rendering, and realize that the background image is only blocked by the circular special effects content. , while other places display the effect normally.
  • the special effects production tool after the special effects production tool produces the original special effects video, it can also perform frame segmentation processing on the original special effects video, that is, extract video frames from the original special effects video to obtain at least one original video frame, and output all of them.
  • Raw video frames After obtaining each original video frame, the color channel information and transparency channel information of each pixel in the original video frame can be extracted.
  • the RGBA channel value corresponding to each pixel can be obtained, the RGB channel value in the RGBA channel value is determined as the color channel information, and the alpha information in the A channel is determined as the transparency channel information.
  • the position information of each pixel in the original video frame can also be obtained, and the position information is combined with the color channel information of the pixel. and transparency channel information for mapping and storage.
  • the color channel information and transparency channel information can be assigned to the color picture and transparency channel information respectively based on the mapping relationship between the position information of each pixel and the color channel information and transparency channel information. Transparency for each pixel in the image.
  • Step S702 Based on the color channel information of each pixel in the original video frame, draw the image corresponding to the original video frame. The corresponding color picture is drawn, and based on the transparency channel information of each pixel in the original video frame, a transparency picture corresponding to the original video frame is drawn.
  • the color channel information and transparency channel information of each pixel can be assigned to the color picture and the transparency picture respectively.
  • two canvases with the same size as the original video frame can be provided respectively, and then the color channel The information and transparency channel information are assigned to each pixel in the canvas to obtain a color image and a transparency image.
  • the canvas also called a blank canvas
  • the blank carrier includes multiple pixels, and on the blank carrier, each pixel is It does not contain any pixel information, that is, each pixel does not render any color channel information and transparency channel information.
  • the color channel information and transparency channel information can be assigned to the two canvases corresponding to each pixel in the original video frame.
  • Each pixel in each canvas carries color channel information or transparency channel information, thereby obtaining a color picture and transparency picture with image content.
  • the canvas may be a virtual canvas. That is to say, after obtaining the color channel information and transparency channel information of each pixel, the color channel information and transparency channel information may be assigned to the virtual canvas respectively. For each pixel, a color image and a transparency image are obtained.
  • Color pictures are pictures that carry color channel information
  • transparency pictures are pictures that carry transparency channel information.
  • a color picture and a transparency picture can be generated for each original video frame in the original special effects video.
  • Step S703 perform splicing processing on the color picture and transparency picture of each original video frame according to the preset configuration information to obtain at least one spliced picture.
  • the preset configuration information is information used to characterize the mapping relationship between each pixel in the transparency image and the pixels in the color image. During the splicing process, the color image and transparency can be determined based on the preset configuration information. The positions of the pictures in the canvas corresponding to the spliced picture (i.e., the spliced canvas).
  • the preset configuration information may include the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture.
  • the first position information of the transparency picture in the splicing canvas refers to where the center position of the transparency picture is located on the splicing canvas corresponding to the splicing picture during the splicing process.
  • the size parameters of the transparency image include the width and height of the transparency image. Based on the width and height of the transparency image, combined with the first position information of the transparency image in the splicing canvas, it can be determined that each pixel in the transparency image is in the splicing canvas. The position in the transparency image can be used to determine the positions of the starting pixel and the ending pixel in the transparency image.
  • the offset value of the transparency image refers to the number of pixels in the spliced image that the transparency image is offset relative to the color image in at least one direction.
  • at least one direction includes but is not limited to: the extension direction along the length of the splicing canvas, The extension direction along the width of the splicing canvas, the extension direction along the diagonal line of the splicing canvas, etc.
  • the splicing canvas corresponding to the spliced picture refers to a blank canvas used to add color pictures and transparency pictures to form a spliced picture. It can be understood that the splicing canvas corresponding to the splicing picture refers to a blank canvas with the same size as the splicing picture. That is to say, when performing splicing processing, you can first provide a blank canvas, and then add color pictures and transparency pictures to the blank canvas in sequence, or add color pictures and transparency pictures to the blank canvas at the same time to form a spliced picture. .
  • the relative positional relationship between the color picture and the transparency picture in the spliced picture may not be fixed.
  • the transparency image can be located at any relative position to the color image, and there is no overlapping area between the color image and the transparency image.
  • the transparency image can be located above, below, left, right, upper right, lower right, upper left, lower left, etc. of the color image.
  • the size of the color picture and the transparency picture are the same, but in the stitched picture, the size of the color picture added to the stitched picture and the size of the transparency picture added to the stitched picture may be the same or different. That is to say, at least one of the color image and the transparency image can be scaled at a specific ratio, and the scaled color image or transparency image can be added to the spliced image.
  • the scaling ratio for scaling processing can be determined based on the information in the preset configuration information. That is to say, when the preset configuration information is given, the scaling ratio of the color picture and transparency picture will also be determined. Color images and transparency images can be scaled based on given preset configuration information. At the same time, since the preset configuration information is known, it can be directly determined based on the preset configuration information in the future, how much the color image and the transparency image are respectively scaled during the splicing process.
  • only the transparency image can be scaled, and the color image can not be scaled. That is to say, the color image is added to the blank canvas in a size ratio of 1:1; the transparency image can be appropriately scaled. The proportional reduction process is then added to the blank canvas. In this way, the amount of transparency information can be greatly reduced, and multiple pixels in the color image can be corresponding to one pixel in the reduced transparency image. In the embodiment of the present application, the transparency image can be reduced by 0.5 times.
  • the color pictures and transparency pictures of each original video frame have the same preset configuration information when splicing them. That is to say, all original video frames in the same original special effects video share the same preset configuration information. In this way, in each obtained spliced picture, the color picture and the transparency picture have the same size, and the color picture and the transparency picture are in the same position.
  • Step S704 Perform video conversion processing on at least one spliced picture to obtain a transparency spliced video.
  • the video conversion process is to perform video compression processing on at least one obtained spliced picture in chronological order to obtain a transparency spliced video.
  • Step S705 Perform special effects video rendering based on preset configuration information and transparency splicing video to obtain a transparency special effects video.
  • the channel information of the pixels can be obtained from each video frame of the transparency spliced video (i.e., the spliced picture) based on the preset configuration information, that is, the color channel information can be obtained from the color picture in the spliced picture, and the color channel information can be obtained from the color picture in the spliced picture. Get the transparency channel information from the transparency image.
  • the video processing method provided by the embodiment of the present application for each original video frame in the original special effects video, is based on the color channel information and transparency channel information of each pixel in the original video frame, correspondingly generating the original video
  • the color picture and transparency picture corresponding to the frame are spliced together to obtain at least one spliced picture, and then the at least one spliced picture is subjected to video conversion processing to obtain a transparency spliced video, thereby based on the preset configuration information Perform special effects video rendering with the transparency splicing video to obtain a transparency special effects video.
  • the generated transparency spliced video also contains the original special effects video.
  • Transparency information so that when rendering special effects video, the transparency special effects video can be rendered based on the transparency information, so that the special effects in the original special effects video can be highly restored when the transparency special effects video is played, greatly reducing the production cost of the transparency special effects video and The memory usage during production.
  • the video processing system at least includes a terminal and a server, and a special effects video production application is installed on the terminal, or a video playback application is installed on the terminal, and the video playback application has a special effects video production function module.
  • the special effects video production function module can use the video processing method provided by the embodiment of the present application to generate a transparency special effects video.
  • FIG 8 is another optional flow diagram of the video processing method provided by the embodiment of the present application. As shown in Figure 8, the method includes the following steps:
  • Step S801 The terminal produces the original special effects video using a special effects production tool.
  • the user can produce the original special effects video through the special effects production tool in the special effects video production function module.
  • the terminal can receive the special effects input by the user through the special effects video production function module of the video playback application.
  • the user when the user inputs the special effects production operation, he or she can input special effect parameters such as the type of special effects, the content of the special effects, the size of the special effects, the delay time of the special effects, the color of the special effects, and the distribution of the special effects.
  • the special effects production tool produces the original special effects video based on the input special effects parameters.
  • Step S802 The terminal exports each original video frame in the original special effects video through the special effects production tool, and encapsulates each original video frame into a video processing request.
  • the original special effects video can be divided into frames or video frame recognition can be performed to obtain each original video frame in the original special effects video and output each original video frame.
  • the original video frames are stored in a specific storage location of the video playback application.
  • each original video frame can also be encapsulated into a video processing request.
  • the video processing request is used to request processing of each original video frame in the original special effects video to obtain Render transparency effects video with special effects.
  • Step S803 The terminal sends the video processing request to the server.
  • Step S804 The server collects the color channel information and transparency channel information of each pixel in each original video frame.
  • the server parses the pixel value of each pixel in each original video frame to extract the color channel information and transparency channel information of each pixel in the original video frame.
  • the original video frames exported by the special effects production tool can be PNG format pictures.
  • the RGBA channel value corresponding to each pixel in the PNG format picture can be obtained, and the RGB channel value in the RGBA channel value can be obtained. It is determined as color channel information, and the color channel information includes R channel value, G channel value and B channel value; the alpha information in the A channel is determined as transparency channel information.
  • Step S805 For each original video frame, the server adds the color channel information of each pixel in the original video frame to the color channel of the first canvas picture to obtain a color picture corresponding to the original video frame.
  • the first canvas picture is a blank canvas to which no pixel value information of any pixel points is added. That is to say, the first canvas picture does not include any pixel point information.
  • the first canvas picture has the same dimensions as the original video frame.
  • the color channel information of each pixel can be added to the color channel in turn according to the position of the pixel in the original video frame. From the color channel of the corresponding position of the first canvas picture, a color picture containing the color channel information of the original video frame is obtained. Among them, the size of the color picture is the same as the size of the original video frame.
  • color channel information may include RGB channel values.
  • the RGB channel value of each pixel in the original video frame can be added to the RGB channel of the first canvas picture to obtain a color picture corresponding to the original video frame.
  • Step S806 For each original video frame, the server converts the transparency channel of each pixel in the original video into The information is added to the color channel of the second canvas image to obtain a transparency image corresponding to the original video frame.
  • the second canvas picture is also a blank canvas to which no pixel value information of any pixel points is added. That is to say, the second canvas picture does not include any pixel point information.
  • the second canvas picture also has the same dimensions as the original video frame.
  • the transparency channel information of each pixel can be added to the color channel in turn according to the position of the pixel in the original video frame. From the color channel of the corresponding position of the second canvas picture, a transparency picture containing transparency channel information of the original video frame is obtained. Among them, the size of the transparency image is also the same as the size of the original video frame.
  • transparency channel information may include transparency values.
  • the transparency value of each pixel in the original video frame can be added to the RGB channel of the second canvas image to obtain a color image corresponding to the original video frame.
  • the transparency value can range from 0 to 255, where 0 means completely transparent and 255 means opaque. The higher the transparency value, the greater the opacity.
  • Step S807 The server determines the starting position of the transparency picture in the splicing canvas and the scaling ratio of the transparency picture based on at least the first position information of the transparency picture in the splicing canvas, the size parameter of the transparency picture, and the offset value of the transparency picture.
  • the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture constitute the above-mentioned preset configuration information.
  • the first position information of the transparency picture in the splicing canvas is used to represent where the center position of the transparency picture is located on the splicing canvas corresponding to the splicing picture during the splicing process.
  • the size parameters of the transparency image include the width and height of the transparency image. Based on the width and height of the transparency image, combined with the first position information of the transparency image in the splicing canvas, it can be determined that each pixel in the transparency image is in the splicing canvas.
  • the position in the transparency image can be used to determine the positions of the starting pixel and the ending pixel in the transparency image.
  • the offset value of the transparency image refers to the number of pixels in the spliced image that the transparency image is offset relative to the color image in at least one direction.
  • at least one direction includes but is not limited to: the extension direction along the length of the splicing canvas, The extension direction along the width of the splicing canvas, the extension direction along the diagonal line of the splicing canvas, etc.
  • the starting position of the transparency picture in the splicing canvas can be determined based on the first position information of the transparency picture in the splicing canvas and the size parameters of the transparency picture.
  • the size parameters of the transparency picture include the width and height of the transparency picture. Therefore, based on the width, height of the transparency image, and the position of the center of the transparency image in the splicing canvas corresponding to the splicing image, the starting pixel (i.e., the first pixel) and the ending pixel in the transparency image can be determined (i.e. the location of the last pixel). In other words, at least the starting position (that is, the position of the starting pixel) and the ending position of the transparency image in the splicing canvas can be determined. What needs to be noted here is that since the size parameters of the transparency image are known, it is usually not necessary to consider the end position. That is to say, it is only necessary to determine the starting position of the transparency image in the splicing canvas.
  • the size parameters of the transparency picture include the width and height of the transparency picture. Therefore, based on the width and height of the transparency image, and the position of the center of the transparency image in the splicing canvas corresponding to the splicing image, it is possible to determine where each pixel on the boundary line of the transparency image (i.e., the frame of the image) is located. Location. In this way, the position of the pixel point on any boundary line in the bounding box can be determined as the starting position of the transparency picture in the splicing canvas, so that using the boundary line as the starting point, each pixel in the transparency picture can be determined. The position of a pixel in the splicing canvas.
  • the scaling ratio of the transparency picture may also be determined based on the size parameter of the transparency picture and the offset value of the transparency picture.
  • the offset value of the transparency image means that in the spliced image, the transparency image is at least The number of pixels offset in one direction, that is, based on the offset value of the transparency image, the pixel in the transparency image corresponding to any pixel in the color image can be determined, and between the two The number of pixels to offset.
  • the transparency image can be reduced by 0.5 times.
  • the size of the transparency image and the color image are in a 1:1 ratio. Therefore, at the (0, 0) position in the color image
  • the pixel corresponds to the pixel at the (0, 0) position in the transparency image.
  • the position of the pixels in the transparency image is shifted. Therefore, the pixels at the (2, 2) position in the color image are different from the (1) position in the transparency image.
  • 1) The pixels at the position correspond to each other.
  • the pixels at the (4, 4) position in the color picture correspond to the pixels at the (2, 2) position in the transparency picture. It can be seen that the number of pixel points offset by the transparency image relative to the color image in the length direction and width direction of the transparency image is both 2.
  • Step S808 The server obtains the second position information of the color picture in the splicing canvas.
  • the second position information refers to where the center position of the color picture is located on the splicing canvas corresponding to the spliced picture during the splicing process.
  • Step S809 The server adds the color picture and transparency picture of each original video frame to the splicing canvas according to the second position information, the starting position and the scaling ratio to form at least one splicing picture.
  • step S809 can be implemented in the following manner: for each original video frame, a color picture can be added to the position corresponding to the second position information in the splicing canvas to obtain a splicing canvas with a color picture. That is to say, taking the center position of the color picture on the splicing canvas as the starting point for expansion, the color picture is expanded from the center position to pixel points, thereby adding each pixel in the color picture to the splicing canvas to form a Mosaic canvas of color pictures.
  • the transparency image can also be scaled according to the scaling ratio to obtain a scaled transparency image. For example, you can reduce the transparency image to half its size using a reduction ratio of 0.5.
  • the starting position is used as the starting position to add the scaled transparency image, and the scaled transparency image is added to the splicing canvas with the color image to obtain a splicing image. That is to say, according to the position information of the starting position, each pixel of the scaled transparency picture is added to the splicing canvas with the color picture.
  • step S809 can also be implemented in the following manner: first add the transparency image to the splicing canvas, that is to say, first scale the transparency image according to the scaling ratio to obtain the scaled transparency image, and then The starting position is the starting position of the scaled transparency image, and the scaled transparency image is added to the splicing canvas; then, the color image is added to the splicing canvas with the scaled transparency image corresponding to the second position information. position to get a stitched picture.
  • Step S810 The server performs video conversion processing on at least one spliced picture to obtain a transparency spliced video.
  • the video conversion process refers to performing video compression processing on at least one obtained spliced picture in chronological order to obtain a transparency spliced video.
  • Step S811 The server performs special effects video rendering based on the preset configuration information and transparency splicing video to obtain a transparency special effects video.
  • Step S812 The server sends the transparency special effect video to the terminal.
  • Step S813 The terminal displays the transparency special effect video on the current interface.
  • the video processing method provided by the embodiment of the present application determines the starting position of the transparency picture in the splicing canvas and the transparency picture based on the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture.
  • the scaling ratio so that according to the second position information, starting position and scaling ratio of the color picture in the splicing canvas, the color picture and transparency picture of each original video frame can be added to the splicing canvas to realize the color picture and transparency Accurate stitching of pictures to form at least one stitched picture.
  • the spliced picture is also a picture that carries the color channel information and transparency channel information of each pixel in the original video frame.
  • the transparency splicing video corresponds to a video compression encoding format
  • the video compression encoding format includes but is not limited to MP4 video format.
  • the video processing method in the embodiment of the present application may also include the following methods: First, the server obtains the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture. Then, the server performs data format conversion on the first position information of the transparency image in the splicing canvas, the size parameters of the transparency image, and the offset value of the transparency image according to the preset data storage format, and obtains the configuration information string. Finally, the server inserts the configuration information string into the preset position of the data header of the video compression encoding format.
  • the data format conversion may be to convert the first position information of the transparency image in the splicing canvas, the size parameters of the transparency image, and the offset value of the transparency image into a JSON string to obtain information in JSON string format
  • the information in JSON string format is the configuration information string.
  • the information in JSON string format can be inserted into the preset position of the data header in the MP4 video format. For example, it can be inserted into the second box position of the data header.
  • the above video processing method can also be implemented by a terminal. That is to say, the terminal serves as the execution subject.
  • the video frame extraction is performed on the original special effects video to obtain at least one original video. frame, and perform information extraction on each original video frame to obtain the color channel information and transparency channel information of each pixel in each original video frame.
  • a color picture and a transparency picture corresponding to the original video frame are generated; according to the preset configuration information, each The color pictures and transparency pictures of the original video frames are spliced to obtain at least one spliced picture; the at least one spliced picture is subjected to video conversion processing to obtain a transparency spliced video; finally, a special effects video is made based on the preset configuration information and the transparency spliced video. Render to obtain the transparency special effects video. When the transparency special effects video is obtained, the transparency special effects video is displayed on the current interface.
  • the terminal obtains the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture and other preset configuration information, and then stores the transparency picture in the splicing canvas according to the preset data storage format.
  • the first position information, the size parameter of the transparency picture and the offset value of the transparency picture are converted into data formats to obtain a configuration information string, and the configuration information string is inserted into a preset position of the data header of the video compression encoding format.
  • the configuration information string is inserted into the preset position of the data header of the video compression encoding format through the above method, that is to say, the preset configuration information is pre-stored to the preset position of the data header of the video compression encoding format, Therefore, the preset configuration information can be directly obtained from the preset position of the data header of the video compression encoding format.
  • the following steps can also be implemented: Obtaining the configuration information string from the preset position of the data header ; And, perform string parsing on the configuration information string to obtain the preset configuration information.
  • FIG 9 is another optional flow diagram of the video processing method provided by the embodiment of the present application. As shown in Figure 9, the method includes the following steps:
  • Step S901 The terminal produces the original special effects video using a special effects production tool.
  • Step S902 The terminal exports each original video frame in the original special effects video through the special effects production tool, and encapsulates each original video frame into a video processing request.
  • Step S903 The terminal sends the video processing request to the server.
  • Step S904 The server collects the color channel information and transparency channel information of each pixel in each original video frame.
  • steps S901 to S904 are the same as the above-mentioned steps S801 to S804, and will not be described again in the embodiment of the present application.
  • Step S905 For each original video frame, the server determines the color of each pixel in the original video frame. Color channel information and transparency channel information are used to generate color pictures and transparency pictures corresponding to the original video frames.
  • the color channel information and transparency channel information of each pixel can be assigned to the color picture and the transparency picture respectively.
  • two canvases with the same size as the original video frame can be provided, and then the color The channel information and transparency channel information are assigned to each pixel in the canvas to obtain a color image and a transparency image.
  • Color pictures are pictures that carry color channel information
  • transparency pictures are pictures that carry transparency channel information.
  • Step S906 The server performs splicing processing on the color picture and transparency picture of each original video frame according to the preset configuration information to obtain at least one spliced picture.
  • the preset configuration information is information used to characterize the mapping relationship between each pixel in the transparency image and the pixels in the color image. During the splicing process, based on the preset configuration information, the positions of the color image and the transparency image in the splicing canvas corresponding to the spliced image can be determined.
  • Step S907 The server performs video conversion processing on at least one spliced picture to obtain a transparency spliced video.
  • Step S908 The server performs video decoding on the transparency spliced video to obtain decoded format data of each spliced picture in the transparency spliced video.
  • the decoded format data of the transparency spliced video can be obtained through hardware decoding or software decoding (for example, using a video player hardware decoding or software decoding), where the decoded format data can be video frame YUV data.
  • Step S909 The server performs data format conversion on the decoded format data of each spliced picture to obtain color format data of each spliced picture; wherein the color format data includes a color channel data part and a transparency channel data part.
  • the YUV data of the video frame corresponding to each spliced picture can be converted into color format data in RGB data format through a conversion formula. Since the spliced picture is formed by splicing a color picture and a transparency picture, and the color picture carries color channel information, and the transparency picture carries transparency channel information, therefore, the color format of the RGB data format after video decoding and data format conversion
  • the data includes the color channel data part and the transparency channel data part. Among them, the color channel data part corresponds to the color channel information carried in the color picture, and the transparency channel data part corresponds to the transparency channel information carried in the transparency picture.
  • Step S910 The server performs special effects video rendering based on the preset configuration information and the color channel data part and the transparency channel data part of the spliced image to obtain a transparency special effect video.
  • step S910 can be implemented in the following manner: first, obtain the information in JSON string format from a preset position of the data header in the MP4 video format, that is, obtain the preset position of the data header stored in the MP4 video format. Default configuration information for the location. Then, based on the preset configuration information, a mapping relationship between each color channel sub-data in the color channel data part and each transparency channel sub-data in the transparency channel data part is determined.
  • the color channel sub-data is the data corresponding to each pixel in the color channel data part
  • the transparency channel sub-data is the data corresponding to each pixel in the transparency channel data part.
  • the color channel sub-data and transparency channel sub-data that have a mapping relationship are determined to be channel data of the same pixel. That is to say, the color channel sub-data and transparency channel sub-data that represent the same pixel information are associated with a mapping relationship to jointly represent the channel data of the same pixel. Furthermore, the position information of the color channel sub-data in the channel data of each pixel point in the color picture is determined as the position information of the pixel point. Since when splicing color pictures and transparency pictures, the transparency pictures are scaled, but the color pictures are not scaled. Therefore, each pixel in the color picture can correspond to the original video frame one-to-one, so The position information of each pixel in the color picture corresponds to the position information of the pixel in the original video frame.
  • the position information of the color channel sub-data in the color picture will also correspond to the position information of the pixels in the original video frame. Therefore, the position information of the color channel sub-data in the color picture can be Position information is determined as the position information of pixels.
  • special effects video is performed on the channel data of each pixel. Render to get transparency effects video.
  • the color channel sub-data in the channel data of the pixel can be used as RGB data according to the position information of each pixel.
  • the transparency channel sub-data in the channel data of the pixel is used as alpha information to implement special effects video rendering and obtain a transparency special effects video.
  • Step S911 the server sends the transparency special effect video to the terminal.
  • Step S912 The terminal renders the background video of the currently running application.
  • the background video can be any type of video, such as short videos, TV series, game videos, etc.
  • the background video can also be a picture, that is, the picture displayed by the terminal on the current interface.
  • Step S913 The terminal receives a special effect triggering operation.
  • Step S914 In response to the special effect triggering operation, the terminal adds the transparency special effect video to a specific position of the background video.
  • the transparency special effects video is added to a specific position of the background video, which may be to display the transparency special effects video on top of the background video.
  • the user can specify a specific location, and the terminal can also perform video analysis on the background video to intelligently determine the specific location for overlaying the transparency special effect video.
  • a special effects video production application is installed on the terminal.
  • the special effects video production application is used to produce and generate a transparency special effects video.
  • the special effects video production application server can input a special effects video production request on the client side of the special effects video production application.
  • the server of the special effects video production application can respond to the special effects video production request and use the video processing method provided by the embodiment of the present application to generate transparency carrying transparency channel information. Splice videos and further generate transparency effects videos.
  • the transparency special effects video can be sent to the client of the terminal as a response result of the special effects video production request, and the transparency special effects video can be displayed on the client; the transparency special effects video can also be used as a special effects video production application to produce products.
  • Output value in other video applications yes other video applications can directly apply the transparency special effects video generated by the special effects video production application.
  • a video playback application is installed on the terminal.
  • the video playback application has a special effects video production function module.
  • the special effects video production function module Through the special effects video production function module, the video processing method provided by the embodiment of the present application can be used to generate a transparency special effect video.
  • the special effects video production function module can be operated to trigger the embodiment of the present application.
  • the video processing process generates a transparency special effects video, and while generating the transparency special effects video, the transparency special effects video is superimposed on the currently playing video and displayed simultaneously with the currently playing video.
  • Scenario 3 A game application is installed on the terminal. During the process of running the game application, if the player wants to add special effects to the game application, the player can operate on the game client and select the special effects button, and then the video of the embodiment of this application can be used.
  • the processing method generates a transparency special effects video, and superimposes the transparency special effects video onto the game screen on the game interface, and plays it simultaneously with the game screen.
  • the server when the user inputs a special effect triggering operation through the client, he or she can also input the playback position of the transparency special effects video.
  • the server After the server generates the transparency special effects video, it feeds back the transparency special effects video to the terminal, and the client of the terminal plays it at the Insert a video with transparency effects at the position, or play a video with transparency effects at this playback position.
  • the server when the user inputs the special effect triggering operation through the client, he or she can also input the display quantity of the special effect content.
  • the server when the server generates the transparency special effects video, it can generate a corresponding number of special effects according to the display quantity of the special effect content.
  • the transparency effect video of the content and plays the transparency effect video.
  • the server can generate a transparency special effects video containing a special effect content, and after generating the transparency special effects video, copy the transparency special effects video to obtain the same number of transparent effects as the displayed number. degree special effect videos and play these number of transparency special effect videos simultaneously.
  • the special effects content refers to the objects in the transparency special effects video.
  • the special effects content can be a pair of shoes, a bouquet of flowers, a book and other objects in the displayed special effects.
  • the server when a user inputs a special effect triggering operation for multiple types of special effect content through the client, the server can respond to the user's special effect triggering operation and simultaneously generate transparency special effect videos of multiple types of special effect content, and will generate Multiple transparency special effects videos are fed back to the client, and multiple transparency special effects videos are played sequentially or multiple transparency special effects videos are played simultaneously on the client. If the playing positions of multiple transparency special effect videos are the same, multiple transparency special effect videos can be played in sequence. If the playing positions of multiple transparency special effect videos are different, multiple transparency special effect videos can be played at the same time.
  • the video processing method provided by the embodiment of the present application can at least realize the production and display of the light and shadow special effects 1001 in Figure 10, the production and display of the game gift special effects (with alpha transparent background) 1102 of the game gift 1101 in Figure 11,
  • the particle special effects 1201 in Figure 12 are produced and displayed, and the flame special effects 1301 in Figure 13 are produced and displayed.
  • the rendering of the game is actually the rendering of each pixel, and each pixel is divided into four channels: R, G, B, and alpha.
  • RGB is the color channel
  • alpha is the transparency channel of the pixel.
  • the entire game special effects rendering, various cool animations, such as light and shadow effects, etc., are all inseparable from the use of alpha blending.
  • MP4 video files obtain video YUV data through hard decoding or soft decoding. Before rendering, the YUV data needs to be converted into RGB data format, and then the RGB data is rendered to display a frame of video.
  • the embodiment of the present application is based on video technology. Since the video data format is YUV, the MP4 video file cannot carry alpha transparency information. Therefore, the embodiment of the present application opens up an additional space specifically for storing alpha transparency information, so the picture needs to be When the video is spliced into upper and lower parts, when rendering, go to the corresponding position and take the corresponding alpha transparency value to complete the alpha blended rendering.
  • Figure 14 is a schematic diagram of the overall process of the video processing method provided by the embodiment of the present application. As shown in Figure 14, the process of the entire solution is divided into two parts: alpha video production process 1401 and alpha usage process 1402, where the alpha video production process is the process from game special effects to generating alpha video, and the alpha usage process is the process from alpha video decoding to rendering.
  • each PNG image includes alpha transparency information.
  • Alpha transparency information is the information stored in the A channel in the RGBA of the image, indicating the transparency of the image. 0 means completely transparent, and 255 means opaque.
  • each PNG image into two images.
  • Alpha information images that are both equal to the alpha transparency value.
  • there are two separated pictures namely RGB picture 161 and alpha picture 162.
  • the two pictures obtained above that is, one is an RGB picture and the other is an alpha picture with RGB channel values equal to the alpha transparency value
  • spliced up and down (of course, in other embodiments, other methods are used Stitching is also possible, and you can also scale the picture.
  • the spliced picture shown in Figure 17 is a picture formed by scaling the alpha picture 0.5 times) to form a new PNG picture, as shown in Figure 17, RGB picture 171 Located in the upper half of the spliced image, the scaled alpha image 172 is located in the lower half of the spliced image.
  • alpha configuration information i.e., the above-mentioned default configuration information
  • the reason why the PNG image is separated into two images and then spliced together to form one image is because the PNG image is an image with four channel information of RGBA.
  • the PNG image is directly converted into a video, and the player What is obtained after decoding the video is YUV data, and then after the YUV data is converted into RGB data, the alpha information is actually lost. Therefore, splitting the PNG image into two images and then splicing them together to form one image can solve this problem, which is equivalent to opening up a picture area to store alpha information.
  • the spliced multiple spliced pictures are spliced into an MP4 video, and the alpha configuration information is inserted into the MP4 video.
  • the alpha configuration information can be inserted into the second box position of the MP4 data header.
  • Figure 18 it is a schematic diagram of the insertion position of the alpha configuration information provided by the embodiment of the present application.
  • the alpha configuration information can be inserted into the second box position of the MP4 data header.
  • the configuration information is inserted into the moov box 181.
  • the MP4 video format is selected as the carrier.
  • other video formats can also be selected as the carrier, that is, a video in other video formats is formed, and the alpha configuration information is inserted into the specific data header of other video formats. Location.
  • the alpha video (the above-mentioned transparency splicing video) has been obtained. If you use an ordinary video player to play the alpha video file, it can be played normally, but you can only see the spliced video with the upper and lower graphics spliced together, and you cannot see the special effects. Therefore, it is necessary to use alpha blending rendering technology in the unity shader of the game application to render the game special effects normally.
  • rendering each pixel requires RGB data and alpha data.
  • the RGB data comes from the upper part of the video frame.
  • the alpha data comes from the lower half of the video frame, but it needs to accurately correspond to the position between the two data.
  • you need to use the alpha configuration information read from the MP4 video which includes the width and height of the video, margins, starting offset position of the lower half, etc. .
  • the video is rendered into an RGBA four-channel video frame (that is, the video frame of the transparency special effect video) and displayed on the screen.
  • the rendering process shown in Figure 19 is based on the data in the RGB picture 1911 in the spliced picture 191 and the alpha picture.
  • the data in 1912 renders the video frame 192 of the transparency special effect video, and then such video frame 192 also has transparency information.
  • the video processing method provided by the embodiment of the present application can develop game special effects programs more efficiently, and supports particle special effects, transparent background display, etc. At the same time, the generated transparency special effect video file takes up little memory and can be achieved using hardware decoding.
  • the alpha data in the lower half can be further compressed, so various shapes of splicing methods can appear.
  • the video processing method of the embodiment of the present application uses MP4 video as a carrier, makes full use of the MP4 mature technology stack, and uses alpha mixing technology as a solution for game special effects rendering. It is a combination of MP4 and alpha rendering technology.
  • Alpha rendering technology is implemented using a unified shader and is fully independently developed and tested.
  • the alpha data compression mode can optimize the upper and lower splicing modes so that the alpha data only accounts for 1/3 of the data of the entire spliced picture.
  • the content related to user information such as original special effects video, color picture, transparency picture, transparency splicing video, transparency special effects video and other information, if it involves user information or enterprise information,
  • user permission or consent needs to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
  • the video processing device 354 includes:
  • the acquisition module is configured to acquire the color channel information of each pixel and the transparency channel information of each pixel in the original video frame for each original video frame of the original special effects video;
  • the picture generation module is configured to obtain the color channel information of each pixel in the original video frame based on the The color channel information of each pixel in the original video frame is used to draw a color picture corresponding to the original video frame, and based on the transparency channel information of each pixel in the original video frame, a color picture corresponding to the original video is drawn.
  • the transparency picture corresponding to the frame; the splicing processing module is configured to perform splicing processing on the color picture and transparency picture of each original video frame according to the preset configuration information to obtain at least one spliced picture; the video conversion module is configured to perform The at least one spliced picture is subjected to video conversion processing to obtain a transparency spliced video; a rendering module is configured to perform special effects video rendering based on the preset configuration information and the transparency spliced video to obtain a transparency special effects video.
  • the picture generation module is further configured to: add the color channel information of each pixel in the original video frame to the color channel of the first canvas picture to obtain the color channel corresponding to the original video frame.
  • Color picture add the transparency channel information of each pixel in the original video to the color channel of the second canvas picture to obtain a transparency picture corresponding to the original video frame; the first canvas picture and the third canvas picture Both canvas pictures are blank canvases without adding any pixel value information.
  • the color channel information includes RGB channel values
  • the transparency channel information includes transparency values
  • the picture generation module is further configured to: separate the RGB channel values of each pixel in the original video frame into Add to the RGB channel of the first canvas picture to obtain a color picture corresponding to the original video frame; and, add the transparency value of each pixel in the original video frame to the second canvas picture respectively In the RGB channel, a color picture corresponding to the original video frame is obtained.
  • the preset configuration information includes at least one of the following: the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture, and the offset value of the transparency picture; the splicing processing module is also configured is: based on the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture and the offset value of the transparency picture, determine the starting position of the transparency picture in the splicing canvas and the scaling ratio of the transparency picture; obtain the second position information of the color picture in the splicing canvas; According to the second position information, the starting position and the scaling ratio, the color picture and transparency picture of each original video frame are added to the splicing canvas to form the at least one splicing picture.
  • the splicing processing module is further configured to: determine the position of the transparency picture in the splicing canvas based on the first position information of the transparency picture in the splicing canvas and the size parameters of the transparency picture. Starting position; determine the scaling ratio of the transparency picture based on the size parameter of the transparency picture and the offset value of the transparency picture.
  • the splicing processing module is further configured to: for each original video frame, add the color picture to a position corresponding to the second position information in the splicing canvas to obtain a spliced picture with a color Canvas; scale the transparency image according to the scaling ratio to obtain a scaled transparency image; use the starting position as the starting adding position of the scaled transparency image, and add the scaled transparency The picture is added to the splicing canvas with color pictures to obtain a splicing picture.
  • the transparency splicing video corresponds to a video compression encoding format
  • the device further includes: an information acquisition module configured to acquire the first position information of the transparency picture in the splicing canvas, the The size parameter and the offset value of the transparency picture;
  • the format conversion module is configured to, according to the preset data storage format, convert the first position information of the transparency picture in the splicing canvas, the size parameters of the transparency picture and the The offset value of the transparency image is converted into a data format to obtain a configuration information string;
  • the insertion module is configured to insert the configuration information string into a preset position of the data header of the video compression encoding format.
  • the device further includes: a string acquisition module configured to obtain the data from the data header before splicing the color picture and transparency picture of each original video frame according to the preset configuration information. Obtain the configuration information string at the preset position; the string parsing module is configured to perform string parsing on the configuration information string to obtain the preset configuration information.
  • the rendering module is further configured to: perform video decoding on the transparency spliced video to obtain decoded format data of each spliced picture in the transparency spliced video; and perform video decoding on the decoded format data of each spliced picture. Convert the data format to obtain the color format data of each spliced picture; wherein the color format data includes a color channel data part and a transparency channel data part; based on the preset configuration information and the color channel data part of the spliced picture; and The transparency channel data part performs special effects video rendering to obtain the transparency special effects video.
  • the rendering module is further configured to: based on the preset configuration information, determine each color channel sub-data in the color channel data part and each transparency channel sub-data in the transparency channel data part
  • the mapping relationship between the color channel sub-data and the transparency channel sub-data with the mapping relationship is determined as the channel data of the same pixel point;
  • the color channel sub-data in the channel data of each pixel point is
  • the position information in the color picture is determined as the position information of the pixel point; according to the position information of each pixel point, special effects video rendering is performed on the channel data of each pixel point to obtain the transparency special effect video.
  • the device further includes: a background video rendering module configured to render the background video of the currently running application; and a video adding module configured to add the transparency special effect video to the background in response to a special effect triggering operation. specific location of the video.
  • Embodiments of the present application provide a computer program product or computer program.
  • the computer program product or computer program includes executable instructions.
  • the executable instructions are computer instructions; the executable instructions are stored in a computer-readable storage medium.
  • the processor of the electronic device reads the executable instructions from the computer-readable storage medium, the processor
  • the computer executes the executable instruction, the electronic device is caused to execute the method described above in the embodiment of the present application.
  • Embodiments of the present application provide a storage medium in which executable instructions are stored. When the executable instructions are executed by a processor, they will cause the processor to execute the method provided by embodiments of the present application. For example, as shown in Figure 7 shows the method.
  • the storage medium may be a computer-readable storage medium, such as ferroelectric memory (FRAM, Ferromagnetic Random Access Memory), read-only memory (ROM, Read Only Memory), programmable read-only memory (PROM, Programmable Read Only Memory), Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory, magnetic surface memory, optical disk, Or memory such as CD-ROM (Compact Disk-Read Only Memory); it can also be various devices including one of the above memories or any combination thereof.
  • FRAM ferroelectric memory
  • ROM Read Only Memory
  • PROM programmable read-only memory
  • EPROM Erasable Programmable Read Only Memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • flash memory magnetic surface memory, optical disk, Or memory such as CD-ROM (Compact Disk-Read Only Memory); it can also be various devices including one of the above memories or any combination thereof
  • executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and their May be deployed in any form, including deployed as a stand-alone program or deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.
  • executable instructions may, but do not necessarily correspond to, files in a file system and may be stored as part of a file holding other programs or data, for example, in a Hyper Text Markup Language (HTML) document. in one or more scripts, in a single file that is specific to the program in question, or in multiple collaborative files (e.g., files that store one or more modules, subroutines, or portions of code).
  • executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected by a communications network. execute on.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Studio Circuits (AREA)

Abstract

本申请实施例提供一种视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品,至少应用于动画制作领域和特效视频处理领域,其中,方法包括:针对原始特效视频的每一原始视频帧,基于其中的每一像素点的颜色通道信息和透明度通道信息,生成颜色图片和透明度图片;按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;对至少一张拼接图片进行视频转化处理,得到透明度拼接视频;基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频并显示。

Description

视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品
相关申请的交叉引用
本申请基于申请号为202211090054.2、申请日为2022年09月07日的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请实施例涉及互联网技术领域,涉及但不限于一种视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品。
背景技术
相关技术中,在进行特效视频制作时,通常采用代码实现,或者用动态图片(例如动画特效)或播放视频的方式播放特效来实现。其中,动态图片的制作通常采用Lottie动画、GIF动画、APNG动画、WebP格式和MP4格式视频等方式实现。
但是,上述代码实现的方式是很费时费力的,需要编写大量的特效代码来实现;而动态图片的方式,通常只支持软件解码,且制作的动画特效存在杂边等边界不清晰的问题,同时动画特效的文件也比较大,会极大的占用内存开销;播放视频的方式不支持透明度,因此,播放的特效视频无法完全融合到当前画面的背景图像中。
由此可见,相关技术中的特效视频制作和播放方案,均存在制作方式不便捷、特效还原度低,且内存开销较大等问题。
发明内容
本申请实施例提供一种视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品,至少应用于动画制作领域和特效视频处理领域,能够简化特效视频的制作方式,支持特效视频的透明度信息,从而实现在透明度特效视频播放时能够还原原始特效视频中的特效效果,降低透明度特效视频制作时对内存的占用量。
本申请实施例的技术方案是这样实现的:
本申请实施例提供一种视频处理方法,所述方法包括:针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;基于所述原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;对所述至少一张拼接图片进行视频转化处理,得到透明度拼接视频;基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频。
本申请实施例提供一种视频处理装置,所述装置包括:获取模块,配置为针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;图片生成模块,配置为基于所述原始视频帧中每一 像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;拼接处理模块,配置为按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;视频转化模块,配置为对所述至少一张拼接图片进行视频转化处理,得到透明度拼接视频;渲染模块,配置为基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频。
本申请实施例提供一种电子设备,包括:存储器,用于存储可执行指令;处理器,用于执行所述存储器中存储的可执行指令时,实现上述视频处理方法。
本申请实施例提供一种计算机程序产品或计算机程序,计算机程序产品或计算机程序包括可执行指令,可执行指令存储在计算机可读存储介质中;其中,电子设备的处理器从计算机可读存储介质中读取可执行指令,并执行可执行指令时,实现上述的视频处理方法。
本申请实施例提供一种计算机可读存储介质,存储有可执行指令,用于引起处理器执行所述可执行指令时,实现上述视频处理方法。
本申请实施例具有以下有益效果:针对原始特效视频中的每一原始视频帧,分别基于原始视频帧中每一像素点的颜色通道信息,绘制与原始视频帧对应的颜色图片,以及,基于原始视频帧中每一像素点的透明度通道信息,绘制与原始视频帧对应的透明度图片,并将颜色图片和透明度图片进行拼接处理得到至少一张拼接图片。如此,由于在拼接图片中,既包含有原始视频帧的颜色通道信息,又包含有原始视频帧的透明度通道信息,这样使得生成的透明度拼接视频中也携带有原始特效视频的透明度信息,从而在进行特效视频渲染时,能够基于透明度信息渲染得到透明度特效视频,实现在透明度特效视频播放时能够高度还原原始特效视频中的特效效果,并且,由于无需编写代码来实现特效实现的制作,因此,极大的降低透明度特效视频的制作成本以及制作时对内存的占用量。
附图说明
图1是采用Lottie动画制作动态图片的实现过程示意图;
图2是在Lottie动画设计中的不同按钮动画效果示意图;
图3是采用APNG技术制作的动画特效示意图;
图4是GIF动态图和WebP动态图大小对比示意图;
图5是本申请实施例提供的视频处理系统的一个可选的架构示意图;
图6是本申请实施例提供的电子设备的结构示意图;
图7是本申请实施例提供的视频处理方法的一个可选的流程示意图;
图8是本申请实施例提供的视频处理方法的另一个可选的流程示意图;
图9是本申请实施例提供的视频处理方法的再一个可选的流程示意图;
图10是本申请实施例提供的光影特效示意图;
图11是本申请实施例提供的游戏礼物特效示意图;
图12是本申请实施例提供的粒子特效示意图;
图13是本申请实施例提供的火焰特效示意图;
图14是本申请实施例提供的视频处理方法的整体流程示意图;
图15是本申请实施例提供的由AE软件导出的游戏特效连续的PNG图片的示意图;
图16是本申请实施例提供的RGBA四个通道信息分离得到的两张图片示意图;
图17是本申请实施例提供的拼接图片示意图;
图18是本申请实施例提供的alpha配置信息的插入位置示意图;
图19是本申请实施例提供的渲染过程示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请作进一步地详细描述,所描述的实施例不应视为对本申请的限制,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例,都属于本申请保护的范围。
在以下的描述中,涉及到“一些实施例”,其描述了所有可能实施例的子集,但是可以理解,“一些实施例”可以是所有可能实施例的相同子集或不同子集,并且可以在不冲突的情况下相互结合。除非另有定义,本申请实施例所使用的所有的技术和科学术语与属于本申请实施例的技术领域的技术人员通常理解的含义相同。本申请实施例所使用的术语只是为了描述本申请实施例的目的,不是旨在限制本申请。
这里,先对本申请实施例涉及的专业术语进行说明。
(1)alpha混合技术:指本申请实施例生成透明度特效视频的整体技术方案。
(2)alpha混合效果:是指运用本申请实施例的方法所做出的一些案例效果。
(3)光影特效:是指显示界面(例如游戏界面)中的光影显示效果。
(4)粒子特效:是指显示界面(例如游戏界面)中很多小物体的动态效果。
(5)硬解:即硬件解码,是视频解码的一种方式,指不使用中央处理器(CPU,Central Processing Unit)资源,利用专用硬件设备,例如,手机硬件设备,如图形处理器(GPU,Graphics Processing Unit)进行解码的方式。
(6)软解:即软件解码,是视频解码的一种方式,是指利用CPU解码。
在解释本申请实施例的视频处理方法之前,首先对相关技术中的方法进行说明。
相关技术中,在进行特效视频制作时,通常采用代码实现,或者用动态图片(例如动画特效)或播放视频的方式播放特效来实现。其中,动态图片的制作通常采用Lottie动画、GIF动画、APNG动画、WebP格式和MP4格式视频等方式实现。
这里,分别对Lottie动画、GIF动画、APNG动画、WebP格式和MP4格式视频等方式进行说明。
Lottie是一套开源的跨平台的完整的动画效果解决方案,设计师可以使用特效制作工具(AE,Adobe After Effects)设计出动画之后,再使用Lottie提供的Bodymovin插件将设计好的动画导出成JS对象简谱(JSON,JavaScript Object Notation)格式,就可以直接运用在不同的系统之上,无需其他额外操作。如图1所示,是采用Lottie动画制作动态图片的实现过程,通过AE 101和Bodymovin插件102将设计好的动画导出成JSON格式的JSON文件103,然后就可以应用到iOS系统、Android系统、Web和React Native(一种开源的跨平台移动应用开发框架)之上。在Lottie动画设计中,通常用于按钮等的动画效果,如图2所示,位置箭头201、指向箭头202和心形203等均可以通过Lottie动画设计实现。
GIF动画是指通过使用专门的动画制作工具或者采用逐帧拍摄对象的方法,让多个GIF图片按一定规律快速、连续播放运动的画面。例如,各种动态表情包都可以是GIF动画做的。
基于PNG的位图动画格式(APNG,Animated Portable Network Graphics),是便携式网络图形(PNG,Portable Network Graphics)格式的动画扩展。APNG的第一帧为标准PNG图像,除了第一帧之外的剩余的动画和帧速等数据被放在PNG扩展数据块里。APNG类似于视频的关键帧,关键帧有完整的图像信息,而两个关键帧之间只保留了变 化的信息。简单来说,APNG支持全彩和透明,且无杂边问题。如图3所示,是采用APNG技术制作的动画特效,其中,动画特效中的对象301的边界清晰无杂边。
WebP主要目标是使图片质量和JPEG(Joint Photographic Experts Group)格式质量相同的情况下,减少图片文件的体积,籍此减少互联网上图片的发送时间和流量消耗。图4是GIF动态图和WebP动态图大小对比,可以看出,WebP动态图402的图片大小远小于GIF动态图401的图片大小。
而MP4视频就是由一帧一帧的动画连接而成的视频。
针对上述几种动态图片的制作,其中,Lottie动画是字符串文件,支持软解,且无粒子特效。GIF动画也支持软解,支持8位色彩,且支持alpha透明背景,但周围杂边非常明显。APNG也支持软解,虽然相比alpha视频文件比较大,但是解决了GIF动态图具有杂边的问题。WebP也支持软解,相比alpha视频文件比较大。MP4视频支持硬解但不支持alpha透明背景。
相关技术中,代码实现的方式,程序员通过写代码是完全可以制作特效视频的,但考虑到效率,像复杂的特效视频(例如,粒子特效、光影特效、水波特效)用代码制作是很费时费力的。而通过播放视频的方案播放特效,有一个问题是不支持透明度,也就是特效视频无法完全融合到画面背景中,看起来就是一个矩形贴在画面背景上。
基于相关技术中所存在的问题,本申请实施例的视频处理方法,能够完美解决这些问题,同时在解码能力、使用便捷、特效还原、透明度支持等方面也都更有优势。
本申请实施例提供的视频处理方法中,首先,针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;基于原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;然后,按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;并对至少一张拼接图片进行视频转化处理,得到透明度拼接视频;最后,基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频,并显示透明度特效视频。如此,由于在拼接图片中既包含有原始视频帧的颜色通道信息,又包含有原始视频帧的透明度通道信息,这样使得生成的透明度拼接视频中可以携带有原始特效视频的透明度信息,从而在进行特效视频渲染时,能够基于透明度信息渲染得到透明度特效视频,实现在透明度特效视频播放时能够高度还原原始特效视频中的特效效果,极大的降低透明度特效视频的制作成本以及制作时对内存的占用量。
下面说明本申请实施例的视频处理设备的示例性应用,该视频处理设备是用于实现视频处理方法的电子设备。本申请实施例提供的视频处理设备(即电子设备)可以实施为终端,也可以实施为服务器。在一种实现方式中,本申请实施例提供的视频处理设备可以实施为笔记本电脑,平板电脑,台式计算机,移动设备(例如,移动电话,便携式音乐播放器,个人数字助理,专用消息设备,便携式游戏设备)、智能机器人、智能家电和智能车载设备等任意的具备视频渲染和显示功能、视频处理功能和游戏功能的终端;在另一种实现方式中,本申请实施例提供的视频处理设备还可以实施为服务器,其中,服务器可以是独立的物理服务器,也可以是多个物理服务器构成的服务器集群或者分布式系统,还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(CDN,Content Delivery Network)、以及大数据和人工智能平台等基础云计算服务的云服务器。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接,本申请实施例中不做限制。下面,将说明视频处理设备实施为服务器时的示例性应用。
参见图5,图5是本申请实施例提供的视频处理系统的一个可选的架构示意图,本申请实施例以视频处理方法应用于特效视频制作为例进行说明,也就是说,最终生成的透明度特效视频可以是任意一种类型的特效视频。本申请实施例中,视频处理系统10中至少包括终端100、网络200和服务器300,为实现支撑任意一个特效视频制作应用,本申请实施例的终端100上至少安装有特效视频制作应用,或者,终端100上安装有视频播放应用,在该视频播放应用中具有特效视频制作功能模块,通过该特效视频制作功能模块能够采用本申请实施例提供的视频处理方法,生成透明度特效视频。服务器300是特效视频制作应用的服务器,服务器300可以构成本申请实施例的视频处理设备。终端100通过网络200连接服务器300,网络200可以是广域网或者局域网,又或者是二者的组合。
在运行特效视频制作应用时,终端100通过特效视频制作应用的客户端获取原始特效视频,并将原始特效视频封装至视频处理请求中,将视频处理请求通过网络200发送给服务器300。服务器300解析视频处理请求得到原始特效视频,并获取原始特效视频的每一原始视频帧,以及,获取原始视频帧中的每一像素点的颜色通道信息和透明度通道信息;然后,针对每一原始视频帧,基于原始视频帧中每一像素点的颜色通道信息,绘制与原始视频帧对应的颜色图片,基于原始视频帧中每一像素点的透明度通道信息,绘制与原始视频帧对应的透明度图片;再然后,按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;对至少一张拼接图片进行视频转化处理,得到透明度拼接视频;最后,基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频。服务器300在得到透明度特效视频之后,将透明度特效视频发送给终端100,并在终端100的当前界面的特定位置显示透明度特效视频。
在一些实施例中,还可以由终端100实现视频处理方法,也就是说,由终端作为执行主体获取原始特效视频的每一原始视频帧,以及,原始视频帧的每一像素点的颜色通道信息和透明度通道信息;针对每一原始视频帧,由终端基于原始视频帧中每一像素点的颜色通道信息和透明度通道信息,生成与原始视频帧对应的颜色图片和透明度图片;以及,由终端按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;并对至少一张拼接图片进行视频转化处理,得到透明度拼接视频;最后,由终端基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频。终端在得到透明度特效视频的同时,还可以在当前界面上显示该透明度特效视频。
本申请实施例所提供的视频处理方法还可以基于云平台并通过云技术来实现,例如,上述服务器300可以是云端服务器。通过云端服务器针对原始特效视频的每一原始视频帧,获取原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息,或者,通过云端服务器生成与原始视频帧对应的颜色图片和透明度图片,或者,通过云端服务器对每一原始视频帧的颜色图片和透明度图片进行拼接处理,或者,通过云端服务器对至少一张拼接图片进行视频转化处理,或者,通过云端服务器基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频等。
在一些实施例中,还可以具有云端存储器,可以将原始特效视频存储至云端存储器中,或者,还可以将每一原始视频帧中的每一像素点的颜色通道信息和透明度通道信息存储至云端存储器中,或者,还可以将颜色图片、透明度图片、拼接图片、透明度拼接视频和透明度特效视频等存储至云端存储器中。这样,在输入原始特效视频时,可以从云端存储器中获取拼接图片,或者从云端存储器中获取透明度拼接视频,进而快速的渲染得到透明度特效视频。
这里需要说明的是,云技术(Cloud technology)是指在广域网或局域网内将硬件、软件、网络等系列资源统一起来,实现数据的计算、储存、处理和共享的一种托管技术。云技术基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术、应用技术等的总称,可以组成资源池,按需所用,灵活便利。云计算技术将变成重要支撑。技术网络系统的后台服务需要大量的计算、存储资源,如视频网站、图片类网站和更多的门户网站。伴随着互联网行业的高度发展和应用,将来每个物品都有可能存在自己的识别标志,都需要传输到后台系统进行逻辑处理,不同程度级别的数据将会分开处理,各类行业数据皆需要强大的系统后盾支撑,只能通过云计算来实现。
图6是本申请实施例提供的电子设备的结构示意图,图6所示的电子设备包括:至少一个处理器310、存储器350、至少一个网络接口320和用户接口330。电子设备中的各个组件通过总线系统340耦合在一起。可理解,总线系统340用于实现这些组件之间的连接通信。总线系统340除包括数据总线之外,还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见,在图6中将各种总线都标为总线系统340。
处理器310可以是一种集成电路芯片,具有信号的处理能力,例如通用处理器、数字信号处理器(DSP,Digital Signal Processor),或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等,其中,通用处理器可以是微处理器或者任何常规的处理器等。
用户接口330包括使得能够呈现媒体内容的一个或多个输出装置331,以及一个或多个输入装置332。
存储器350可以是可移除的,不可移除的或其组合。示例性的硬件设备包括固态存储器,硬盘驱动器,光盘驱动器等。存储器350可选地包括在物理位置上远离处理器310的一个或多个存储设备。存储器350包括易失性存储器或非易失性存储器,也可包括易失性和非易失性存储器两者。非易失性存储器可以是只读存储器(ROM,Read Only Memory),易失性存储器可以是随机存取存储器(RAM,Random Access Memory)。本申请实施例描述的存储器350旨在包括任意适合类型的存储器。在一些实施例中,存储器350能够存储数据以支持各种操作,这些数据的示例包括程序、模块和数据结构或者其子集或超集,下面示例性说明。
操作系统351,包括用于处理各种基本系统服务和执行硬件相关任务的系统程序,例如框架层、核心库层、驱动层等,用于实现各种基础业务以及处理基于硬件的任务;网络通信模块352,用于经由一个或多个(有线或无线)网络接口320到达其他计算设备,示例性的网络接口320包括:蓝牙、无线相容性认证(WiFi)、和通用串行总线(USB,Universal Serial Bus)等;输入处理模块353,用于对一个或多个来自一个或多个输入装置332之一的一个或多个用户输入或互动进行检测以及翻译所检测的输入或互动。
在一些实施例中,本申请实施例提供的装置可采用软件方式实现,图6示出了存储在存储器350中的一种视频处理装置354,该视频处理装置354可以是电子设备中的视频处理装置,其可以是程序和插件等形式的软件,包括以下软件模块:获取模块3541、图片生成模块3542、拼接处理模块3543、视频转化模块3544和渲染模块3545,这些模块是逻辑上的,因此根据所实现的功能可以进行任意的组合或进一步拆分。将在下文中说明各个模块的功能。
在另一些实施例中,本申请实施例提供的装置可以采用硬件方式实现,作为示例,本申请实施例提供的装置可以是采用硬件译码处理器形式的处理器,其被编程以执行本申请实施例提供的视频处理方法,例如,硬件译码处理器形式的处理器可以采用一个或多个应用专用集成电路(ASIC,Application Specific Integrated Circuit)、DSP、可编程逻辑器件(PLD,Programmable Logic Device)、复杂可编程逻辑器件(CPLD,Complex  Programmable Logic Device)、现场可编程门阵列(FPGA,Field-Programmable Gate Array)或其他电子元件。
本申请各实施例提供的视频处理方法可以由电子设备来执行,其中,该电子设备可以是任意一种具备视频渲染和显示功能、视频处理功能和游戏功能的终端,或者也可以是服务器,即本申请各实施例的视频处理方法可以通过终端来执行,也可以通过服务器来执行,或者还可以通过终端与服务器进行交互来执行。
参见图7,图7是本申请实施例提供的视频处理方法的一个可选的流程示意图,下面将结合图7示出的步骤进行说明,需要说明的是,图7中的视频处理方法是通过服务器作为执行主体为例来说明的。
步骤S701,针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息。
原始特效视频可以是采用特效制作工具制作得到的特效视频,其中,特效制作工具可以是AE软件(Adobe After Effects),通过AE软件可以制作并输出原始特效视频。原始特效视频可以包括任意类型的特效内容,例如,特效内容包括:粒子特效、光影特效、水波特效、游戏礼物特效、火焰特效等。在原始特效视频中,特效内容是具有一定透明度信息的内容,也就是说,原始视频帧中的每一像素点包括颜色通道信息和透明度通道信息。如果终端显示该原始特效视频的话,则原始特效视频中的特效内容的显示效果是边界分明且无杂边的。举例来说,当原始视频帧为RGB图像时,原始视频帧的RGB通道存储有颜色通道信息,原始视频帧的alpha通道(即透明度通道)存储有透明度通道信息。
原始特效视频并不是能直接显示的视频,因为视频解码渲染的流程是对一个MP4格式的视频文件先通过硬解或软解,得到视频YUV数据,在进行视频渲染前,需要将视频YUV数据转换成RGB格式数据,然后渲染RGB格式数据,从而显示一帧视频画面。由于渲染前的视频数据格式是YUV格式,YUV格式是不携带alpha信息(即透明度通道信息)的,因此渲染前的MP4视频并不能携带alpha信息,导致原始特效视频的透明度信息丢失,从而造成渲染后的特效视频并不能与背景有效的融合,呈现的效果将是背景被原始特效视频中特效内容之外的其他区域遮挡。举例来说,原始特效视频中的原始视频帧都是矩形的,而原始视频帧的特效内容如果是一个圆形的话,采用上述的渲染方式渲染后,由于透明度信息丢失,背景会被整个矩形遮挡,不是只显示一个圆形。本申请实施例的方法,由于将透明度信息提取出来,并在后续的步骤中进行有效的存储,可以保证渲染时基于透明度信息对特效视频的准确渲染,实现背景图像仅被圆形的特效内容遮挡,而其他地方正常显示的效果。
本申请实施例中,特效制作工具在制作得到原始特效视频后,还可以对原始特效视频进行分帧处理,即对原始特效视频进行视频帧提取,得到至少一张原始视频帧,并输出全部的原始视频帧。在获取到每一原始视频帧之后,可以对原始视频帧中的每一像素点的颜色通道信息和透明度通道信息进行提取。在提取的时候,可以获取每一像素点对应的RGBA通道值,将RGBA通道值中的RGB通道值确定为颜色通道信息,将A通道中的alpha信息确定为透明度通道信息。
在一些实施例中,在提取每一像素点的颜色通道信息和透明度通道信息时,还可以获取每一像素点在原始视频帧中的位置信息,并将位置信息与该像素点的颜色通道信息和透明度通道信息进行映射存储。这样,在后续进行颜色图片和透明度图片生成时,可以基于每一像素点的位置信息与颜色通道信息和透明度通道信息之间的映射关系,将颜色通道信息和透明度通道信息分别赋值给颜色图片和透明度图片中的每一像素点。
步骤S702,基于原始视频帧中每一像素点的颜色通道信息,绘制与原始视频帧对 应的颜色图片,以及,基于原始视频帧中每一像素点的透明度通道信息,绘制与原始视频帧对应的透明度图片。
本申请实施例中,可以将每一像素点的颜色通道信息和透明度通道信息分别赋值给颜色图片和透明度图片,例如,可以分别提供两张与原始视频帧具有相同大小的画布,然后将颜色通道信息和透明度通道信息分别赋值至画布中的每一像素点,得到颜色图片和透明度图片。
这里,画布(也称空白画布)是指与原始视频帧具有相同尺寸大小且未渲染任何图像内容的空白载体,该空白载体包括多个像素点,且在该空白载体上,每一像素点均不包含任何像素信息,即每一像素点均没有渲染任何颜色通道信息和透明度通道信息。通过对空白载体上每一像素点的像素信息进行渲染,即可得到具有图像内容的任意图像。本申请实施例中,由于画布的尺寸大小与原始视频帧的尺寸大小相同,因此,可以对应于原始视频帧中的每一像素点,将颜色通道信息和透明度通道信息分别赋值给两张画布中每一画布中的每一像素点,使得画布中的每一像素点携带颜色通道信息或者透明度通道信息,从而得到具有图像内容的颜色图片和透明度图片。在一些实施例中,上述画布可以是一虚拟画布,也就是说,可以在得到每一像素点的颜色通道信息和透明度通道信息之后,将颜色通道信息和透明度通道信息分别赋值至虚拟画布中的每一像素点,得到颜色图片和透明度图片。
颜色图片是携带有颜色通道信息的图片,透明度图片是携带有透明度通道信息的图片。通过对颜色图片进行像素解析,可以得到原始视频帧中的每一像素点的颜色像素值;通过对透明度图片进行像素解析,可以得到原始视频帧中的每一像素点的透明度信息。因此,结合颜色图片和透明度图片中的信息,就可以获取到原始视频帧中的每一像素点的全部信息。
本申请实施例中,可以按照原始视频帧在原始特效视频中的先后顺序,依次针对原始特效视频中的每一原始视频帧,均生成一张颜色图片和一张透明度图片。
步骤S703,按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片。
预设配置信息是用于表征透明度图片中的每一像素点与颜色图片中的像素点之间的映射关系的信息,在进行拼接处理时,基于预设配置信息,能够确定出颜色图片和透明度图片分别在拼接图片对应的画布(即拼接画布)中的位置。
预设配置信息可以包括透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值。其中,透明度图片在拼接画布中的第一位置信息,是指在拼接处理时,透明度图片的中心位置位于拼接图片对应的拼接画布的什么位置。透明度图片的尺寸参数包括透明度图片的宽度和高度,基于透明度图片的宽度和高度,再结合透明度图片在拼接画布中的第一位置信息,则可以确定出透明度图片中的每一像素点在拼接画布中的位置,即可以确定出透明度图片中的起始像素点和结束像素点分别所在的位置。透明度图片的偏移值是指在拼接图片中,透明度图片相对于颜色图片在至少一个方向上偏移的像素点的数量,这里,至少一个方向包括但不限于:沿拼接画布长度的延伸方向、沿拼接画布宽度的延伸方向、沿拼接画布对角线的延伸方向等。
需要说明的是,拼接图片对应的拼接画布是指用于添加颜色图片和透明度图片,以形成拼接图片的空白画布。可以理解的,拼接图片对应的拼接画布是指与拼接图片具有相同尺寸的空白画布。也就是说,在进行拼接处理时,可以先提供一空白画布,然后将颜色图片和透明度图片依次添加至该空白画布上,或者将颜色图片和透明度图片同时添加至该空白画布上,形成拼接图片。
在一些实施例中,颜色图片和透明度图片在拼接图片中的相对位置关系可以不固定, 也就是说,透明度图片可以位于颜色图片的任意相对位置上,且颜色图片和透明度图片不存在重叠区域。例如,透明度图片可以位于颜色图片的上方、下方、左方、右方、右上方、右下方、左上方、左下方等。
在一些实施例中,颜色图片和透明度图片的大小相同,但是在拼接图片中,添加到拼接图片中的颜色图片与添加到拼接图片中的透明度图片之间的大小可以相同,也可以不同。也就是说,可以对颜色图片和透明度图片中的至少一个进行特定比例的缩放处理,并将缩放处理后的颜色图片或透明度图片添加至拼接图片中。本申请实施例中,进行缩放处理的缩放比例可以基于预设配置信息中的信息来确定,也就是说,当给定预设配置信息时,颜色图片和透明度图片缩放比例也将是确定的,可以基于给定的预设配置信息对颜色图片和透明度图片进行缩放处理。同时由于预设配置信息已知,因此后续可以直接基于预设配置信息确定出在进行拼接处理时,对颜色图片和透明度图片分别进行了多大比例的缩放处理。
在另一些实施例中,可以仅对透明度图片进行缩放处理,不对颜色图片进行缩放处理,也就是说,将颜色图片按照尺寸大小1:1的比例,添加到空白画布中;将透明度图片进行适当比例的缩小处理再添加到空白画布中。这样,可以极大的缩小透明度信息的数量,可以将颜色图片中的多个像素点与缩小处理后的透明度图片中的一个像素点对应。本申请实施例中,可以对透明度图片进行0.5倍的缩小处理。
需要说明的是,每一原始视频帧的颜色图片和透明度图片在进行拼接处理时的预设配置信息相同,也就是说,同一原始特效视频中的全部原始视频帧共用同一预设配置信息。这样,在得到的每一张拼接图片中,颜色图片和透明度图片的大小相同,且颜色图片和透明度图片各自所在的位置也相同。
步骤S704,对至少一张拼接图片进行视频转化处理,得到透明度拼接视频。
这里,视频转化处理是将得到的至少一张拼接图片按照时间先后顺序进行视频压缩处理,得到一段透明度拼接视频。
步骤S705,基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频。
这里,可以基于预设配置信息从透明度拼接视频的每一视频帧(即拼接图片)中获取像素点的通道信息,即可以从拼接图片中的颜色图片中获取颜色通道信息,从拼接图片中的透明度图片中获取透明度通道信息。
为了保证特效视频渲染的准确进行,会在视频特效渲染之前进行一系列的视频格式转换等处理,而由于透明度拼接视频中的每一拼接图片既保留了颜色通道信息,还保留了透明度通道信息,因此,即使经过视频格式转换等处理,也不会丢失原始视频帧的透明度通道信息。从而使得在进行特效视频渲染时,可以考虑到原始视频帧的透明度通道信息,渲染得到的透明度特效视频在特效内容的边界处边界清晰,不会对透明度特效视频背后的背景图像在特效内容之外的其他区域的内容进行遮挡。并且,由于整个过程无需通过编写代码来实现,极大的简化了透明度特效视频的处理过程。同时,制作得到的透明度特效视频也不会占用过多的内存开销。
由此可见,本申请实施例提供的视频处理方法,针对原始特效视频中的每一原始视频帧,分别基于原始视频帧中每一像素点的颜色通道信息和透明度通道信息,对应生成与原始视频帧对应的颜色图片和透明度图片,并将颜色图片和透明度图片进行拼接处理得到至少一张拼接图片,之后再对至少一张拼接图片进行视频转化处理,得到透明度拼接视频,从而基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频。如此,由于在拼接图片中,既包含有原始视频帧的颜色通道信息,又包含有原始视频帧的透明度通道信息,这样使得生成的透明度拼接视频中也携带有原始特效视频的 透明度信息,从而在进行特效视频渲染时,能够基于透明度信息渲染得到透明度特效视频,实现在透明度特效视频播放时能够高度还原原始特效视频中的特效效果,极大的降低透明度特效视频的制作成本以及制作时对内存的占用量。
在一些实施例中,视频处理系统中至少包括终端和服务器,终端上安装有特效视频制作应用,或者,终端上安装有视频播放应用,在该视频播放应用中具有特效视频制作功能模块,通过该特效视频制作功能模块能够采用本申请实施例提供的视频处理方法,生成透明度特效视频。
图8是本申请实施例提供的视频处理方法的另一个可选的流程示意图,如图8所示,方法包括以下步骤:
步骤S801,终端通过特效制作工具制作得到原始特效视频。
本申请实施例中,用户可以通过特效视频制作功能模块中的特效制作工具制作得到原始特效视频,在制作得到原始特效视频时,终端可以接收用户通过视频播放应用的特效视频制作功能模块输入的特效制作操作,在用户输入特效制作操作时,可以输入特效的类型、特效的内容、特效的尺寸、特效的延时时长、特效的颜色、特效的分布等特效参数。特效制作工具基于输入的特效参数制作得到原始特效视频。
步骤S802,终端通过特效制作工具导出原始特效视频中的每一原始视频帧,并将每一原始视频帧封装至视频处理请求中。
通过特效制作工具在制作完原始特效视频之后,可以对原始特效视频进行分帧处理或者进行视频帧识别,得到原始特效视频中的每一原始视频帧,并输出每一原始视频帧,将每一原始视频帧存储至视频播放应用的特定存储位置。
在通过特效制作工具得到每一原始视频帧之后,还可以将每一原始视频帧封装至视频处理请求中,视频处理请求用于请求对原始特效视频中的每一原始视频帧进行处理,得到能够渲染出特效效果的透明度特效视频。
步骤S803,终端将视频处理请求发送给服务器。
步骤S804,服务器采集每一原始视频帧中的每一像素点的颜色通道信息和透明度通道信息。
这里,服务器对每一原始视频帧中的每一像素点的像素值进行解析,以提取原始视频帧中的每一像素点的颜色通道信息和透明度通道信息。
举例来说,特效制作工具导出的原始视频帧可以是PNG格式图片,在信息提取的时候,可以获取PNG格式图片中的每一像素点对应的RGBA通道值,将RGBA通道值中的RGB通道值确定为颜色通道信息,及颜色通道信息包括R通道值、G通道值和B通道值;将A通道中的alpha信息确定为透明度通道信息。
步骤S805,服务器针对每一原始视频帧,将原始视频帧中每一像素点的颜色通道信息添加至第一画布图片的颜色通道中,得到与原始视频帧对应的颜色图片。
这里,第一画布图片是未添加任何像素点的像素值信息的空白画布,也就是说,第一画布图片中不包括任何像素点的信息。第一画布图片与原始视频帧具有相同的尺寸。
在将原始视频帧中每一像素点的颜色通道信息添加至第一画布图片的颜色通道中时,可以按照像素点在原始视频帧的位置,依次将每一像素点的颜色通道信息,添加至第一画布图片的对应位置的颜色通道中,得到包含有原始视频帧的颜色通道信息的颜色图片。其中,颜色图片的尺寸与原始视频帧的尺寸相同。
在一些实施例中,颜色通道信息可以包括RGB通道值。可以将原始视频帧中每一像素点的RGB通道值分别对应添加至第一画布图片的RGB通道中,得到与原始视频帧对应的颜色图片。
步骤S806,服务器针对每一原始视频帧,将原始视频中每一像素点的透明度通道 信息添加至第二画布图片的颜色通道中,得到与原始视频帧对应的透明度图片。
这里,第二画布图片也是未添加任何像素点的像素值信息的空白画布,也就是说,第二画布图片中不包括任何像素点的信息。第二画布图片与原始视频帧也具有相同的尺寸。
在将原始视频帧中每一像素点的透明度通道信息添加至第二画布图片的颜色通道中时,可以按照像素点在原始视频帧的位置,依次将每一像素点的透明度通道信息,添加至第二画布图片的对应位置的颜色通道中,得到包含有原始视频帧的透明度通道信息的透明度图片。其中,透明度图片的尺寸也与原始视频帧的尺寸相同。
在一些实施例中,透明度通道信息可以包括透明度值。可以将原始视频帧中每一像素点的透明度值分别添加至第二画布图片的RGB通道中,得到与原始视频帧对应的颜色图片。这里,透明度值可以取值0至255,其中,0表示全部透明,255表示不透明,透明度值越高,不透明度越大。
步骤S807,服务器至少基于透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值,确定透明度图片在拼接画布中的起始位置和透明度图片的缩放比例。
这里,透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值,构成上述预设配置信息。其中,透明度图片在拼接画布中的第一位置信息,用于表征在拼接处理时透明度图片的中心位置位于拼接图片对应的拼接画布的哪个位置。透明度图片的尺寸参数包括透明度图片的宽度和高度,基于透明度图片的宽度和高度,再结合透明度图片在拼接画布中的第一位置信息,则可以确定出透明度图片中的每一像素点在拼接画布中的位置,即可以确定出透明度图片中的起始像素点和结束像素点分别所在的位置。透明度图片的偏移值是指在拼接图片中,透明度图片相对于颜色图片在至少一个方向上偏移的像素点的数量,这里,至少一个方向包括但不限于:沿拼接画布长度的延伸方向、沿拼接画布宽度的延伸方向、沿拼接画布对角线的延伸方向等。
在一些实施例中,可以基于透明度图片在拼接画布中的第一位置信息和透明度图片的尺寸参数,确定透明度图片在拼接画布中的起始位置。
在一种实现方式中,由于第一位置信息能够表征透明度图片的中心位置在拼接图片对应的拼接画布中的位置,透明度图片的尺寸参数包括透明度图片的宽度和高度。因此,基于透明度图片的宽度、高度、透明度图片的中心位置在拼接图片对应的拼接画布中的位置,则可以确定出透明度图片中的起始像素点(即第一个像素点)和结束像素点(即最后一个像素点)分别所在的位置。也就是说,至少可以确定出透明度图片在拼接画布中的起始位置(即起始像素点所在的位置)和结束位置。这里需要说明的是,由于透明度图片的尺寸参数已知,通常可以不用考虑结束位置,也就是说只需确定出透明度图片在拼接画布中的起始位置即可。
在另一种实现方式中,由于第一位置信息能够表征透明度图片的中心位置在拼接图片对应的拼接画布中的位置,透明度图片的尺寸参数包括透明度图片的宽度和高度。因此,基于透明度图片的宽度、高度、透明度图片的中心位置在拼接图片对应的拼接画布中的位置,则可以确定出透明度图片的边界线(即图片的边框)上的每一像素点分别所在的位置。这样,可以将边界框中的任意一条边界线上的像素点所在的位置,确定为透明度图片在拼接画布中的起始位置,从而以该边界线为起点,即可确定出透明度图片中的每一像素点在拼接画布中的位置。
在一些实施例中,还可以基于透明度图片的尺寸参数和透明度图片的偏移值,确定透明度图片的缩放比例。
这里,透明度图片的偏移值是指在拼接图片中,透明度图片相对于颜色图片在至少 一个方向上偏移的像素点的数量,也就是说,基于透明度图片的偏移值,可以确定出透明度图片中与颜色图片中的任一像素点对应的透明度图片中像素点,两者之间偏移的像素点的数量。
举例来说,可以对透明度图片进行0.5倍的缩小处理,在缩小处理之前,透明度图片和颜色图片的尺寸是1:1的比例关系,因此,在颜色图片中的(0,0)位置上的像素点与透明度图片中的(0,0)位置上的像素点对应。但是,由于对透明度图片进行0.5倍的缩小处理,使得透明度图片中的像素点的位置发生了偏移,因此,颜色图片中的(2,2)位置上的像素点与透明度图片中的(1,1)位置上的像素点对应,颜色图片中的(4,4)位置上的像素点与透明度图片中的(2,2)位置上的像素点对应。由此可见,透明度图片相对于颜色图片在透明度图片的长度方向和宽度方向上偏移的像素点的数量均为2。
步骤S808,服务器获取颜色图片在拼接画布中的第二位置信息。
这里,第二位置信息是指在拼接处理时,颜色图片的中心位置位于拼接图片对应的拼接画布的什么位置。
步骤S809,服务器按照第二位置信息、起始位置和缩放比例,将每一原始视频帧的颜色图片和透明度图片添加至拼接画布中,形成至少一张拼接图片。
在一些实施例中,步骤S809可以通过以下方式实现:针对每一原始视频帧,可以先将颜色图片添加至拼接画布中第二位置信息对应的位置,得到具有颜色图片的拼接画布。也就是说,以颜色图片的中心位置位于拼接画布的位置为扩展的起始点,对颜色图片从中心位置进行像素点扩展,从而将颜色图片中的每一像素点添加至拼接画布中,形成具有颜色图片的拼接画布。同时,还可以按照缩放比例对透明度图片进行缩放处理,得到缩放后的透明度图片。例如,可以按照0.5缩小比例,将透明度图片缩小至一半尺寸大小。最后以起始位置为缩放后的透明度图片的起始添加位置,将缩放后的透明度图片添加至具有颜色图片的拼接画布中,得到一张拼接图片。也就是说,按照起始位置的位置信息,将缩放后的透明度图片的每一像素点添加至具有颜色图片的拼接画布中。
在另一些实施例中,步骤S809还可以通过以下方式实现:先将透明度图片添加至拼接画布中,也就是说,先按照缩放比例对透明度图片进行缩放处理,得到缩放后的透明度图片,并以起始位置为缩放后的透明度图片的起始添加位置,将缩放后的透明度图片添加至拼接画布中;然后,将颜色图片添加至具有缩放后的透明度图片的拼接画布中第二位置信息对应的位置,得到一张拼接图片。
步骤S810,服务器对至少一张拼接图片进行视频转化处理,得到透明度拼接视频。
这里,视频转化处理是指对得到的至少一张拼接图片,按照时间先后顺序进行视频压缩处理,得到一段透明度拼接视频。
步骤S811,服务器基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频。
步骤S812,服务器将透明度特效视频发送给终端。
步骤S813,终端在当前界面上显示透明度特效视频。
本申请实施例提供的视频处理方法,基于透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值,确定透明度图片在拼接画布中的起始位置和透明度图片的缩放比例,从而按照颜色图片在拼接画布中的第二位置信息、起始位置和缩放比例,能够将每一原始视频帧的颜色图片和透明度图片添加至拼接画布中,实现对颜色图片和透明度图片的准确拼接处理,形成至少一张拼接图片。同时,该拼接图片中也是携带有原始视频帧中的每一像素点的颜色通道信息和透明度通道信息的图片,因此,通过进一步对拼接图片进行视频转化处理和特效视频渲染,能够得到携带有透明 度通道信息的透明度特效视频,实现在透明度特效视频播放时高度还原原始特效视频中的特效效果。
在一些实施例中,透明度拼接视频对应一视频压缩编码格式,这里的视频压缩编码格式包括但不限于MP4视频格式。本申请实施例的视频处理方法还可以包括以下方式:首先,服务器获取透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值。然后,服务器按照预设数据存储格式,对透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值进行数据格式转换,得到配置信息字符串。最后,服务器将配置信息字符串,插入至视频压缩编码格式的数据头的预设位置。
在一些实施例中,数据格式转换可以是将透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值转换成JSON字符串,得到JSON字符串格式的信息,该JSON字符串格式的信息即配置信息字符串。本申请实施例中,可以将JSON字符串格式的信息插入至MP4视频格式的数据头的预设位置。例如,可以插入至数据头的第二个盒子位置。
在另一些实施例中,上述视频处理方法也可以由终端来实现,也就是说,由终端作为执行主体,在获取到原始特效视频之后,对原始特效视频进行视频帧提取,得到至少一个原始视频帧,并对每一原始视频帧进行信息提取,得到每一原始视频帧中的每一像素点的颜色通道信息和透明度通道信息。之后,针对每一原始视频帧,分别基于原始视频帧中每一像素点的颜色通道信息和透明度通道信息,生成与原始视频帧对应的颜色图片和透明度图片;按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;对至少一张拼接图片进行视频转化处理,得到透明度拼接视频;最终,基于预设配置信息和透明度拼接视频进行特效视频渲染,得到透明度特效视频,在得到透明度特效视频时,在当前界面显示该透明度特效视频。并且,由终端获取透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值等预设配置信息,然后按照预设数据存储格式,对透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值进行数据格式转换,得到配置信息字符串,并将配置信息字符串插入至视频压缩编码格式的数据头的预设位置。
由于通过上述方式实现了将配置信息字符串插入至视频压缩编码格式的数据头的预设位置,也就是说,预先将预设配置信息预先存储至视频压缩编码格式的数据头的预设位置,因此,可以从视频压缩编码格式的数据头的预设位置直接获取预设配置信息。本申请实施例中,在按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理之前,还可以实现以下步骤:从数据头的预设位置获取所述配置信息字符串;并且,对配置信息字符串进行字符串解析,得到预设配置信息。
图9是本申请实施例提供的视频处理方法的再一个可选的流程示意图,如图9所示,方法包括以下步骤:
步骤S901,终端通过特效制作工具制作得到原始特效视频。
步骤S902,终端通过特效制作工具导出原始特效视频中的每一原始视频帧,并将每一原始视频帧封装至视频处理请求中。
步骤S903,终端将视频处理请求发送给服务器。
步骤S904,服务器采集每一原始视频帧中的每一像素点的颜色通道信息和透明度通道信息。
需要说明的是,步骤S901至步骤S904与上述步骤S801至步骤S804相同,本申请实施例不再赘述。
步骤S905,服务器针对每一原始视频帧,分别基于原始视频帧中每一像素点的颜 色通道信息和透明度通道信息,生成与原始视频帧对应的颜色图片和透明度图片。
本申请实施例中,可以通过将每一像素点的颜色通道信息和透明度通道信息分别赋值给颜色图片和透明度图片,例如,可以分别提供两张与原始视频帧具有相同大小的画布,然后将颜色通道信息和透明度通道信息分别赋值至画布中的每一像素点,得到颜色图片和透明度图片。颜色图片是携带有颜色通道信息的图片,透明度图片是携带有透明度通道信息的图片。
步骤S906,服务器按照预设配置信息,对每一原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片。
预设配置信息是用于表征透明度图片中的每一像素点与颜色图片中的像素点之间的映射关系的信息。在进行拼接处理时,基于预设配置信息,能够确定出颜色图片和透明度图片分别在拼接图片对应的拼接画布中的位置。
步骤S907,服务器对至少一张拼接图片进行视频转化处理,得到透明度拼接视频。
步骤S908,服务器对透明度拼接视频进行视频解码,得到透明度拼接视频中每一拼接图片的解码格式数据。
这里,可以通过硬件解码或者软件解码的方式(例如,使用视频播放器硬件解码或软件解码),得到透明度拼接视频的解码格式数据,其中,解码格式数据可以是视频帧YUV数据。
步骤S909,服务器对每一拼接图片的解码格式数据进行数据格式转换,得到每一拼接图片的颜色格式数据;其中,颜色格式数据包括颜色通道数据部分以及透明度通道数据部分。
这里,可以是将每一拼接图片对应的视频帧YUV数据,通过转换公式转换成RGB数据格式的颜色格式数据。由于拼接图片是由颜色图片和透明度图片拼接形成的,而颜色图片中携带有颜色通道信息,透明度图片中携带有透明度通道信息,因此,经过视频解码和数据格式转换后的RGB数据格式的颜色格式数据包括颜色通道数据部分以及透明度通道数据部分。其中,颜色通道数据部分对应颜色图片中携带的颜色通道信息,透明度通道数据部分对应透明度图片中携带的透明度通道信息。
步骤S910,服务器基于预设配置信息和拼接图片的颜色通道数据部分以及透明度通道数据部分进行特效视频渲染,得到透明度特效视频。
在一些实施例中,步骤S910可以通过以下方式实现:首先,从可以MP4视频格式的数据头的预设位置获取JSON字符串格式的信息,即获取到存储在MP4视频格式的数据头的预设位置的预设配置信息。然后,基于预设配置信息,确定颜色通道数据部分中的每一颜色通道子数据与透明度通道数据部分中的每一透明度通道子数据之间的映射关系。这里,颜色通道子数据是颜色通道数据部分中对应于每一像素点的数据;透明度通道子数据是透明度通道数据部分中对应于每一像素点的数据。再然后,将具有映射关系的颜色通道子数据和透明度通道子数据,确定为同一像素点的通道数据。也就是说,将具有映射关系的表征同一像素点信息的颜色通道子数据和透明度通道子数据关联起来,共同来表征同一像素点的通道数据。并且,将每一像素点的通道数据中的颜色通道子数据在颜色图片中的位置信息,确定为像素点的位置信息。由于在进行颜色图片和透明度图片的拼接处理时,是对透明度图片进行缩放处理,而不对颜色图片进行缩放处理,因此,颜色图片中的每一像素点与原始视频帧均可以一一对应,所以颜色图片中的每一像素点的位置信息与原始视频帧中的像素点的位置信息对应。如此,在得到的颜色通道子数据中,颜色通道子数据在颜色图片中的位置信息也将与原始视频帧中的像素点的位置信息对应,所以,可以将颜色通道子数据在颜色图片中的位置信息,确定为像素点的位置信息。最后,按照每一像素点的位置信息,对每一像素点的通道数据进行特效视频 渲染,得到透明度特效视频。
本申请实施例中,在对每一像素点的通道数据进行特效视频渲染时,可以按照每一像素点的位置信息,将该像素点的通道数据中的颜色通道子数据作为RGB数据,将该像素点的通道数据中的透明度通道子数据作为alpha信息,实现特效视频渲染,得到透明度特效视频。
步骤S911,服务器将透明度特效视频发送给终端。
步骤S912,终端渲染当前运行应用的背景视频。
这里,背景视频可以是任意一种类型的视频,例如,短视频、电视剧、游戏视频等。当然,背景视频也可以是图片,即终端在当前界面上显示的图片。
步骤S913,终端接收特效触发操作。
步骤S914,终端响应于特效触发操作,将透明度特效视频添加至背景视频的特定位置。
这里,将透明度特效视频添加至背景视频的特定位置,可以是将透明度特效视频显示在背景视频之上。在一些实施例中,用户可以指定特定位置,终端也可以对背景视频进行视频分析,智能的确定出用于叠加透明度特效视频的特定位置。
本申请实施例的方法,至少可以应用于以下场景:
场景一:终端上安装有特效视频制作应用,特效视频制作应用用于制作并生成透明度特效视频。用于可以在特效视频制作应用的客户端输入特效视频制作请求,特效视频制作应用的服务器可以响应于特效视频制作请求,采用本申请实施例提供的视频处理方法,生成携带有透明度通道信息的透明度拼接视频,并进一步生成透明度特效视频。在生成透明度特效视频之后,可以将透明度特效视频作为特效视频制作请求的响应结果发送给终端的客户端,并在客户端上显示透明度特效视频;也可以将透明度特效视频作为特效视频制作应用制作产品输出值其他视频应用中,是的其他视频应用可以直接应用特效视频制作应用生成的透明度特效视频。
场景二:终端上安装有视频播放应用,在该视频播放应用中具有特效视频制作功能模块,通过该特效视频制作功能模块能够采用本申请实施例提供的视频处理方法,生成透明度特效视频。当用户在视频播放应用的客户端进行视频播放时,为了增加视频播放的趣味性,如果想在当前播放视频之上显示特效视频,则可以对特效视频制作功能模块进行操作,触发本申请实施例的视频处理流程,生成透明度特效视频,并在生成透明度特效视频的同时,将透明度特效视频叠加至当前播放视频之上,与当前播放视频同时显示。
场景三:终端上安装有游戏应用,在运行游戏应用的过程中,如果想要在游戏应用中添加特效,玩家可以在游戏客户端进行操作,选中特效按钮,则可以采用本申请实施例的视频处理方法生成透明度特效视频,并在游戏界面上,将透明度特效视频叠加至游戏画面中,与游戏画面同时播放。
在一些实施例中,当用户通过客户端输入特效触发操作时,还可以输入透明度特效视频的播放位置,服务器在生成透明度特效视频之后,将透明度特效视频反馈给终端,终端的客户端在该播放位置插入透明度特效视频,或者在该播放位置播放透明度特效视频。
在一些实施例中,当用户通过客户端输入特效触发操作时,还可以输入特效内容的显示数量,这样,服务器在生成透明度特效视频时,可以根据特效内容的显示数量,生成具有相应数量的特效内容的透明度特效视频,并播放该透明度特效视频。或者,在另一些实施例中,服务器可以生成包含有一个特效内容的透明度特效视频,在生成该透明度特效视频之后,对该透明度特效视频进行复制,得到与显示数量具有相同数量的透明 度特效视频,并对这些数量的透明度特效视频进行同时播放。这里需要说明的是,特效内容是指透明度特效视频中的对象,举例来说,对于游戏特效,特效内容可以是所显示特效中的一双鞋子、一束花、一本书等对象。
在一些实施例中,当用户通过客户端输入针对多种类型的特效内容的特效触发操作时,服务器可以响应于用户的特效触发操作,同时生成多种类型特效内容的透明度特效视频,并将生成的多个透明度特效视频反馈给客户端,在客户端上依次播放多个透明度特效视频或者同时播放多个透明度特效视频。如果多个透明度特效视频的播放位置相同,则可以依次播放多个透明度特效视频,如果多个透明度特效视频的播放位置不同,则可以同时播放多个透明度特效视频。
下面,将说明本申请实施例在一个实际的应用场景中的示例性应用。
在游戏特效使用场景中,编程实现特效让游戏更加炫酷好玩,但对于粒子特效、光影特效、水波特效中出现的物体分散性,使得编程实现类似特效并不是最好的手段。使用本申请实施例提供的视频处理方法,使粒子特效、光影特效、水波特效等等,能够像播放视频一样简单,不仅游戏特效开发时间大大减少,性能提升也得到保障。在视频处理效率、性能和用户体验上更优于相关技术中的方案。
本申请实施例提供的视频处理方法,至少能够实现对图10中的光影特效1001的制作和显示、对图11中的游戏礼物1101的游戏礼物特效(带alpha透明背景)1102的制作和显示、对图12中的粒子特效1201的制作和显示、对图13中的火焰特效1301的制作和显示。
游戏的渲染其实就是每一个像素点的渲染,每个像素点分成R、G、B、alpha四个通道。RGB是颜色通道,alpha是像素的透明度通道。整个游戏特效渲染,各种炫酷的动画,例如光影效果等等,都离不开alpha混合的运用。
这里对为什么特效需要使用到alpha混合来进行说明:举个例子,由于所有的图片都是矩形的,图片内容是一个圆形的话,使用这种图片的时候,其实背景会被整个矩形遮挡,不是只显示一个圆形。这个时候就需要使用alpha通道了,让圆形周围的地方都设置成透明。
这里先简单介绍下视频解码渲染流程:MP4视频文件通过硬解或软解,得到视频YUV数据,渲染前需要将YUV数据转换成RGB数据格式,然后渲染RGB数据,显示一帧视频画面。本申请实施例就是基于视频技术,由于视频数据格式是YUV,所以MP4视频文件并不能携带alpha透明度信息,因此,本申请实施例另外开辟一块空间专门用来存储alpha透明度信息,所以就需要把图片拼接成上下两部分构成的视频,在渲染的时候,到相应位置取对应的alpha透明度值,就可以完成alpha混合的渲染。
下面介绍本申请实施例提供的视频处理方法的整体流程,图14是本申请实施例提供的视频处理方法的整体流程示意图,如图14所示,整个方案的流程分成两部分:alpha视频生产过程1401和alpha使用过程1402,其中,alpha视频生产过程是游戏特效到生成alpha视频的过程,alpha使用过程是alpha视频解码到渲染的过程。
(1)在alpha视频生产过程1401中,首先,通过AE软件导出游戏特效连续的PNG图片,如图15所示,是本申请实施例提供的由AE软件导出的游戏特效连续的PNG图片的示意图,每张PNG图片都是包括alpha透明度信息的。alpha透明度信息是存储在图片RGBA中的A通道中的信息,表示图片的透明度,0就是全部透明,255就是不透明。
然后,将每张PNG图片分离成两张图片,可以把PNG图片的RGBA四个通道信息分离成RGB和alpha两部分,并形成两张图片,一张是RGB图片,另一张是RGB通道值都等于alpha透明度值的alpha信息图片。例如,一张PNG图片的RGBA=(255, 0,0,128),将该PNG图片分离成RGB=(255,0,0)和RGB=(128,128,128)的两张图片。如图16所示,是分离得到的两张图片,即RGB图片161和alpha图片162。
再然后,将上述得到的两张图片(即一张是RGB图片,另一张是RGB通道值都等于alpha透明度值的alpha图片)按上下拼接起来(当然,在其他实施例中,采用其他方式拼接也可以,还可以对图片进行缩放,如图17所示的拼接图片,就是将alpha图片缩放0.5倍后形成的图片),形成一张新的PNG图片,如图17所示,RGB图片171位于拼接图片的上半部分,缩放后的alpha图片172位于拼接图片的下半部分。与此同时,会把拼接的alpha图片所在的位置、宽高、偏移值(alpha图片在整个拼接图片中的起始位置)等信息记录下来,形成alpha配置信息(即上述预设配置信息),并将alpha配置信息以JSON字符串的形成保留。
需要说明的是,之所以要将PNG图片分离成两张图片,之后又拼接起来形成一张图片,是因为PNG图片是具有RGBA四个通道信息的图片,PNG图片直接转成视频,播放器在对视频解码后得到的是YUV数据,然后YUV数据转换成RGB数据后,其实就丢失了alpha信息。所以将PNG图片分离成两张图片,之后又拼接起来形成一张图片,就能解决这个问题,相当于开辟了一个图片区域来存放alpha信息。
最后,将拼接后的多张拼接图片拼接成一段MP4视频,并把alpha配置信息插入到该MP4视频中。考虑到读取效率,可以将alpha配置信息插入到MP4数据头的第二个盒子(box)位置,如图18所示,是本申请实施例提供的alpha配置信息的插入位置示意图,可以将alpha配置信息插入至moov盒子181中。
需要说明的是,由于MP4格式技术栈成熟稳定且在不同系统端都能够得到非常好的支持,无兼容性问题。因此,本申请实施例中选择MP4视频格式作为载体,当然,也可以选择其他的视频格式作为载体,即形成其他视频格式的一段视频,并将alpha配置信息插入到其他视频格式的数据头的特定位置。
此时,就已经得到了alpha视频(即上述透明度拼接视频)。如果使用普通视频播放器播放alpha视频的文件,可以正常播放,但只能看到上下拼接图形的拼接视频,无法看到特效。因此,需要在游戏应用的统一着色器(unity shader)中,使用alpha混合渲染技术,才能正常渲染出游戏特效。
(2)在alpha使用过程过程1402中,首先,对alpha视频进行硬解或软解(使用视频播放器硬解或软件),得到视频帧YUV数据;然后,从alpha视频文件中的第二个盒子中读取出alpha配置信息的JSON字符串。再然后,把视频帧YUV数据通过转换公式转换成RGB数据格式,其中,YUV数据格式与RGB数据格式之间的转换如以下公式(1)至(6):
RGB数据格式转YUV数据格式如以下公式(1)至(3):
Y=0.257R+0.504G+0.098B+16      (1)
Cr=V=0.439R-0.368G-0.071B+16    (2)
Cb=U=-0.148R-0.291G+0.439B+128    (3)
YUV数据格式转RGB数据格式如以下公式(4)至(6):
B=1.164(Y-16)+2.018(U-128)    (4)
G=1.164(Y-16)-0.813(V-128)-0.391(U-128)    (5)
R=1.164(Y-16)+1.596(V-128)     (6)
之后,渲染每个像素点,需要RGB数据和alpha数据,RGB数据来自视频帧的上 半部分,alpha数据来自视频帧的下半部分,但是需要精确对应这两个数据之间位置。在对应着两个数据之间位置的时候,就需要用上从MP4视频中读取到的alpha配置信息,里面包括了视频的宽高、边距、下半部分起始偏移位置等等信息。这样视频就被渲染成RGBA四通道的视频帧(即透明度特效视频的视频帧)显示在屏幕上,如图19所示的渲染过程,基于拼接图片191中的RGB图片1911中的数据和alpha图片1912中的数据,渲染得到透明度特效视频的视频帧192,那么这样的视频帧192也就具备了透明度信息。
本申请实施例提供的视频处理方法,能够更加高效地开发游戏特效程序,且支持粒子特效,支持透明背景显示等。同时,生成的透明度特效视频文件内存占用小,采用硬件解码即可实现。
需要说明的是,在拼接上图片中,下半部分的alpha数据可以进行进一步压缩,因此,可以出现各种形状的拼接方式。
本申请实施例的视频处理方法,使用MP4视频作为载体,充分利用了MP4成熟技术栈,把alpha混合技术作为游戏特效渲染的方案,是MP4与alpha渲染技术的结合体。alpha渲染技术是使用统一着色器实现,充分自主研发和测试。alpha数据压缩形式,能够实现从上下拼接模式优化到alpha数据只占整个拼接图片的数据的1/3。
可以理解的是,在本申请实施例中,涉及到用户信息的内容,例如,原始特效视频、颜色图片、透明度图片、透明度拼接视频和透明度特效视频等信息,如果涉及与用户信息或企业信息相关的数据,当本申请实施例运用到具体产品或技术中时,需要获得用户许可或者同意,且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
下面继续说明本申请实施例提供的视频处理装置354实施为软件模块的示例性结构,在一些实施例中,如图6所示,视频处理装置354包括:
获取模块,配置为针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;图片生成模块,配置为基于所述原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;拼接处理模块,配置为按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;视频转化模块,配置为对所述至少一张拼接图片进行视频转化处理,得到透明度拼接视频;渲染模块,配置为基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频。
在一些实施例中,所述图片生成模块还配置为:将所述原始视频帧中每一像素点的颜色通道信息添加至第一画布图片的颜色通道中,得到与所述原始视频帧对应的颜色图片;将所述原始视频中每一像素点的透明度通道信息添加至第二画布图片的颜色通道中,得到与所述原始视频帧对应的透明度图片;所述第一画布图片和所述第二画布图片均是未添加任何像素点的像素值信息的空白画布。
在一些实施例中,所述颜色通道信息包括RGB通道值,所述透明度通道信息包括透明度值;所述图片生成模块还配置为:将所述原始视频帧中每一像素点的RGB通道值分别添加至所述第一画布图片的RGB通道中,得到与所述原始视频帧对应的颜色图片;以及,将所述原始视频帧中每一像素点的透明度值分别添加至所述第二画布图片的RGB通道中,得到与所述原始视频帧对应的颜色图片。
在一些实施例中,所述预设配置信息包括以下至少之一:透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值;所述拼接处理模块还配置为:基于所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参 数和所述透明度图片的偏移值,确定所述透明度图片在拼接画布中的起始位置和所述透明度图片的缩放比例;获取所述颜色图片在所述拼接画布中的第二位置信息;按照所述第二位置信息、所述起始位置和所述缩放比例,将每一所述原始视频帧的颜色图片和透明度图片添加至所述拼接画布中,形成所述至少一张拼接图片。
在一些实施例中,所述拼接处理模块还配置为:基于所述透明度图片在拼接画布中的第一位置信息和所述透明度图片的尺寸参数,确定所述透明度图片在所述拼接画布中的起始位置;基于所述透明度图片的尺寸参数和所述透明度图片的偏移值,确定所述透明度图片的缩放比例。
在一些实施例中,所述拼接处理模块还配置为:针对每一所述原始视频帧,将所述颜色图片添加至所述拼接画布中第二位置信息对应的位置,得到具有颜色图片的拼接画布;按照所述缩放比例对所述透明度图片进行缩放处理,得到缩放后的透明度图片;以所述起始位置为所述缩放后的透明度图片的起始添加位置,将所述缩放后的透明度图片添加至所述具有颜色图片的拼接画布中,得到一张拼接图片。
在一些实施例中,所述透明度拼接视频对应一视频压缩编码格式;所述装置还包括:信息获取模块,配置为获取所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值;格式转换模块,配置为按照预设数据存储格式,对所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值进行数据格式转换,得到配置信息字符串;插入模块,配置为将所述配置信息字符串,插入至所述视频压缩编码格式的数据头的预设位置。
在一些实施例中,所述装置还包括:字符串获取模块,配置为在按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理之前,从所述数据头的预设位置获取所述配置信息字符串;字符串解析模块,配置为对所述配置信息字符串进行字符串解析,得到所述预设配置信息。
在一些实施例中,所述渲染模块还配置为:对所述透明度拼接视频进行视频解码,得到所述透明度拼接视频中每一拼接图片的解码格式数据;对每一拼接图片的解码格式数据进行数据格式转换,得到每一拼接图片的颜色格式数据;其中,所述颜色格式数据包括颜色通道数据部分以及透明度通道数据部分;基于所述预设配置信息和所述拼接图片的颜色通道数据部分以及透明度通道数据部分进行特效视频渲染,得到所述透明度特效视频。
在一些实施例中,所述渲染模块还配置为:基于所述预设配置信息,确定所述颜色通道数据部分中的每一颜色通道子数据与透明度通道数据部分中的每一透明度通道子数据之间的映射关系;将具有所述映射关系的颜色通道子数据和透明度通道子数据,确定为同一像素点的通道数据;将每一所述像素点的通道数据中的颜色通道子数据在所述颜色图片中的位置信息,确定为所述像素点的位置信息;按照每一像素点的位置信息,对每一像素点的通道数据进行特效视频渲染,得到所述透明度特效视频。
在一些实施例中,所述装置还包括:背景视频渲染模块,配置为渲染当前运行应用的背景视频;视频添加模块,配置为响应于特效触发操作,将所述透明度特效视频添加至所述背景视频的特定位置。
需要说明的是,本申请实施例装置的描述,与上述方法实施例的描述是类似的,具有同方法实施例相似的有益效果,因此不做赘述。对于本装置实施例中未披露的技术细节,请参照本申请方法实施例的描述而理解。
本申请实施例提供了一种计算机程序产品或计算机程序,该计算机程序产品或计算机程序包括可执行指令,该可执行指令是一种计算机指令;该可执行指令存储在计算机可读存储介质中。当电子设备的处理器从计算机可读存储介质读取该可执行指令,处理 器执行该可执行指令时,使得该电子设备执行本申请实施例上述的方法。
本申请实施例提供一种存储有可执行指令的存储介质,其中存储有可执行指令,当可执行指令被处理器执行时,将引起处理器执行本申请实施例提供的方法,例如,如图7示出的方法。
在一些实施例中,存储介质可以是计算机可读存储介质,例如,铁电存储器(FRAM,Ferromagnetic Random Access Memory)、只读存储器(ROM,Read Only Memory)、可编程只读存储器(PROM,Programmable Read Only Memory)、可擦除可编程只读存储器(EPROM,Erasable Programmable Read Only Memory)、带电可擦可编程只读存储器(EEPROM,Electrically Erasable Programmable Read Only Memory)、闪存、磁表面存储器、光盘、或光盘只读存储器(CD-ROM,Compact Disk-Read Only Memory)等存储器;也可以是包括上述存储器之一或任意组合的各种设备。
在一些实施例中,可执行指令可以采用程序、软件、软件模块、脚本或代码的形式,按任意形式的编程语言(包括编译或解释语言,或者声明性或过程性语言)来编写,并且其可按任意形式部署,包括被部署为独立的程序或者被部署为模块、组件、子例程或者适合在计算环境中使用的其它单元。
作为示例,可执行指令可以但不一定对应于文件系统中的文件,可以可被存储在保存其它程序或数据的文件的一部分,例如,存储在超文本标记语言(HTML,Hyper Text Markup Language)文档中的一个或多个脚本中,存储在专用于所讨论的程序的单个文件中,或者,存储在多个协同文件(例如,存储一个或多个模块、子程序或代码部分的文件)中。作为示例,可执行指令可被部署为在一个电子设备上执行,或者在位于一个地点的多个电子设备上执行,又或者,在分布在多个地点且通过通信网络互连的多个电子设备上执行。
以上所述,仅为本申请的实施例而已,并非用于限定本申请的保护范围。凡在本申请的精神和范围之内所作的任何修改、等同替换和改进等,均包含在本申请的保护范围之内。

Claims (15)

  1. 一种视频处理方法,所述方法由电子设备执行,所述方法包括:
    针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;
    基于所述原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;
    按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;
    对所述至少一张拼接图片进行视频转化处理,得到透明度拼接视频;
    基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频。
  2. 根据权利要求1所述的方法,其中,所述基于所述原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片,包括:
    将所述原始视频帧中每一像素点的颜色通道信息添加至第一画布图片的颜色通道中,得到与所述原始视频帧对应的颜色图片;
    将所述原始视频中每一像素点的透明度通道信息添加至第二画布图片的颜色通道中,得到与所述原始视频帧对应的透明度图片;所述第一画布图片和所述第二画布图片均是未添加任何像素点的像素值信息的空白画布。
  3. 根据权利要求2所述的方法,其中,所述颜色通道信息包括RGB通道值,所述透明度通道信息包括透明度值;
    所述将所述原始视频帧中每一像素点的颜色通道信息添加至第一画布图片的颜色通道中,得到与所述原始视频帧对应的颜色图片,包括:
    将所述原始视频帧中每一像素点的RGB通道值分别添加至所述第一画布图片的RGB通道中,得到与所述原始视频帧对应的颜色图片;
    所述将所述原始视频中每一像素点的透明度通道信息添加至第二画布图片的颜色通道中,得到与所述原始视频帧对应的透明度图片,包括:
    将所述原始视频帧中每一像素点的透明度值分别添加至所述第二画布图片的RGB通道中,得到与所述原始视频帧对应的颜色图片。
  4. 根据权利要求1所述的方法,其中,所述预设配置信息包括以下至少之一:透明度图片在拼接画布中的第一位置信息、透明度图片的尺寸参数和透明度图片的偏移值;
    所述按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片,包括:
    基于所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值,确定所述透明度图片在拼接画布中的起始位置和所述透明度图片的缩放比例;
    获取所述颜色图片在所述拼接画布中的第二位置信息;
    按照所述第二位置信息、所述起始位置和所述缩放比例,将每一所述原始视频帧的颜色图片和透明度图片添加至所述拼接画布中,形成所述至少一张拼接图片。
  5. 根据权利要求4所述的方法,其中,所述基于所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值,确定所述透明度图片在拼接画布中的起始位置和所述透明度图片的缩放比例,包括:
    基于所述透明度图片在拼接画布中的第一位置信息和所述透明度图片的尺寸参数,确定所述透明度图片在所述拼接画布中的起始位置;
    基于所述透明度图片的尺寸参数和所述透明度图片的偏移值,确定所述透明度图片的缩放比例。
  6. 根据权利要求4所述的方法,其中,所述按照所述第二位置信息、所述起始位置和所述缩放比例,将每一所述原始视频帧的颜色图片和透明度图片添加至所述拼接画布中,形成所述至少一张拼接图片,包括:
    针对每一所述原始视频帧,将所述颜色图片添加至所述拼接画布中第二位置信息对应的位置,得到具有颜色图片的拼接画布;
    按照所述缩放比例对所述透明度图片进行缩放处理,得到缩放后的透明度图片;
    以所述起始位置为所述缩放后的透明度图片的起始添加位置,将所述缩放后的透明度图片添加至所述具有颜色图片的拼接画布中,得到一张拼接图片。
  7. 根据权利要求4所述的方法,其中,所述透明度拼接视频对应一视频压缩编码格式;所述方法还包括:
    获取所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值;
    按照预设数据存储格式,对所述透明度图片在拼接画布中的第一位置信息、所述透明度图片的尺寸参数和所述透明度图片的偏移值进行数据格式转换,得到配置信息字符串;
    将所述配置信息字符串,插入至所述视频压缩编码格式的数据头的预设位置。
  8. 根据权利要求7所述的方法,其中,所述方法还包括:
    在按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理之前,从所述数据头的预设位置获取所述配置信息字符串;
    对所述配置信息字符串进行字符串解析,得到所述预设配置信息。
  9. 根据权利要求1所述的方法,其中,所述基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频,包括:
    对所述透明度拼接视频进行视频解码,得到所述透明度拼接视频中每一拼接图片的解码格式数据;
    对每一拼接图片的解码格式数据进行数据格式转换,得到每一拼接图片的颜色格式数据;其中,所述颜色格式数据包括颜色通道数据部分以及透明度通道数据部分;
    基于所述预设配置信息和所述拼接图片的颜色通道数据部分以及透明度通道数据部分进行特效视频渲染,得到所述透明度特效视频。
  10. 根据权利要求9所述的方法,其中,所述基于所述预设配置信息和所述拼接图片的颜色通道数据部分以及透明度通道数据部分进行特效视频渲染,得到所述透明度特效视频,包括:
    基于所述预设配置信息,确定所述颜色通道数据部分中的每一颜色通道子数据与透明度通道数据部分中的每一透明度通道子数据之间的映射关系;
    将具有所述映射关系的颜色通道子数据和透明度通道子数据,确定为同一像素点的通道数据;
    将每一所述像素点的通道数据中的颜色通道子数据在所述颜色图片中的位置信息,确定为所述像素点的位置信息;
    按照每一像素点的位置信息,对每一像素点的通道数据进行特效视频渲染,得到所述透明度特效视频。
  11. 根据权利要求1至10任一项所述的方法,其中,所述方法还包括:
    渲染当前运行应用的背景视频;
    响应于特效触发操作,将所述透明度特效视频添加至所述背景视频的特定位置。
  12. 一种视频处理装置,所述装置包括:
    获取模块,配置为针对原始特效视频的每一原始视频帧,获取所述原始视频帧中的每一像素点的颜色通道信息和每一像素点的透明度通道信息;
    图片生成模块,配置为基于所述原始视频帧中每一像素点的颜色通道信息,绘制与所述原始视频帧对应的颜色图片,以及,基于所述原始视频帧中每一像素点的透明度通道信息,绘制与所述原始视频帧对应的透明度图片;
    拼接处理模块,配置为按照预设配置信息,对每一所述原始视频帧的颜色图片和透明度图片进行拼接处理,得到至少一张拼接图片;
    视频转化模块,配置为对所述至少一张拼接图片进行视频转化处理,得到透明度拼接视频;
    渲染模块,配置为基于所述预设配置信息和所述透明度拼接视频进行特效视频渲染,得到透明度特效视频。
  13. 一种电子设备,包括:
    存储器,用于存储可执行指令;处理器,用于执行所述存储器中存储的可执行指令时,实现权利要求1至11任一项所述的视频处理方法。
  14. 一种计算机可读存储介质,存储有可执行指令,用于引起处理器执行所述可执行指令时,实现权利要求1至11任一项所述的视频处理方法。
  15. 一种计算机程序产品或计算机程序,所述计算机程序产品或计算机程序包括可执行指令,所述可执行指令存储在计算机可读存储介质中;
    当电子设备的处理器从所述计算机可读存储介质读取所述可执行指令,并执行所述可执行指令时,实现权利要求1至11任一项所述的视频处理方法。
PCT/CN2023/110290 2022-09-07 2023-07-31 视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品 WO2024051394A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211090054.2 2022-09-07
CN202211090054.2A CN117218269A (zh) 2022-09-07 2022-09-07 视频处理方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024051394A1 true WO2024051394A1 (zh) 2024-03-14

Family

ID=89033998

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/110290 WO2024051394A1 (zh) 2022-09-07 2023-07-31 视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品

Country Status (2)

Country Link
CN (1) CN117218269A (zh)
WO (1) WO2024051394A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110102456A1 (en) * 2009-10-30 2011-05-05 Synopsys, Inc. Drawing an image with transparent regions on top of another image without using an alpha channel
CN107770618A (zh) * 2017-11-02 2018-03-06 腾讯科技(深圳)有限公司 一种图像处理方法、装置及存储介质
CN112070863A (zh) * 2019-06-11 2020-12-11 腾讯科技(深圳)有限公司 动画文件处理方法、装置、计算机可读存储介质和计算机设备
CN113645469A (zh) * 2020-05-11 2021-11-12 腾讯科技(深圳)有限公司 图像处理方法、装置、智能终端及计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110102456A1 (en) * 2009-10-30 2011-05-05 Synopsys, Inc. Drawing an image with transparent regions on top of another image without using an alpha channel
CN107770618A (zh) * 2017-11-02 2018-03-06 腾讯科技(深圳)有限公司 一种图像处理方法、装置及存储介质
CN112070863A (zh) * 2019-06-11 2020-12-11 腾讯科技(深圳)有限公司 动画文件处理方法、装置、计算机可读存储介质和计算机设备
CN113645469A (zh) * 2020-05-11 2021-11-12 腾讯科技(深圳)有限公司 图像处理方法、装置、智能终端及计算机可读存储介质

Also Published As

Publication number Publication date
CN117218269A (zh) 2023-12-12

Similar Documents

Publication Publication Date Title
CN111193876B (zh) 视频中添加特效的方法及装置
CN106611435B (zh) 动画处理方法和装置
CN111899155B (zh) 视频处理方法、装置、计算机设备及存储介质
CN111899322B (zh) 视频处理方法、动画渲染sdk和设备及计算机存储介质
CN108010112B (zh) 动画处理方法、装置及存储介质
US20230144094A1 (en) Multimedia data processing method, multimedia data generation method, and related device
US11507727B2 (en) Font rendering method and apparatus, and computer-readable storage medium
TW202004674A (zh) 在3d模型上展示豐富文字的方法、裝置及設備
CN107767437B (zh) 一种多层混合异步渲染方法
CN113946402A (zh) 基于渲染分离的云手机加速方法、系统、设备及存储介质
US20180143741A1 (en) Intelligent graphical feature generation for user content
CN112929627A (zh) 虚拟现实场景实现方法、装置、存储介质及电子设备
US10460490B2 (en) Method, terminal, and computer storage medium for processing pictures in batches according to preset rules
CN114222185B (zh) 视频播放方法、终端设备及存储介质
CN112954452B (zh) 视频生成方法、装置、终端及存储介质
WO2024051394A1 (zh) 视频处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品
CN111526420A (zh) 一种视频渲染方法、电子设备及存储介质
CN115391692A (zh) 视频处理方法和装置
CN115250335A (zh) 视频处理方法、装置、设备及存储介质
CN115131470A (zh) 一种图文素材合成方法、装置、电子设备和存储介质
CN117065357A (zh) 媒体数据处理方法、装置、计算机设备和存储介质
CN113938572A (zh) 图片传输方法、显示方法、装置、电子设备及存储介质
WO2024131222A1 (zh) 信息处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品
US11599338B2 (en) Model loading method and apparatus for head-mounted display device, and head-mounted display device
CN116527983A (zh) 页面显示方法、装置、设备、存储介质及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23862094

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023862094

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023862094

Country of ref document: EP

Effective date: 20240809