CN111641838A - Browser video playing method and device and computer storage medium - Google Patents
Browser video playing method and device and computer storage medium Download PDFInfo
- Publication number
- CN111641838A CN111641838A CN202010404258.3A CN202010404258A CN111641838A CN 111641838 A CN111641838 A CN 111641838A CN 202010404258 A CN202010404258 A CN 202010404258A CN 111641838 A CN111641838 A CN 111641838A
- Authority
- CN
- China
- Prior art keywords
- video
- browser
- thread
- video data
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012545 processing Methods 0.000 claims abstract description 54
- 238000009877 rendering Methods 0.000 claims abstract description 44
- 230000003139 buffering effect Effects 0.000 claims abstract description 17
- 239000000872 buffer Substances 0.000 claims description 33
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012546 transfer Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 description 15
- 230000008569 process Effects 0.000 description 13
- 230000000694 effects Effects 0.000 description 8
- 238000006243 chemical reaction Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 241000023320 Luma <angiosperm> Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23406—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving management of server-side video buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application provides a browser video playing method and device and a computer storage medium. The browser video playing method comprises the following steps: creating a main thread and a sub-thread; buffering the video data by the main thread; compiling the code language of the browser into a preset code language by adopting a preset compiler; and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser. Through the mode, the browser video playing method avoids influencing the fluency of the playing page by creating the main thread and the sub thread and enabling the main thread and the sub thread to respectively and independently work; and compiling the code language of the browser to expand the video processing capacity of the browser.
Description
Technical Field
The present application relates to the field of audio/video stream decoding and playing technologies, and in particular, to a method and an apparatus for playing a browser video, and a computer storage medium.
Background
With the advent of short video, live broadcast, and AI (Artificial Intelligence) technologies, more and more applications and scenes are needed to be able to process and preview video in real time, such as splicing multiple videos, adding filters, background sound, subtitles, and the like. The video processing has a great limitation from the specification of w3c (World Wide Web Consortium) to the specific implementation of the browser at the browser end, and the current technologies and defects for implementing real-time processing and previewing mainly include:
1. simulating a video processing preview effect by using a JavaScript operation Dom (Document object model) on the basis of the native video; the capability is limited and only simple graph-like layer add operations can be handled.
2. The video preview system comprises a native video plus canvas (canvas) for realizing real-time video processing and preview by processing images and controlling rendering frequency; the effect is realized by simulation, and the key frame cannot be accurately realized.
3. The server side processes the video, and returns the processed video to the browser side for rendering; to ensure real-time performance, the performance and bandwidth of the server are extremely high.
Therefore, the browser end needs to process the video and preview the video in real time, and needs to break through the native media technology of the current browser, and meanwhile, the performance problem needs to be solved and the stability needs to be ensured.
Disclosure of Invention
The application provides a browser video playing method and device and a computer storage medium.
The application provides a browser video playing method, which comprises the following steps:
creating a main thread and a sub-thread;
buffering video data by the main thread;
compiling the code language of the browser into a preset code language by adopting a preset compiler;
and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser.
Through the mode, the main thread and the sub-thread are created and respectively and independently work, so that the influence on the fluency of the playing page is avoided; and compiling the code language of the browser to expand the video processing capacity of the browser.
The step of compiling the code language of the browser into the preset code language by adopting the preset encoder comprises the following steps:
and compiling the code language of the computer program, the video decoding interface, the image processing interface and the audio processing interface of the browser into WebAssembly by adopting an Emscript compiler.
By the mode, the video processing capacity of the browser end is expanded by compiling the C code of the browser into the WebAssembly.
Before the step of buffering video data by the main thread, the browser video playing method includes:
creating a buffer and determining a size of the buffer based on a bitrate of the video.
Through the mode, the appropriate size of the buffer area is confirmed based on the code stream of the video to be played, and the problems of frequent blocking and discontinuous data caused by too large and too small buffer areas are avoided.
Wherein the decoding and rendering, by the child thread, the video data based on the preset code language comprises:
and the sub-thread transfers the video data of the main thread by reference of a transferable interface.
By the mode, the sub-thread does not need to directly buffer the video data, the reference transmission of the video data of the main thread can be realized through the transferable interface, and the performance problems of high performance consumption and low speed caused by data sharing are avoided.
Before the step of compiling the code languages of the computer program, the video decoding interface, the image processing interface and the audio processing interface of the browser into the WebAssembly by using the emscript compiler, the browser video playing method further includes:
unnecessary program code in the computer program of the browser is deleted based on the demand information input by the user.
By the method, the browser function required by the user is obtained according to the requirement information input by the user, so that the necessary program and the unnecessary program in the computer program are analyzed, and the unnecessary program code is deleted, so that the size of the compiled computer program is reduced.
Wherein the decoding and rendering, by the child thread, the video data based on the preset code language comprises:
acquiring the size of video data buffered by the main thread;
judging whether the size of the buffered video data is larger than or equal to a preset video data amount or not;
and rendering the video data by the sub thread under the condition that the size of the buffered video data is larger than or equal to a preset video data amount.
By the mode, the size of the video data buffered by the buffer area is monitored, and the decoding and rendering work is started only when the size of the video data reaches a certain degree, so that the problems of data discontinuity and playing pause are avoided.
Wherein the decoding and rendering, by the child thread, the video data based on the preset code language to play the rendered video at the browser includes:
acquiring a screen refreshing frequency;
determining a video frame rate in video rendering based on the screen refresh frequency.
By the mode, the screen refreshing frequency and the video frame rate of the browser are matched, and the problem that video blocking is caused by video frame loss can be effectively solved.
The browser video playing method further comprises the following steps:
before the sub-thread renders the audio in the video data, sending the time stamp of the audio to the main thread;
the main thread acquires timestamps of video pictures in the video data and aligns the timestamps of the video pictures based on the timestamps of the audio.
By the method, the audio and video clock synchronization method is provided, so that stable output of the video stream and the audio stream is ensured, and the final video presentation effect is ensured.
The present application further provides a video playback device comprising a memory and a processor, wherein the memory is coupled to the processor;
the memory is used for storing program data, and the processor is used for executing the program data to realize the browser video playing method.
The application also provides a computer storage medium for storing program data, and the program data is used for realizing the browser video playing method when being executed by a processor.
The present application further provides a video playback device, the video playback device includes:
the creating module is used for creating a main thread and a sub-thread;
the buffer module is used for buffering video data in the main thread;
the compiling module is used for compiling the code language of the browser into a preset code language by adopting a preset compiler;
and the playing module is used for decoding and rendering the video data on the basis of the preset code language in the sub-thread so as to play the rendered video in the browser.
The compiling module is used for compiling the code language of the computer program, the video decoding interface, the image processing interface and the audio processing interface of the browser into WebAssembly by adopting an Emscript compiler.
The buffer module is used for creating a buffer area and determining the size of the buffer area based on the code rate of the video.
The playing module is used for transmitting the video data of the main thread in the sub-thread through a transferable interface.
The compiling module is used for deleting unnecessary program codes in the computer program of the browser based on the requirement information input by the user.
The playing module is used for acquiring the size of the video data buffered by the main thread and judging whether the size of the buffered video data is larger than or equal to a preset video data amount or not; and rendering the video data at the sub thread under the condition that the size of the buffered video data is larger than or equal to a preset video data amount.
The playing module is used for acquiring the screen refreshing frequency and determining the video frame rate in video rendering based on the screen refreshing frequency.
The playing module is used for sending the time stamp of the audio to the main thread before the sub-thread renders the audio in the video data; and acquiring the time stamp of the video picture in the video data at the main thread, and aligning the time stamp of the video picture based on the time stamp of the audio.
The beneficial effect of this application is: the video playing device creates a main thread and a sub thread; buffering the video data by the main thread; compiling the code language of the browser into a preset code language by adopting a preset compiler; and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser. Through the mode, the browser video playing method improves the fluency of playing pages by creating the main thread and the sub-thread, and enabling the main thread and the sub-thread to respectively and independently work; by compiling the code language of the browser, the video data can be decoded and rendered at the browser end, the video processing capacity of the browser end is expanded, and the dependence on server resources is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
fig. 1 is a schematic flowchart of an embodiment of a browser video playing method provided in the present application;
FIG. 2 is a block diagram of an embodiment of a thread system provided herein;
fig. 3 is a schematic flowchart of another embodiment of a browser video playing method provided in the present application;
fig. 4 is a schematic structural diagram of an embodiment of a video playback device provided in the present application;
fig. 5 is a schematic structural diagram of another embodiment of a video playback device provided in the present application;
FIG. 6 is a schematic structural diagram of an embodiment of a computer storage medium provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to achieve the technical effects of processing a video at a browser end and previewing the video in real time, the present application provides a browser video playing method, and specifically refer to fig. 1, where fig. 1 is a schematic flowchart of an embodiment of a browser video playing method provided in the present application.
The browser video playing method is applied to a video playing device, wherein the video playing device can be a server, a terminal device, or a system formed by the server and the terminal device in a mutual matching mode. Accordingly, each part, such as each unit, sub-unit, module, and sub-module, included in the electronic device may be all disposed in the server, may be all disposed in the terminal device, and may be disposed in the server and the terminal device, respectively.
Further, the server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules, for example, software or software modules for providing distributed servers, or as a single software or software module, and is not limited herein. The browser video playing method can also be realized by running computer executable codes through a processor.
As shown in fig. 1, the method for playing a browser video in the embodiment of the present application specifically includes the following steps:
s101: a main thread and a sub-thread are created.
S102: the video data is buffered by the main thread.
In which the video playback apparatus first creates a main thread, i.e., main thread, and buffers video data through the main thread. Videos played by browsers include, but are not limited to: live broadcast video, movie and television video recorded by websites, short video and the like.
The video playing device creates a sub-thread (Worker) based on the main thread, and the sub-thread is used for bearing part of work tasks in the main thread. The sub threads and the main thread run independently without interference. The relationship between the main thread and the sub thread can be specifically seen in fig. 2, and fig. 2 shows a thread system applied to browser video playing.
As shown in FIG. 2, the thread system includes a main thread and a sub-thread, the sub-thread including a download sub-thread and a decode sub-thread. The downloading sub-thread is responsible for downloading video data needing to be played by the browser from a server of a video source, and the main thread is responsible for caching the video data of the downloading sub-thread in a buffer area. Then, the decoding sub-thread decodes the video data in the buffer area to obtain an audio stream and a video stream; the main thread further renders and synchronizes the audio stream and the video stream, and finally plays the video on the page of the browser.
As can be seen from the thread system in fig. 2, compared with the prior art in which one thread completes the complete video downloading, video buffering, video decoding, and video rendering processes, the sub-thread undertakes part of the processing tasks, such as video downloading and video decoding, and the like, so that the workload of the main thread can be reduced, the work efficiency of the thread system can be improved, and the load of a Central Processing Unit (CPU) can be reduced.
Steps 101 and 102 disclose a video buffering control step in the browser video playing method, please continue to refer to fig. 3 as an initial step of the browser video playing method, and steps 101 and 102 may specifically include:
s201: a main thread and a sub-thread are created.
S202: and creating a buffer area in the main thread, and determining the size of the buffer area based on the video code rate.
Before the main thread creates the buffer area, the video playing device uses http-range to request the relevant information of the video, and the relevant information can be the video playing length, the video data size and/or the video code rate. The conversion relation between the video playing length and the video data size can be obtained by calculating the video code rate, so that the video playing device can calculate the size of the buffer area only by acquiring any two information of the video playing length, the video data size and/or the video code rate, and the video buffering control is completed. Then, the video playing device requests the complete data of the video by using http-range.
The video buffering control is to download video data at a reasonable speed for decoding and rendering, so as to reduce the problems of frequent jamming and data discontinuity caused by too large and too small buffer area. The video playing device controls the buffer size according to the video playing length and the video data size, for example, the control buffer has a video preloading time length targetSecondsBufferdahead 10s, and the video data size interval is [2, 50] (unit: MB).
It should be noted that, in order to ensure that the main thread is not affected, the video buffering control may be implemented in the download sub-thread.
S203: the video data is buffered by the main thread.
S204: the sub-thread transfers the video data of the main thread by reference of a transferable interface.
The video data among the sub-threads cannot be directly shared, and the video data need to be transmitted through the main thread. The video playing device of the application buffers video data to the main thread, and then schedules the execution of the downloading sub-thread through the postMessage interface. In the prior art, the default value transmission mode between the sub-thread and the main thread is numerical value copy, and the video playing process cannot be influenced in a scene with small data volume; in a large data volume scene, the problems of high performance consumption and low speed are revealed.
The method and the device adopt transferable object to realize data transfer between the sub-thread and the main thread, and particularly, the sub-thread can transfer video data of the main thread by reference of a transferable interface. In the process, the sub-thread acquires the mapping address of the main thread for storing the video data, and then accesses the mapping address of the main thread through the transferable interface, so as to decode the video data. Compared with a numerical value copying mode, the sub-thread does not need to store video data, only the mapping address of the main thread for storing the video data is needed to be stored, and the data transmission performance can be effectively improved.
S103: and compiling the code language of the browser into a preset code language by adopting a preset compiler.
After the video buffering control is finished, the video playing device enters the steps of video decoding and processing.
The multimedia video processing tool (FFmpeg, Fast Forward Mpeg) carried in the browser is used as a set of open source computer programs which can be used for recording and converting digital audio and video and can convert the digital audio and video into streams, and a complete solution for recording, converting and streaming audio and video is provided. The multimedia video processing tool FFmpeg has very powerful functions including video capture function, video format conversion, video capture, watermarking of video, etc.
However, the multimedia video processing tool FFmpeg is bulky and not beneficial to improving the efficiency of video processing. All functions of the multimedia video processing tool FFmpeg are not generally used each time the browser plays a video, and therefore some functions are unnecessary in a certain video play. In order to reduce the compiling volume of the multimedia video processing tool, the video playing device can delete unnecessary program codes in a computer program (such as FFmpeg) of the browser according to the requirement information input by the user, the unnecessary program codes are used for realizing the unnecessary functions, such as network, ffplay, doc and the like, and the deletion of the unnecessary program codes does not influence the normal use of other functions of the multimedia video processing tool. The requirement information input by the user refers to a function that the user expects the browser to be capable of realizing in the video playing process or a function that the user expects to delete. In one possible implementation, the user may delete unnecessary modules, such as network, ffplay, doc, etc., by modifying a setting item (configure) of FFmpeg.
In a possible implementation manner, the code language of the browser is defaulted to the C language, and the preset code language of the compiling result is the WebAssembly language. The WebAssembly is a technical scheme that codes can be written by using a non-JavaScript programming language and can be run on a browser, the FFmpeg can be directly run at the browser end through the WebAssembly, and videos can be decoded and edited frame by frame at the browser end, so that the video processing capability of the browser end can be expanded, and the dependence on server resources can be reduced. Specifically, the video playback apparatus compiles a code language (such as C language) of a computer program, a video decoding interface, an image processing interface, and an audio processing interface of the browser into WebAssembly using an emscript compiler. Wherein, the Emscript is an open source LLVM compiler facing JavaScript. The browser side runs a multimedia video processing tool through WebAssembly to realize video decoding, video picture processing and audio processing, and breaks through the dilemma that the native capability of the browser cannot support video editing at present.
In specific application, the video processing device runs the compiled WebAssembly in the decoding sub-thread, and actively takes out video buffer data from the main thread through the postMessage interface for decoding and processing, such as adding a filter, adding a watermark, changing voice and the like, so as to finally obtain completely separated image data and audio data. The image data may be in a yuv format, and the audio data may be in a pcm (Pulse Code Modulation) format. In yuv, "Y" represents brightness (Luma or Luma), i.e., a gray scale value, and "U" and "V" represent chromaticity (Chroma or Chroma).
Furthermore, the video decoding and processing steps play a role in the browser video playing method, and are used as intermediate links for downloading and rendering video data, and are important for real-time processing and previewing of videos. Therefore, in order to improve the real-time effect of video processing and previewing, the video playing device can also control the buffer.
Specifically, on the one hand, in an initial state, decoding is performed at a decoding sub-thread if and only if the video playback apparatus successfully acquires metadata information, i.e., header information, of the video. On the other hand, the video playing device monitors the size of the video data buffered by the main thread in real time, and when the buffered video data is smaller than the preset video data amount, namely the video rendering data is insufficient, the video playing device enters a buffering state; and starting or resuming the video rendering by the video playing device until the buffered video data is larger than or equal to the preset video data amount.
S104: and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser.
The audio and video rendering of the embodiment of the application is mainly divided into two parts, namely, image rendering and audio rendering, and the contents of the two parts are respectively described below.
For the rendering of a single image, the video playing device adopts WebGL (Web Graphics Library) to start GPU (Graphics Processing Unit) hardware acceleration, and converts the image format from yuv format to rgba format through vertex coloring, Graphics assembly, rasterization, texture coordinate conversion and transmission to the fragment shader, and finally renders the image format on the canvas. Wherein rgba is a color space representing Red (Red) Green (Green) Blue (Blue) and Alpha. The image output uses WebGL to render yuv, so that overhead, such as the overhead of rgba conversion, can be reduced. Specifically, because the video playing device cannot directly process the image in the yuv format, the video playing device needs to convert the image format from the yuv format to the rgba format, and in the prior art, the yuv format is directly converted into the rgba format by a decoder in the decoding process, that is, the image output by the decoder is the image in the rgba format, but because the image data volume in the rgba format is larger than that in the yuv format, the transmission speed of the subsequent image data is affected, and because the decoder also takes over the task of format conversion, the decoding speed of the decoder is slowed down. As shown in fig. 2 of the embodiment of the present disclosure, the decoding sub-thread outputs an image in a yuv format during decoding, and the main thread acquires the image in the yuv format and renders the yuv through WebGL to convert the image in the yuv format into an image in an rgba format, and outputs a video time stamp (pts), which can effectively reduce the overhead caused by the rgba conversion during the decoding sub-thread and reduce the data transmission amount for transmitting the image in the rgba format.
For continuously played images, in order to ensure smooth playing of the video, it is necessary to ensure that the frame rate of the video is above 25fps, i.e. the time interval between two frames of images does not exceed 0.04 s. In the prior art, the timer (setTimeout, setInterval) cannot completely guarantee the aging and the performance overhead is large. However, in the embodiment of the present application, when performing video image rendering, a screen refresh frequency is first obtained, and a video frame rate in video rendering is determined based on the screen refresh frequency. For example, in the embodiment of the present application, the video playback apparatus uses a requestanimation frame (an application program interface dedicated to request animation) to implement continuous redrawing of images, and matches the frame rate of the redrawed continuous images with the refresh frequency of the browser screen, where the refresh frequency of the browser screen is typically 60 fps.
Compared with the timer, the most advantage of the requestAnimationFrame is that the execution time of the callback function is decided by the system. The pace of the requestAnimationFrame follows the refresh pace of the system. The method can ensure that the callback function is executed only once in each refreshing interval of the screen, thereby avoiding the frame loss phenomenon and causing the problem of jamming of the animation. Further, the requestAnimationFrame has the advantages of CPU power saving and function throttling, and is not described herein again.
In the embodiment of the application, for the video with the frame rate exceeding 60fps, the video playing device can perform equal-difference frame loss processing on the video.
In one possible implementation, as shown in fig. 2, the audio processing comprises two parts: an audio processing node AudioWorkletNode and an audio processor audioworkletpprocessor. Wherein, the audioWorkletProcessor is used for actual audio processing and rendering, and processes 128 frame data each time; the audioworkdelenodeon is connected with the main thread, transmits audio data to the audioworkdeleprocessor, and binds audio output equipment.
When the main thread acquires the decoded and processed audio data, the audio data is immediately transmitted to the audioWorkletprocessor through the audioWorkletNode. Wherein, the AudioWrokeletProcessor maintains the buffer data, and the AudioBufferLength of the buffer area should satisfy the following formula (1):
channel x output sampling length x 2+ readIndex ≦ AudioBufferLength ≦ channel x output sampling length x max Num equation (1)
Wherein channels is the number of channels of the audio data, outputSampleLength is the size of the audio frame processed by the audioWorkletProcessor each time, readIndex is the current reading index of the buffer, and maxNum can be configured according to the actual memory limitation.
Since the image rendering and the audio rendering are rendered independently, the rendering rate cannot be guaranteed to be consistent. Therefore, the video playing apparatus according to the embodiment of the present application calibrates the video frame with the time stamp of the audio rendering as a reference.
Specifically, before the AudioWorkletProcessor processes audio data each time, an audio timestamp to be processed is returned to the main thread, and the difference between the audio timestamp and the video picture timestamp is compared in the main thread. If the comparison result is that the audio time stamp is faster, performing video frame image rendering in an equal-ratio frame skipping mode; and if the comparison result is that the video picture timestamp is faster, waiting for audio rendering until the audio timestamp and the video picture timestamp are synchronized, so that the audio and the video output by the browser end are aligned on a time axis, and the effect of audio and video synchronization is achieved.
For example, as shown in fig. 2, the decode sub-thread processes audio and video frames, respectively, and sends audio and video picture timestamps to the main thread. The main thread compares the audio time stamp with the video picture time stamp, sends the comparison result to the audioWorkletprocessor through the audioWorkletNode, and adjusts the frame number of the video picture according to the comparison result by the audioWorkletprocessor so as to synchronize the video picture with the audio clock.
In the embodiment of the application, the video playing device creates a main thread and a sub-thread; buffering the video data by the main thread; compiling the code language of the browser into a preset code language by adopting a preset compiler; and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser. Through the mode, the browser video playing method provided by the embodiment of the application improves the fluency of playing pages by creating the main thread and the sub-thread, and enabling the main thread and the sub-thread to respectively and independently work; by compiling the code language of the browser, the video data can be decoded and rendered at the browser end, the video processing capacity of the browser end is expanded, and the dependence on server resources is reduced. By the browser video playing method, real-time processing and previewing of the video can be achieved at the browser end, decoding, image processing, audio processing and rendering can be operated in independent sub-threads, so that the influence on the fluency of a main page is reduced, and an effective solution can be provided for video editing and special effects in live broadcasting of the browser end and video content identification and marking. For example, by using the browser video playing method of the embodiment of the application, video editing and decoding can be performed in the browser, and a user can directly implement functions such as video splicing, background sound adding, subtitle adding, flower character adding, filter, transition, video clipping, sound changing and the like at the browser end through a plug-in provided by the browser. For another example, in a live scene of a browser, the browser end supports FFmpeg, supports video conversion between different video formats, realizes playing of most live streaming formats, and can incorporate an AI special effect into a live screen. Furthermore, the decoding and frame-by-frame editing of the video are supported by compiling the code language of the browser into WebAssembly, so that each frame of video can be marked and rendered and output while the normal video is played, and the real-time performance and the precision are improved. For example, the video playing apparatus may compile a code language of a decoder into WebAssembly, so that the decoder of the browser can support a video decapsulation function and a frame reading function, and thus the browser can implement video decoding and video frame-by-frame editing.
To implement the browser video playing method of the foregoing embodiment, the present application further provides a video playing device, and please refer to fig. 4 specifically, where fig. 4 is a schematic structural diagram of an embodiment of the video playing device provided in the present application.
The video playback apparatus 300 of the embodiment of the present application includes a processor 31, a memory 32, an input/output device 33, and a bus 34.
The processor 31, the memory 32, and the input/output device 33 are respectively connected to the bus 34, the memory 32 stores program data, and the processor 31 is configured to execute the program data to implement the browser video playing method according to the above embodiment.
In the embodiment of the present application, the processor 31 may also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip having signal processing capabilities. The processor 31 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor 31 may be any conventional processor or the like.
To implement the browser video playing method of the foregoing embodiment, the present application further provides a video playing device, and please refer to fig. 5 specifically, where fig. 5 is a schematic structural diagram of another embodiment of the video playing device provided in the present application.
The video playback apparatus 400 of the embodiment of the present disclosure includes:
a creation module 41 for creating a main thread and a sub thread; a buffer module 42 for buffering video data in the main thread; a compiling module 43, configured to compile a code language of the browser into a preset code language by using a preset compiler; and the playing module 44 is configured to decode and render the video data based on the preset code language in the sub-thread, so as to play the rendered video in the browser.
The compiling module 43 is further configured to compile a code language of a computer program, a video decoding interface, an image processing interface, and an audio processing interface of the browser into webassemb by using an emscript compiler.
The buffer module 42 is further configured to create a buffer and determine the size of the buffer based on the bitrate of the video.
The playing module 44 is further configured to transfer the video data of the main thread in the sub-thread by reference through a transferable interface.
Wherein, the compiling module 43 is used for deleting unnecessary program codes in the computer program of the browser based on the requirement information input by the user.
The playing module 44 is configured to obtain the size of the video data buffered by the main thread, and determine whether the size of the buffered video data is greater than or equal to a preset video data amount; and rendering the video data on the sub thread under the condition that the size of the buffered video data is larger than or equal to the preset video data amount.
The playing module 44 is configured to obtain a screen refresh frequency, and determine a video frame rate in video rendering based on the screen refresh frequency.
The playing module 44 is configured to send a timestamp of the audio to the main thread before the sub-thread renders the audio in the video data; and acquiring the time stamp of the video picture in the video data at the main thread, and aligning the time stamp of the video picture based on the time stamp of the audio.
The present application further provides a computer storage medium, as shown in fig. 6, the computer storage medium 500 is used to store program data 51, and when the program data 51 is executed by the processor, the browser video playing method according to the above embodiment is implemented by the computer storage medium 500.
The browser video playing method according to the foregoing embodiment of the present application may be implemented in a form of a software functional unit, and may be stored in a device, for example, a computer readable storage medium, when the browser video playing method is sold or used as an independent product. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (11)
1. A browser video playing method is characterized by comprising the following steps:
creating a main thread and a sub-thread;
buffering video data by the main thread;
compiling the code language of the browser into a preset code language by adopting a preset compiler;
and decoding and rendering the video data by the sub-thread based on the preset code language so as to play the rendered video in the browser.
2. The browser video playback method according to claim 1,
the step of compiling the code language of the browser into the preset code language by adopting a preset encoder comprises the following steps:
and compiling the code language of the computer program, the video decoding interface, the image processing interface and the audio processing interface of the browser into WebAssembly by adopting an Emscript compiler.
3. The browser video playback method according to claim 1 or 2,
before the step of buffering the video data by the main thread, the browser video playing method comprises the following steps:
creating a buffer and determining a size of the buffer based on a bitrate of the video.
4. The browser video playback method according to any one of claims 1 to 3,
the decoding and rendering, by the child thread, the video data based on the preset code language includes:
and the sub-thread transfers the video data of the main thread by reference of a transferable interface.
5. The browser video playback method according to claim 2,
before the step of compiling the code languages of the computer program, the video decoding interface, the image processing interface and the audio processing interface of the browser into the WebAssembly by using the emscript compiler, the browser video playing method further includes:
unnecessary program code in the computer program of the browser is deleted based on the demand information input by the user.
6. The browser video playback method according to any one of claims 1 to 5,
the decoding and rendering, by the child thread, the video data based on the preset code language includes:
acquiring the size of video data buffered by the main thread;
judging whether the size of the buffered video data is larger than or equal to a preset video data amount or not;
and rendering the video data by the sub thread under the condition that the size of the buffered video data is larger than or equal to a preset video data amount.
7. The browser video playback method according to any one of claims 1 to 6,
the decoding and rendering the video data by the child thread based on the preset code language to play the rendered video in the browser includes:
acquiring a screen refreshing frequency;
determining a video frame rate in video rendering based on the screen refresh frequency.
8. The browser video playback method according to claim 7,
the browser video playing method further comprises the following steps:
before the sub-thread renders the audio in the video data, sending the time stamp of the audio to the main thread;
the main thread acquires timestamps of video pictures in the video data and aligns the timestamps of the video pictures based on the timestamps of the audio.
9. A video playback device, comprising a memory and a processor, wherein the memory is coupled to the processor;
wherein the memory is used for storing program data, and the processor is used for executing the program data to realize the browser video playing method according to any one of claims 1-8.
10. A computer storage medium for storing program data which, when executed by a processor, is adapted to implement a browser video playback method as claimed in any one of claims 1 to 8.
11. A video playback apparatus, comprising:
the creating module is used for creating a main thread and a sub-thread;
the buffer module is used for buffering video data in the main thread;
the compiling module is used for compiling the code language of the browser into a preset code language by adopting a preset compiler;
and the playing module is used for decoding and rendering the video data on the basis of the preset code language in the sub-thread so as to play the rendered video in the browser.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010404258.3A CN111641838A (en) | 2020-05-13 | 2020-05-13 | Browser video playing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010404258.3A CN111641838A (en) | 2020-05-13 | 2020-05-13 | Browser video playing method and device and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111641838A true CN111641838A (en) | 2020-09-08 |
Family
ID=72332046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010404258.3A Pending CN111641838A (en) | 2020-05-13 | 2020-05-13 | Browser video playing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111641838A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112261377A (en) * | 2020-10-23 | 2021-01-22 | 青岛以萨数据技术有限公司 | Web version monitoring video playing method, electronic equipment and storage medium |
| CN112291628A (en) * | 2020-11-25 | 2021-01-29 | 杭州视洞科技有限公司 | Multithreading video decoding playing method based on web browser |
| CN112929741A (en) * | 2021-01-21 | 2021-06-08 | 杭州雾联科技有限公司 | Video frame rendering method and device, electronic equipment and storage medium |
| CN112929733A (en) * | 2021-01-18 | 2021-06-08 | 稿定（厦门）科技有限公司 | Video preview playing method and device |
| CN113225616A (en) * | 2021-04-27 | 2021-08-06 | 杭州海康威视数字技术股份有限公司 | Video playing method and device, computer equipment and readable storage medium |
| CN113301038A (en) * | 2021-05-20 | 2021-08-24 | 中国工商银行股份有限公司 | Web end video playing method and device |
| CN113342506A (en) * | 2021-07-05 | 2021-09-03 | 湖南快乐阳光互动娱乐传媒有限公司 | Data processing method and device |
| CN113434110A (en) * | 2021-06-29 | 2021-09-24 | 稿定（厦门）科技有限公司 | Real-time audio processing method, system and storage device based on browser |
| CN113613064A (en) * | 2021-07-19 | 2021-11-05 | 深圳Tcl新技术有限公司 | Video processing method, device, storage medium and terminal |
| CN114666514A (en) * | 2022-03-18 | 2022-06-24 | 稿定（厦门）科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115134658A (en) * | 2022-05-27 | 2022-09-30 | 阿里巴巴云计算(北京)有限公司 | Video processing method, device, equipment and storage medium |
| CN115134659A (en) * | 2022-06-15 | 2022-09-30 | 阿里巴巴云计算(北京)有限公司 | Video editing and configuring method and device, browser, electronic equipment and storage medium |
| CN115243089A (en) * | 2022-07-30 | 2022-10-25 | 天翼云科技有限公司 | Audio and video synchronous rendering method and device and electronic equipment |
| CN115379257A (en) * | 2021-05-20 | 2022-11-22 | 阿里巴巴新加坡控股有限公司 | Rendering method, device, system, storage medium and program product |
| WO2023083064A1 (en) * | 2021-11-15 | 2023-05-19 | 北京字跳网络技术有限公司 | Video processing method and apparatus, electronic device, and readable storage medium |
| CN116931864A (en) * | 2023-09-18 | 2023-10-24 | 广东保伦电子股份有限公司 | Screen sharing method and intelligent interaction panel |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102802075A (en) * | 2012-08-24 | 2012-11-28 | 乐视网信息技术（北京）股份有限公司 | Online play buffering system and method |
| CN103152651A (en) * | 2013-01-31 | 2013-06-12 | 广东欧珀移动通信有限公司 | Method and system of automatically adjusting play threshold of streaming media buffering area |
| CN103841456A (en) * | 2013-12-10 | 2014-06-04 | 珠海全志科技股份有限公司 | Webcast data buffering method |
| US20180213295A1 (en) * | 2017-01-20 | 2018-07-26 | Hanwha Techwin Co., Ltd. | Apparatus and method for simultaneous playback and backup of media in a web browser |
| US20180213301A1 (en) * | 2017-01-20 | 2018-07-26 | Hanwha Techwin Co., Ltd. | Media playback apparatus and method for synchronously reproducing video and audio on a web browser |
| CN109088887A (en) * | 2018-09-29 | 2018-12-25 | 北京金山云网络技术有限公司 | A kind of decoded method and device of Streaming Media |
| CN109819315A (en) * | 2019-03-21 | 2019-05-28 | 广州华多网络科技有限公司 | Playback method, device, terminal and the storage medium of HEVC video |
| CN110198479A (en) * | 2019-05-24 | 2019-09-03 | 浪潮软件集团有限公司 | A kind of browser audio/video decoding playback method based on webassembly |
| CN110620959A (en) * | 2018-06-20 | 2019-12-27 | 杭州海康威视数字技术股份有限公司 | Data processing method, device, electronic equipment, system and storage medium |
-
2020
- 2020-05-13 CN CN202010404258.3A patent/CN111641838A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102802075A (en) * | 2012-08-24 | 2012-11-28 | 乐视网信息技术（北京）股份有限公司 | Online play buffering system and method |
| CN103152651A (en) * | 2013-01-31 | 2013-06-12 | 广东欧珀移动通信有限公司 | Method and system of automatically adjusting play threshold of streaming media buffering area |
| CN103841456A (en) * | 2013-12-10 | 2014-06-04 | 珠海全志科技股份有限公司 | Webcast data buffering method |
| US20180213295A1 (en) * | 2017-01-20 | 2018-07-26 | Hanwha Techwin Co., Ltd. | Apparatus and method for simultaneous playback and backup of media in a web browser |
| US20180213301A1 (en) * | 2017-01-20 | 2018-07-26 | Hanwha Techwin Co., Ltd. | Media playback apparatus and method for synchronously reproducing video and audio on a web browser |
| CN110620959A (en) * | 2018-06-20 | 2019-12-27 | 杭州海康威视数字技术股份有限公司 | Data processing method, device, electronic equipment, system and storage medium |
| CN109088887A (en) * | 2018-09-29 | 2018-12-25 | 北京金山云网络技术有限公司 | A kind of decoded method and device of Streaming Media |
| CN109819315A (en) * | 2019-03-21 | 2019-05-28 | 广州华多网络科技有限公司 | Playback method, device, terminal and the storage medium of HEVC video |
| CN110198479A (en) * | 2019-05-24 | 2019-09-03 | 浪潮软件集团有限公司 | A kind of browser audio/video decoding playback method based on webassembly |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112261377A (en) * | 2020-10-23 | 2021-01-22 | 青岛以萨数据技术有限公司 | Web version monitoring video playing method, electronic equipment and storage medium |
| CN112291628A (en) * | 2020-11-25 | 2021-01-29 | 杭州视洞科技有限公司 | Multithreading video decoding playing method based on web browser |
| CN112929733B (en) * | 2021-01-18 | 2022-06-28 | 稿定（厦门）科技有限公司 | Video preview playing method and device |
| CN112929733A (en) * | 2021-01-18 | 2021-06-08 | 稿定（厦门）科技有限公司 | Video preview playing method and device |
| CN112929741A (en) * | 2021-01-21 | 2021-06-08 | 杭州雾联科技有限公司 | Video frame rendering method and device, electronic equipment and storage medium |
| CN112929741B (en) * | 2021-01-21 | 2023-02-03 | 杭州雾联科技有限公司 | Video frame rendering method and device, electronic equipment and storage medium |
| CN113225616A (en) * | 2021-04-27 | 2021-08-06 | 杭州海康威视数字技术股份有限公司 | Video playing method and device, computer equipment and readable storage medium |
| CN113225616B (en) * | 2021-04-27 | 2022-11-15 | 杭州海康威视数字技术股份有限公司 | Video playing method and device, computer equipment and readable storage medium |
| CN115379257B (en) * | 2021-05-20 | 2024-05-14 | 阿里巴巴创新公司 | Rendering method, device, system, storage medium and program product |
| CN113301038A (en) * | 2021-05-20 | 2021-08-24 | 中国工商银行股份有限公司 | Web end video playing method and device |
| CN115379257A (en) * | 2021-05-20 | 2022-11-22 | 阿里巴巴新加坡控股有限公司 | Rendering method, device, system, storage medium and program product |
| CN113434110A (en) * | 2021-06-29 | 2021-09-24 | 稿定（厦门）科技有限公司 | Real-time audio processing method, system and storage device based on browser |
| WO2023273360A1 (en) * | 2021-06-29 | 2023-01-05 | 稿定（厦门）科技有限公司 | Browser-based real-time audio processing method and system, and storage device |
| CN113342506A (en) * | 2021-07-05 | 2021-09-03 | 湖南快乐阳光互动娱乐传媒有限公司 | Data processing method and device |
| CN113613064A (en) * | 2021-07-19 | 2021-11-05 | 深圳Tcl新技术有限公司 | Video processing method, device, storage medium and terminal |
| CN113613064B (en) * | 2021-07-19 | 2023-06-27 | 深圳Tcl新技术有限公司 | Video processing method, device, storage medium and terminal |
| WO2023083064A1 (en) * | 2021-11-15 | 2023-05-19 | 北京字跳网络技术有限公司 | Video processing method and apparatus, electronic device, and readable storage medium |
| CN114666514B (en) * | 2022-03-18 | 2024-02-02 | 稿定（厦门）科技有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN114666514A (en) * | 2022-03-18 | 2022-06-24 | 稿定（厦门）科技有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115134658A (en) * | 2022-05-27 | 2022-09-30 | 阿里巴巴云计算(北京)有限公司 | Video processing method, device, equipment and storage medium |
| CN115134658B (en) * | 2022-05-27 | 2024-05-14 | 阿里巴巴云计算(北京)有限公司 | Video processing method, device, equipment and storage medium |
| CN115134659A (en) * | 2022-06-15 | 2022-09-30 | 阿里巴巴云计算(北京)有限公司 | Video editing and configuring method and device, browser, electronic equipment and storage medium |
| CN115243089A (en) * | 2022-07-30 | 2022-10-25 | 天翼云科技有限公司 | Audio and video synchronous rendering method and device and electronic equipment |
| CN115243089B (en) * | 2022-07-30 | 2024-01-02 | 天翼云科技有限公司 | Audio and video synchronous rendering method and device and electronic equipment |
| CN116931864A (en) * | 2023-09-18 | 2023-10-24 | 广东保伦电子股份有限公司 | Screen sharing method and intelligent interaction panel |
| CN116931864B (en) * | 2023-09-18 | 2024-02-09 | 广东保伦电子股份有限公司 | Screen sharing method and intelligent interaction panel |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111641838A (en) | Browser video playing method and device and computer storage medium | |
| US10930318B2 (en) | Gapless video looping | |
| EP1239674B1 (en) | Recording broadcast data | |
| US20070169158A1 (en) | Method and system for creating and applying dynamic media specification creator and applicator | |
| US20090103835A1 (en) | Method and system for combining edit information with media content | |
| WO2020097857A1 (en) | Media stream processing method and apparatus, storage medium, and program product | |
| CN112073810B (en) | Multi-layout cloud conference recording method and system and readable storage medium | |
| CN111899322A (en) | Video processing method, animation rendering SDK, device and computer storage medium | |
| CN111669645B (en) | Video playing method and device, electronic equipment and storage medium | |
| US20040075678A1 (en) | Multimedia contents editing apparatus and multimedia contents playback apparatus | |
| CN112291628A (en) | Multithreading video decoding playing method based on web browser | |
| US20190327425A1 (en) | Image processing device, method and program | |
| CN112653904A (en) | Rapid video clipping method based on PTS and DTS modification | |
| JPH10283715A (en) | System and method for high-speed reproduction of sub-picture unit in digital video disk | |
| KR100246762B1 (en) | Decoding method for video data | |
| WO2007084870A2 (en) | Method and system for recording edits to media content | |
| CN113490047A (en) | Android audio and video playing method | |
| US8514949B1 (en) | Synchronous, multi-stream decoder | |
| US20240086451A1 (en) | Information processing apparatus, reception apparatus, information processing method, and storage medium | |
| JP2004289629A (en) | Apparatus and method for sending image data out | |
| US20230224557A1 (en) | Auxiliary mpds for mpeg dash to support prerolls, midrolls and endrolls with stacking properties | |
| JP2015510727A (en) | Method and system for providing file data for media files | |
| US11799943B2 (en) | Method and apparatus for supporting preroll and midroll during media streaming and playback | |
| CN115243089B (en) | Audio and video synchronous rendering method and device and electronic equipment | |
| CN113556595B (en) | Miracast-based playback method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200908 |