CN111147908B - Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback - Google Patents
Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback Download PDFInfo
- Publication number
- CN111147908B CN111147908B CN202010255735.4A CN202010255735A CN111147908B CN 111147908 B CN111147908 B CN 111147908B CN 202010255735 A CN202010255735 A CN 202010255735A CN 111147908 B CN111147908 B CN 111147908B
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- frame rate
- frame image
- html5
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000007704 transition Effects 0.000 claims abstract description 11
- 230000003139 buffering effect Effects 0.000 claims abstract description 8
- 230000001360 synchronised effect Effects 0.000 claims abstract description 4
- 230000001427 coherent effect Effects 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4782—Web browsing, e.g. WebTV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
- H04N21/8173—End-user applications, e.g. Web browser, game
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Television Systems (AREA)
Abstract
The invention discloses an audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback, which comprises the following steps: step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5; step 2: directly playing the audio stream; and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video. The invention solves the problem that the playback of the HTML5 video can only adopt the original frame rate to perform playback but not adopt the dynamic frame rate in the prior art, and ensures that the picture played back by the HTML5 video is more coherent, smoother and clearer.
Description
Technical Field
The invention relates to the related fields of video playing, editing and recommending of HTML5, in particular to an accurate audio and video synchronization method based on dynamic frame rate playback of HTML5 video.
Background
HTML5 is the most dramatic leap in Web standards in the last decade. HTML5 is not intended to be a mere representation of Web content, but it also brings the Web to a sophisticated application platform where video, audio, animation, and interaction with computers are standardized. With the development of HTML5, various browsers have or will support HTML 5.
However, when the HTML5 video in the prior art wants to be played back, only the original frame rate can be used for playing back, but the dynamic frame rate cannot be used for playing back, and when the playback frame rate needs to be changed, the video stream must be reproduced, which takes time and obviously feels that the picture is not smooth, and the user experience is affected.
Disclosure of Invention
The invention aims to: the method for accurately synchronizing the audio and video based on the dynamic frame rate playback of the HTML5 video solves the problem that the HTML5 video playback in the prior art can only adopt the original frame rate for playback and cannot adopt the dynamic frame rate for playback, and enables the picture played back by the HTML5 video to be more coherent, smoother and clearer.
The technical scheme adopted by the invention is as follows:
an audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback comprises the following steps:
step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
Wherein, the corresponding buffered frame image calculated in the step 3 can always be correctly acquired in view of the step 1.
Further, the specific method for converting the video stream into the video frame image and buffering in step 1 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the previous adjacent nearest key frame.
S3: and after the playback of one frame image is finished, immediately updating the cache content, wherein the cached frame image is not cached any more, and only a new frame image is cached so as to ensure that more than half of available frame image data is always in the cache queue.
Step S2 is to save system consumption; step S3 requires timely updating of the cache content in order to save system consumption and guarantee validity and availability of the cache data.
Further, the specific steps of step 3 are:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
Further, the transitionDuration parameter of the transition method in the step b is a half of the time corresponding to the dynamic playback frame rate.
Further, the field type of the binary data format in step 1 is a BLOB type.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. an accurate audio and video synchronization method based on dynamic frame rate playback of an HTML5 video solves the problem that in the prior art, the playback of the HTML5 video can only adopt an original frame rate to perform playback, but not a dynamic frame rate, and enables pictures of the playback of the HTML5 video to be more coherent, smoother and clearer.
2. The invention realizes the dynamic frame rate playback of the HTML5 video.
3. The invention solves the problem of adjusting the dynamic frame rate when editing videos manufactured based on the HTML5 technology by the dynamic frame rate playback technology of the HTML5 videos.
4. The invention solves the problem of accurate synchronization of audio stream and video stream when the dynamic frame rate of HTML5 video is played back by processing the key frame and non-key frame during frame image buffer.
5. The invention solves the problem of image jitter when the sequential pictures are displayed at fixed time based on the HTML5 technology by a transition method.
Detailed Description
All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.
Example 1
An audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback comprises the following steps:
step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
Wherein, the corresponding buffered frame image calculated in the step 3 can always be correctly acquired in view of the step 1.
Example 2
The difference between this embodiment and embodiment 1 is that the specific method for converting the video stream into the video frame image and buffering in step 1 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the previous adjacent nearest key frame.
S3: and after the playback of one frame image is finished, immediately updating the cache content, wherein the cached frame image is not cached any more, and only a new frame image is cached so as to ensure that more than half of available frame image data is always in the cache queue.
Step S2 is to save system consumption; step S3 requires timely updating of the cache content in order to save system consumption and guarantee validity and availability of the cache data.
Example 3
The difference between this embodiment and the above embodiment is that the specific steps of step 3 are:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
Example 4
The present embodiment is different from the above embodiments in that the transitionDuration parameter of the transition method in the step b is half of the time corresponding to the dynamic playback frame rate.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be made by those skilled in the art without inventive work within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope defined by the claims.
Claims (4)
1. An audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback is characterized by comprising the following steps:
step 1: the specific method for converting the video stream into the video frame image and caching the video frame image in the binary data format of HTML5 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the adjacent and nearest key frame in front;
s3: after playing back a frame image, updating the buffer content immediately, at this time, the buffered frame image is not buffered to save the system consumption, and only the new frame image is buffered to ensure that more than half of the available frame image data is always in the buffer queue;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
2. The method for accurately synchronizing the audio and video played back based on the dynamic frame rate of the HTML5 video according to claim 1, wherein the specific steps in the step 3 are as follows:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
3. The method for accurately synchronizing the audio and video based on the dynamic frame rate playback of the HTML5 video, according to claim 2, wherein: and b, taking the transitionDuration parameter value of the transition method in the step b as half of the time corresponding to the dynamic playback frame rate.
4. The method for accurately synchronizing the audio and video based on the dynamic frame rate playback of the HTML5 video, according to claim 1, wherein: the field type of the binary data format in step 1 is a BLOB type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255735.4A CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255735.4A CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111147908A CN111147908A (en) | 2020-05-12 |
| CN111147908B true CN111147908B (en) | 2021-04-06 |
Family
ID=70528735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010255735.4A Active CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111147908B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7188353B1 (en) * | 1999-04-06 | 2007-03-06 | Sharp Laboratories Of America, Inc. | System for presenting synchronized HTML documents in digital television receivers |
| WO2016178320A1 (en) * | 2015-05-06 | 2016-11-10 | Sharp Kabushiki Kaisha | Dynamic event signaling |
| CN105677884A (en) * | 2016-01-15 | 2016-06-15 | 曾戟 | Method for displaying browser window content through html file |
| CN106484823A (en) * | 2016-09-28 | 2017-03-08 | 山东浪潮商用系统有限公司 | A kind of amending method of CSS file, device and computer-readable medium |
| CN106506525B (en) * | 2016-11-29 | 2019-10-25 | 北京旷视科技有限公司 | For playing the method and device of video flowing on a web browser |
| CN108600777B (en) * | 2018-04-09 | 2021-02-12 | 北京大米科技有限公司 | Streaming media playback server and client |
| CN109819310B (en) * | 2019-01-25 | 2021-07-27 | 青岛海信传媒网络技术有限公司 | Streaming media data playing method and device |
| CN109862384A (en) * | 2019-03-13 | 2019-06-07 | 北京河马能量体育科技有限公司 | A kind of audio-video automatic synchronous method and synchronization system |
-
2020
- 2020-04-02 CN CN202010255735.4A patent/CN111147908B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
Non-Patent Citations (1)
| Title |
|---|
| html5-清理CSS抖动;编程之家;《https://www.jb51.cc/html5/168396.html》;20190504;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111147908A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107613357B (en) | Sound and picture synchronous optimization method and device and readable storage medium | |
| CN114866817B (en) | Method, system, and storage medium for improving video transitions | |
| US10930318B2 (en) | Gapless video looping | |
| CN103747287B (en) | Be applied to the method and system of the video playout speed adjustment of flash | |
| US8683341B2 (en) | Multimedia presentation editor for a small-display communication terminal or computing device | |
| JP3852568B2 (en) | Apparatus and method for creating multimedia presentation | |
| CN110505511B (en) | Method, device and system for playing video in webpage and computing equipment | |
| AU2020325292B2 (en) | Frame rate conversion | |
| WO2022116962A1 (en) | Video playback method and apparatus, and electronic device | |
| CN112055254B (en) | Video playing method, device, terminal and storage medium | |
| CN103581741B (en) | Special effect play equipment and method | |
| CN111147908B (en) | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback | |
| JP4994623B2 (en) | Text editing / playback device, content editing / playback device, and text editing / playback method | |
| US20200401496A1 (en) | Method and device of displaying video comments, computing device, and readable storage medium | |
| CN114339289B (en) | Video playing processing method | |
| CN110290143B (en) | Video online editing method and device, electronic equipment and storage medium | |
| US20240089555A1 (en) | System and method for playing a subtitles file separately from a multimedia content file | |
| CN113556576B (en) | Video generation method and device | |
| JP6343451B2 (en) | Video playback device, video playback method, and video playback program | |
| CN115334334B (en) | Video frame inserting method and device | |
| CN102263922A (en) | Static state image extraction apparatus | |
| US20220247891A1 (en) | Processing method and processing device | |
| Engley | Being and timeouts: live sports in the psyche | |
| CN118741223A (en) | Game video processing method, computing device, and storage medium | |
| JP2024510139A (en) | Methods, apparatus and computer programs for supporting pre-roll and mid-roll during media streaming and playback |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |