CN111147908A - Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback - Google Patents
Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback Download PDFInfo
- Publication number
- CN111147908A CN111147908A CN202010255735.4A CN202010255735A CN111147908A CN 111147908 A CN111147908 A CN 111147908A CN 202010255735 A CN202010255735 A CN 202010255735A CN 111147908 A CN111147908 A CN 111147908A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- frame rate
- frame image
- html5
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4782—Web browsing, e.g. WebTV
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
- H04N21/8173—End-user applications, e.g. Web browser, game
Abstract
The invention discloses an audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback, which comprises the following steps: step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5; step 2: directly playing the audio stream; and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video. The invention solves the problem that the playback of the HTML5 video can only adopt the original frame rate to perform playback but not adopt the dynamic frame rate in the prior art, and ensures that the picture played back by the HTML5 video is more coherent, smoother and clearer.
Description
Technical Field
The invention relates to the related fields of video playing, editing and recommending of HTML5, in particular to an accurate audio and video synchronization method based on dynamic frame rate playback of HTML5 video.
Background
HTML5 is the most dramatic leap in Web standards in the last decade. HTML5 is not intended to be a mere representation of Web content, but it also brings the Web to a sophisticated application platform where video, audio, animation, and interaction with computers are standardized. With the development of HTML5, various browsers have or will support HTML 5.
However, when the HTML5 video in the prior art wants to be played back, only the original frame rate can be used for playing back, but the dynamic frame rate cannot be used for playing back, and when the playback frame rate needs to be changed, the video stream must be reproduced, which takes time and obviously feels that the picture is not smooth, and the user experience is affected.
Disclosure of Invention
The invention aims to: the method for accurately synchronizing the audio and video based on the dynamic frame rate playback of the HTML5 video solves the problem that the HTML5 video playback in the prior art can only adopt the original frame rate for playback and cannot adopt the dynamic frame rate for playback, and enables the picture played back by the HTML5 video to be more coherent, smoother and clearer.
The technical scheme adopted by the invention is as follows:
an audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback comprises the following steps:
step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
Wherein, the corresponding buffered frame image calculated in the step 3 can always be correctly acquired in view of the step 1.
Further, the specific method for converting the video stream into the video frame image and buffering in step 1 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the previous adjacent nearest key frame.
S3: and after the playback of one frame image is finished, immediately updating the cache content, wherein the cached frame image is not cached any more, and only a new frame image is cached so as to ensure that more than half of available frame image data is always in the cache queue.
Step S2 is to save system consumption; step S3 requires timely updating of the cache content in order to save system consumption and guarantee validity and availability of the cache data.
Further, the specific steps of step 3 are:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
Further, the transitionDuration parameter of the transition method in the step b is a half of the time corresponding to the dynamic playback frame rate.
Further, the field type of the binary data format in step 1 is a BLOB type.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. an accurate audio and video synchronization method based on dynamic frame rate playback of an HTML5 video solves the problem that in the prior art, the playback of the HTML5 video can only adopt an original frame rate to perform playback, but not a dynamic frame rate, and enables pictures of the playback of the HTML5 video to be more coherent, smoother and clearer.
2. The invention realizes the dynamic frame rate playback of the HTML5 video.
3. The invention solves the problem of adjusting the dynamic frame rate when editing videos manufactured based on the HTML5 technology by the dynamic frame rate playback technology of the HTML5 videos.
4. The invention solves the problem of accurate synchronization of audio stream and video stream when the dynamic frame rate of HTML5 video is played back by processing the key frame and non-key frame during frame image buffer.
5. The invention solves the problem of image jitter when the sequential pictures are displayed at fixed time based on the HTML5 technology by a transition method.
Detailed Description
All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.
Example 1
An audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback comprises the following steps:
step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
Wherein, the corresponding buffered frame image calculated in the step 3 can always be correctly acquired in view of the step 1.
Example 2
The difference between this embodiment and embodiment 1 is that the specific method for converting the video stream into the video frame image and buffering in step 1 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the previous adjacent nearest key frame.
S3: and after the playback of one frame image is finished, immediately updating the cache content, wherein the cached frame image is not cached any more, and only a new frame image is cached so as to ensure that more than half of available frame image data is always in the cache queue.
Step S2 is to save system consumption; step S3 requires timely updating of the cache content in order to save system consumption and guarantee validity and availability of the cache data.
Example 3
The difference between this embodiment and the above embodiment is that the specific steps of step 3 are:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
Example 4
The present embodiment is different from the above embodiments in that the transitionDuration parameter of the transition method in the step b is half of the time corresponding to the dynamic playback frame rate.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be made by those skilled in the art without inventive work within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope defined by the claims.
Claims (5)
1. An audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback is characterized by comprising the following steps:
step 1: converting the video stream into video frame images and buffering in a binary data format of HTML 5;
step 2: directly playing the audio stream;
and step 3: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; and (3) extracting the cached frame image data corresponding to the sequence number from the step (1) for image display, eliminating image jitter by using a transition method of CSS3 contained in HTML5, and finally presenting accurately synchronized audio and video.
2. The method according to claim 1, wherein the specific method for converting the video stream into the video frame image and buffering in step 1 is as follows:
s1: converting the video stream into corresponding images of each frame according to the frame rate of the video stream, and buffering the images in a binary data format of HTML5 in a data queue, wherein the queue length is 1 to 2 seconds of image data, namely: the original frame rate of the video stream is cache duration;
s2: when the frame image is cached, caching the frame image data according to a playback frame rate, and not caching the frame image data according to an original frame rate; the frame image data corresponding to the playback frame rate becomes a key frame, the frame image data corresponding to the non-playback frame rate becomes a non-key frame, and the non-key frame directly adopts the frame image data of the adjacent and nearest key frame in front;
s3: and updating the cache content immediately after playing back one frame image, wherein the cached frame image is not cached to save system consumption, and only the new frame image is cached to ensure that more than half of available frame image data is always in the cache queue.
3. The method for accurately synchronizing the audio and video played back based on the dynamic frame rate of the HTML5 video according to claim 1, wherein the specific steps in the step 3 are as follows:
step a: acquiring the current time of the audio stream played in the step 2, and converting the current time into a current frame image sequence number; namely: at the current moment, the original frame rate of the video stream is integrated to correspond to the cached frame image sequence number;
step b: the buffered frame image data corresponding to the sequence number is extracted from the step 1, and is displayed as an image by the createObjectURL method, and image blurring is removed by the transition method of the CSS3 included in the HTML 5.
4. The accurate audio and video synchronization method based on HTML5 video dynamic frame rate playback according to claim 3, wherein: and b, taking the transitionDuration parameter value of the transition method in the step b as half of the time corresponding to the dynamic playback frame rate.
5. The method for accurately synchronizing the audio and video based on the dynamic frame rate playback of the HTML5 video, according to claim 1, wherein: the field type of the binary data format in step 1 is a BLOB type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255735.4A CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010255735.4A CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111147908A true CN111147908A (en) | 2020-05-12 |
| CN111147908B CN111147908B (en) | 2021-04-06 |
Family
ID=70528735
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010255735.4A Active CN111147908B (en) | 2020-04-02 | 2020-04-02 | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111147908B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7188353B1 (en) * | 1999-04-06 | 2007-03-06 | Sharp Laboratories Of America, Inc. | System for presenting synchronized HTML documents in digital television receivers |
| CN105677884A (en) * | 2016-01-15 | 2016-06-15 | 曾戟 | Method for displaying browser window content through html file |
| CN106484823A (en) * | 2016-09-28 | 2017-03-08 | 山东浪潮商用系统有限公司 | A kind of amending method of CSS file, device and computer-readable medium |
| CN106506525A (en) * | 2016-11-29 | 2017-03-15 | 北京旷视科技有限公司 | For playing the method and device of video flowing on a web browser |
| US20180139476A1 (en) * | 2015-05-06 | 2018-05-17 | Sharp Kabushiki Kaisha | Dynamic event signaling |
| CN108600777A (en) * | 2018-04-09 | 2018-09-28 | 北京大米科技有限公司 | A kind of stream media playback server, client |
| CN109819310A (en) * | 2019-01-25 | 2019-05-28 | 青岛海信传媒网络技术有限公司 | A kind of stream medium data playback method and device |
| CN109862384A (en) * | 2019-03-13 | 2019-06-07 | 北京河马能量体育科技有限公司 | A kind of audio-video automatic synchronous method and synchronization system |
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
-
2020
- 2020-04-02 CN CN202010255735.4A patent/CN111147908B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7188353B1 (en) * | 1999-04-06 | 2007-03-06 | Sharp Laboratories Of America, Inc. | System for presenting synchronized HTML documents in digital television receivers |
| US20180139476A1 (en) * | 2015-05-06 | 2018-05-17 | Sharp Kabushiki Kaisha | Dynamic event signaling |
| CN105677884A (en) * | 2016-01-15 | 2016-06-15 | 曾戟 | Method for displaying browser window content through html file |
| CN106484823A (en) * | 2016-09-28 | 2017-03-08 | 山东浪潮商用系统有限公司 | A kind of amending method of CSS file, device and computer-readable medium |
| CN106506525A (en) * | 2016-11-29 | 2017-03-15 | 北京旷视科技有限公司 | For playing the method and device of video flowing on a web browser |
| CN108600777A (en) * | 2018-04-09 | 2018-09-28 | 北京大米科技有限公司 | A kind of stream media playback server, client |
| CN109819310A (en) * | 2019-01-25 | 2019-05-28 | 青岛海信传媒网络技术有限公司 | A kind of stream medium data playback method and device |
| CN109862384A (en) * | 2019-03-13 | 2019-06-07 | 北京河马能量体育科技有限公司 | A kind of audio-video automatic synchronous method and synchronization system |
| CN110944225A (en) * | 2019-11-20 | 2020-03-31 | 武汉长江通信产业集团股份有限公司 | HTML 5-based method and device for synchronizing audio and video with different frame rates |
Non-Patent Citations (3)
| Title |
|---|
| 偶尔平凡_偶尔: "CSS3 transition 过渡属性", 《HTTPS://WWW.JIANSHU.COM/P/5EA9545670CA》 * |
| 凤舞九天: "CSS3 transition的背后", 《HTTPS://WWW.CNBLOGS.COM/ICHENCHAO/ARTICLES/11169519.HTML》 * |
| 编程之家: "html5-清理CSS抖动", 《HTTPS://WWW.JB51.CC/HTML5/168396.HTML》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111147908B (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107613357B (en) | Sound and picture synchronous optimization method and device and readable storage medium | |
| CN114866817B (en) | Method, system, and storage medium for improving video transitions | |
| US8683341B2 (en) | Multimedia presentation editor for a small-display communication terminal or computing device | |
| EP3357253B1 (en) | Gapless video looping | |
| CN103747287A (en) | Video playing speed regulation method and system applied to flash | |
| CN110505511B (en) | Method, device and system for playing video in webpage and computing equipment | |
| AU2020325292B2 (en) | Frame rate conversion | |
| CN112055254B (en) | Video playing method, device, terminal and storage medium | |
| CN115398924A (en) | Method and apparatus for media streaming | |
| CN103581741B (en) | Special effect play equipment and method | |
| CN111147908B (en) | Audio and video accurate synchronization method based on HTML5 video dynamic frame rate playback | |
| US20200401496A1 (en) | Method and device of displaying video comments, computing device, and readable storage medium | |
| JP2007067895A (en) | Text editor/reproducer, content editor/reproducer, and text editing/reproducing method | |
| CN114339289B (en) | Video playing processing method | |
| CN114339291A (en) | Method for zooming and playing streaming media video | |
| CN110290143B (en) | Video online editing method and device, electronic equipment and storage medium | |
| US20240089555A1 (en) | System and method for playing a subtitles file separately from a multimedia content file | |
| US11799943B2 (en) | Method and apparatus for supporting preroll and midroll during media streaming and playback | |
| CN113556576B (en) | Video generation method and device | |
| JP6343451B2 (en) | Video playback device, video playback method, and video playback program | |
| CN115334334B (en) | Video frame inserting method and device | |
| CN114666648B (en) | Video playing method and electronic equipment | |
| Engley | Being and timeouts: live sports in the psyche | |
| CN112887774A (en) | Method and device for playing content seamlessly | |
| JP2021069100A (en) | Delivery system, program, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |