CN114786052A - Academic live video fast browsing method based on key frame extraction - Google Patents
Academic live video fast browsing method based on key frame extraction Download PDFInfo
- Publication number
- CN114786052A CN114786052A CN202210464596.5A CN202210464596A CN114786052A CN 114786052 A CN114786052 A CN 114786052A CN 202210464596 A CN202210464596 A CN 202210464596A CN 114786052 A CN114786052 A CN 114786052A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- academic
- frames
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/432—Content retrieval operation from a local storage medium, e.g. hard-disk
- H04N21/4325—Content retrieval operation from a local storage medium, e.g. hard-disk by playing back content from the storage medium
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/74—Browsing; Visualisation therefor
- G06F16/745—Browsing; Visualisation therefor the internal structure of a single video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
Abstract
The invention discloses a method for quickly browsing academic live video based on key frame extraction, which comprises the following steps: the live video is backed up in real time by landing in the academic live broadcasting process; extracting video key frames of the backed-up video; and presenting the extracted video key frames to a user as the main content of one video. The method provided by the invention can enable a user to more quickly acquire the main content of an academic live video with longer duration under the condition of time fragmentation, and the reviewing and comparing are faster than the video operation; on the other hand, the method can help the user to learn key knowledge more efficiently from massive academic live videos under the condition of limited time.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to a method for quickly browsing academic live video based on key frame extraction.
Background
Academic live broadcast is one of network live broadcast, and academic spread and communication are carried out in a live broadcast mode. Interested users can participate in live interactive communication on line and can watch live playback after live broadcasting is finished. Due to the influence of epidemic situations in recent years, live broadcasting has become an important way for academic dissemination and communication in order to avoid the gathering of people, such as academic conferences, academic lectures, teacher lectures (online lessons), and the like. As time goes on, the amount and variety of academic live videos also start to increase dramatically, but the live features of academic categories are very obvious: first, academic live broadcasts give the main screen to ppt or pdf, with the typical lecturer occupying a corner (usually top left, top right or bottom right); secondly, the video duration is long; third, the profession is very demanding for the reader to think or even deduce. The user faces the following problems when reviewing live video:
1. how to judge whether the video is the video required by the user and how to quickly define whether the video content has the knowledge required by the user.
2. The video is very long, and some video content users do not need to pay attention to the video content users, so that the knowledge that the users want to know can be quickly located.
3. In the face of massive videos, a user can know hotspots, key points and the like of the academic field in a limited time.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a method for quickly browsing academic live video based on key frame extraction.
The purpose of the invention is realized by the following technical scheme:
a fast academic live video browsing method based on key frame extraction comprises the following steps:
A. the method comprises the steps of (1) landing and backing up live video in real time in an academic live broadcasting process;
B. extracting video key frames of the backed-up video;
C. and presenting the extracted video key frames to a user as the main content of the video.
One or more embodiments of the invention may have the following advantages over the prior art:
compared with the prior art, the method has the advantages that on one hand, a user can rapidly acquire the main content of the academic live video with long time under the condition of time fragmentation, and the reviewing and comparing are faster than the video operation; on the other hand, the method can help the user to learn key knowledge more efficiently from massive academic live videos under the condition of limited time.
Drawings
Fig. 1 is a flowchart of a method for fast browsing academic live video based on key frame extraction.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings.
As shown in fig. 1, a method for fast browsing academic live video based on key frame extraction includes the following steps:
step 10, recording live video in real time: the module is mainly used for backing up live video in real time in the academic video live broadcasting process, wherein v _ backup is used for storage, and v _ hand is used for processing;
step 20, extracting video key frames: processing v _ hand to extract key frame
(1) Selecting a video frame contrast area: the influence of irrelevant factors in the image is avoided, and a proper fixed area is selected for feature extraction, for example, the user head portrait in the video area is excluded from the area;
(2) video decoding into frames: decoding the video into frames frams _ src [ [ f _0, f _1, ]...,. f _ n-1], assuming that n frames are obtained by co-decoding;
(3) and filtering the noise frame: for the decoded video frames _ src, removing the noise frames according to the characteristics of the academic video
1) Removing black/white screen frames: removing white screen/black screen in the video, filtering out the white/black ratio of the selected frame area larger than Ww, wherein Ww is 0.8 by default, and can be set according to needs or scenes, and the calculation process is as follows:
wherein: gray represents a Gray value, B, G, R represents the blue, green and red components of the pixel point;
when the area occupied by the pixel points belonging to Gray E [230,255] is more than or equal to Ww, judging that the frame is a white screen;
when the area occupied by the pixel points of Gray belonging to [0,20] is more than or equal to Ww, judging that the frame is a black screen;
2) removing the switching frame: the academic video has the great characteristics that the ppt is demonstrated, the ppt can be switched along with the rhythm of a speaker, the switching can not be too fast under the general condition, the switching frame is defined according to the characteristics that all frames in a certain time (1 second) are filtered when the amount of different frames is larger than fmax, the fmax is defaulted to be 10 frames/second, and the academic video can be set according to the requirements or scenes;
3) finally, obtaining a frame set frame _ shifted without noise [ ff _0, ff _1, a.... multidot.ff _ m-1], m < n;
step 30 extracting key frames:
(1) extracting frames at a certain interval I: default I — 5, and a frame set fframes _ I may also be obtained as needed or according to a scene setting [ ff _0, ff _5, ff _10, ff _15, ·;
(2) extracting frame characteristics, namely extracting characteristics of each frame in the fframs _ I (the default is hash, and other methods can be adopted);
(3) and (3) judging key frames:
1) comparing 64-bit characteristic values of each frame with the previous frame, comparing Ic times at most (the Ic defaults to 3, and can be set according to needs and scenes), calculating a difference value and taking a minimum value Vdmin _ i;
2) judging that the Vdmin _ i is larger than Wv and then is used as a key frame (picture), wherein the Wv is set to 10 by default or set according to requirements and scenes;
3) traversing all frames in fframs _ I, and extracting all key frames fframes _ key of a video by executing I and ii;
step 40 presents to the user: presenting the extracted video key frame fframes _ key as the main content of a video to a user;
the above embodiment is specifically implemented as follows:
scene: some known expert a has made a 120 minute live broadcast of "the integrity and writing of a treatise" and the entire video would take 90 minutes to playback if it were watched.
The task requires: the user wishes to quickly browse the main contents and knowledge points of the video in a short time.
The treatment method comprises the following steps:
1) recording live video in real time: the module is mainly used for backing up live video in real time in the academic video live broadcasting process, wherein v _ backup is used for storage, and v _ hand is used for processing;
2) extracting video key frames: processing v _ hand to extract key frame
a. Selecting a video frame comparison area to avoid interference factors;
b. video decoding into frames: the frame set frames _ src obtained by transcoding the video has 162000 frames;
c. noise frame filtering: for the decoded video frame frames _ src, denoising according to the characteristics of academic video to obtain a frame set frames _ flitted with no noise and 159690 frames in total;
d. extracting key frames: finally, 79 key frames (pictures) fframes _ key are extracted from the frames _ warped collectively as the main content of the video;
presenting to the user: the extracted video key frame fframes _ key is presented to the user as the main content of one video.
Although the embodiments of the present invention have been described above, the above descriptions are only for the convenience of understanding the present invention, and are not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. A fast academic live video browsing method based on key frame extraction is characterized by comprising the following steps:
A. the method comprises the steps of (1) landing and backing up live video in real time in an academic live broadcasting process;
B. extracting video key frames of the backed-up video;
C. and presenting the extracted video key frames to a user as the main content of one video.
2. The method for fast browsing academic live video based on key frame extraction according to claim 1, wherein A is mainly real-time backup of live video in the academic video live broadcasting process, v _ backup is used for storage, and v _ hand is used for processing.
3. The method for fast browsing of academic live video based on key frame extraction as claimed in claim 1, wherein the B-video key frame extraction comprises:
b1 selects video frame contrast regions: avoiding influence of irrelevant factors in the image, and selecting a proper fixed area for feature extraction;
b2 decodes the video into frames frams _ src [ f _0, f _1,.. the.. f _ n-1], assuming co-decoding into n frames;
b3 filters noise frames: for the decoded video frame frames _ src, removing the noise frame according to the characteristics of the academic video;
b4 extracts key frames.
4. The method for browsing academic live video quickly based on key frame extraction as claimed in claim 3, wherein said step B3 specifically includes:
1) removing black/white screen frames: removing white screen/black screen in the video, filtering out the white/black ratio of the selected frame area larger than Ww, wherein Ww is 0.8 by default, and the calculation process can be set according to the requirement or the scene as follows:
wherein: gray represents a Gray value, B, G, R represents the blue, green and red components of the pixel point;
when the area occupied by the pixel points belonging to Gray E [230,255] is more than or equal to Ww, judging that the frame is a white screen;
when the area occupied by the pixel points of Gray belonging to [0,20] is more than or equal to Ww, judging that the frame is a black screen;
2) removing the switching frame: defining a switching frame as that the 'amount of different frames appearing in a certain time' is greater than fmax according to the academic video ppt switching rhythm, filtering all frames in the certain time, wherein the fmax is default to 10 frames/second, and can also be set according to needs or scenes;
3) a noise-free frame set frame _ shifted is obtained [ ff _0, ff _1,.. once, ff _ m-1], m < ═ n.
5. The method for browsing academic live video quickly based on key frame extraction as claimed in claim 3, wherein said B4 specifically comprises:
1) extracting frames at a certain interval I: default I — 5, and a frame set fframes _ I may also be obtained as needed or according to a scene setting [ ff _0, ff _5, ff _10, ff _15, ·;
2) extracting frame features: extracting the characteristics of each frame in the fframs _ I;
3) judging the key frame:
a) comparing 64-bit characteristic values of each frame with the previous frame, comparing Ic for the most times, wherein the value of Ic is 3, calculating a difference value and taking the minimum value Vdmin _ i;
b) judging that the picture is taken as a key frame picture if Vdmin _ i is larger than Wv, wherein the Wv is defaulted to 10;
c) after all frames in fframs _ I are traversed, and a) and b) are executed, all key frames fframs _ key of one video are extracted.
6. The method for fast browsing of academic live video based on key frame extraction as claimed in claim 1, wherein the key frame fframes _ key of the extracted video in C is presented to the user as the main content of a video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210464596.5A CN114786052A (en) | 2022-04-29 | 2022-04-29 | Academic live video fast browsing method based on key frame extraction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210464596.5A CN114786052A (en) | 2022-04-29 | 2022-04-29 | Academic live video fast browsing method based on key frame extraction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114786052A true CN114786052A (en) | 2022-07-22 |
Family
ID=82434613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210464596.5A Pending CN114786052A (en) | 2022-04-29 | 2022-04-29 | Academic live video fast browsing method based on key frame extraction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114786052A (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713964A (en) * | 2016-12-05 | 2017-05-24 | 乐视控股（北京）有限公司 | Method of generating video abstract viewpoint graph and apparatus thereof |
| CN108804980A (en) * | 2017-04-28 | 2018-11-13 | 合信息技术（北京）有限公司 | Switching detection method of video scene and device |
| CN109168020A (en) * | 2018-10-22 | 2019-01-08 | 广州虎牙科技有限公司 | Method for processing video frequency, device, calculating equipment and storage medium based on live streaming |
| CN110381366A (en) * | 2019-07-09 | 2019-10-25 | 新华智云科技有限公司 | Race automates report method, system, server and storage medium |
| CN110866510A (en) * | 2019-11-21 | 2020-03-06 | 山东浪潮人工智能研究院有限公司 | Video description system and method based on key frame detection |
| CN111597911A (en) * | 2020-04-22 | 2020-08-28 | 成都运达科技股份有限公司 | Method and system for rapidly extracting key frame based on image characteristics |
| CN111918085A (en) * | 2020-08-06 | 2020-11-10 | 腾讯科技（深圳）有限公司 | Live broadcast processing method and device, electronic equipment and computer readable storage medium |
| CN112019905A (en) * | 2019-05-30 | 2020-12-01 | 上海哔哩哔哩科技有限公司 | Live broadcast playback method, computer equipment and readable storage medium |
| CN112261425A (en) * | 2020-10-20 | 2021-01-22 | 成都中科大旗软件股份有限公司 | Video live broadcast and video recording playing method and system |
| CN113569668A (en) * | 2021-07-12 | 2021-10-29 | 杭州网易云音乐科技有限公司 | Method, medium, apparatus and computing device for determining highlight segments in video |
| CN114245232A (en) * | 2021-12-14 | 2022-03-25 | 推想医疗科技股份有限公司 | Video abstract generation method and device, storage medium and electronic equipment |
-
2022
- 2022-04-29 CN CN202210464596.5A patent/CN114786052A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713964A (en) * | 2016-12-05 | 2017-05-24 | 乐视控股（北京）有限公司 | Method of generating video abstract viewpoint graph and apparatus thereof |
| CN108804980A (en) * | 2017-04-28 | 2018-11-13 | 合信息技术（北京）有限公司 | Switching detection method of video scene and device |
| CN109168020A (en) * | 2018-10-22 | 2019-01-08 | 广州虎牙科技有限公司 | Method for processing video frequency, device, calculating equipment and storage medium based on live streaming |
| CN112019905A (en) * | 2019-05-30 | 2020-12-01 | 上海哔哩哔哩科技有限公司 | Live broadcast playback method, computer equipment and readable storage medium |
| CN110381366A (en) * | 2019-07-09 | 2019-10-25 | 新华智云科技有限公司 | Race automates report method, system, server and storage medium |
| CN110866510A (en) * | 2019-11-21 | 2020-03-06 | 山东浪潮人工智能研究院有限公司 | Video description system and method based on key frame detection |
| CN111597911A (en) * | 2020-04-22 | 2020-08-28 | 成都运达科技股份有限公司 | Method and system for rapidly extracting key frame based on image characteristics |
| CN111918085A (en) * | 2020-08-06 | 2020-11-10 | 腾讯科技（深圳）有限公司 | Live broadcast processing method and device, electronic equipment and computer readable storage medium |
| CN112261425A (en) * | 2020-10-20 | 2021-01-22 | 成都中科大旗软件股份有限公司 | Video live broadcast and video recording playing method and system |
| CN113569668A (en) * | 2021-07-12 | 2021-10-29 | 杭州网易云音乐科技有限公司 | Method, medium, apparatus and computing device for determining highlight segments in video |
| CN114245232A (en) * | 2021-12-14 | 2022-03-25 | 推想医疗科技股份有限公司 | Video abstract generation method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7808555B2 (en) | Image display method and image display apparatus with zoom-in to face area of still image | |
| US9226048B2 (en) | Video delivery and control by overwriting video data | |
| WO2021244440A1 (en) | Method, apparatus, and system for adjusting image quality of television, and television set | |
| US8169497B2 (en) | Method of segmenting videos into a hierarchy of segments | |
| US6525774B1 (en) | Inverse telecine converting device and inverse telecine converting method | |
| CN104519401A (en) | Video division point acquiring method and equipment | |
| US20090110366A1 (en) | Image processing apparatus and image processing method, program, and recording medium | |
| US9426385B2 (en) | Image processing based on scene recognition | |
| CN101674489A (en) | Filter device, image correction circuit, image correction method and image display device | |
| JP2017511627A (en) | Raw scene recognition that allows scene-dependent image modification before image recording or display | |
| CN111405339A (en) | Split screen display method, electronic equipment and storage medium | |
| CN102111577B (en) | A kind of system for playing stock information subtitle in real time | |
| TWI229562B (en) | Apparatus and method for signal processing of format conversion and combination of video signals | |
| CN114596259A (en) | Method, device, equipment and storage medium for determining reference-free video quality | |
| CN114222149A (en) | Plug flow method, device, medium and computer equipment | |
| JP2003298981A (en) | Digest image generating apparatus, digest image generating method, digest image generating program, and computer-readable storage medium for storing the digest image generating program | |
| CN107948718B (en) | Program information processing method, device and system | |
| CN114786052A (en) | Academic live video fast browsing method based on key frame extraction | |
| JPH1051770A (en) | Image coding system and method, and image division system | |
| CN114666654A (en) | Comparison method for confirming video image content consistency through rgb color mode | |
| US20040213547A1 (en) | Method and system for video compression and resultant media | |
| CN107534785A (en) | Method for the grade of the definition of the image that sets multimedia programming | |
| JP2002152660A (en) | Device and method for reproducing image | |
| US11328398B2 (en) | Method and system of reducing block boundary artifacts in digital image processing | |
| Plutino et al. | Work memories in super 8: the dawn of paper recycling in Brescia |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |