CN114071226B - Video preview graph generation method and device, storage medium and electronic equipment - Google Patents
Video preview graph generation method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114071226B CN114071226B CN202210041115.XA CN202210041115A CN114071226B CN 114071226 B CN114071226 B CN 114071226B CN 202210041115 A CN202210041115 A CN 202210041115A CN 114071226 B CN114071226 B CN 114071226B
- Authority
- CN
- China
- Prior art keywords
- video
- thread
- preview
- preset
- reading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
Abstract
The invention provides a method and a device for generating a video preview picture, a storage medium and electronic equipment, wherein the method comprises the following steps: under the condition that a user plays a local video by using a video player, extracting key image frames from the local video by using a packet reading thread, and storing each extracted key image frame into a first cache queue; and the decoding thread decodes the key image frames read from the first cache queue, stores decoded image data obtained by decoding into a second cache queue, and converts the decoded image data read from the second cache queue into a video preview image. The key image frames are extracted from the local video by using the packet reading thread, the key image frames are processed by using the decoding thread and the storage thread to obtain a video preview of the local video, the video content is displayed to a user through the video preview, the user knows the playing content of the video in advance, the interested playing content can be quickly and accurately found, and the user experience is better.
Description
Technical Field
The invention relates to the technical field of picture processing, in particular to a method and a device for generating a video preview picture, a storage medium and electronic equipment.
Background
With the rapid development of multimedia technology, people live more and more abundantly, and watching videos becomes a common entertainment mode in daily life of people. With the development of computer technology, people can watch videos by installing video software in a mobile terminal and watching the videos by using the video software instead of watching the videos by using a television.
The selectivity of watching videos by using video software is very wide, for example, downloaded local videos can be watched by using the video software, and videos can also be watched online. When a user wants to jump to a content which the user is interested in during the process of watching the video, the user needs to continuously drag the progress bar of the video to adjust the playing progress of the video until the content played in the video shows the favorite content. When the progress bar is dragged, the user cannot know the content of the video playing in advance, and the user can adjust the content interested by the user by adjusting the playing progress for many times, so that the experience of watching the video by the user is very poor.
Disclosure of Invention
In view of this, the present invention provides a method and an apparatus for generating a video preview image, a storage medium, and an electronic device, where the method and the apparatus can be used to generate a video preview image of a video, and when a user finds an interesting video content in a video, the content in the video can be previewed through the video preview image, so that the interesting video content can be accurately and quickly found, and the experience of the user in watching the video is improved.
In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:
the first aspect of the invention discloses a method for generating a video preview picture, which comprises the following steps:
under the condition that a user plays a local video by using a video player, extracting key image frames from the local video by using a preset packet reading thread, and storing each extracted key image frame into a preset first cache queue;
reading each key image frame in the first cache queue one by using a preset decoding thread, and taking the read key image frames as target key image frames;
the decoding thread decodes the target key image frame to obtain decoded image data, and stores the decoded image data into a preset second cache queue;
and reading each decoded image data in the second cache queue one by using a preset disk storage thread, determining the read decoded image data as target image data, and converting the target image data into a video preview.
The above method, optionally, further includes:
and the storage thread determines a storage directory of the local video and saves the video preview picture based on the storage directory.
Optionally, in the method, the extracting, from the local video, the key image frame by using a preset packet reading thread includes:
the packet reading thread determines the total video duration of the local video based on the video information of the local video;
determining a reading time step length based on the total video duration and the number of preset preview pictures;
and reading each key image frame from the local video based on the reading time step.
In the foregoing method, optionally, the decoding thread performs decoding processing on the target key image frame to obtain decoded image data, and includes:
and decompressing the target key image frame by using a preset decoding function to obtain decoded image data of the target key image frame.
The above method, optionally, further includes:
when a preview instruction sent by a user is received, determining a preview position corresponding to the preview instruction in the local video, and determining a video playing time length corresponding to the preview position;
determining a preview number corresponding to the preview instruction based on the video playing duration and the reading time step;
and determining a target preview image in each video preview image based on the preview number, and rendering the target preview image in a display control of a display page of the video player so as to show a video picture of the local video at the preview position to the user.
The second aspect of the present invention discloses a device for generating a video preview image, including:
the extraction unit is used for extracting key image frames from the local video by using a preset packet reading thread under the condition that a user plays the local video by using a video player, and storing each extracted key image frame into a preset first cache queue;
the first reading unit is used for reading each key image frame in the first cache queue one by using a preset decoding thread and taking the read key image frames as target key image frames;
the decoding unit is used for decoding the target key image frame by the decoding thread to obtain decoded image data and storing the decoded image data into a preset second cache queue;
and the second reading unit is used for reading the decoded image data in the second cache queue one by using a preset disk storage thread, determining the read decoded image data as target image data, and converting the target image data into a video preview.
The above apparatus, optionally, further comprises:
and the saving unit is used for determining a storage directory of the local video by the saving thread and saving the video preview based on the storage directory.
In the apparatus provided in the embodiment of the present invention, the extracting unit may be configured to:
the first determining subunit is used for determining the total video duration of the local video by the packet reading thread based on the video information of the local video;
the second determining subunit is used for determining a reading time step length based on the total video duration and the preset number of preview pictures;
and the reading subunit is used for reading each key image frame from the local video based on the reading time step.
The above apparatus, optionally, the decoding unit includes:
and the decompression subunit is used for decompressing the target key image frame by using a preset decoding function to obtain decoded image data of the target key image frame.
The above apparatus, optionally, further comprises:
the first determining unit is used for determining a preview position corresponding to a preview instruction in the local video and determining a video playing time length corresponding to the preview position when the preview instruction sent by a user is received;
a second determining unit, configured to determine a preview number corresponding to the preview instruction based on the video playing time length and the reading time step;
and the rendering unit is used for determining a target preview image in each video preview image based on the preview number, and rendering the target preview image in a display control of a display page of the video player so as to show a video picture of the local video at the preview position to the user.
The third aspect of the present invention discloses a storage medium, which includes stored instructions, wherein when the instructions are executed, the apparatus on which the storage medium is located is controlled to execute the method for generating the video preview image.
In a fourth aspect, the present invention discloses an electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the method for generating a video preview image as described above.
Compared with the prior art, the invention has the following advantages:
the invention provides a method and a device for generating a video preview picture, a storage medium and electronic equipment, wherein the method comprises the following steps: under the condition that a user plays a local video by using a video player, extracting key image frames from the local video by using a packet reading thread, and storing each extracted key image frame into a first cache queue; reading each key image frame in the first cache queue one by using a decoding thread, and taking the read key image frames as target key image frames; and the decoding thread decodes the target key image frame to obtain decoded image data, stores the decoded image data into a second cache queue, reads the decoded image data in the second cache queue one by using the disk storage thread, determines the read decoded image data as target image data, and converts the target image data into a video preview image. The key image frames are extracted from the local video by using the package reading thread, and the key image frames are processed by using the decoding thread and the storage thread, so that a video preview of the local video can be obtained, video content can be displayed to a user through the video preview, the user can quickly know playing content of the video through the video preview, the interested playing content can be quickly and accurately found, and good video watching experience is provided for the user.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a method for generating a video preview image according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for extracting a key image frame from a local video according to an embodiment of the present invention;
fig. 3 is a scene illustration diagram of a method for generating a video preview image according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a video preview provided by an embodiment of the present invention;
fig. 5 is a diagram illustrating an example of a video preview provided in an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a device for generating a video preview image according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Technical terms used in the present invention are explained:
frame: the encoded pictures are stored in a file according to a certain format to form a video file, and each encoded picture is called a frame and is equivalent to each frame of shot on a motion picture film.
And (3) key image frame: the frame where the key action in the movement change of the character or the object is located is equivalent to a stop motion original picture in the two-dimensional animation, and other frames can be calculated through an interpolation algorithm according to the frame and can also be called as a key frame.
Transition frame: the frame compressed by the difference value with the key frame occupies small space, and the decoding needs to depend on the key image frame.
Frame extraction: refers to the process of extracting a frame from a video and decoding it into an image.
Video frame rate: the number of frames recorded per second, usually around 30 frames of video, is more consistent with a high frame rate picture.
Thread: the multi-core CPU scheduling method is the minimum unit for operation scheduling of an operating system, modern operating systems support multi-thread programming, and time-consuming tasks are generally dispersed into different threads to be processed concurrently, so that the processing capacity of the multi-core CPU is fully utilized, and the processing of the tasks is accelerated.
FFmpeg: a set of tool set for processing audio and video coding and decoding comprises eight libraries and four executable programs, which are the general names of the tool libraries. The method has the functions of transcoding, packaging, decapsulating, playing, pushing direct-current broadcast stream, viewing video information and the like.
RGB: a common color space, the display displays images requiring data in this format.
YUV: a color space commonly used in the video field is saved by a YUV format compared with an RGB format, and YUV can be converted into RGB by a display card through a matrix formula during display.
JPG: is a very common high compression rate image format.
VTP: the manner in which key picture frames are extracted from a video provided in the present invention is referred to as VTP.
VTP application: the Mac application developed based on the invention provides VTP scheme, which is used for drawing a picture for local video and is called as VTP application program.
The video player: the application for playing the video can play an online video and also can play a local video.
A rear end: the client is directly used by the user, however, data such as images, characters and the like displayed to the user are all from a server of the video player, and one end which cannot be seen by the user who provides service for the video player is called as a back end.
When a user plays a local video downloaded from a network to a computer terminal by using a video player, the user drags the progress bar of the video to check the contents of the video at different playing time points when searching for the interesting contents in the video, the user cannot preview the story line in the video, so that the user can find the interesting video clip by dragging the progress bar of the video for many times, the whole process takes a long time, and poor watching experience is brought to the user.
In order to solve the problems, the invention provides a method and a device for generating a video preview image, a storage medium and electronic equipment.
The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like.
Referring to fig. 1, a flowchart of a method for generating a video preview image according to an embodiment of the present invention is specifically described as follows:
s101, under the condition that a user plays a local video by using a video player, a preset packet reading thread is used for extracting key image frames from the local video, and the extracted key image frames are stored in a preset first cache queue.
The video player is installed on the intelligent terminal, the local video is a video downloaded to the intelligent terminal by a user through a network, and the intelligent terminal can be Mac, IPAD and other devices.
The video players are various, different suppliers have different video players, and the video players can play local videos and also can play copyright videos in a video database of the video players on line.
The local video is composed of a plurality of video frames, and it should be noted that the video frames in the local video include a plurality of types of frames such as key video frames, reference video frames, and the like.
The read-package thread is used for extracting key image frames from the video, the number of the extracted key image frames is a preset number, the read-package thread uses a program written by 4 library files of FFmpeg instead of an FFmpeg command line execution program, and the 4 library files of FFmpeg are respectively: libavformat, libavcodec, libswscale, libutil library.
And storing the key image frame into a first buffer queue every time the packet reading thread extracts the key image frame from the local video.
Referring to fig. 2, a flowchart of a method for extracting a key image frame from a local video according to an embodiment of the present invention is specifically described as follows:
s201, determining the total video duration of the local video by the packet reading thread based on the video information of the local video.
And the packet reading thread acquires video information of the local video, wherein the video information comprises information such as data format, file name, file size, total video duration and the like of the local video.
From the video information, a total video duration of the local video may be determined, for example, 1 hour.
S202, determining a reading time step based on the total video duration and the preset number of preview pictures.
The preview picture quantity is the quantity of key image frames extracted from a local video, the preview picture data quantity can be set according to actual requirements, preferably, the preview picture quantity can also be determined based on the total video duration, different durations correspond to different preview picture quantities, illustratively, a duration interval to which the total video duration belongs is determined, and a numerical value corresponding to the duration interval is determined as the preview picture quantity; further, there are a plurality of duration intervals, each duration interval corresponds to a different value, the value is the number of key image frames that need to be extracted from the video in the duration interval, and exemplarily, the duration interval is: and if the value of the time interval is 600 within one hour to one half hour, the video belonging to the time interval needs to extract 600 key image frames.
After the total video duration and the number of preview pictures are obtained, the reading time step needs to be determined, specifically, the total video duration may be divided by the number of preview pictures, so that the reading time step may be obtained.
And S203, reading each key image frame from the local video based on the reading time step.
When each key image frame is read from the local video according to the reading time step, each key image frame is periodically read from the local video based on the reading time step.
For example, considering the reading time step as T, the first key video frame encountered after the interval T is taken as the key video frame.
Preferably, the local video can be segmented based on the reading time step to obtain a plurality of video segments, and the key image frame is extracted from each video segment; preferably, for each video segment, the last key video frame in the video segment can be used as a key video frame, and the other video frames can be discarded; preferably, for each video clip, any one key video frame in the video clip can be used as a key picture frame, and other video frames are discarded.
According to the embodiment of the invention, each key picture frame is read from the local video by using the packet reading thread, the read key picture frames are stored in the first cache queue, and useless video frames are discarded by extracting the key picture frames, so that the processing amount of the video frames can be reduced, and the speed of generating the video preview image is increased.
Preferably, the buffer space of the first buffer queue is limited, and in the process that the packet reading thread extracts the key image frames from the local video and stores the key image frames in the first buffer queue, the packet reading thread also needs to determine whether the first buffer queue is full of data, and if the first buffer queue is full of data, the packet reading thread needs to continue to extract the key image frames from the local video after a preset waiting time.
And S102, reading the key image frames in the first cache queue one by using a preset decoding thread, and taking the read key image frames as target key image frames.
The decoding thread is used for reading the key image frames from the first buffer queue and determining the read key image frames as target key image frames.
It should be noted that the first buffer queue is a first-in first-out queue, the decoding thread reads the key image frames of the first buffer queue one by one and determines the read key image frames as target key image frames, and further, a process of reading the key image frames in the first buffer queue by the decoding thread is actually a process of consuming data in the first buffer queue.
The work of the packet reading thread and the work of the decoding thread are mutually independent, and the process of extracting the key image frame from the local data by the packet reading thread and the process of reading the key image frame from the first cache queue by the decoding thread are not influenced mutually.
S103, decoding the target key image frame by the decoding thread to obtain decoded image data, and storing the decoded image data into a preset second buffer queue.
After the decoding thread obtains the target key image frame, the key image frame needs to be decoded, so that the decoded image data of the target key image frame can be obtained, and the decoded image data is stored in a second buffer queue, wherein the second buffer queue is a first-in first-out queue.
The decoding thread decompresses the target key image frame by using a decoding function, so as to obtain decoded image data, wherein the decoded image data can be YUV image data, and the decoding function can be a decoding algorithm in FFmpeg.
And S104, reading the decoded image data in the second cache queue one by using a preset disk storage thread, determining the read decoded image data as target image data, and converting the target image data into a video preview.
And the disk storing thread reads the decoded image data in the second buffer queue one by one.
The storage thread converts the target image data into a video preview meeting a preset picture format; for example, the target image data may be converted into an RGB image, and the RGB image may be converted into a picture in a JPG format, which is determined as a video preview; note that, as many decoded image data as there are video preview images.
Specifically, the picture format may be set according to actual requirements, and may be any picture format that can be read by the mobile terminal.
Further, the saving thread may store the video preview image after processing the decoded image data to obtain the video preview image, for example, the saving thread determines a storage target of the local video, and stores the video preview image based on the storage directory, preferably, when the saving thread stores the video preview image, it is necessary to assign a number to the video preview image, where the number is used for searching the video preview image subsequently, and preferably, when assigning a number to the video preview image, it may assign numbers according to the order of the video preview image, for example, the number of the first video preview image is 1, the number of the second video preview image is 2, and so on.
In the method provided by the embodiment of the invention, under the condition that a user plays a local video by using a video player, a packet reading thread is used for extracting key image frames from the local video and storing each extracted key image frame into a first cache queue; reading each key image frame in the first cache queue one by using a decoding thread, and taking the read key image frames as target key image frames; and the decoding thread decodes the target key image frame to obtain decoded image data, stores the decoded image data into a second cache queue, reads the decoded image data in the second cache queue one by using the disk storage thread, determines the read decoded image data as target image data, and converts the target image data into a video preview image. The key image frames are extracted from the local video by using the package reading thread, and the key image frames are processed by using the decoding thread and the storage thread, so that a video preview of the local video can be obtained, video content can be displayed to a user through the video preview, the user can quickly know playing content of the video through the video preview, the interested playing content can be quickly and accurately found, and good video watching experience is provided for the user.
To specifically describe the process flows of the packet reading thread, the decoding thread, and the disk saving thread provided by the embodiment of the present invention, referring to fig. 3, a scene example diagram for generating a video preview image is provided for the embodiment of the present invention, which is specifically described as follows:
as shown in the figure, the process of generating the video preview map may be named as VTP, where the read package thread, the decode thread, and the store disk thread are all programs written using 4 library files of FFmpeg, and different threads perform different operations.
The packet reading thread continuously puts the key image frames read from the local video into the first buffer queue, if the first buffer queue is full of data, after a preset waiting time (for example, 3 milliseconds), whether the first buffer queue is full of data is inquired, if so, after the waiting time, the first buffer queue is inquired again, until the first buffer queue is not full of data, the packet reading thread continues to read the key image frames in the local video, until an instruction of canceling the frame extraction is received or the end of the video is read, the packet reading thread is quitted, namely the packet reading thread is ended. Therefore, under normal conditions, the packet reading thread keeps the first buffer queue as saturated as possible.
The method provided by the invention can be executed when the user watches the video, the playing content of the video is previewed in the limited space of the player, so that excessive video preview images are extracted, the experience is not good, the adjacent images are possibly similar, and the user can flash a plurality of images by slightly moving the mouse, thereby not only wasting the disk space and the memory space, but also the user cannot see the flashed images. To avoid this problem, an upper limit number of the video preview images may be set, for example, N, that is, the user switches N preview images at most when moving the mouse to view the preview images, and further, the upper limit number of the video preview images may be set according to the duration of the videos, where the videos with different durations correspond to different upper limit numbers. In order to meet the limitation of the number of pictures and enable uniform frame extraction, the operation of discarding the video frames is required, and the time interval S of discarding the video frames can be obtained by dividing the total video duration by the extraction upper limit N. Therefore, the packet reading thread reserves a frame of key image frame every S, and the frames in the period are discarded without processing, preferably, the saved key image frame may be any one of the video frames in the video clip in the time interval S.
The other side of the first buffer queue is a decoding thread consuming the key image frames, the decoding thread continuously reads the key image frames from the first buffer queue, the key image frames are waited for 3 milliseconds before being fetched, the decoding is started if the key image frames are fetched, and the decoding thread is quitted if the first buffer queue is empty and the reading packet thread is finished. Therefore, normally, the decoding thread consumes the key image frames in the first buffer queue as much as possible.
And continuously decoding by the decoding thread, putting the decoded YUV data into a second cache queue, waiting for 3 milliseconds until the second cache queue is full, inquiring whether the cache space of the second cache queue is full again, if so, inquiring again after waiting for 3 milliseconds until the inquiry is not full, continuously storing the video preview image into the second cache queue until an instruction of canceling frame extraction is received or no key picture frame needs to be processed and the decoding thread exits when the packet reading thread is finished. Therefore, normally, the decoding thread keeps the second buffer queue as saturated as possible.
In summary, the decoding thread consumes the first buffer queue as much as possible and fills the second buffer queue, that is, the key image frames in the first buffer queue are converted into decoded image data and then placed into the second buffer queue.
And the disk storage thread continuously reads the YUV data from the second cache queue, if the YUV data cannot be read, the YUV data is read in the second cache queue after 3 milliseconds, if the YUV data is read, the YUV data is converted into a video preview image, the video preview image is written into a disk for storage based on a storage directory, further, when an instruction of canceling frame extraction is received, or the second cache queue is empty and the decoding thread is finished, the storage thread is quitted, and under the normal condition, the disk storage thread can consume the video preview image in the second cache queue as much as possible.
The invention considers the dependency relationship of the input and the output among the packet reading, the decoding and the disk storage, and abstracts the three tasks to three threads to complete in order to avoid the tight coupling among the packet reading, the decoding and the disk storage, and the data is transmitted between the threads through two cache regions, thereby fully utilizing the multi-core CPU resource and accelerating the processing speed.
Usually, the FFmpeg is the first choice for video processing, but the frame extraction efficiency is found to be low through actual use, the experience of the FFmpeg is poor when the FFmpeg is used by a client, a common video needs to be extracted for tens of seconds, the good experience cannot be given to a user at the first time, and even the user may be misled, and the illusion that the client does not support the preview function is given to the user.
In the invention, the key frames are screened out firstly when reading the packet, and then the intermediate frames are discarded according to the time interval S; in order to filter unnecessary frames in advance so as to avoid unnecessary time and memory consumption for subsequent operation, a multithreading and double-cache model is adopted to accelerate the speed of generating the video preview image.
For detailed description, the present invention further provides related experimental data, and specifically, with reference to table 1, a comparison parameter table for generating a video preview image by extracting a video frame using FFmpeg, using FFmpeg-I and using the method provided by the present invention in the prior art is specifically as follows:
TABLE 1
The FFmpeg-I refers to using the same FFmpeg program, and taking parameters of filtering the key frame when extracting the frame, so that the key frame is also used when extracting the frame by the FFmpeg.
In the prior art, a single thread is used to generate a video preview of a video, but the present invention uses a three-thread mode, and in order to illustrate the advantages of the present invention, an experimental data table of the single thread and the three threads is provided here, specifically referring to table 2, the contents of table 2 are as follows:
TABLE 2
The parameters in the table 1 and the table 2 can be seen visually that the time spent on extracting the key image frames of the video and generating the video preview image of the video is very short, the key image frames are extracted by filtering the invalid video frames, and a multithreading and double-cache model is used, so that the overall performance is improved, and the speed of generating the video preview image is increased.
After the video preview of the video is generated by using the method provided by the embodiment of the invention, the video preview can be displayed to the user so that the user can preview the content of the video, and the specific flow refers to fig. 4, which specifically describes the following:
s401, when a preview instruction sent by a user is received, determining a preview position corresponding to the preview instruction in the local video, and determining a video playing time length corresponding to the preview position.
When the user moves the mouse to the progress bar of the video, a preview instruction is generated.
Determining a preview position in a progress bar of a local video, and determining video playing time according to the preview position, wherein the video playing time is the video time from an initial playing position of the video to the preview position. Referring to fig. 5, which is a scene illustration showing a video preview provided in the embodiment of the present invention, a position where a mouse is placed on a video progress bar in the view is a preview position, and it can be known from the view that a video playing time corresponding to the preview position is 19 minutes and 40 seconds.
S402, determining a preview number corresponding to the preview instruction based on the video playing time length and the reading time step length.
Illustratively, the preview number may be obtained by dividing the video playing duration by the reading time step, rounding the obtained value, and then adding one to the rounded value.
The preview number is mainly used to specify a video preview image corresponding to the preview position.
And S403, determining a target preview image in each video preview image based on the preview number, and rendering the target preview image in a display control of a display page of the video player so as to show a video picture of the local video at a preview position to the user.
And determining the picture number of each video preview picture, and determining the video preview picture corresponding to the picture number which is the same as the preview number as the target preview picture.
The display page can be understood as a video playing page, and referring to fig. 5, a video preview image is rendered in the display control in the image, so that a user can know the video content at the preview position in advance, and it should be noted that the invention supports previewing the video content at other progress during the playing of the video.
In the method provided by the embodiment of the invention, the corresponding video preview image can be determined according to the preview instruction input by the user. And the video preview image is rendered to the user, so that the user can know the storyline in the video in advance, and interested video content can be quickly and accurately inquired.
Corresponding to the method shown in fig. 1, an embodiment of the present invention further provides a device for generating a video preview, where the device is used to support implementation of the method shown in fig. 1 in life, and the device may be disposed in an intelligent terminal, and referring to fig. 6, a schematic structural diagram of the device provided in the embodiment of the present invention is specifically described as follows:
the extraction unit 601 is configured to extract key image frames from a local video by using a preset packet reading thread under the condition that a user uses a video player to play the local video, and store each extracted key image frame into a preset first cache queue;
a first reading unit 602, configured to read each key image frame in the first cache queue one by one using a preset decoding thread, and use the read key image frame as a target key image frame;
the decoding unit 603 is configured to decode the target key image frame by the decoding thread to obtain decoded image data, and store the decoded image data in a preset second buffer queue;
a second reading unit 604, configured to read, one by one, each decoded image data in the second buffer queue using a preset disk thread, determine the read decoded image data as target image data, and convert the target image data into a video preview.
In the device provided by the embodiment of the invention, under the condition that a user plays a local video by using a video player, a packet reading thread is used for extracting key image frames from the local video and storing each extracted key image frame into a first cache queue; reading each key image frame in the first cache queue one by using a decoding thread, and taking the read key image frames as target key image frames; and the decoding thread decodes the target key image frame to obtain decoded image data, stores the decoded image data into a second cache queue, reads the decoded image data in the second cache queue one by using the disk storage thread, determines the read decoded image data as target image data, and converts the target image data into a video preview image. The key image frames are extracted from the local video by using the package reading thread, and the key image frames are processed by using the decoding thread and the storage thread, so that a video preview of the local video can be obtained, video content can be displayed to a user through the video preview, the user can quickly know playing content of the video through the video preview, the interested playing content can be quickly and accurately found, and good video watching experience is provided for the user.
In the apparatus provided in the embodiment of the present invention, the apparatus may be further configured to:
and the saving unit is used for determining a storage directory of the local video by the saving thread and saving the video preview based on the storage directory.
In the apparatus provided in the embodiment of the present invention, the extracting unit 601 may be configured to:
the first determining subunit is used for determining the total video duration of the local video by the packet reading thread based on the video information of the local video;
the second determining subunit is used for determining a reading time step length based on the total video duration and the preset number of preview pictures;
and the reading subunit is used for reading each key image frame from the local video based on the reading time step.
In the apparatus provided in the embodiment of the present invention, the decoding unit 603 may be configured to:
and the decompression subunit is used for decompressing the target key image frame by using a preset decoding function to obtain decoded image data of the target key image frame.
In the apparatus provided in the embodiment of the present invention, the apparatus may be further configured to:
the first determining unit is used for determining a preview position corresponding to a preview instruction in the local video and determining a video playing time length corresponding to the preview position when the preview instruction sent by a user is received;
a second determining unit, configured to determine a preview number corresponding to the preview instruction based on the video playing time length and the reading time step;
and the rendering unit is used for determining a target preview image in each video preview image based on the preview number, and rendering the target preview image in a display control of a display page of the video player so as to show a video picture of the local video at the preview position to the user.
The embodiment of the invention also provides a storage medium, which comprises stored instructions, wherein when the instructions are executed, the equipment where the storage medium is located is controlled to execute the generation method of the video preview image.
An electronic device is provided in an embodiment of the present invention, and its structural diagram is shown in fig. 7, which specifically includes a memory 701 and one or more instructions 702, where the one or more instructions 702 are stored in the memory 701, and are configured to be executed by one or more processors 703 to perform the following operations according to the one or more instructions 702:
under the condition that a user plays a local video by using a video player, extracting key image frames from the local video by using a preset packet reading thread, and storing each extracted key image frame into a preset first cache queue;
reading each key image frame in the first cache queue one by using a preset decoding thread, and taking the read key image frames as target key image frames;
the decoding thread decodes the target key image frame to obtain decoded image data, and stores the decoded image data into a preset second cache queue;
and reading each decoded image data in the second cache queue one by using a preset disk storage thread, determining the read decoded image data as target image data, and converting the target image data into a video preview.
The specific implementation procedures and derivatives thereof of the above embodiments are within the scope of the present invention.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for generating a video preview picture is characterized by comprising the following steps:
under the condition that a user plays a local video by using a video player, determining whether a preset first cache queue is full of data or not by using a packet reading thread preset in a VTP application program, if the data is not full of data, extracting key image frames from the local video by using the packet reading thread, and storing each extracted key image frame into the first cache queue; if the data is full, after a preset waiting time, determining whether a preset first cache queue is full of data or not by using a preset packet reading thread; quitting the packet reading thread until receiving an instruction of canceling the frame extraction or reading the end of the video;
reading each key image frame in the first cache queue one by one according to a first-in first-out rule by using a decoding thread preset in a VTP application program, and taking the read key image frames as target key image frames; if the key frame image is not read, reading the key frame image from the first cache queue again after a preset second waiting time; exiting the decode thread until the first cache queue is empty and the read packet thread has ended;
the decoding thread decodes the target key image frame to obtain decoded image data, and stores the decoded image data into a preset second cache queue; if the second cache queue is full of data, after waiting for a preset third waiting time, inquiring whether the cache space of the second cache is full again, and continuing to store the decoded image data into the preset second cache queue until the inquiry is not full;
reading each decoded image data in the second cache queue one by one according to a first-in first-out rule by using a disk storage thread preset in a VTP application program, determining the read decoded image data as target image data, and converting the target image data into a video preview; if the decoded image data are not read, extracting the decoded image data from the second cache queue again after a preset fourth waiting time; exiting the stub thread until the second cache queue is empty and the decode thread has ended;
the VTP application program is a program compiled by four library files, namely libavformat, libavcodec, libswscale and libutil based on FFmpeg; and the read packet thread, the decode thread and the save disk thread in the VTP application are three mutually independent threads.
2. The method of claim 1, further comprising:
and the storage thread determines a storage directory of the local video and saves the video preview picture based on the storage directory.
3. The method of claim 1, wherein the extracting key image frames from the local video using the packet reading thread comprises:
the packet reading thread determines the total video duration of the local video based on the video information of the local video;
determining a reading time step length based on the total video duration and the number of preset preview pictures;
and reading each key image frame from the local video based on the reading time step.
4. The method according to claim 1, wherein the decoding thread performs decoding processing on the target key image frame to obtain decoded image data, and comprises:
and decompressing the target key image frame by using a preset decoding function to obtain decoded image data of the target key image frame.
5. The method of claim 3, further comprising:
when a preview instruction sent by a user is received, determining a preview position corresponding to the preview instruction in the local video, and determining a video playing time length corresponding to the preview position;
determining a preview number corresponding to the preview instruction based on the video playing duration and the reading time step;
and determining a target preview image in each video preview image based on the preview number, and rendering the target preview image in a display control of a display page of the video player so as to show a video picture of the local video at the preview position to the user.
6. An apparatus for generating a video preview map, comprising:
the extraction unit is used for determining whether a preset first cache queue is full of data or not by using a packet reading thread preset in a VTP application program under the condition that a user plays a local video by using a video player, if the preset first cache queue is not full of data, extracting key image frames from the local video by using the packet reading thread, and storing each extracted key image frame into the first cache queue; if the data is full, after a preset waiting time, determining whether a preset first cache queue is full of data or not by using a preset packet reading thread; quitting the packet reading thread until receiving an instruction of canceling the frame extraction or reading the end of the video;
the first reading unit is used for reading the key image frames in the first cache queue one by one according to a first-in first-out rule by using a decoding thread preset in a VTP application program, and taking the read key image frames as target key image frames; if the key frame image is not read, reading the key frame image from the first cache queue again after a preset second waiting time; exiting the decode thread until the first cache queue is empty and the read packet thread has ended;
the decoding unit is used for decoding the target key image frame by the decoding thread to obtain decoded image data and storing the decoded image data into a preset second cache queue; if the second cache queue is full of data, after waiting for a preset third waiting time, inquiring whether the cache space of the second cache is full again, and continuing to store the decoded image data into the preset second cache queue until the inquiry is not full;
the second reading unit is used for reading the decoding image data in the second cache queue one by one according to a first-in first-out rule by using a disk storage thread preset in a VTP application program, determining the read decoding image data as target image data, and converting the target image data into a video preview; if the decoded image data are not read, extracting the decoded image data from the second cache queue again after a preset fourth waiting time; exiting the stub thread until the second cache queue is empty and the decode thread has ended;
the VTP application program is a program compiled by four library files, namely libavformat, libavcodec, libswscale and libutil based on FFmpeg; and the read packet thread, the decode thread and the save disk thread in the VTP application are three mutually independent threads.
7. The apparatus of claim 6, further comprising:
and the saving unit is used for determining a storage directory of the local video by the saving thread and saving the video preview based on the storage directory.
8. The apparatus of claim 6, wherein the extraction unit comprises:
the first determining subunit is used for determining the total video duration of the local video by the packet reading thread based on the video information of the local video;
the second determining subunit is used for determining a reading time step length based on the total video duration and the preset number of preview pictures;
and the reading subunit is used for reading each key image frame from the local video based on the reading time step.
9. A storage medium, characterized in that the storage medium comprises stored instructions, wherein when the instructions are executed, a device on which the storage medium is located is controlled to execute the method for generating a video preview image according to any one of claims 1 to 5.
10. An electronic device comprising a memory, and one or more instructions stored in the memory and configured to be executed by the one or more processors to perform a method of generating a video preview image according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210041115.XA CN114071226B (en) | 2022-01-14 | 2022-01-14 | Video preview graph generation method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210041115.XA CN114071226B (en) | 2022-01-14 | 2022-01-14 | Video preview graph generation method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114071226A CN114071226A (en) | 2022-02-18 |
| CN114071226B true CN114071226B (en) | 2022-04-26 |
Family
ID=80230874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210041115.XA Active CN114071226B (en) | 2022-01-14 | 2022-01-14 | Video preview graph generation method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114071226B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114786039B (en) * | 2022-04-25 | 2024-03-26 | 海信电子科技(武汉)有限公司 | Server and video preview drawing manufacturing method |
| CN115361579A (en) * | 2022-07-28 | 2022-11-18 | 珠海全志科技股份有限公司 | Video transmitting and displaying method and device, electronic equipment and storage medium |
| CN115278366B (en) * | 2022-09-28 | 2023-03-24 | 天津卓朗昆仑云软件技术有限公司 | Data processing method and device for video stream of virtual machine and electronic equipment |
| CN115314654B (en) * | 2022-10-12 | 2023-01-06 | 飞狐信息技术（天津）有限公司 | Video processing method, system, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142933A (en) * | 2013-05-07 | 2014-11-12 | 深圳市快播科技有限公司 | Audio and video file tag display method and system |
| CN106713964A (en) * | 2016-12-05 | 2017-05-24 | 乐视控股（北京）有限公司 | Method of generating video abstract viewpoint graph and apparatus thereof |
| CN107820115A (en) * | 2017-09-30 | 2018-03-20 | 中兴通讯股份有限公司 | Realize the method, apparatus and client and storage medium of video information preview |
| CN108810622A (en) * | 2018-07-09 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Extracting method, device, computer-readable medium and the electronic equipment of video frame |
| CN110392306A (en) * | 2019-07-29 | 2019-10-29 | 腾讯科技（深圳）有限公司 | A kind of data processing method and equipment |
| CN113596582A (en) * | 2021-08-04 | 2021-11-02 | 杭州海康威视系统技术有限公司 | Video preview method and device and electronic equipment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090150784A1 (en) * | 2007-12-07 | 2009-06-11 | Microsoft Corporation | User interface for previewing video items |
| CN101453649B (en) * | 2008-12-30 | 2011-01-05 | 浙江大学 | Key frame extracting method for compression domain video stream |
| CN103020076B (en) * | 2011-09-23 | 2017-02-08 | 深圳市快播科技有限公司 | Dynamic preview method and device for player video file |
| CN102547213A (en) * | 2011-12-23 | 2012-07-04 | 南京超然科技有限公司 | Video imaging preview method for video conference system |
| CN103414944B (en) * | 2013-07-16 | 2017-07-25 | 深圳Tcl新技术有限公司 | The method and apparatus of rapid preview file destination |
| US10440316B2 (en) * | 2015-12-03 | 2019-10-08 | Hisense USA Corporation | Device and method for displaying a video over a network |
| CN105872714A (en) * | 2016-05-26 | 2016-08-17 | 北京金山安全软件有限公司 | Video previewing method and device |
| CN110113621B (en) * | 2018-02-01 | 2023-03-03 | 腾讯科技（深圳）有限公司 | Media information playing method and device, storage medium and electronic device |
| CN110971926B (en) * | 2019-12-02 | 2022-01-25 | 咪咕视讯科技有限公司 | Video playing method, video processing device and storage medium |
-
2022
- 2022-01-14 CN CN202210041115.XA patent/CN114071226B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104142933A (en) * | 2013-05-07 | 2014-11-12 | 深圳市快播科技有限公司 | Audio and video file tag display method and system |
| CN106713964A (en) * | 2016-12-05 | 2017-05-24 | 乐视控股（北京）有限公司 | Method of generating video abstract viewpoint graph and apparatus thereof |
| CN107820115A (en) * | 2017-09-30 | 2018-03-20 | 中兴通讯股份有限公司 | Realize the method, apparatus and client and storage medium of video information preview |
| CN108810622A (en) * | 2018-07-09 | 2018-11-13 | 腾讯科技（深圳）有限公司 | Extracting method, device, computer-readable medium and the electronic equipment of video frame |
| CN110392306A (en) * | 2019-07-29 | 2019-10-29 | 腾讯科技（深圳）有限公司 | A kind of data processing method and equipment |
| CN113596582A (en) * | 2021-08-04 | 2021-11-02 | 杭州海康威视系统技术有限公司 | Video preview method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114071226A (en) | 2022-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114071226B (en) | Video preview graph generation method and device, storage medium and electronic equipment | |
| US20210344978A1 (en) | Methods, systems, processors and computer code for providing video clips | |
| US8676952B2 (en) | User adaptive HTTP stream manager and method for using same | |
| US10225613B2 (en) | Method and apparatus for video playing processing and television | |
| EP3357253B1 (en) | Gapless video looping | |
| US8473628B2 (en) | Dynamically altering playlists | |
| JP5425322B2 (en) | Event queuing in an interactive media environment | |
| EP2267606A2 (en) | Method and device for displaying individualized user interface | |
| CN113891113A (en) | Video clip synthesis method and electronic equipment | |
| WO2007005268A2 (en) | Synchronization aspects of interactive multimedia presentation management | |
| CN110505511B (en) | Method, device and system for playing video in webpage and computing equipment | |
| WO2016207735A1 (en) | A system and methods thereof for auto-playing video content on mobile devices | |
| US9055272B2 (en) | Moving image reproduction apparatus, information processing apparatus, and moving image reproduction method | |
| CN113535063A (en) | Live broadcast page switching method, video page switching method, electronic device and storage medium | |
| CN101489027B (en) | Moving picture viewing system, moving picture viewing device, control method thereof | |
| CN113542765B (en) | Media data jump continuous playing method and display device | |
| CN113923507B (en) | Low-delay video rendering method and device for Android terminal | |
| CN114827724A (en) | Video playing method and related equipment | |
| JP2010204892A (en) | Video analysis device, video analysis method and video analysis program | |
| CN112019858B (en) | Video playing method and device, computer equipment and storage medium | |
| WO2014024255A1 (en) | Terminal and video playback program | |
| CN110381370B (en) | Animation processing method and device, terminal and storage medium | |
| CN113709574A (en) | Video screenshot method and device, electronic equipment and computer-readable storage medium | |
| CN108111900B (en) | Playing speed control method and device | |
| CN115118921B (en) | Method and system for video screen-combining self-adaptive output in cloud conference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |