CN106686445B - The method that multimedia file is jumped on demand - Google Patents
The method that multimedia file is jumped on demand Download PDFInfo
- Publication number
- CN106686445B CN106686445B CN201510745515.9A CN201510745515A CN106686445B CN 106686445 B CN106686445 B CN 106686445B CN 201510745515 A CN201510745515 A CN 201510745515A CN 106686445 B CN106686445 B CN 106686445B
- Authority
- CN
- China
- Prior art keywords
- frame
- video
- audio
- timestamp
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4343—Extraction or processing of packetized elementary streams [PES]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The present invention provides the methods that a kind of pair of multimedia file is jumped on demand, comprising: is demultiplexed by multimedia file of the ffmpeg to input, obtains the ES stream of the multimedia file, and obtain the audiovisual information of the multimedia file；Intel's decoder is initialized using the audiovisual information；According to the information for jumping accuracy requirement information and target frame, the timestamp of target frame in ES stream is determined；Stream operation is carried out to the ES stream by ffmpeg, and reads video and audio frame data since the position of a left side key frame of the target frame；It is decoded using the video and audio frame data of Intel's decoder after initialization to reading, obtains the timestamp of the video and audio frame；When judging that the timestamp of the video and audio frame is greater than or equal to the timestamp of the target frame, which is put into video and audio output queue, to be exported.The present invention can be jumped on demand based on Intel's decoder using ffmpeg.
Description
Technical field
The method jumped on demand the present invention relates to a kind of pair of multimedia file more particularly to a kind of comprehensive utilization English are special
Your Multimedia Software Development packet (intel media sdk) realized with ffmpeg software development kit video and audio synchronize jump
(seek) method.
Background technique
Have many multimedia video audio processing software development kits on the market at present, these development kits respectively have the advantage of oneself with
Disadvantage, for example, ffmpeg software development kit, it be capable of providing including video and audio acquisition, the conversion of video and audio format, video capture,
The functions such as video with watermark have the advantages that powerful, broad covered area.But ffmpeg in terms of hardware compression acceleration compared with
It is related to less, it is still very not perfect, far away from the superior performance of intel media sdk.Intel media sdk is in hardware-accelerated solution
Code, encoding context provide brilliant performance, and graphics processor (GPU) can be made full use of hardware-accelerated to carry out.By surveying
Examination, for same section of material, compared with ffmpeg sdk, using intel media sdk come transcoding, speed can improve percent
30 or so and its CPU occupy efficiency substantially reduce.Under identical running environment, using ffmpeg sdk and intel is used
Media sdk is as follows come the performance test results being decoded to same story:
Obvious intel media sdk has a clear superiority at hardware-accelerated aspect.However, intel media sdk is but
Seek operation cannot be carried out based on multimedia file stream, the hardware-accelerated sequence decoding operate based on ES stream can only be provided.
In broadcasting and TV field, multimedia file is being decoded in playing process or nonlinear editing system, Ren Menjing
It will often play or preview location from a time point of file drags to another time point.To check the content of oneself care.
This process is known as seek.This operation is relatively conventional, if cannot seek on-demand to file stream, even if having superior hard
Part acceleration function can be also very limited in the application of product.
Summary of the invention
The present invention provides the method that a kind of pair of multimedia file is jumped on demand, with solve one in the prior art or
Multinomial missing.
The embodiment of the present invention provides the method that a kind of pair of multimedia file is jumped on demand, comprising: receives user's input
Multimedia file, the multimedia file is demultiplexed by ffmpeg, obtain the multimedia file ES stream, and
Obtain the audiovisual information of the multimedia file；Intel's decoder is initialized using the audiovisual information；Root
The information for jumping accuracy requirement information and a target frame according to one determines the timestamp of target frame in the ES stream；By described
Ffmpeg carries out stream operation to the ES stream, and video and audio frame is read since the position of a left side key frame of the target frame
Data；It is decoded, is obtained described using the video and audio frame data of the Intel's decoder after initialization to reading
The timestamp of video and audio frame；When judging that the timestamp of the video and audio frame is greater than or equal to the timestamp of the target frame, by institute
It states video and audio frame data and is put into video and audio output queue, to be exported.
In one embodiment, the information for jumping accuracy requirement information and a target frame according to one is determined in the ES stream
The timestamp of target frame, comprising: the threshold jumped according to the frame jumped in the stream of ES described in accuracy requirement information setting
Value；Judge whether the frame number difference of current decoded output frame in the target frame and ES stream is greater than or equal to the threshold value；
If so, the timestamp of the target frame is calculated according to the position of the current decoded output frame.
In one embodiment, further includes: according to the timestamp of the first frame video frame in ES stream to the target frame
In the timestamp of video frame be corrected, according to the timestamp of the first frame audio frame in ES stream in the target frame
The timestamp of audio frame be corrected.
In one embodiment, further includes: when judging that the timestamp of the video and audio frame is less than the timestamp of the target frame,
The video and audio frame data that timestamp is less than the timestamp of the target frame are abandoned, and continue to read the video and audio frame number of next frame
According to until the timestamp of the video and audio frame of reading is greater than or equal to the timestamp of the target frame.
In one embodiment, when the video and audio frame data are video data, Intel's decoder is video solution
Code device；When the video and audio frame data are audio data, Intel's decoder is audio decoder.
In one embodiment, when the video and audio frame data are video data, the timestamp of the target frame are as follows: vt=
(inverse/video flowing time base of (target video frame frame number * (1.0/ video frame rate) * fiducial time) * fiducial time).
In one embodiment, when the video and audio frame data are audio data, the timestamp of the target frame are as follows: at=
((target audio frame frame number * (hits * fiducial time/target audio sample rate of mono- frame audio of 1.0*) * fiducial time fall
The time base of number/audio stream), wherein the hits of a frame audio=target audio sample rate/video frame rate, target audio frame
Frame number=target audio sample index value/frame audio hits, target audio sample index value can be operated by seek and be sent out
It is obtained in the incoming interface parameters of interface caller when raw.
In one embodiment, the video and audio frame data of reading are put into video and audio output queue, to be exported, packet
Include: it is defeated that video data timestamp being greater than or equal in the video and audio frame data of the reading of the target frame is put into video
Dequeue, to carry out video output；Timestamp is greater than or equal in the video and audio frame data of the reading of the target frame
Audio data be put into audio output queue, to carry out audio output；Judge the video output queue and the audio output
When queue contains the video and audio frame data exported, the ffmpeg stops reading video and audio frame data.
In one embodiment, the audiovisual information of the multimedia file includes: multimedia file total length, multimedia text
Part encapsulation format, Video coding mode, video frame rate, video flowing length, vedio color format, video code rate, audio coding lattice
One or more of formula, audio sample rate, audio sample format and audio stream length.
In one embodiment, the left side key frame is the pass on the left of the target frame and closest to the target frame
Key frame.
The embodiment of the present invention can both make full use of Intel's decoder (intel media sdk) to carry out hardware
Accelerate decoding, and user can be allowed to be jumped (seek) on demand to the multimedia file played by decoding, it can be well
Realize that the hardware-accelerated decoding of the multimedia file based on Intel decoder intel media sdk is played and jumped on demand
seek.The present invention can comprehensively utilize intel media sdk with ffmpeg sdk come the method for realizing the synchronous seek of video and audio.
Simultaneously as the matching to target video frame, target audio frame is used closest to principle (leftmost side key frame), thus for having
Some there are the Multi-media Materials of timestamp drift phenomenon, equally can be carried out accurate seek operation.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.In the accompanying drawings:
Fig. 1 is the flow diagram for the method for the embodiment of the present invention jumped on demand to multimedia file；
Fig. 2 is the flow diagram of the acquisition methods of the target frame timestamp in one embodiment of the invention；
Fig. 3 is the flow diagram of the acquisition methods of the target frame timestamp of another embodiment of the present invention；
Fig. 4 is that the process of the method that video and audio frame data are put into video and audio output queue in one embodiment of the invention is shown
It is intended to；
Fig. 5 is the flow diagram for the method that one embodiment of the invention jumps multimedia file on demand.
Specific embodiment
Understand in order to make the object, technical scheme and advantages of the embodiment of the invention clearer, with reference to the accompanying drawing to this hair
Bright embodiment is described in further details.Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but simultaneously
It is not as a limitation of the invention.
The embodiment of the present invention provides the method that a kind of pair of multimedia file is jumped on demand, and this method both can use English
Te Er decoder (intel media sdk) carries out hardware-accelerated decoding, and can carry out to the multimedia file played by decoding
(seek) is jumped on demand.
Fig. 1 is the flow diagram for the method for the embodiment of the present invention jumped on demand to multimedia file.Such as Fig. 1 institute
Show, the method that multimedia file is jumped on demand, it may include step:
S110: the multimedia file of user's input is received, the multimedia file is demultiplexed by ffmpeg, is obtained
ES to the multimedia file flows, and obtains the audiovisual information of the multimedia file；
S120: Intel's decoder is initialized using the audiovisual information；
S130: the information for jumping accuracy requirement information and a target frame according to one determines target frame in the ES stream
Timestamp；
S140: carrying out stream operation to the ES stream by the ffmpeg, and from a left side key frame of the target frame
Position start read video and audio frame data；
S150: being decoded using the video and audio frame data of the Intel's decoder after initialization to reading,
Obtain the timestamp of the video and audio frame；
S160: when judging that the timestamp of the video and audio frame is greater than or equal to the timestamp of the target frame, by the view
Audio frame number is according to video and audio output queue is put into, to be exported.
In above-mentioned steps S120, the audiovisual information of the multimedia file may include much information.For example, above-mentioned more matchmakers
The audiovisual information of body file may include multimedia file total length, multimedia file encapsulation format, Video coding mode, video
Frame per second, video flowing length, vedio color format, video code rate, audio coding formats, audio sample rate, audio sample format and
Audio stream length being capable of one or more of information.
S120 through the above steps of the embodiment of the present invention decodes Intel using the audiovisual information that step S110 is obtained
Device is initialized, and hardware-accelerated decoding context can be got out.
In the embodiment of the present invention, such as in above-mentioned S120 and S150, for different multi-medium datas, it can be used different
Decoder.For example, above-mentioned Intel's decoder is Video Decoder when the video and audio frame data of reading are video data；When
When the video and audio frame data of reading are audio data, above-mentioned Intel's decoder is audio decoder.Similarly, believed using video
Breath initializes Video Decoder, is initialized using audio-frequency information to audio decoder.
It may include many groups of frames being made of multiple frames in above-mentioned steps S140, on the left of the target frame, it is each
The first frame of the leftmost side of framing can be key frame, then view sound can be read since each different key frame on the left of target frame
Frequency evidence.Preferably, the left side key frame in step S140 is the pass on the left of the target frame and closest to the target frame
Key frame can reduce the stream operation task amount of ffmpeg, to improve seek efficiency with this.Moreover, for having, some there are the times
The Multi-media Material for stabbing drift phenomenon equally can be carried out accurate seek operation.
The embodiment of the present invention accelerates defect and Intel's decoder for the hardware encoding of ffmpeg in the prior art
(intel media sdk) cannot be jumped the defect of (seek) based on stream, by effectively being carried out together using ffmpeg
Stream operates and reads video and audio frame data using Intel's decoder (intel media sdk), can both can use Ying Te
Your decoder (intel media sdk) carries out hardware-accelerated decoding, but the multimedia file played by decoding can be carried out by
(seek) need to be jumped.
Fig. 2 is the flow diagram of the acquisition methods of the target frame timestamp in one embodiment of the invention.As shown in Fig. 2,
In above-mentioned steps S130, the information for jumping accuracy requirement information and a target frame according to one determines target in the ES stream
The method of the timestamp of frame, it may include step:
S131: the threshold value jumped according to the frame jumped in the stream of ES described in accuracy requirement information setting；
S132: judge whether the target frame and the frame number difference of current decoded output frame in ES stream are greater than or equal to
The threshold value；
S133: if so, the timestamp of the target frame is calculated according to the position of the current decoded output frame.
In above-mentioned steps S131, between the threshold value that frame jumps, e.g. minimum interval or minimum frame number difference
Every.In above-mentioned steps S132, it is above-mentioned jump accuracy requirement information can be jump before (seek) video and audio frame frame number with
The difference DELTA t of the frame number of the required target frame for jumping (seek), such as can be indicated with time difference or frame number difference.If frame is jumped
The threshold value turned is Δ m, the timestamp of target frame can be just calculated, to carry out seek operation as Δ t >=Δ m.
In the embodiment of the present invention, by the threshold value jumped according to accuracy requirement information setting is jumped, and according to threshold
Value judges whether to jump, and can satisfy and various jumps accuracy requirement.By can according to the position of current decoded output frame
Effectively to calculate the timestamp of target frame.
Fig. 3 is the flow diagram of the acquisition methods of the target frame timestamp of another embodiment of the present invention.As shown in figure 3,
The acquisition methods of target frame timestamp as shown in Figure 2, may further comprise the step of:
S134: according to the timestamp of the first frame video frame in ES stream to the time of the video frame in the target frame
Stamp is corrected, according to the timestamp of the first frame audio frame in ES stream to the timestamp of the audio frame in the target frame
It is corrected.
In the preferred embodiment, it is corrected by the timestamp to target frame, available more accurate target frame
Timestamp.In above-mentioned steps S140~S160, video and audio frame data can be read according to the target frame timestamp after correction.
Again as shown in Figure 1, the method for the embodiment of the present invention jumped on demand to multimedia file, may also include step
It is rapid:
S170: it when judging that the timestamp of the video and audio frame is less than the timestamp of the target frame, abandons timestamp and is less than
The video and audio frame data of the timestamp of the target frame, and continue to read the video and audio frame data of next frame, until the view of reading
The timestamp of audio frame is greater than or equal to the timestamp of the target frame.
In the embodiment of the present invention, the timestamp of target frame is less than according to the timestamp of video and audio frame, it can be determined that read
Video and audio frame also in the left side of target frame, redirect broadcasting position has not yet been reached.Abandon the timestamp that timestamp is less than target frame
Video and audio frame data, facilitate save decoder spatial cache.
Fig. 4 is that the process of the method that video and audio frame data are put into video and audio output queue in one embodiment of the invention is shown
It is intended to.As shown in figure 4, the video and audio frame data of reading are put into video and audio output queue in above-mentioned steps S160, with
It is exported, it may include step:
S161: video data timestamp being greater than or equal in the video and audio frame data of the reading of the target frame
It is put into video output queue, to carry out video output；
S162: audio data timestamp being greater than or equal in the video and audio frame data of the reading of the target frame
It is put into audio output queue, to carry out audio output；
S163: judge that the video output queue and the audio output queue contain the video and audio frame number exported
According to when, the ffmpeg stops reading video and audio frame data.
In the embodiment of the present invention, in above-mentioned steps S161 and S162, the video/audio for being put into queue is defeated for waiting
Out.In above-mentioned steps S163, when according to having output depending on, audio, stopping reading video and audio frame data, that is, terminating seek, no
Can only jump at target frame, and can be used family by sound or picture simultaneously any one can determine and has jumped
It completes, and then the preferable sense of hearing of user and visual experience can be brought.
It is worth noting that the description as described in " video and audio " in various embodiments of the present invention, such as video and audio frame, video and audio frame
Data, audiovisual information, can be at least one of audio and video.In each step, operation or processing, audio number
According to can be optionally corresponding with audio operation, video data can be optionally corresponding with vision operation, such as audio data passes through
Audio decoder is decoded, and video data is decoded by video, and the timestamp of target frame sound intermediate frequency frame can pass through first
The timestamp of audio frame in frame is corrected, and the timestamp of the video frame in target frame can pass through the video frame in first frame
Timestamp is corrected.
Fig. 5 is the flow diagram for the method that one embodiment of the invention jumps multimedia file on demand.Such as Fig. 5 institute
Show, firstly, carrying out de-multiplex operation using multimedia file of the ffmpeg to input, obtains ES stream, it can by de-multiplex operation
To get file total length, Document encapsulation format, in a manner of Video coding, video frame rate, video flowing length, vedio color lattice
Formula, video code rate, the code-related informations such as audio coding formats, audio sample rate, audio sample format, audio stream length (step
Rapid S201).
Then, intel decoder is initialized using these code-related informations got, gets out hardware and adds
Fast decoding context relevant information (step S202).Next, according to the demand to seek operation accuracy, setting judges seek
The threshold value that operation occurs is Δ m, for example, setting Δ m=5 frame (step S203).Calculate current goal frame and previous frame frame number
Difference DELTA t enters seek operation logic (step S204) if Δ t >=Δ m.Target video and audio is calculated according to target frame frame number
The timestamp of frame, and (step S205~S206) is corrected to the value using the timestamp of first frame video and audio.
Specifically, the timestamp vt that initial target video frame can be calculated according to target frame frame number recycles first frame video
Timestamp vs is corrected the timestamp vt of the initial target video frame, obtains the timestamp VT of target video frame.For example, mesh
The calculation formula for marking the timestamp VT of video frame can are as follows:
VT=vt+vs, (1)
The calculation formula that the timestamp vt of initial target video frame is calculated according to target frame frame number can be a variety of expression-forms
Or calculation, such as the calculation formula can are as follows:
Vt=(inverse/video flowing of (target video frame frame number * (1.0/ video frame rate) * fiducial time) * fiducial time
Time base), (2)
Wherein, target video frame frame number can obtain in the information according to a target frame, video frame rate, fiducial time AV_
The time base of TIME_BASE and video flowing can be obtained by demultiplexing the audiovisual information of obtained multimedia file.Benchmark
The occurrence of time AV_TIME_BASE can be selected optionally, for example, AV_TIME_BASE=1000000.
Similarly, the timestamp at that initial target audio frame can be calculated according to target frame frame number recycles first frame audio
Timestamp as is corrected the timestamp at of the initial target audio frame, obtains the timestamp AT of target audio frame.For example, mesh
The calculation formula for marking the timestamp AT of audio frame can are as follows:
AT=at+as. (3)
The calculation formula that the timestamp at of initial target audio frame is calculated according to target frame frame number can be a variety of expression-forms
Or calculation, such as the calculation formula can are as follows:
At=((target audio frame frame number * (hits * fiducial time/target audio sample rate of mono- frame audio of 1.0*) *
Inverse/audio stream time base of fiducial time) (4)
Wherein, formula (4) can meet: the hits of a frame audio=target audio sample rate/video frame rate, target audio
Frame frame number=target audio sample index value/frame audio hits.Target audio sample index value can be operated by seek
It is obtained in the incoming interface parameters of interface caller when generation.Target audio sample rate refers to audio sample required for caller
Rate, target audio sample rate can obtain in the target decoder argument structure body according to the setting of interface caller.Fiducial time AV_
The occurrence of TIME_BASE can be selected optionally, for example, AV_TIME_BASE=1000000.
In above-described embodiment, by step S205~S206, for example, accurate base can be obtained by formula (1)~(2)
In the target video timestamp VT of video flowing time base；Similarly, by formula (3)~(4), accurate target audio can be obtained
The timestamp AT of frame.
After obtaining the timestamp AT of timestamp VT and target audio frame of accurate target video frame, using ffmpeg
Carry out stream operation.Preferably, it is nearest first to jump to distance objective video and audio frame (can be target video frame or target audio frame)
Left side key frame position, and intel decoder can be notified to empty its inner buffer, so that subsequent reads are fetched according to (step S207).
Using ffmpeg by the nearest left side key frame position, audio-video frame number is read from the file stream (ES stream)
According to (step S208).If reading is video frame, it is sent into the decoding of intel Video Decoder, judges the timestamp of the video frame
Whether it is less than VT, if so, directly abandoning, otherwise, it is (step S209~S211) to be output is put into video output queue etc..If reading
Arriving is audio frame, then the decoding of intel audio decoder is sent into, and judge whether the audio frame of output is less than AT, if so, directly
It connects and abandons, otherwise, be put into audio output queue, wait (step S212~S214) to be output.When video frame and audio frame have it is defeated
When out, seek terminates.
The embodiment of the present invention can both make full use of Intel's decoder (intel media sdk) to carry out hardware
Accelerate decoding, and user can be allowed to be jumped (seek) on demand to the multimedia file played by decoding, it can be well
Realize that the hardware-accelerated decoding of the multimedia file based on Intel decoder intel media sdk is played and jumped on demand
seek.The present invention can comprehensively utilize intel media sdk with ffmpeg sdk come the method for realizing the synchronous seek of video and audio.
Simultaneously as the matching to target video frame, target audio frame is used closest to principle (leftmost side key frame), thus for having
Some there are the Multi-media Materials of timestamp drift phenomenon, equally can be carried out accurate seek operation.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing devices, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Particular embodiments described above has carried out further in detail the purpose of the present invention, technical scheme and beneficial effects
Describe in detail it is bright, it should be understood that the above is only a specific embodiment of the present invention, the guarantor being not intended to limit the present invention
Range is protected, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should be included in this
Within the protection scope of invention.
Claims (10)
1. the method that a kind of pair of multimedia file is jumped on demand characterized by comprising
The multimedia file for receiving user's input, demultiplexes the multimedia file by ffmpeg, obtains described more
The ES of media file flows, and obtains the audiovisual information of the multimedia file；
Intel's decoder is initialized using the audiovisual information；
The information for jumping accuracy requirement information and a target frame according to one determines the timestamp of target frame in the ES stream；
Stream operation is carried out to the ES stream by the ffmpeg, and since the position of a left side key frame of the target frame
Read video and audio frame data；
It is decoded using the video and audio frame data of the Intel's decoder after initialization to reading, obtains the view
The timestamp of audio frame；
When judging that the timestamp of the video and audio frame is greater than or equal to the timestamp of the target frame, by the video and audio frame data
It is put into video and audio output queue, to be exported.
2. the method jumped on demand to multimedia file as described in claim 1, which is characterized in that jump essence according to one
The information of true property demand information and a target frame determines the timestamp of target frame in the ES stream, comprising:
The threshold value jumped according to the frame jumped in the stream of ES described in accuracy requirement information setting；
Judge whether the frame number difference of current decoded output frame in the target frame and ES stream is greater than or equal to the threshold value；
If so, the timestamp of the target frame is calculated according to the position of the current decoded output frame.
3. the method jumped on demand to multimedia file as claimed in claim 2, which is characterized in that further include: according to
The timestamp of first frame video frame in the ES stream is corrected the timestamp of the video frame in the target frame, according to institute
The timestamp for stating the first frame audio frame in ES stream is corrected the timestamp of the audio frame in the target frame.
4. the method jumped on demand to multimedia file as described in claim 1, which is characterized in that further include: judgement
When the timestamp of the video and audio frame is less than the timestamp of the target frame, the timestamp that timestamp is less than the target frame is abandoned
Video and audio frame data, and continue read next frame video and audio frame data, until the timestamp of the video and audio frame of reading is greater than
Or the timestamp equal to the target frame.
5. the method jumped on demand to multimedia file as described in claim 1, which is characterized in that
When the video and audio frame data are video data, Intel's decoder is Video Decoder；
When the video and audio frame data are audio data, Intel's decoder is audio decoder.
6. the method jumped on demand to multimedia file as claimed in claim 3, which is characterized in that when the video and audio
When frame data are video data, the timestamp of the target frame are as follows:
Vt=(inverse/video flowing time of (target video frame frame number * (1.0/ video frame rate) * fiducial time) * fiducial time
Base).
7. the method jumped on demand to multimedia file as claimed in claim 6, which is characterized in that when the video and audio
When frame data are audio data, the timestamp of the target frame are as follows:
At=is (when target audio frame frame number * (hits * fiducial time/target audio sample rate of mono- frame audio of 1.0*) * benchmark
Between inverse/audio stream time base), wherein the hits of a frame audio=target audio sample rate/video frame rate, target
Audio frame frame number=target audio sample index value/frame audio hits.
8. the method jumped on demand to multimedia file as claimed in claim 5, which is characterized in that will be described in reading
Video and audio frame data are put into video and audio output queue, to be exported, comprising:
Video data timestamp being greater than or equal in the video and audio frame data of the reading of the target frame is put into video
Output queue, to carry out video output；
Audio data timestamp being greater than or equal in the video and audio frame data of the reading of the target frame is put into audio
Output queue, to carry out audio output；
It is described when judging that the video output queue and the audio output queue contain the video and audio frame data exported
Ffmpeg stops reading video and audio frame data.
9. the method jumped on demand to multimedia file as described in claim 1, which is characterized in that the multimedia text
The audiovisual information of part include: multimedia file total length, multimedia file encapsulation format, Video coding mode, video frame rate,
Video flowing length, vedio color format, video code rate, audio coding formats, audio sample rate, audio sample format and audio stream
One or more of length.
10. the method jumped on demand to multimedia file as claimed in claim 7, which is characterized in that close in the left side
Key frame is the key frame on the left of the target frame and closest to the target frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510745515.9A CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510745515.9A CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106686445A CN106686445A (en) | 2017-05-17 |
| CN106686445B true CN106686445B (en) | 2019-06-11 |
Family
ID=58857766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510745515.9A Active CN106686445B (en) | 2015-11-05 | 2015-11-05 | The method that multimedia file is jumped on demand |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106686445B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11146608B2 (en) * | 2017-07-20 | 2021-10-12 | Disney Enterprises, Inc. | Frame-accurate video seeking via web browsers |
| CN107566890B (en) * | 2017-09-15 | 2020-05-22 | 深圳国微技术有限公司 | Method, device, computer device and computer readable storage medium for processing audio stream playing abnormity |
| CN107566889B (en) * | 2017-09-15 | 2020-06-16 | 深圳国微技术有限公司 | Audio stream flow velocity error processing method and device, computer device and computer readable storage medium |
| CN108401188A (en) * | 2018-03-05 | 2018-08-14 | 青岛海信传媒网络技术有限公司 | A kind of method and device of media play |
| CN108737874A (en) * | 2018-06-05 | 2018-11-02 | 武汉斗鱼网络科技有限公司 | A kind of video broadcasting method and electronic equipment |
| CN110418186B (en) * | 2019-02-01 | 2022-03-29 | 腾讯科技（深圳）有限公司 | Audio and video playing method and device, computer equipment and storage medium |
| CN113141522B (en) * | 2020-01-17 | 2022-09-20 | 北京达佳互联信息技术有限公司 | Resource transmission method, device, computer equipment and storage medium |
| CN111918121B (en) * | 2020-06-23 | 2022-02-18 | 南斗六星系统集成有限公司 | Accurate editing method for streaming media file |
| CN114466238B (en) * | 2020-11-09 | 2023-09-29 | 华为技术有限公司 | Frame demultiplexing method, electronic device and storage medium |
| CN113726778A (en) * | 2021-08-30 | 2021-11-30 | 咪咕视讯科技有限公司 | Streaming media seek method, device, computing equipment and computer storage medium |
| CN113965786B (en) * | 2021-09-29 | 2024-03-26 | 杭州当虹科技股份有限公司 | Method for precisely controlling video output playing |
| CN114245231B (en) * | 2021-12-21 | 2023-03-10 | 威创集团股份有限公司 | Multi-video synchronous skipping method, device and equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150729A (en) * | 2006-09-22 | 2008-03-26 | 康佳集团股份有限公司 | Audio and video synchronization playing method |
| CN102368835A (en) * | 2011-06-28 | 2012-03-07 | 上海盈方微电子有限公司 | Audio and video Seek synchronization strategy |
| CN102404624A (en) * | 2011-11-07 | 2012-04-04 | 深圳市佳创视讯技术股份有限公司 | All-format media player capable of supporting hardware decoding for digital STB (Set Top Box) |
| CN104754349A (en) * | 2013-12-25 | 2015-07-01 | 炫一下（北京）科技有限公司 | Method and device for hardware decoding of audio/video |
| CN104766608A (en) * | 2014-01-07 | 2015-07-08 | 深圳市中兴微电子技术有限公司 | Voice control method and voice control device |
-
2015
- 2015-11-05 CN CN201510745515.9A patent/CN106686445B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101150729A (en) * | 2006-09-22 | 2008-03-26 | 康佳集团股份有限公司 | Audio and video synchronization playing method |
| CN102368835A (en) * | 2011-06-28 | 2012-03-07 | 上海盈方微电子有限公司 | Audio and video Seek synchronization strategy |
| CN102404624A (en) * | 2011-11-07 | 2012-04-04 | 深圳市佳创视讯技术股份有限公司 | All-format media player capable of supporting hardware decoding for digital STB (Set Top Box) |
| CN104754349A (en) * | 2013-12-25 | 2015-07-01 | 炫一下（北京）科技有限公司 | Method and device for hardware decoding of audio/video |
| CN104766608A (en) * | 2014-01-07 | 2015-07-08 | 深圳市中兴微电子技术有限公司 | Voice control method and voice control device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106686445A (en) | 2017-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106686445B (en) | The method that multimedia file is jumped on demand | |
| CN103959802B (en) | Image provides method, dispensing device and reception device | |
| US7139470B2 (en) | Navigation for MPEG streams | |
| JP6562992B2 (en) | Trick playback in digital video streaming | |
| RU2011106942A (en) | PROCESSING 3D SUBTITLE DISPLAY | |
| US20140161172A1 (en) | Software hardware hybrid video encoder | |
| US9154834B2 (en) | Fast switching of synchronized media using time-stamp management | |
| RU2013158832A (en) | REDUCED DELAY IN VIDEO CODING AND DECODING | |
| WO2012106272A1 (en) | System and method for custom segmentation for streaming | |
| US9615130B2 (en) | Method and device for processing multimedia frame and storage medium | |
| RU2016118985A (en) | Audio decoder, device for forming output encoded audio data and methods allowing the initialization of decoder | |
| CN109788212A (en) | A kind of processing method of segmenting video, device, terminal and storage medium | |
| US7974281B2 (en) | Multiplexer and multiplexing method, program, and recording medium | |
| KR20130137632A (en) | Method for semantics based trick mode play in video system | |
| WO2019128668A1 (en) | Method and apparatus for processing video bitstream, network device, and readable storage medium | |
| CN105052060B (en) | For from the first data flow handoff to the apparatus and method of the second data flow | |
| JP2024079768A (en) | Information processor and method, program, and information processing system | |
| CN114363648A (en) | Method, equipment and storage medium for audio and video alignment in mixed flow process of live broadcast system | |
| CN113490047A (en) | Android audio and video playing method | |
| CN108989905A (en) | Media stream control method, calculates equipment and storage medium at device | |
| WO2014200539A1 (en) | Remultiplexing bitstreams of encoded video for video playback | |
| CN103165157A (en) | Method and device for locating playing position of no-indexing audio video interleaved (AVI) file and player | |
| WO2018076899A1 (en) | Data switching method, device, terminal and computer-readable storage medium | |
| US20220124419A1 (en) | Information processing device, information processing method, and information processing program | |
| EP1871108B1 (en) | Recording device, reproducing device, recording medium, recording method, and lsi |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |