CN106060628A - DirectShow-based method and system supporting variable coding - Google Patents
DirectShow-based method and system supporting variable coding Download PDFInfo
- Publication number
- CN106060628A CN106060628A CN201610491470.1A CN201610491470A CN106060628A CN 106060628 A CN106060628 A CN 106060628A CN 201610491470 A CN201610491470 A CN 201610491470A CN 106060628 A CN106060628 A CN 106060628A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- sound
- video data
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/426—Internal components of the client ; Characteristics thereof
- H04N21/42607—Internal components of the client ; Characteristics thereof for processing the incoming bitstream
- H04N21/42615—Internal components of the client ; Characteristics thereof for processing the incoming bitstream involving specific demultiplexing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
- H04N21/4381—Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
- H04N21/4385—Multiplex stream processing, e.g. multiplex stream decrypting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
Abstract
The invention discloses a DirectShow-based method supporting variable coding. The method comprises the following steps: acquiring audio and video data based on DirectShow, performing separation, decoding and re-sampling on the acquired audio and video data in sequence, playing the processed audio and video data in a rendering way, and recording current re-sampling output format parameters; acquiring audio and video decoder parameters of audio and video data of which coding formats are changed; creating audio and video decoders and audio and video re-samplers of the audio and video data of which the coding formats are changed; setting audio and video re-sampling output format parameters of the audio and video data of which the coding formats are changed according to the re-sampling output format parameters, and linking to audio and video renderers; and performing separation, decoding and re-sampling on the audio and video data of which the coding formats are changed, transmitting the processed audio and video data to the audio and video renderers, and playing the processed audio and video data in a rendering way. Through adoption of the method, existing renderers do not need to be disconnected and updated, so that the audio data playing continuity is ensured, and pause during audio listening and video watching of a user is avoided.
Description
Technical field
The present invention relates to the technical field of video playback, support variable more particularly, to one based on DirectShow
The method and system of coding.
Background technology
When playing existing formatted video file based on DirectShow, once change (as increased video code model,
Mp4 increases can increase " hevc " structure to after H.265 supporting), it is necessary to upgrading separator, otherwise None-identified, broadcasting.
DirectShow be Microsoft release on the basis of ActiveMovie and Video for Windows new
The kit that generation Streaming Media based on COM (Component Object Model) processes, rises with DirectX kit one
Cloth.DirectShow uses Filter Graph (filter figure) model to manage the processing procedure of whole data stream,
The when of DirectShow work, carried out between Filter (functional module elements of DirectShow) by MediaType
Link, is carried out data transmission by MediaSample.
The mode that DirectShow runs is typically: developer creates Filter Graph, can turn corresponding customizedly
Scheme of changing adds Filter Graph, then played file.Use DirectShow, can be easily from supporting that WDM drives
Capture data on the capture card of model, and carry out corresponding later stage process or even store in file.Make at multi-medium data
In base management system (MDBMS), the access of multi-medium data becomes convenient.DirectShow supports various media widely
Form, including Asf, Mpeg, Avi, Dv, Mp3, Wave etc., seizure and playback for media stream provide strong support.
Therefore, DirectShow leads in video playback as audio and video playing scheme widely used in a kind of windows operating system
Territory is widely used.DirectShow defines a series of standard interface, in that context it may be convenient to carry out Function Extension, as play
The separator of needs, decoder etc..
At present, carry out Internet video broadcasting based on DirectShow, during especially carrying out network direct broadcasting, may
Along with the change of video type, cause the situation that the parameter carrying out encoding based on DirectShow changes, such as, directly
When inserting advertisement during broadcasting, it may occur that the change of audio frequency and video form, the change of encoder sample rate and the resolution of audio frequency and video
The change etc. of rate.For example, based on DirectShow carrying out audio and video playing when, if coding parameter changes
(become 44.1k from 48k or video resolution size is become by 1920*1080 as audio format types is become aac, sample rate from mp3
For 720*576 etc.) time, it is decoded broadcasting according further to original decoder and is clearly the most not all right, if Fig. 1 is base in traditional scheme
Carry out the system structure schematic diagram of audio/video decoding broadcasting in DirectShow, in traditional scheme, sound based on DirectShow regards
Frequently player method is: first carry out the separation of audio frequency and video in separator 101 based on DirectShow；Decode at video the most respectively
Device 102 and audio decoder 103 carry out video and audio decoder；Video after decoding process and audio frequency are respectively at video
Play after renderer 104 and sound renderer 105 render.Traditional carries out the variable volume of audio frequency and video based on DirectShow
Code play solution step as shown in Figure 2:
Step 201, when detecting that audio and video playing generation coding parameter based on DirectShow changes, stop current
Audio and video playing based on DirectShow；
Step 202, disconnect former sound, Video Decoder link, and disconnect former sound, video renderer link；
Step 203, discharge former sound, Video Decoder and former sound, video renderer；
Step 204, create new sound, Video Decoder and new renderer according to the audio frequency and video after change.
Step 205, link new sound, Video Decoder, and start decoding after linking new sound, video renderer and play.
From figure 2 it can be seen that traditional solution carrying out code-change audio/video decoding broadcasting based on DirectShow
Scheme needs again link decoder and renderer, be then further continued for playing new audio frequency and video.Again link decoder and
Can produce during renderer pause, picture can blackening or the phenomenon such as beat, thus affect the viewing experience of user；And
And disconnecting former decoder and renderer, the link process carrying out new decoder and renderer needs the regular hour, and this can increase
Add the waiting time of user, be unfavorable for the viewing experience of user.
Therefore it provides a kind of based on DirectShow quickly, the method that carries out code-change playing audio-video glibly be
This area problem demanding prompt solution.
Summary of the invention
In view of this, the invention provides a kind of method and system supporting code-change based on DirectShow, solve
Prior art carries out based on DirectShow code-change audio/video decoding broadcasting can produce pause, picture can blackening or send out
The problem giving birth to phenomenons such as beating.
In order to solve above-mentioned technical problem, the present invention proposes a kind of method supporting code-change based on DirectShow,
Including:
Obtain audio, video data based on DirectShow to carry out successively separating, decode and transmitting after resampling to sound, video
Renderer carries out rendering broadcasting, and records current resampling output format parameter；Wherein, described audio, video data includes: compile
Code information, video data and voice data；
The coding information that coded format changes from described acquisition audio, video data obtains coded format become
The sound of audio, video data of change, Video Decoder parameter；
The sound that changes according to described coded format, Video Decoder parameter create the sound that coded format changes and regard
The sound of frequency evidence, Video Decoder；
The sound that changes according to described coded format, Video Decoder parameter create the sound that coded format changes and regard
The sound of frequency evidence, video re-sampling device；
And according to described resampling output format parameter set the audio, video data that changes of described coded format sound,
Video re-sampling output format parameter；
Utilize the sound of audio, video data that described coded format changes, video re-sampling output format parameter by described
The sound of the audio, video data that coded format changes, video re-sampling device are linked to described sound, video renderer；
The audio, video data changing described coded format uses described coded format to change after separating
The sound of audio, video data, Video Decoder is decoded；And utilize audio, video data that described coded format changes
After sound, video re-sampling device carry out resampling, transmission carries out rendering broadcasting to described sound, video renderer.
Further, wherein, described obtain audio, video data based on DirectShow and carry out successively separating, decode, heavily adopt
Sample and playing after rendering, and record the output format parameter after current resampling, including:
After separating based on DirectShow acquisition audio, video data, obtain according to the coding information of described audio, video data
Take sound, video track information, and from described sound, video track information, obtain the relevant sound of described audio, video data, Video Decoder
Parameter；
Create sound, Video Decoder according to described sound, Video Decoder parameter, and set described sound, Video Decoder
Output format parameter；
Create resampler according to described sound, Video Decoder parameter, and record current resampling output format parameter.
Further, wherein, the method also includes:
The coding information that coded format does not changes from described acquisition audio, video data obtains coded format do not send out
The sound of the audio, video data of changing, Video Decoder parameter；
The audio, video data not changing described coded format transmits to utilizing described coded format after separating
The audio, video data that the coded format of the sound of the audio, video data not changed, Video Decoder parameter link does not changes
Sound, video decoding be decoded；
And use the sound of audio, video data that coded format do not changes, the resampler of video data to carry out resampling
Rear transmission carries out rendering broadcasting to described sound, video renderer.
Further, wherein, described set described code form according to described resampling output format parameter and change
The sound of audio, video data, video re-sampling output format parameter, also include:
Set the sound of the audio, video data that described coded format changes, video re-sampling output format parameter with described
Resampling output format parameter is consistent.
Further, wherein, decoder and the coded format of the audio, video data that described coded format changes occurs
The resampler of the audio, video data of change is provided by third party's decoder of non-DirectShow.
On the other hand, the present invention also provides for a kind of system supporting code-change based on DirectShow, described system bag
Include: player, detection module, decoder creation module, resampling creation module and link module, wherein,
Described player, carries out separating, decoding and resampling for obtaining audio, video data based on DirectShow successively
Rear transmission carries out rendering broadcasting to sound, video renderer, and records current resampling output format parameter；Wherein, described sound
Video data includes: coding information, video data and voice data；
Described acquisition module, obtains in the coding information that coded format changes from described acquisition audio, video data
Take the sound of the audio, video data that coding information changes, Video Decoder parameter；
Described decoder creation module, for the sound changed according to described coded format, Video Decoder parameter wound
Build the sound of the audio, video data that coded format changes, Video Decoder；
Described resampling creation module, sound, Video Decoder parameter wound after changing according to described coded format
Build the sound of the audio, video data that coded format changes, video re-sampling device；
And according to described resampling output format parameter set the audio, video data that changes of described coded format sound,
Video re-sampling output format parameter；
Described link module, for the sound of audio, video data utilizing described coded format to change, video re-sampling
The sound of the audio, video data that described coded format is changed by output format parameter, video re-sampling device be linked to described sound,
Video renderer；
Described player, the audio, video data being additionally operable to change described coded format uses described after separating
The sound of the audio, video data that coded format changes, Video Decoder are decoded；And utilize described coded format to become
The sound of audio, video data, the video re-sampling device changed transmit extremely described sound, video renderer and carry out rendering broadcasting after carrying out resampling
Put.
Further, wherein, described player, it is additionally operable to:
After separating based on DirectShow acquisition audio, video data, obtain according to the coding information of described audio, video data
Take sound, video track information, and from described sound, video track information, obtain the relevant sound of described audio, video data, Video Decoder
Parameter；
Create sound, Video Decoder according to described sound, Video Decoder parameter, and set described sound, Video Decoder
Output format parameter；
Create resampler according to described sound, Video Decoder parameter, and record the output format ginseng of current resampling
Number.
Further, wherein, described acquisition module, it is additionally operable to: from the audio, video data of described acquisition, encode information lattice
The coding information that formula does not changes obtains the sound of audio, video data, Video Decoder ginseng that coded format does not changes
Number；
Described decoder creation module, is additionally operable to: the audio, video data not changing described coded format is carried out point
Sound, the coding of Video Decoder parameter link from rear transmission to the audio, video data utilizing described coded format not change
The decoding of the sound of the audio, video data that form does not changes, video is decoded；
Described resampling creation module, is additionally operable to: use coded format do not change the sound of audio, video data, video
Sound, video data that decoded coded format is not changed by the resampler of data carry out resampling；
Described player, is additionally operable to: the sound of audio, video data that the coded format after resampling do not changed, video
Data transmission carries out rendering broadcasting to described sound, video renderer.
Further, wherein, described resampling creation module, it is additionally operable to:
Set the sound of the audio, video data that described coded format changes, video re-sampling output format parameter with described
Resampling output format parameter is consistent.
Further, wherein, decoder and the coded format of the audio, video data that described coded format changes occurs
The resampler of the audio, video data of change is provided by third party's decoder of non-DirectShow.
Compared with prior art, the method and system supporting code-change based on DirectShow of the present invention, it is achieved that
Following beneficial effect:
(1) method and system supporting code-change based on DirectShow of the present invention, by arranging resampling
After output format consistent so that audio, video data encodes information format when changing, after new audio, video data decoding
Resampling output format is consistent with the output format of earlier set, it is not necessary to disconnect and update existing renderer, it is ensured that sound
Frequently the seriality of data playback, will not impact Consumer's Experience.
(2) method and system supporting code-change based on DirectShow of the present invention so that audio, video data
When middle coding information format changes, resampling output format and the output lattice of earlier set after new audio, video data decoding
Formula is consistent, saves the link time of new decoder and renderer, decreases the time that user waits, improves Consumer's Experience.
Certainly, the arbitrary product implementing the present invention specific needs must not reach all the above technique effect simultaneously.
By detailed description to the exemplary embodiment of the present invention referring to the drawings, the further feature of the present invention and
Advantage will be made apparent from.
Accompanying drawing explanation
Combined in the description and the accompanying drawing of the part that constitutes description shows embodiments of the invention, and even
With its explanation together for explaining the principle of the present invention.
Fig. 1 is the system structure schematic diagram carrying out audio/video decoding broadcasting in prior art based on DirectShow；
Fig. 2 is the method flow schematic diagram carrying out audio frequency and video code-change broadcasting in prior art based on DirectShow；
Fig. 3 is the system structure schematic diagram carrying out audio/video decoding broadcasting in the present invention based on DirectShow；
Fig. 4 is the method flow schematic diagram supporting code-change based on DirectShow of the present invention；
Fig. 5 is the format structure schematic diagram of heretofore described audio, video data；
Fig. 6 is an optional reality of the method supporting code-change based on DirectShow described in the embodiment of the present invention 2
Execute the schematic flow sheet of example；
Fig. 7 is to use FFmpeg to carry out audio, video data decoding and the schematic flow sheet of resampling；
The flow process using FFmpeg to be decoded with resampling when Fig. 8 is to carry out audio and video playing based on DirectShow is shown
It is intended to；
Fig. 9 is at the beginning of an instantiation of the method supporting code-change based on DirectShow described in the present embodiment
Beginningization schematic flow sheet；
Figure 10 is a concrete reality of the method supporting code-change based on DirectShow described in the embodiment of the present invention 3
Example is not detected by schematic flow sheet when coded format changes；
Figure 11 is a concrete reality of the method supporting code-change based on DirectShow described in the embodiment of the present invention 3
Example detects schematic flow sheet when coded format changes；
Figure 12 is the system structure schematic diagram supporting code-change based on DirectShow of the present invention.
Detailed description of the invention
The various exemplary embodiments of the present invention are described in detail now with reference to accompanying drawing.It should also be noted that unless additionally have
Body illustrates, the parts illustrated the most in these embodiments and positioned opposite, the numerical expression of step and numerical value are not intended to this
The scope of invention.
Description only actually at least one exemplary embodiment is illustrative below, never as to the present invention
And any restriction applied or use.
May be not discussed in detail for technology, method and apparatus known to person of ordinary skill in the relevant, but suitable
In the case of when, described technology, method and apparatus should be considered a part for description.
It is shown here that any occurrence should be construed as merely exemplary with in all examples discussed, and not
It is as restriction.Therefore, other example of exemplary embodiment can have different values.
It should also be noted that similar label and letter represent similar terms, therefore, the most a certain Xiang Yi in following accompanying drawing
Individual accompanying drawing is defined, then need not it is further discussed in accompanying drawing subsequently.
Embodiment 1
As shown in Figures 3 and 4, the system knot of audio/video decoding broadcasting is carried out during Fig. 3 is the present invention based on DirectShow
Structure schematic diagram；Wherein, separator 301 is integrated with separative element 311, Video Decoder 302, audio decoder 303, video weight
Sampling 304 and audio resampling 305, audio, video data transmits after carrying out separator process to video renderer 306 and audio frequency wash with watercolours
Dye device 307 carries out rendering broadcasting.Fig. 4 is the method flow signal supporting code-change based on DirectShow of the present invention
Figure.The method supporting code-change based on DirectShow of the present invention, it is to avoid disconnected between decoder and renderer
Open and again link the slack problem of the audio and video playing brought.The present embodiment supports code-change based on DirectShow
Method comprise the following steps:
Step 401, obtain based on DirectShow audio, video data carry out successively separating, decode and transmit after resampling to
Sound, video renderer carry out rendering broadcasting, and record current resampling output format parameter；Wherein, described audio, video data
Including: coding information, video data and voice data.As it is shown in figure 5, be the format structure schematic diagram of audio, video data, including:
Coding information 501, video data 502 and voice data 503.
Step 402, the coding information that coded format changes from described acquisition audio, video data obtain coding lattice
The sound of the audio, video data that formula changes, Video Decoder parameter.
Step 403, disconnect current sound, Video Decoder, the sound that changes according to described coded format, video decoding
Device parameter creates the sound of audio, video data, the Video Decoder that coded format changes.
Step 404, disconnect current sound, video re-sampling device, the sound that changes according to described coded format, video solution
Code device parameter creates the sound of audio, video data, the video re-sampling device that coded format changes；And it is defeated according to described resampling
Go out format parameter and set the sound of audio, video data, the video re-sampling output format parameter that described coded format changes.Its
In, resampling refers to uncompressed data is carried out form or Parameters Transformation, as being yuv format by the video conversion of rgb format
Video, or audio frequency is carried out sample rate conversion.
Step 405, utilize the sound of audio, video data, video re-sampling output format ginseng that described coded format changes
The sound of audio, video data, video re-sampling device that described coded format is changed by number are linked to described sound, video renderer.
Step 406, the audio, video data changing described coded format use described coded format after separating
The sound of the audio, video data changed, Video Decoder are decoded；And the sound utilizing described coded format to change regards
After the sound of frequency evidence, video re-sampling device carry out resampling, transmission carries out rendering broadcasting to described sound, video renderer.
Described in the present embodiment based on DirectShow support code-change method, in the separator be integrated with decoding and
Resampling function, directly transmits the audio, video data without compression to renderer.Ensure that the output format of separator remains system
One, do not change, therefore need not between separator with renderer again link and just can normally work.Ensure that when coding lattice
The when that formula or parameter changing, the seriality of played data, Consumer's Experience will not be impacted.Further, in separator
The decoder in portion and resampling can also be provided (as used FFmpeg or other third party's decoders etc.) by non-DirectShow.
During this, all operations is all carried out at separator interior, Consumer's Experience will not be produced impact.
Embodiment 2
As shown in Figure 6, for one of the method supporting code-change based on DirectShow described in the embodiment of the present invention 2
The schematic flow sheet of alternative embodiment.The present embodiment, on the basis of embodiment 1, describes and occurs sound to regard based on DirectShow
Frequently, during data encoding information change, again decode, the particular content of resampling.Described in the present embodiment based on
DirectShow supports that the step of the method for code-change is as follows:
Step 601, obtain after audio, video data separates based on DirectShow, according to the volume of described audio, video data
Code acquisition of information sound, video track information (including multiple sound, video track information in sound, video information), and from described sound, video
Rail information obtains the relevant sound of described audio, video data, Video Decoder parameter；Wherein, described audio, video data includes: compile
Code information, video data and voice data.
Step 602, create sound, Video Decoder according to described sound, Video Decoder parameter, and set described sound, video
The output format parameter of decoder.
Step 603, create resampler according to described sound, Video Decoder parameter, and record current resampling output
Format parameter.
Step 604, the sound after resampling, video data transmission to sound, video renderer are carried out rendering broadcasting, and record
Current resampling output format parameter.
Step 605, the coding information that coded format changes from described acquisition audio, video data obtain coding lattice
The sound of the audio, video data that formula changes, Video Decoder parameter.
Step 606, disconnect current sound, Video Decoder, the sound that changes according to described coded format, video decoding
Device parameter creates the sound of audio, video data, the Video Decoder that coded format changes.
Step 607, disconnect current sound, video re-sampling device, the sound that changes according to described coded format, video solution
Code device parameter creates the sound of audio, video data, the video re-sampling device that coded format changes, and defeated according to described resampling
Go out format parameter and set the sound of audio, video data, the video re-sampling output format parameter that described coded format changes；Excellent
Selection of land, sets the sound of the audio, video data that described coded format changes, video re-sampling output format parameter heavy with described
Sampling output format parameter is consistent.
Step 608, utilize the sound of audio, video data, video re-sampling output format ginseng that described coded format changes
The sound of audio, video data, video re-sampling device that described coded format is changed by number are linked to described sound, video renderer.
Step 609, the coding information that coded format does not changes from described acquisition audio, video data obtain coding
The sound of the audio, video data that form does not changes, Video Decoder parameter.
After step 610, the audio, video data not changing described coded format separate, transmission is described to utilizing
The sound that the coded format of the sound of the audio, video data that coded format does not changes, Video Decoder parameter link does not changes
The decoding of the sound of video data, video is decoded；And use the sound of audio, video data, the video counts that coded format do not changes
According to resampler carry out resampling after transmission to described sound, video renderer carry out rendering broadcasting.
Preferably, the decoding described in the present embodiment and resampling are provided by third party's decoder of non-DirectShow,
As used FFmpeg or other third party's decoders, FFmpeg is that a kind of tool is with there being opening of the function such as audio/video decoding, resampling
Storehouse, source.The present embodiment illustrates as a example by using FFmpeg, as it is shown in fig. 7, for using FFmpeg to carry out audio, video data decoding
With the schematic flow sheet of resampling, comprise the steps:
Step 701, input audio, video data；
Step 702, judge whether audio frequency and video rail；
Step 703, when there is audio frequency and video rail, obtain the coding information of this audio frequency and video rail；
Step 704, according to obtain coding information creating FFmpeg decoder；
Step 705, the output parameter of setting FFmpeg decoder；
Step 706, establishment FFmpeg resampling；
Step 707, record resampling format parameter；
Step 708, establishment output transport function (pin), and continue to determine whether to there is next audio frequency and video rail；
Step 709, when there is not audio frequency and video rail, end-sound video data decoding and resampling.
During such as Fig. 8 for carrying out audio and video playing based on DirectShow, FFmpeg is used to be decoded the stream with resampling
Journey schematic diagram, it specifically comprises the following steps that
Step 801, start to input audio, video data；
Step 802, it is successively read audio, video data；
Step 803, judge whether to read audio, video data；
If step 804 does not reads audio, video data, then terminate FFmpeg decoding and resampling；
If step 805 reads audio, video data, compare whether this audio, video data coding changes；
If step 806 audio, video data coding does not changes, then carry out FFmpeg decoding；
Step 807, carry out FFmpeg resampling；
Export data after step 808, FFmpeg resampling to carry out rendering and carrying out next digital independent；
If step 809 audio, video data coding changes, then close FFmpeg decoder；
Step 810, closedown FFmpeg resampling；
Step 811, create new FFmpeg decoder；
Step 812, create new FFmpeg resampling；
Step 813, record FFmpeg resampling format parameter, and carry out FFmpeg decoding.
Embodiment 3
As it is shown in figure 9, one for the method based on DirectShow support code-change described in the present embodiment concrete
The initialization schematic flow sheet of example.Method based on DirectShow support code-change described in the present embodiment includes as follows
Step:
The initialization step of audio, video data:
Step 901, acquisition video decoder types are 1920*1080 for H.264 parameter.Wherein, currently playing video
H.264 form is, 1920*1080, and audio format is mp3,2 sound channel 16bit 44.1k.
Step 902, create H.264 decoder, set decoder output format as YV12,1920*1080.
Step 903, create video re-sampling, set output format as RGB32,1920*1080, and record current format and be
H.264,1920*1080, resampling output format is RGB32,1920*1080；Create video frequency output transport function
, and specified format is RGB32,1920*1080, later with this form for video frequency output transport function format standard (Pin).
Step 904, establishment mp3 decoder, set decoder output format as PCM 2 sound channel, 16bit 44.1k.
Step 905, create audio resampling, sets output format as PCM 2 sound channel, 16bit 44.1k, and record currently
Form is mp3,2 sound channel 16bit 44.1k, and resampling output format is PCM, 2 sound channel 16bit 44.1k；Establishment audio frequency exports
Pin, and specified format is PCM 2 sound channel 16bit 44.1k.Later with this form as standard.
As shown in Figure 10, for supporting of method of code-change concrete based on DirectShow described in the present embodiment
Example is not detected by schematic flow sheet when coded format changes.Audio frequency and video number when playing out based on DirectShow
The process step not changed according to coded format is as follows:
Step 1001, reading video data, H.264 form is, 1920*1080 detects audio, video data coded format
Do not change.
Step 1002, decoding video data, be output as YV12,1920*1080.
Step 1003, resampling video data, be output as RGB32,1920*1080, and export video data by Pin,
Form is RGB32 1920*1080.
Step 1004, reading voice data, form is mp3, and 2 sound channel 16bit 44.1k detect audio data coding lattice
Formula and parameter do not change.
Step 1005, decoding audio data, be output as PCM, 2 sound channel 16bit 44.1k.
Step 1006, resampling voice data, be output as PCM, 2 sound channel 16bit 44.1k, exports audio frequency number by Pin
According to, form is PCM, 2 sound channel 16bit 44.1k.
As shown in figure 11, for supporting of method of code-change concrete based on DirectShow described in the present embodiment
Example detects schematic flow sheet when coded format changes.After playing a period of time, detect that audio, video data is compiled
Code form changes, and video format becomes Divx, 720*576, and audio format becomes aac, 6 sound channel 16bit 48k, now presses
The step carrying out playing based on DirectShow according to the inventive method is as follows:
Step 1101, the coding information read in audio, video data, detect that video code model and parameter become Divx,
720*576, with original H.264,1920*1080 is different.
Step 1102, discharge former H.264 decoder, create Divx decoder, set Video Decoder output format as RGB
720*576。
Step 1103, discharge former video re-sampling, and create new video resampling, set output format as RGB32,
1920*1080, and to record current video form be Divx 720*576.
Step 1104, audio coding formats and parameter aac being detected, 6 sound channel 16bit 48k, with original mp3,2 sound channels
16bit 44.1k is different.
Step 1105, discharge former mp3 decoder, create aac decoder, and setting audio decoder output format is PCM
6 sound channel 16bit 48k.
Step 1106, discharge former audio resampling, create new audio resampling, set output format as PCM 2 sound channel
16bit 44.1k, and to record present video form be aac 6 sound channel 16bit 48k.
Step 1107, reading format are the video data of Divx 720*576, and after decoding, output format is RGB 720*
576。
Step 1108, to decoding after output video data carry out resampling, output format is RGB32 1920*1080,
And exporting video data by Pin, form is RGB32 1920*1080.
Step 1109, reading format are aac, the voice data of 6 sound channel 16bit 48k, are decoded rear output format and are
PCM, 6 sound channel 16bit 48k.
Step 1110, decoded voice data being carried out resampling, output format is PCM 2 sound channel 16bit 44.1k,
Exporting voice data by Pin, form is PCM, 2 sound channel 16bit 44.1k.
Embodiment 4
As shown in figure 12, for the system structure schematic diagram supporting code-change based on DirectShow of the present invention.
The present embodiment based on DirectShow support code-change system for realize in above-described embodiment based on
DirectShow supports the method for code-change, and this system includes: player 1201, acquisition module 1202, decoder create mould
Block 1203, resampling creation module 1204 and link module 1205.
Wherein, described player 1201, connect with described acquisition module 1202 phase Rhizoma Nelumbinis, for obtaining sound based on DirectShow
Video data carries out successively separating, decode and transmit after resampling to sound, video renderer and carries out rendering broadcasting, and record current
Resampling output format parameter；Wherein, described audio, video data includes: coding information, video data and voice data；
Described acquisition module 1202, connects with described player 1201 and decoder creation module 1203 phase Rhizoma Nelumbinis, for from institute
State to obtain and the coding information that in audio, video data, coded format changes obtains the audio frequency and video number that coding information changes
According to sound, Video Decoder parameter；
Described decoder creation module 1203, connects with described acquisition module 1202 and resampling creation module 1204 phase Rhizoma Nelumbinis,
The audio frequency and video number that coded format changes is created for the sound changed according to described coded format, Video Decoder parameter
According to sound, Video Decoder；
Described resampling creation module 1204, connects with described decoder creation module 1203 and link module 1205 phase Rhizoma Nelumbinis,
After changing according to described coded format, sound, Video Decoder parameter create the audio frequency and video number that coded format changes
According to sound, video re-sampling device；
And according to described resampling output format parameter set the audio, video data that changes of described coded format sound,
Video re-sampling output format parameter.
Described link module 1205, connects with described resampling creation module 1204 and player 1201 phase Rhizoma Nelumbinis, is used for utilizing
Described coded format is occurred by the sound of the audio, video data that described coded format changes, video re-sampling output format parameter
The sound of audio, video data of change, video re-sampling device are linked to described sound, video renderer.
Described player 1201, also connects with described link module 1205 phase Rhizoma Nelumbinis, for changing described coded format
Audio, video data separate after the sound of audio, video data that uses described coded format to change, Video Decoder carry out
Decoding；And utilize the sound of audio, video data that described coded format changes, video re-sampling device to transmit after carrying out resampling
Carry out rendering broadcasting to described sound, video renderer.
Further, described player 1201, it is additionally operable to:
After separating based on DirectShow acquisition audio, video data, obtain according to the coding information of described audio, video data
Take sound, video track information, and from described sound, video track information, obtain the relevant sound of described audio, video data, Video Decoder
Parameter；
Create sound, Video Decoder according to described sound, Video Decoder parameter, and set described sound, Video Decoder
Output format parameter；
Create resampler according to described sound, Video Decoder parameter, and record the output format ginseng of current resampling
Number.
Described acquisition module 1202 is additionally operable to:
Encode from the audio, video data of described acquisition in the coding information that information format does not changes and obtain coding lattice
The sound of the audio, video data that formula does not changes, Video Decoder parameter；
Described decoder creation module 1203 is additionally operable to: the audio, video data not changing described coded format is carried out
The sound to the audio, video data utilizing described coded format not change, the volume of Video Decoder parameter link is transmitted after separation
The sound of audio, video data, video decoding that code form does not changes are decoded；
Described resampling creation module 1204 is additionally operable to: use the audio, video data that do not changes of coded format sound,
Sound, video data that decoded coded format is not changed by the resampler of video data carry out resampling；
Described player 1201 is additionally operable to: the sound of the audio, video data that the coded format after resampling is not changed,
Video data transmission carries out rendering broadcasting to described sound, video renderer.
Described resampling creation module 1203 is additionally operable to:
Set the sound of the audio, video data that described coded format changes, video re-sampling output format parameter with described
Resampling output format parameter is consistent.
Preferably, decoder and the coded format of the audio, video data that the described coded format in this enforcement changes is sent out
The resampler of the audio, video data of changing is provided by third party's decoder of non-DirectShow, such as, passes through FFmpeg
The decoding of the audio, video data that offer coded format changes and resampling.
By each embodiment above, the method and system supporting code-change based on DirectShow of the present invention
Exist and provide the benefit that:
(1) method and system supporting code-change based on DirectShow of the present invention, by arranging resampling
After output format consistent so that audio, video data encodes information format when changing, after new audio, video data decoding
Resampling output format is consistent with the output format of earlier set, it is not necessary to disconnect and update existing decoder and renderer,
Ensure that the seriality that voice data is play, Consumer's Experience will not be impacted.
(2) method and system supporting code-change based on DirectShow of the present invention so that audio, video data
When middle coding information format changes, resampling output format and the output lattice of earlier set after new audio, video data decoding
Formula is consistent, saves the link time of new decoder and renderer, decreases the time that user waits, improves Consumer's Experience.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, device or computer program
Product.Therefore, the reality in terms of the present invention can use complete hardware embodiment, complete software implementation or combine software and hardware
Execute the form of example.And, the present invention can use at one or more computers wherein including computer usable program code
The upper computer program product implemented of usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.)
The form of product.
Although by example, some specific embodiments of the present invention have been described in detail, but the skill of this area
Art personnel are it should be understood that example above is merely to illustrate rather than in order to limit the scope of the present invention.The skill of this area
Art personnel are it should be understood that can modify to above example without departing from the scope and spirit of the present invention.This
Bright scope is defined by the following claims.
Claims (10)
1. the method supporting code-change based on DirectShow, it is characterised in that including:
Obtain audio, video data based on DirectShow to carry out successively separating, decode and transmitting after resampling to sound, Video Rendering
Device carries out rendering broadcasting, and records current resampling output format parameter；Wherein, described audio, video data includes: coding letter
Breath, video data and voice data；
The coding information that coded format changes from described acquisition audio, video data obtains what coded format changed
The sound of audio, video data, Video Decoder parameter；
The sound, the Video Decoder parameter that change according to described coded format create the audio frequency and video number that coded format changes
According to sound, Video Decoder；
The sound, the Video Decoder parameter that change according to described coded format create the audio frequency and video number that coded format changes
According to sound, video re-sampling device；
And set, according to described resampling output format parameter, the sound of audio, video data, the video that described coded format changes
Resampling output format parameter；
Utilize the sound of audio, video data that described coded format changes, video re-sampling output format parameter by described coding
The sound of the audio, video data that form changes, video re-sampling device are linked to described sound, video renderer；
The audio, video data changing described coded format uses the sound that described coded format changes after separating
The sound of video data, Video Decoder are decoded；And utilize the sound of audio, video data that described coded format changes, regard
Frequently after resampler carries out resampling, transmission carries out rendering broadcasting to described sound, video renderer.
The most according to claim 1 based on DirectShow support code-change method, it is characterised in that described based on
DirectShow obtain audio, video data carry out successively separating, decode, resampling and playing after rendering, and record current heavily adopting
Output format parameter after sample, including:
After separating based on DirectShow acquisition audio, video data, according to the coding acquisition of information of described audio, video data
Sound, video track information, and from described sound, video track information, obtain the relevant sound of described audio, video data, Video Decoder ginseng
Number；
Create sound, Video Decoder according to described sound, Video Decoder parameter, and set the output of described sound, Video Decoder
Format parameter；
Create resampler according to described sound, Video Decoder parameter, and record current resampling output format parameter.
The method supporting code-change based on DirectShow the most according to claim 1, it is characterised in that also include:
The coding information that coded format does not changes from described acquisition audio, video data obtains coded format become
The sound of audio, video data of change, Video Decoder parameter；
The audio, video data not changing described coded format transmits after separating to utilizing described coded format not send out
The audio, video data that the coded format of the sound of the audio, video data of changing, Video Decoder parameter link does not changes
The decoding of sound, video is decoded；
And use the sound of audio, video data that coded format do not changes, the resampler of video data to pass after carrying out resampling
Transport to described sound, video renderer carries out rendering broadcasting.
The method supporting code-change based on DirectShow the most according to claim 1, it is characterised in that described basis
Described resampling output format parameter sets the sound of audio, video data, the video re-sampling output lattice that described code form changes
Formula parameter, also includes:
Set the sound of the audio, video data that described coded format changes, video re-sampling output format parameter heavy is adopted with described
Sample output format parameter is consistent.
The method supporting code-change based on DirectShow the most according to claim 1, it is characterised in that described coding
The resampler of the audio, video data that the decoder of the audio, video data that form changes and coded format change is by non-
Third party's decoder of DirectShow provides.
6. the system supporting code-change based on DirectShow, it is characterised in that including: player, detection module, solution
Code device creation module, resampling creation module and link module, wherein,
Described player, carries out separating, decode and passing after resampling for obtaining audio, video data based on DirectShow successively
Transport to sound, video renderer carries out rendering broadcasting, and records current resampling output format parameter；Wherein, described audio frequency and video
Data include: coding information, video data and voice data；
Described acquisition module, obtains in the coding information that coded format changes from described acquisition audio, video data and compiles
Code information change the sound of audio, video data, Video Decoder parameter；
Described decoder creation module, creates compile for the sound changed according to described coded format, Video Decoder parameter
Code form change the sound of audio, video data, Video Decoder；
Described resampling creation module, after changing according to described coded format, sound, Video Decoder parameter create and compile
Code form change the sound of audio, video data, video re-sampling device；
And set, according to described resampling output format parameter, the sound of audio, video data, the video that described coded format changes
Resampling output format parameter；
Described link module, for the sound of audio, video data utilizing described coded format to change, video re-sampling output
The sound of the audio, video data that described coded format is changed by format parameter, video re-sampling device are linked to described sound, video
Renderer；
Described player, the audio, video data being additionally operable to change described coded format uses described coding after separating
The sound of the audio, video data that form changes, Video Decoder are decoded；And utilize described coded format to change
After the sound of audio, video data, video re-sampling device carry out resampling, transmission carries out rendering broadcasting to described sound, video renderer.
The system supporting code-change based on DirectShow the most according to claim 6, it is characterised in that described broadcasting
Device, is additionally operable to:
After separating based on DirectShow acquisition audio, video data, according to the coding acquisition of information of described audio, video data
Sound, video track information, and from described sound, video track information, obtain the relevant sound of described audio, video data, Video Decoder ginseng
Number；
Create sound, Video Decoder according to described sound, Video Decoder parameter, and set the output of described sound, Video Decoder
Format parameter；
Create resampler according to described sound, Video Decoder parameter, and record the output format parameter of current resampling.
The system supporting code-change based on DirectShow the most according to claim 6, it is characterised in that described acquisition
Module, is additionally operable to: encodes to obtain in the coding information that information format does not changes from the audio, video data of described acquisition and compiles
Code form do not change the sound of audio, video data, Video Decoder parameter；
Described decoder creation module, is additionally operable to: after the audio, video data not changing described coded format separates
Transmit the sound to the audio, video data utilizing described coded format not change, the coded format of Video Decoder parameter link
The decoding of the sound of the audio, video data not changed, video is decoded；
Described resampling creation module, is additionally operable to: use coded format do not change the sound of audio, video data, video data
Resampler sound that decoded coded format is not changed, video data carry out resampling；
Described player, is additionally operable to: the sound of audio, video data that the coded format after resampling do not changed, video data
Transmission carries out rendering broadcasting to described sound, video renderer.
The system supporting code-change based on DirectShow the most according to claim 6, it is characterised in that described heavy adopt
Sample creation module, is additionally operable to:
Set the sound of the audio, video data that described coded format changes, video re-sampling output format parameter heavy is adopted with described
Sample output format parameter is consistent.
The system supporting code-change based on DirectShow the most according to claim 6, it is characterised in that described volume
The resampler of the decoder of audio, video data that code form changes and the audio, video data that coded format changes by
Third party's decoder of non-DirectShow provides.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610491470.1A CN106060628A (en) | 2016-06-28 | 2016-06-28 | DirectShow-based method and system supporting variable coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610491470.1A CN106060628A (en) | 2016-06-28 | 2016-06-28 | DirectShow-based method and system supporting variable coding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106060628A true CN106060628A (en) | 2016-10-26 |
Family
ID=57167298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610491470.1A Pending CN106060628A (en) | 2016-06-28 | 2016-06-28 | DirectShow-based method and system supporting variable coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106060628A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106878290A (en) * | 2017-01-22 | 2017-06-20 | 北京奇艺世纪科技有限公司 | The player method and device of audio frequency and video |
| CN108010534A (en) * | 2016-11-01 | 2018-05-08 | 北京酷我科技有限公司 | A kind of audio file play method and device |
| CN109218815A (en) * | 2018-09-10 | 2019-01-15 | 苏宁智能终端有限公司 | A kind of video switching method and system |
| CN109257655A (en) * | 2018-09-29 | 2019-01-22 | 视联动力信息技术股份有限公司 | Audio-frequency processing method and device when a kind of program switches |
| CN110536096A (en) * | 2018-05-25 | 2019-12-03 | 视联动力信息技术股份有限公司 | A kind of decoded method and apparatus of data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101272390A (en) * | 2008-05-12 | 2008-09-24 | 腾讯科技（深圳）有限公司 | Filter link establishing method and device for media file |
| CN101552791A (en) * | 2009-05-22 | 2009-10-07 | 腾讯科技（深圳）有限公司 | Method and system for playing multiple media file |
| CN103067759A (en) * | 2011-10-19 | 2013-04-24 | 深圳市快播科技有限公司 | Establishment method and devices of playback link and player |
| CN104837052A (en) * | 2014-06-10 | 2015-08-12 | 腾讯科技（北京）有限公司 | Playing method of multimedia data and device |
| CN105657524A (en) * | 2016-01-13 | 2016-06-08 | 上海视云网络科技有限公司 | Seamless video switching method |
-
2016
- 2016-06-28 CN CN201610491470.1A patent/CN106060628A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101272390A (en) * | 2008-05-12 | 2008-09-24 | 腾讯科技（深圳）有限公司 | Filter link establishing method and device for media file |
| CN101552791A (en) * | 2009-05-22 | 2009-10-07 | 腾讯科技（深圳）有限公司 | Method and system for playing multiple media file |
| CN103067759A (en) * | 2011-10-19 | 2013-04-24 | 深圳市快播科技有限公司 | Establishment method and devices of playback link and player |
| CN104837052A (en) * | 2014-06-10 | 2015-08-12 | 腾讯科技（北京）有限公司 | Playing method of multimedia data and device |
| CN105657524A (en) * | 2016-01-13 | 2016-06-08 | 上海视云网络科技有限公司 | Seamless video switching method |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108010534A (en) * | 2016-11-01 | 2018-05-08 | 北京酷我科技有限公司 | A kind of audio file play method and device |
| CN106878290A (en) * | 2017-01-22 | 2017-06-20 | 北京奇艺世纪科技有限公司 | The player method and device of audio frequency and video |
| CN110536096A (en) * | 2018-05-25 | 2019-12-03 | 视联动力信息技术股份有限公司 | A kind of decoded method and apparatus of data |
| CN109218815A (en) * | 2018-09-10 | 2019-01-15 | 苏宁智能终端有限公司 | A kind of video switching method and system |
| CN109218815B (en) * | 2018-09-10 | 2021-10-22 | 苏宁智能终端有限公司 | Video switching method and system |
| CN109257655A (en) * | 2018-09-29 | 2019-01-22 | 视联动力信息技术股份有限公司 | Audio-frequency processing method and device when a kind of program switches |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106060628A (en) | DirectShow-based method and system supporting variable coding | |
| EP1648172A1 (en) | System and method for embedding multimedia editing information in a multimedia bitstream | |
| CN1723703B (en) | Data processing device and method | |
| CN102014262A (en) | Hard disk video recorder and system and method for converting multimedia formats | |
| US20150062353A1 (en) | Audio video playback synchronization for encoded media | |
| CN103210655B (en) | Content-data generating means and content data generation method | |
| CN1862970A (en) | Audio decoding device | |
| US9800880B2 (en) | Configurable transcoder and methods for use therewith | |
| JPWO2006137425A1 (en) | Audio encoding apparatus, audio decoding apparatus, and audio encoded information transmission apparatus | |
| CN1941144B (en) | Data recording and reproducing apparatus and method therefor | |
| JP4613674B2 (en) | Audio playback device | |
| TW200836510A (en) | Method and system for switching elementary streams on a decoder with zero delay | |
| CN101312460A (en) | Method for converting media file of multiple formats into target device supported media file | |
| US20060059509A1 (en) | System and method for embedding commercial information in a video bitstream | |
| CN102196268B (en) | Method, device and system for processing multimedia data | |
| CN103841451A (en) | Multimedia playing method, device and terminal | |
| CN101808242B (en) | Buffer control device and buffer control method | |
| US7822569B2 (en) | Specific-condition-section detection apparatus and method of detecting specific condition section | |
| CN102572339A (en) | Decoding apparatus for encoded video signals | |
| CN102955809A (en) | Method and system for editing and playing media files | |
| JP2004280994A (en) | Data creation method and data recorder | |
| US20060056506A1 (en) | System and method for embedding multimedia compression information in a multimedia bitstream | |
| US7664373B2 (en) | Program, data processing method, and system of same | |
| JP2007243501A (en) | Video image reproducer | |
| CN106060656A (en) | Method and system for generating BHD file based on DirectShow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161026 |
|
| RJ01 | Rejection of invention patent application after publication |