CN106375821B - A kind of audio and video playing method and device - Google Patents
A kind of audio and video playing method and device Download PDFInfo
- Publication number
- CN106375821B CN106375821B CN201610780671.3A CN201610780671A CN106375821B CN 106375821 B CN106375821 B CN 106375821B CN 201610780671 A CN201610780671 A CN 201610780671A CN 106375821 B CN106375821 B CN 106375821B
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- sub
- stream
- switched
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention discloses a kind of audio and video playing method and device, method includes: to obtain two-way media stream, wherein media stream includes target video subflow all the way, and another way media stream includes target audio subflow；It will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；The target video subflow and the target audio subflow are decoded and are rendered respectively；The target video subflow and the target audio subflow after rendering is played simultaneously.Using the embodiment of the present invention, it can mix and play two-way media stream.
Description
Technical field
The present invention relates to audio and video playing technical field, in particular to a kind of audio and video playing method and device.
Background technique
With the rapid development of internet and multimedia technology, user watches Online Video by network, very convenient,
It is more and more common in people's lives.With the promotion of quality of the life, requirement of the people to player plays audio-video
It is more and more.
During online playing audio-video, is influenced, made by network bandwidth factor, user's factor, service side's factor etc.
Obtaining multimedia streaming data provided by service side cannot meet the needs of users, and existing player can only play media stream and mention
The single-pass data of confession cannot mix and play two-way stream.For example, the data that service side provides include two-way media stream, it is more all the way
Media Stream A, including the video data that encryption algorithm is h264, resolution ratio is 480p and encryption algorithm be AAC, sample rate is
22.05KHz audio data；Another way media stream B, including the video data that encryption algorithm is h265, resolution ratio is 720p,
With the audio data that encryption algorithm is AAC, sample rate is 22.05KHz；Existing playback method can only individually play media stream
The audio, video data or individually play the audio, video data that media stream B includes that A includes, and cannot be h264 by encryption algorithm,
The audio data that the video data and encryption algorithm that resolution ratio is 480p are AAC, sample rate is 22.05KHz, which combines, to be played, or will
The audio data and encryption algorithm that encryption algorithm is AAC, sample rate is 22.05KHz be h265, the video counts that resolution ratio is 720p
It is played according to combination.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of audio and video playing method and device, plays two-way to realize to mix
The purpose of media stream.
In order to achieve the above objectives, the embodiment of the invention discloses a kind of audio and video playing method and devices.Technical solution is such as
Under:
In a first aspect, a kind of audio and video playing method provided in an embodiment of the present invention, comprising:
Obtain two-way media stream, wherein media stream includes target video subflow all the way, and another way media stream includes
Target audio subflow；
It will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；
The target video subflow and the target audio subflow are decoded and are rendered respectively；
The target video subflow and the target audio subflow after rendering is played simultaneously.
Preferably, the acquisition two-way media stream, comprising:
Two downloaders are called, download the two-way media stream respectively.
Preferably, described to be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way, it wraps
It includes:
Two resolvers are called, are parsed respectively to per media stream all the way, obtaining every media stream all the way includes
Video sub-streams and/or audio sub-stream.
Preferably, described that the target video subflow and the target audio subflow are decoded and are rendered respectively, packet
It includes:
It calls the first Video Decoder to be decoded the target video subflow, calls the first audio decoder to described
Target audio subflow is decoded；
It calls video renderer to render the target video subflow, calls sound renderer to the target audio
Subflow is rendered.
Preferably, the method also includes:
Receive the switching command for target video subflow or target audio subflow；
Judge in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video sub-streams or to be switched
Audio sub-stream；If it does not, obtaining the target comprising the to be switched video sub-streams or the to be switched audio sub-stream
Media stream；Video sub-streams and/or audio sub-stream that the destination multimedia stream includes are separated, to be switched view is obtained
Frequency subflow or to be switched audio sub-stream；
First Video Decoder is closed, the second Video Decoder corresponding with the to be switched video sub-streams is called
The to be switched video sub-streams are decoded, the to be switched video sub-streams after rendering simultaneously broadcast decoder；Or it closes
First audio decoder calls the second audio decoder corresponding with the to be switched audio sub-stream to described to be switched
Audio sub-stream be decoded, render and broadcast decoder after the to be switched audio sub-stream.
Preferably, before the to be switched audio sub-stream after rendering and broadcast decoder, further includes:
Anti- sonic boom processing is carried out to the decoded to be switched audio sub-stream.
Preferably, before the target video subflow and the target audio subflow are decoded and are rendered respectively,
The method also includes:
The target video subflow and the target audio subflow are packaged and cached.
Preferably, in calling the second Video Decoder corresponding with the to be switched video sub-streams to described to be switched
Video sub-streams are decoded, or call the second audio decoder corresponding with the to be switched audio sub-stream to described to be switched
Audio sub-stream be decoded before, the method also includes:
By currently playing not decoded audio sub-stream and the to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and the to be switched audio sub-stream group packet and caches.
Second aspect, a kind of audio and video display device provided in an embodiment of the present invention, comprising:
First obtains module, for obtaining two-way media stream, wherein and media stream includes target video subflow all the way,
Another way media stream includes target audio subflow；
Separation module, for will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；
First processing module, for being decoded simultaneously wash with watercolours to the target video subflow and the target audio subflow respectively
Dye；
Playing module, for the target video subflow and the target audio subflow after rendering to be played simultaneously.
Preferably, described first module is obtained, is specifically used for calling two downloaders, downloads the two-way multimedia respectively
Stream.
Preferably, the separation module specifically calls two resolvers, parses respectively to per media stream all the way,
It obtains per the video sub-streams and/or audio sub-stream that media stream includes all the way.
Preferably, the first processing module, comprising:
First calling module calls for calling the first Video Decoder to be decoded the target video subflow
One audio decoder is decoded the target audio subflow；
Second calling module calls audio wash with watercolours for calling video renderer to render the target video subflow
Dye device renders the target audio subflow.
Preferably, further includes:
Switching command receiver module, for receiving the switching command for being directed to target video subflow or target audio subflow；
Judgment module, for judging in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video
Stream or to be switched audio sub-stream；
Second obtains module, in the absence of being for the judging result when the judgment module, obtains comprising described to be cut
The destination multimedia stream of the video sub-streams or the to be switched audio sub-stream changed；The video for including by the destination multimedia stream
Subflow and/or audio sub-stream are separated, and to be switched video sub-streams or to be switched audio sub-stream are obtained；
Switching module is called corresponding with the to be switched video sub-streams for closing first Video Decoder
Second Video Decoder is decoded the to be switched video sub-streams, the to be switched view after rendering simultaneously broadcast decoder
Frequency subflow；Or first audio decoder is closed, call the second audio decoder corresponding with the to be switched audio sub-stream
Device is decoded the to be switched audio sub-stream, the to be switched audio sub-stream after rendering simultaneously broadcast decoder.
Preferably, described device further include:
Second processing module, for rendering and broadcast decoder after the to be switched audio sub-stream before, to decoding
The to be switched audio sub-stream afterwards carries out anti-sonic boom processing.
Preferably, described device further include:
Third processing module, for being decoded simultaneously to the target video subflow and the target audio subflow respectively
Before rendering, the target video subflow and the target audio subflow are packaged and cached.
Preferably, further includes:
Fourth processing module, for call corresponding with the to be switched video sub-streams the second Video Decoder to institute
It states to be switched video sub-streams to be decoded, or calls the second audio decoder pair corresponding with the to be switched audio sub-stream
Before the to be switched audio sub-stream is decoded,
By currently playing not decoded audio sub-stream and the to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and the to be switched audio sub-stream group packet and caches.
Using the embodiment of the present invention, by obtaining two-way media stream, media stream includes target video subflow all the way, separately
Media stream includes target audio subflow all the way；It respectively will be per the video sub-streams and/or audio sub-stream that media stream includes all the way
It is separated；Target video subflow and target audio subflow can be isolated in this way, respectively to target video subflow and target sound
Frequency subflow is decoded and renders；Target video subflow and target audio subflow after rendering is played simultaneously, so as to realize
Mixing plays the purpose of two-way media stream.
Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach all the above excellent simultaneously
Point.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the first flow diagram of audio and video playing method provided in an embodiment of the present invention；
Fig. 2 is second of flow diagram of audio and video playing method provided in an embodiment of the present invention；
Fig. 3 is the first structural schematic diagram of audio and video display device provided in an embodiment of the present invention；
Fig. 4 is second of structural schematic diagram of audio and video display device provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
In order to realize that mixing plays the purpose of two-way media stream, the embodiment of the invention provides a kind of audio-videos to broadcast
Put method and device.
A kind of audio and video playing method provided in an embodiment of the present invention is introduced first below.
As shown in FIG. 1, FIG. 1 is the first flow diagrams of audio and video playing method provided in an embodiment of the present invention, should
Method the following steps are included:
S101, two-way media stream is obtained, wherein media stream includes target video subflow, another way multimedia all the way
Stream includes target audio subflow；
In this step, this two-way media stream can be downloaded respectively by calling two different downloaders.It can also lead to
It crosses same downloader and creates two downloading examples, download this two-way media stream respectively.Wherein, target video subflow is to be played
Video sub-streams, target audio subflow is audio sub-stream to be played.
Here acquisition two-way media stream can carry out subsequent operation to be directed to the data downloaded in downloading；Certainly,
It is also possible to after two-way media stream whole downloading is cached to local, then carries out subsequent operation.
Media stream is the data flow for encapsulating video sub-streams and/or audio sub-stream, and network server is pre-stored more
Media Stream may include: the media stream comprising video sub-streams and audio sub-stream, the media stream comprising video sub-streams include sound
The media stream of frequency subflow.In practical applications, it can be stored according to the actual situation, conventional player can only play one
Road includes therefore the media stream of video sub-streams and audio sub-stream can store in the server comprising video sub-streams and audio
The media stream of subflow, and the media stream comprising audio sub-stream, specifically, video sub-streams include that encryption algorithm is H264, divides
Resolution is the video sub-streams of 1080P, encryption algorithm H264, the video sub-streams that encryption algorithm H265, resolution ratio are 720P are divided
Resolution is the video sub-streams of 720P, and encryption algorithm H264, resolution ratio are video sub-streams of 480P etc.；Audio sub-stream includes compiling
Code algorithm be EAC3, the audio sub-stream that sample rate is 48KH, the audio sub-stream that encryption algorithm EAC3, sample rate are 44.1KH,
The audio sub-stream that encryption algorithm is AAC, sample rate is 44.1KH, audio that encryption algorithm AAC, sample rate are 22.05KH
Stream etc., it is without being limited thereto.
It should be noted that all the way include the media stream of video sub-streams and audio sub-stream in the two-way stream of acquisition,
Another way is the media stream for including audio sub-stream, is played by the mixing of two-way media stream, and same video had both been able to achieve
Stream and the mixing broadcasting of the audio sub-streams of different audios, can be improved the experience of user, simultaneously as the data of audio sub-stream compared with
Small, shared bandwidth is little, will not flow all the way because of more downloadings and cause bandwidth nervous, so that playing slack problem.
S102, it will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；
When it is implemented, pre-setting multiple resolvers, carried out respectively for the media stream of different encapsulation format
Parsing.When the encapsulation format difference of the two-way media stream of acquisition, call corresponding two different resolvers respectively to two
Road media stream is parsed；When the encapsulation format of the two-way media stream of acquisition is identical, a corresponding resolver is called
It creates two parsing examples respectively to parse two-way media stream, only includes the media stream quilt of video sub-streams after parsing
Corresponding video sub-streams are parsed, only the media stream comprising audio sub-stream is resolved corresponding audio sub-stream, includes video
The media stream of subflow and audio sub-stream can be resolved corresponding video sub-streams and audio sub-stream.For example, the envelope of media stream
Filling format or extension suffix is .MOV, then calls the resolver for the parsing of such media stream.
S103, target video subflow and target audio subflow are decoded and are rendered respectively；
When it is implemented, before this step, can according to the timestamp of two-way media stream, by target video subflow and
Target audio subgroup packet, can not also organize packet, if not organizing packet, need to video sub-streams, the push-mechanism of audio sub-stream and sound
It draws synchronization mechanism to adjust, is to reach that requirement is played simultaneously.
Target video subflow, target audio subflow can also be cached before this, only need to set if organizing packet
A buffer area is found, if not organizing packet needs that a buffer area is respectively set for target video subflow, target audio subflow, delay
Deposit data can make to play more smooth.
This step when it is implemented, call the first Video Decoder target video subflow is decoded, call the first sound
Frequency decoder is decoded target audio subflow；
It calls video renderer to render target video subflow, sound renderer is called to carry out target audio subflow
Rendering.
Wherein, multiple Video Decoders and multiple audio decoders are pre-set, for example, Video Decoder includes H263
Video Decoder, H264 Video Decoder, H265 Video Decoder etc., audio decoder include AAC audio decoder, EAC3
Audio decoder etc., these Video Decoders belong to soft solution decoder, such as if be directly decoded using CPU
The video acceleration function that fruit uses is decoded, and belongs to hardware decoding；The CPU of these audio decoders, use is directly solved
Code, belongs to soft solution decoder.
S104, the target video subflow and target audio subflow being played simultaneously after rendering.
When this step is implemented, target video subflow and target audio subflow can be decoded respectively and rendering it
Before, target video subflow and target audio subflow are packaged and cached.
When it is implemented, being to be carried out target video subflow and target audio subflow according to the timestamp of two-way media stream
Group packet, it is so-called to be played simultaneously, it is primarily referred to as video sub-streams synchronization corresponding with the time of audio sub-stream.
Using embodiment shown in FIG. 1, by obtaining two-way media stream, media stream includes target video all the way
Stream, another way media stream include target audio subflow；It respectively will be per the video sub-streams and/or audio that media stream includes all the way
Subflow is separated；Target video subflow and target audio subflow can be isolated in this way, respectively to target video subflow and mesh
Mark audio sub-stream is decoded and renders；Target video subflow and target audio subflow after rendering is played simultaneously, so as to
Realize that mixing plays the purpose of two-way media stream.
It is second of flow diagram of audio and video playing method provided in an embodiment of the present invention, this hair referring to Fig. 2, Fig. 2
Increase following four step on the basis of bright embodiment illustrated in fig. 2 embodiment shown in Fig. 1:
S105, the switching command for being directed to target video subflow or target audio subflow is received；
In this step, in the playing process of audio-video, receive for target video subflow or target audio subflow
Switching command, switching command is that user is issued by client, when it is implemented, can be a variety of to user's displaying in client
Audio sub-stream switching push button, audio sub-stream or different special efficacy audio sub-streams for different sound quality, is arranged different buttons, example
Such as, the button for switching to child's voice audio sub-stream switches to the button etc. of hoarse characteristic audio sub-stream.It, can for video sub-streams
To show various video subflow switching push button to user in client, pressed for the difference of the video sub-streams setting of different qualities
Button switches to the button of the video sub-streams of SD picture, switches to for example, switching to the button of the video sub-streams of high definition picture
Encryption algorithm is H264, resolution ratio is button of video sub-streams of 720P etc..
S106, judge in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video sub-streams or to be cut
The audio sub-stream changed；If it does not, first carrying out step step S107, then step S108 is executed, if it does, directly executing step
Rapid S108.
In this step, if there are to be switched video sub-streams or to be switched audio in the media stream parsed
Stream, then can directly use, if it does not, just needing to download, parsing to obtain to be switched video sub-streams or to be switched
Audio sub-stream.
S107, the destination multimedia stream comprising to be switched video sub-streams or to be switched audio sub-stream is obtained；By target
The video sub-streams and/or audio sub-stream that media stream includes are separated, and to be switched video sub-streams or to be switched sound are obtained
Frequency subflow；
To be switched video sub-streams, to be switched audio sub-stream are not present in step S106, need to be implemented this step again
It obtains, when it is implemented, downloader can be called, downloading includes the mesh of to be switched video sub-streams or to be switched audio sub-stream
Mark media stream；Then, resolver is recalled to be divided the video sub-streams and/or audio sub-stream that destination multimedia stream includes
From to obtain to be switched video sub-streams or to be switched audio sub-stream.
S108, the first Video Decoder is closed, calls the second Video Decoder pair corresponding with to be switched video sub-streams
To be switched video sub-streams are decoded, the to be switched video sub-streams after rendering simultaneously broadcast decoder；Or close the first audio solution
Code device, calls the second audio decoder corresponding with to be switched audio sub-stream to be decoded to be switched audio sub-stream, wash with watercolours
To be switched audio sub-stream after contaminating simultaneously broadcast decoder.
There are to be switched video sub-streams or audio sub-stream, or regain to be switched video sub-streams or
In the case where to be switched audio sub-stream, following steps are executed, the first Video Decoder is closed and closes the decoding that will be replaced
Device is called for decoding the second Video Decoder of to be switched video sub-streams, for example, what is played originally is that encryption algorithm is
H265, resolution ratio are 720P video sub-streams, and to be switched is the video sub-streams that encryption algorithm is H264, resolution ratio is 1080P, are closed
The former corresponding Video Decoder is closed, calls Video Decoder corresponding with the latter, to encryption algorithm be H264, resolution ratio is
The video sub-streams of 1080P are decoded.
It closes the first audio decoder and closes the decoder that will be replaced, call for decoding to be switched audio sub-stream
The second audio decoder, to be switched is the audio sub-stream that encryption algorithm is EAC3, sample rate is 48KH, close the former correspond to
Audio decoder, corresponding with the latter audio decoder is called, to the audio that encryption algorithm is AAC, sample rate is 44.1KH
Subflow is decoded.
When it is implemented, the decoding for video sub-streams can solve decoder, both sides using soft solution decoder or firmly
The switching of formula can satisfy the switching of the video sub-streams less than frame per second 50fps all within 20S；For the switching of audio sub-stream,
Soft solution decoder can be used, switching time within 1ms, can satisfy audio decoder demand.
Video sub-streams, audio sub-stream independently of each other, are independent of each other in switching.Here switching method belongs to lightweight
Switching can guarantee that smooth playing is experienced, and picture will not will not generate serious flicker problem by clear black.
In addition, can individually close or switch video sub-streams or audio therein since two-way subflow is to separate to play
Subflow closes the downloading of video sub-streams if it is under audio mode in time, can be to avoid using excessive flow.
Before the to be switched audio sub-stream after rendering and broadcast decoder, audio and video playing provided in an embodiment of the present invention
Method further include:
Anti- sonic boom processing is carried out to decoded to be switched audio sub-stream.
After audio decoder, two different audio sub-streams may generate sonic boom phenomenon, therefore, it is necessary to right in switching
Decoded audio sub-stream is post-processed, and specific sonic boom processing method belongs to the prior art, and this will not be repeated here.
The second Video Decoder corresponding with to be switched video sub-streams is being called to solve to be switched video sub-streams
Code, or the second audio decoder corresponding with to be switched audio sub-stream is called to be decoded it to be switched audio sub-stream
Before, audio and video playing method provided in an embodiment of the present invention further include:
By currently playing not decoded audio sub-stream and to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and to be switched audio sub-stream group packet and caches.
When it is implemented, according to the timestamp of currently playing not decoded audio sub-stream and to be switched video sub-streams,
This two-way subflow is packaged, meanwhile, it can be by the way of generating empty data packet, the management mould of notice management decoder
Block, video sub-streams or the changed information of audio sub-stream can save video sub-streams or audio sub-stream that judgement plays in this way
The step of whether changing, it is possible to reduce the consumption of CPU.By way of caching, playing process can be made more smooth.
Corresponding with above-mentioned embodiment of the method, the embodiment of the present invention also provides a kind of audio and video display device.
Referring to Fig. 3, the first structural schematic diagram of this audio and video display device provided in an embodiment of the present invention of Fig. 3, comprising:
First obtains module 21, for obtaining two-way media stream, wherein media stream includes target video all the way
Stream, another way media stream include target audio subflow；
Separation module 22, for will be divided respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way
From；
First processing module 23, for target video subflow and target audio subflow to be decoded and rendered respectively；
Playing module 24, for target video subflow and target audio subflow after rendering to be played simultaneously.
Using embodiment shown in Fig. 3, by obtaining two-way media stream, media stream includes target video all the way
Stream, another way media stream include target audio subflow；It respectively will be per the video sub-streams and/or audio that media stream includes all the way
Subflow is separated；Target video subflow and target audio subflow can be isolated in this way, respectively to target video subflow and mesh
Mark audio sub-stream is decoded and renders；Target video subflow and target audio subflow after rendering is played simultaneously, so as to
Realize that mixing plays the purpose of two-way media stream.
In the present embodiment, first obtains module 21, is specifically used for calling two downloaders, downloads two-way multimedia respectively
Stream.
In the present embodiment, separation module 22 is specifically used for calling two resolvers, carry out respectively to per media stream all the way
Parsing is obtained per the video sub-streams and/or audio sub-stream that media stream includes all the way.
In the present embodiment, first processing module 23, comprising:
First calling module calls the first sound for calling the first Video Decoder to be decoded target video subflow
Frequency decoder is decoded target audio subflow；
Second calling module calls sound renderer for calling video renderer to render target video subflow
Target audio subflow is rendered.
It is second of structural schematic diagram of audio and video display device provided in an embodiment of the present invention, this hair referring to Fig. 4, Fig. 4
Increase following four module on the basis of bright embodiment illustrated in fig. 2 embodiment shown in Fig. 3:
Switching command receiver module 25, for receiving the switching command for being directed to target video subflow or target audio subflow；
Judgment module 26, for judging in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video
Subflow or to be switched audio sub-stream；
Second obtains module 27, in the absence of being for the judging result when the judgment module, obtains comprising to be switched
Video sub-streams or to be switched audio sub-stream destination multimedia stream；The video sub-streams that include by destination multimedia stream and/or
Audio sub-stream is separated, and to be switched video sub-streams or to be switched audio sub-stream are obtained；
Switching module 28 calls the second view corresponding with to be switched video sub-streams for closing the first Video Decoder
Frequency decoder is decoded to be switched video sub-streams, the to be switched video sub-streams after rendering simultaneously broadcast decoder；Or it closes
First audio decoder, call corresponding with to be switched audio sub-stream the second audio decoder to be switched audio sub-stream into
Row decoding, the to be switched audio sub-stream after rendering simultaneously broadcast decoder.
In the present embodiment, further includes:
Second processing module, for rendering and broadcast decoder after to be switched audio sub-stream before, to decoded
To be switched audio sub-stream carries out anti-sonic boom processing.
In the present embodiment, further includes:
Third processing module, for being decoded respectively to target video subflow and target audio subflow and rendering it
Before, target video subflow and target audio subflow are packaged and cached.
In the present embodiment, further includes:
Fourth processing module, for call corresponding with to be switched video sub-streams the second Video Decoder to be switched
Video sub-streams be decoded, or call corresponding with to be switched audio sub-stream the second audio decoder to be switched audio
Before subflow is decoded,
By currently playing not decoded audio sub-stream and to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and to be switched audio sub-stream group packet and caches.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device reality
For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method
Part explanation.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention
It is interior.
Claims (12)
1. a kind of audio and video playing method characterized by comprising
Obtain two-way media stream, wherein media stream includes target video subflow all the way, and another way media stream includes target
Audio sub-stream；
It will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；
It calls the first Video Decoder to be decoded the target video subflow, calls the first audio decoder to the target
Audio sub-stream is decoded；
It calls video renderer to render the target video subflow, calls sound renderer to the target audio subflow
It is rendered；
The target video subflow and the target audio subflow after rendering is played simultaneously；
Receive the switching command for target video subflow or target audio subflow；
Judge in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video sub-streams or to be switched audio
Subflow；If it does not, obtaining the more matchmakers of target comprising the to be switched video sub-streams or the to be switched audio sub-stream
Body stream；Video sub-streams and/or audio sub-stream that the destination multimedia stream includes are separated, to be switched video is obtained
Stream or to be switched audio sub-stream；
First Video Decoder is closed, calls the second Video Decoder corresponding with the to be switched video sub-streams to institute
It states to be switched video sub-streams to be decoded, the to be switched video sub-streams after rendering simultaneously broadcast decoder；Or described in closing
First audio decoder calls the second audio decoder corresponding with the to be switched audio sub-stream to the to be switched sound
Frequency subflow is decoded, the to be switched audio sub-stream after rendering simultaneously broadcast decoder.
2. the method according to claim 1, wherein the acquisition two-way media stream, comprising:
Two downloaders are called, download the two-way media stream respectively.
3. the method according to claim 1, wherein described respectively will be per video that media stream includes all the way
Stream and/or audio sub-stream are separated, comprising:
Two resolvers are called, are parsed respectively to per media stream all the way, are obtained per the video that media stream includes all the way
Subflow and/or audio sub-stream.
4. the method according to claim 1, wherein the to be switched audio after rendering and broadcast decoder
Before subflow, further includes:
Anti- sonic boom processing is carried out to the decoded to be switched audio sub-stream.
5. the method according to claim 1, wherein respectively to the target video subflow and the target sound
Before frequency subflow is decoded and renders, the method also includes:
The target video subflow and the target audio subflow are packaged and cached.
6. the method according to claim 1, wherein calling corresponding with the to be switched video sub-streams the
Two Video Decoders are decoded the to be switched video sub-streams, or call corresponding with the to be switched audio sub-stream
Before second audio decoder is decoded the to be switched audio sub-stream, the method also includes:
By currently playing not decoded audio sub-stream and the to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and the to be switched audio sub-stream group packet and caches.
7. a kind of audio and video display device characterized by comprising
First obtains module, for obtaining two-way media stream, wherein media stream includes target video subflow all the way, another
Road media stream includes target audio subflow；
Separation module, for will be separated respectively per the video sub-streams and/or audio sub-stream that media stream includes all the way；
First processing module, comprising: the first calling module, for call the first Video Decoder to the target video subflow into
Row decoding, calls the first audio decoder to be decoded the target audio subflow；Second calling module, for calling video
Renderer renders the target video subflow, and sound renderer is called to render the target audio subflow；
Playing module, for the target video subflow and the target audio subflow after rendering to be played simultaneously；
Switching command receiver module, for receiving the switching command for being directed to target video subflow or target audio subflow；
Judgment module, for judging in isolated video sub-streams and audio sub-stream with the presence or absence of to be switched video sub-streams or
To be switched audio sub-stream；
Second obtains module, in the absence of being for the judging result when the judgment module, obtains comprising described to be switched
The destination multimedia stream of video sub-streams or the to be switched audio sub-stream；The video sub-streams for including by the destination multimedia stream
And/or audio sub-stream is separated, and to be switched video sub-streams or to be switched audio sub-stream are obtained；
Switching module is called and the to be switched video sub-streams corresponding second for closing first Video Decoder
Video Decoder is decoded the to be switched video sub-streams, to be switched video after rendering simultaneously broadcast decoder
Stream；Or first audio decoder is closed, call the second audio decoder pair corresponding with the to be switched audio sub-stream
The to be switched audio sub-stream is decoded, the to be switched audio sub-stream after rendering simultaneously broadcast decoder.
8. device according to claim 7, which is characterized in that described first obtains module, is specifically used for calling under two
Device is carried, downloads the two-way media stream respectively.
9. device according to claim 7, which is characterized in that the separation module specifically calls two resolvers, respectively
It parses, is obtained per the video sub-streams and/or audio sub-stream that media stream includes all the way to per media stream all the way.
10. device according to claim 7, which is characterized in that further include:
Second processing module, for rendering and broadcast decoder after the to be switched audio sub-stream before, to decoded
The to be switched audio sub-stream carries out anti-sonic boom processing.
11. device according to claim 7, which is characterized in that further include:
Third processing module, for the target video subflow and the target audio subflow to be decoded and rendered respectively
Before, the target video subflow and the target audio subflow are packaged and is cached.
12. device according to claim 7, which is characterized in that further include:
Fourth processing module, for call corresponding with the to be switched video sub-streams the second Video Decoder to it is described to
The video sub-streams of switching are decoded, or call the second audio decoder corresponding with the to be switched audio sub-stream to described
Before to be switched audio sub-stream is decoded,
By currently playing not decoded audio sub-stream and the to be switched video sub-streams group packet and cache；
Or it by currently playing not decoded video sub-streams and the to be switched audio sub-stream group packet and caches.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610780671.3A CN106375821B (en) | 2016-08-30 | 2016-08-30 | A kind of audio and video playing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610780671.3A CN106375821B (en) | 2016-08-30 | 2016-08-30 | A kind of audio and video playing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106375821A CN106375821A (en) | 2017-02-01 |
| CN106375821B true CN106375821B (en) | 2019-04-12 |
Family
ID=57900527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610780671.3A Active CN106375821B (en) | 2016-08-30 | 2016-08-30 | A kind of audio and video playing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106375821B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108055565A (en) * | 2017-12-26 | 2018-05-18 | 合肥中科云巢科技有限公司 | The reorientation method and device of audio and video |
| CN110072066B (en) * | 2019-05-30 | 2021-11-30 | 北京淳中科技股份有限公司 | Audio and video signal transmission device and system |
| CN112203116A (en) * | 2019-07-08 | 2021-01-08 | 腾讯科技（深圳）有限公司 | Video generation method, video playing method and related equipment |
| CN114710475B (en) * | 2022-04-11 | 2024-03-15 | 三星电子（中国）研发中心 | Streaming media audio fusion method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103167347A (en) * | 2011-12-12 | 2013-06-19 | 中兴通讯股份有限公司 | Mobile terminal television program audio and video rematching method and mobile terminal television program audio and video rematching system |
| CN103268763A (en) * | 2013-06-05 | 2013-08-28 | 广州市花都区中山大学国光电子与通信研究院 | Wireless media system based on synchronous audio extraction and real-time transmission |
| CN104333802A (en) * | 2013-12-13 | 2015-02-04 | 乐视网信息技术（北京）股份有限公司 | Video playing method and video player |
| CN104837052A (en) * | 2014-06-10 | 2015-08-12 | 腾讯科技（北京）有限公司 | Playing method of multimedia data and device |
-
2016
- 2016-08-30 CN CN201610780671.3A patent/CN106375821B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103167347A (en) * | 2011-12-12 | 2013-06-19 | 中兴通讯股份有限公司 | Mobile terminal television program audio and video rematching method and mobile terminal television program audio and video rematching system |
| CN103268763A (en) * | 2013-06-05 | 2013-08-28 | 广州市花都区中山大学国光电子与通信研究院 | Wireless media system based on synchronous audio extraction and real-time transmission |
| CN104333802A (en) * | 2013-12-13 | 2015-02-04 | 乐视网信息技术（北京）股份有限公司 | Video playing method and video player |
| CN104837052A (en) * | 2014-06-10 | 2015-08-12 | 腾讯科技（北京）有限公司 | Playing method of multimedia data and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106375821A (en) | 2017-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106375821B (en) | A kind of audio and video playing method and device | |
| US10805650B2 (en) | Signaling important video information in network video streaming using mime type parameters | |
| JP5964972B2 (en) | Stream multimedia data from multiple sources | |
| JP6474830B2 (en) | Continuous multi-period content processing | |
| CN103843301B (en) | The switching between expression during the network crossfire of decoded multi-medium data | |
| KR102168596B1 (en) | Low latency video streaming | |
| CN101321275B (en) | System and method for processing video stream | |
| JP5788101B2 (en) | Network streaming of media data | |
| US20070039025A1 (en) | Method for application sharing | |
| KR101224256B1 (en) | Method and apparatus for controlling scene structure of multiple channel for a mobile terminal based on light application scene representation | |
| JP5524193B2 (en) | Media stream processing | |
| KR20110089309A (en) | Method and apparatus for interleaving a data block | |
| CN106303682B (en) | Method, apparatus, terminal and the server of channel switching | |
| EP3358849B1 (en) | Transmission device, transmission method, reception device, and reception method | |
| US20130204973A1 (en) | Method for transmitting a scalable http stream for natural reproduction upon the occurrence of expression-switching during http streaming | |
| JP4308555B2 (en) | Receiving device and information browsing method | |
| WO2011075876A1 (en) | Picture in picture for mobile tv | |
| KR20160086863A (en) | Signal transmission and reception apparatus and signal transmission and reception method for providing trick play service | |
| CN103635938B (en) | For processing the method for video streaming data, streaming client in streaming client | |
| CN102447950A (en) | Method and equipment for video play | |
| Chernock et al. | Next-generation broadcast television: ATSC 3.0 [standards in a nutshell] | |
| KR101700626B1 (en) | Multi angle view processing apparatus | |
| US20220321946A1 (en) | Video system | |
| KR102349451B1 (en) | The method for transmitting or receiving multimedia and apparatus thereof | |
| JP4794640B2 (en) | Transmitting apparatus and media data transmitting method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |