CN108174242A - Live data multiplexing method, device, storage medium and equipment - Google Patents
Live data multiplexing method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN108174242A CN108174242A CN201810018715.8A CN201810018715A CN108174242A CN 108174242 A CN108174242 A CN 108174242A CN 201810018715 A CN201810018715 A CN 201810018715A CN 108174242 A CN108174242 A CN 108174242A
- Authority
- CN
- China
- Prior art keywords
- frame
- data packet
- data
- lyrics
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23614—Multiplexing of additional data and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/488—Data services, e.g. news ticker
- H04N21/4884—Data services, e.g. news ticker for displaying subtitles
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
Abstract
The present invention relates to live streaming plug-flow technical field more particularly to live data multiplexing method, device, storage medium and equipment, and live data includes video data, audio data and lyrics data, including step S1：Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet are separately encoded out with preset frame rate；Step S2：Time order and function sequence according to timestamp stores respectively to be obtained in video buffer queue, Audio Buffer queue and lyrics buffering queue；Step S3：The first frame video data packet positioned at queue first place, first frame packets of audio data and first frame lyrics data packet are extracted from video buffer queue, Audio Buffer queue and lyrics buffering queue respectively；Step S4：Extract the data packet of timestamp minimum in first frame video data packet, first frame packets of audio data and first frame lyrics data packet, and the data packet for timestamp minimum adds frame originating point information and carries out live transmission, jumps to step S3 up to the equal end of transmission of data packet in all queues.
Description
Technical field
The present invention relates to live streaming plug-flow technical field more particularly to live data multiplexing method, device, storage medium and set
It is standby.
Background technology
With the fast development of science and technology, more and more electronic multimedia technologies are applied to daily life
In, wherein, net cast is exactly the emerging application of one of which.Net cast can penetrate network system the same time in difference
Intercommunion platform broadcast live in a manner of video signal, have the characteristics that intuitive, quick, interactivity is strong.
There is currently a kind of net cast modes, that is, main broadcaster plays song by net cast, and the lyrics of song also can be same
Step live streaming is to spectators.For how synchronous lyrics being broadcast live to spectators, the prior art is by the way of：The lyrics are superimposed upon and are regarded
Frequency encodes, and be transmitted in encoded together in flowing, however is declined the clarity for leading to the lyrics using aforesaid way.
Invention content
In view of the above problems, it is proposed that the present invention overcomes the above problem in order to provide one kind or solves at least partly
State live data multiplexing method, device, storage medium and the equipment of problem.
The embodiment of the present invention provides a kind of live data multiplexing method, and the live data includes video data, audio number
According to lyrics data corresponding with the audio data, the method includes：
Step S1：The video data, the audio data and the lyrics data are encoded into respectively with preset frame rate
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the audio number
Timestamp is carried according to packet and the lyrics data Bao Jun；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio
Data packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is carried from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Take out first frame video data packet, first frame packets of audio data and the first frame lyrics data packet positioned at queue first place；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics
The size of the timestamp of data packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame head
Information is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step S3 up to the number in all queues
According to the equal end of transmission of packet.
Preferably, the queue type of the video buffer queue, the Audio Buffer queue and the lyrics buffering queue
It is First Input First Output.
Preferably, the size of data volume that the frame originating point information includes frame type, frame time stamp and frame include.
Preferably, before step S3, the method further includes：
The one-frame video data packet positioned at queue first place is extracted from the video buffer queue, and is the frame video counts
Frame originating point information is added according to packet, transmission is broadcast live to the video data packet for being added with frame originating point information.
The embodiment of the present invention provides a kind of live data multiplexer, and the live data includes video data, audio number
According to lyrics data corresponding with the audio data, described device includes：
Coding module, for performing step S1：With preset frame rate respectively by the video data, the audio data and institute
It states lyrics data and is encoded into multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video counts
Timestamp is carried according to packet, the packets of audio data and the lyrics data Bao Jun；
Memory module, for performing step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video number
According to packet, the multiframe packets of audio data and the multiframe lyrics data packet store to video buffer queue, Audio Buffer queue and
In lyrics buffering queue；
Extraction module, for performing step S3：Respectively from the video buffer queue, the Audio Buffer queue and described
First frame video data packet, first frame packets of audio data and the first frame song positioned at queue first place are extracted in lyrics buffering queue
Word data packet；
First transmission module, for performing step S4：Compare the first frame video data packet, the first frame audio number
According to the size of packet and the timestamp of the first frame lyrics data packet, the data packet of timestamp minimum is extracted, and is timestamp
Minimum data packet addition frame originating point information, is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step
S3 is up to the equal end of transmission of data packet in all queues.
Preferably, the queue type of the video buffer queue, the Audio Buffer queue and the lyrics buffering queue
It is First Input First Output.
Preferably, the size of data volume that the frame originating point information includes frame type, frame time stamp and frame include.
Preferably, described device further includes：
Second transmission module, for extracting the one-frame video data positioned at queue first place from the video buffer queue
Packet, and frame originating point information is added for the frame video data packet, transmission is broadcast live to the video data packet for being added with frame originating point information.
The embodiment of the present invention provides a kind of computer readable storage medium, is stored thereon with computer program, the program quilt
Processor realizes following steps when performing：
Step S1：Video data, audio data and lyrics data are encoded by multi-frame video data with preset frame rate respectively
Packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the packets of audio data and the song
Word data packet carries timestamp；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio
Data packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is carried from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Take out first frame video data packet, first frame packets of audio data and the first frame lyrics data packet positioned at queue first place；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics
The size of the timestamp of data packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame head
Information is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step S3 up to the number in all queues
According to the equal end of transmission of packet.
The embodiment of the present invention provides a kind of computer equipment, including memory, processor and storage on a memory and can
The computer program run on a processor, the processor realize following steps when performing described program：
Step S1：Video data, audio data and lyrics data are encoded by multi-frame video data with preset frame rate respectively
Packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the packets of audio data and the song
Word data packet carries timestamp；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio
Data packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is carried from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Take out first frame video data packet, first frame packets of audio data and the first frame lyrics data packet positioned at queue first place；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics
The size of the timestamp of data packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame head
Information is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step S3 up to the number in all queues
According to the equal end of transmission of packet.
One or more of embodiment of the present invention technical solution, has at least the following technical effects or advantages：
The live data multiplexing method of the application, live data include video data, audio data and with the audio number
According to corresponding lyrics data, the application is by performing step S1：With preset frame rate respectively by video data, audio data and the lyrics
Data encoding into multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, video data packet, audio
Data packet and lyrics data Bao Jun carry timestamp, then perform step S2：According to the time order and function sequence of timestamp, difference
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet are stored to video buffer queue, Audio Buffer team
In row and lyrics buffering queue, step S3 is then performed：Respectively team is buffered from video buffer queue, Audio Buffer queue and the lyrics
The first frame video data packet positioned at queue first place, first frame packets of audio data and first frame lyrics data packet are extracted in row,
Then step S4 is performed：Compare the time of first frame video data packet, first frame packets of audio data and first frame lyrics data packet
The size of stamp extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame originating point information, to being added with
The data packet of frame originating point information is broadcast live transmission, jump to step S3 until all queues in the equal end of transmission of data packet, from
And after being broadcast live transmission through the above way, improve the display resolution of the lyrics.
Description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this field
Technical staff will become clear.Attached drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, identical component is represented with identical reference pattern.In the accompanying drawings：
Fig. 1 shows the flow chart of the live data multiplexing method in the embodiment of the present invention；
Fig. 2 shows the structure charts of the live data multiplexer in the embodiment of the present invention；
Fig. 3 shows the entity structure schematic diagram of the computer equipment in the embodiment of the present invention.
Specific embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Completely it is communicated to those skilled in the art.
The embodiment of the present application provides a kind of live data multiplexing method, as shown in Figure 1, live data include video data,
Audio data and lyrics data corresponding with the audio data, the live data multiplexing method are applied to main broadcaster end, Zhu Boduan
Electronic equipment be the live data multiplexing method executive agent, the method includes：
Step S1：Video data, audio data and lyrics data are encoded by multi-frame video data with preset frame rate respectively
Packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, video data packet, packets of audio data and lyrics data Bao Jun take
With timestamp；
Step S2：According to timestamp time order and function sequence, respectively by multi-frame video data packet, multiframe packets of audio data and
Multiframe lyrics data packet is stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is extracted from video buffer queue, Audio Buffer queue and lyrics buffering queue positioned at queue respectively
The first first frame video data packet, first frame packets of audio data and first frame lyrics data packet；
Step S4：Compare the time of first frame video data packet, first frame packets of audio data and first frame lyrics data packet
The size of stamp extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame originating point information, to being added with
The data packet of frame originating point information is broadcast live transmission, jumps to step S3 up to the equal end of transmission of data packet in all queues.
For step S1, frame per second can in advance be set according to actual conditions, for example, preset frame rate is per second 60
Frame.Further, the application respectively will encode video data, audio data and lyrics data with preset frame rate, that is, with pre-
If frame per second encodes video data, multi-frame video data packet is obtained, audio data is encoded with preset frame rate, obtained
Multiframe packets of audio data, encodes lyrics data with preset frame rate, obtains multiframe lyrics data packet, wherein, multi-frame video
Data packet includes multi-frame video data, and multiframe packets of audio data includes multiframe audio data, and multiframe lyrics data packet includes multiframe
Lyrics data.For example, being encoded with the video data of a length of 1 minute during the preset frame rate pair of 60 frame per second, then can obtain
360 frame video data packets.Further, the size of data volume included per frame data packet can be different.Meanwhile video data packet,
Packets of audio data and lyrics data Bao Jun carry timestamp, for example, the timestamp that the first video requency frame data packet carries is 00:
01, the timestamp that the second frame video data packet carries is 00:02.
For step S2, according to the time order and function sequence of timestamp, multi-frame video data packet is stored to video and is delayed
It rushes in queue, multiframe packets of audio data is stored into Audio Buffer queue, multiframe lyrics data packet is stored to the lyrics and is buffered
In queue.The queue type of video buffer queue, Audio Buffer queue and lyrics buffering queue is First Input First Output.First enter
The queue of first dequeue passes in and out mode：The data packet being introduced into first is extracted and is retired from office, and and then just extracts data packet below.
By taking video buffer queue as an example, in video buffer queue, timestamp 00:It is 00 that 01 video data packet, which is located at timestamp,:
Before 02 video data packet, when extracting data packet, first extraction time stamp is 00:01 video data packet, then when extracting
Between stamp for 00:02 video data packet.
Usually, for the live TV stream at main broadcaster end is pushed to spectators, it is necessary first to one-frame video data packet is pushed, because
This, before step S3, the method further includes：
The one-frame video data packet positioned at queue first place is extracted from video buffer queue, and is the frame video data packet
Frame originating point information is added, transmission is broadcast live to the video data packet for being added with frame originating point information.Wherein, frame originating point information includes frame class
The size of data volume that type, frame time stamp and frame include.
Then step S3 and step S4 is performed, specifically, is extracted from video buffer queue positioned at queue first place
First frame video data packet extracts the first frame packets of audio data positioned at queue first place, from the lyrics from Audio Buffer queue
The first frame lyrics data packet positioned at queue first place is extracted in buffering queue, then compares first frame video data packet, first
The size of the timestamp of frame packets of audio data and first frame lyrics data packet, extracts the data packet of timestamp minimum, and for when
Between stab minimum data packet addition frame originating point information, transmission is broadcast live to the data packet for being added with frame originating point information, is then jumped
Step S3 is gone back to, continues to extract the first frame video data packet positioned at queue first place from video buffer queue, delay from audio
The first frame packets of audio data extracted positioned at queue first place is rushed in queue, is extracted from lyrics buffering queue positioned at queue head
The first frame lyrics data packet of position, is further continued for performing follow-up comparison and transmission process, until the data packet in all queues passes
It is totally lost complete.
Based on same inventive concept, the embodiment of the present application also provides a kind of live data multiplexer, the live data
Comprising video data, audio data and lyrics data corresponding with the audio data, as shown in Fig. 2, described device includes：
Coding module 201, for performing step S1：With preset frame rate respectively by the video data, the audio data
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet are encoded into the lyrics data, wherein, it is described to regard
Frequency data packet, the packets of audio data and the lyrics data Bao Jun carry timestamp；
Memory module 202, for performing step S2：According to the time order and function sequence of timestamp, the multiframe is regarded respectively
Frequency data packet, the multiframe packets of audio data and the multiframe lyrics data packet are stored to video buffer queue, Audio Buffer team
In row and lyrics buffering queue；
Extraction module 203, for performing step S3：Respectively from the video buffer queue, the Audio Buffer queue and
The first frame video data packet positioned at queue first place, first frame packets of audio data and first are extracted in the lyrics buffering queue
Frame lyrics data packet；
First transmission module 204, for performing step S4：Compare the first frame video data packet, the first frame sound
The size of the timestamp of frequency data packet and the first frame lyrics data packet, extracts the data packet of timestamp minimum, and for when
Between stab minimum data packet addition frame originating point information, transmission is broadcast live to the data packet for being added with frame originating point information, is jumped to
Step S3 is up to the equal end of transmission of data packet in all queues.
Preferably, the queue type of the video buffer queue, the Audio Buffer queue and the lyrics buffering queue
It is First Input First Output.
Preferably, the size of data volume that the frame originating point information includes frame type, frame time stamp and frame include.
Preferably, described device further includes：
Second transmission module, for extracting the one-frame video data positioned at queue first place from the video buffer queue
Packet, and frame originating point information is added for the frame video data packet, transmission is broadcast live to the video data packet for being added with frame originating point information.
Based on same inventive concept, the embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with
Computer program, the program realize following steps when being executed by processor：
Step S1：The video data, the audio data and the lyrics data are encoded into respectively with preset frame rate
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the audio number
Timestamp is carried according to packet and the lyrics data Bao Jun；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio
Data packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is carried from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Take out first frame video data packet, first frame packets of audio data and the first frame lyrics data packet positioned at queue first place；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics
The size of the timestamp of data packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame head
Information is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step S3 up to the number in all queues
According to the equal end of transmission of packet.
The embodiment of the present invention additionally provides a kind of computer equipment, as shown in figure 3, for convenience of description, illustrate only with
The relevant part of the embodiment of the present invention, particular technique details do not disclose, please refer to present invention method part.The calculating
Machine equipment can be include mobile phone, tablet computer, PDA (Personal Digital Assistant, personal digital assistant),
The arbitrary terminal devices such as POS (Point of Sales, point-of-sale terminal), vehicle-mounted computer, by computer equipment for for mobile phone：
Fig. 3 is illustrated that the block diagram with the relevant part-structure of computer equipment provided in an embodiment of the present invention.Reference chart
3, which includes：Memory 301 and processor 302.It will be understood by those skilled in the art that the meter shown in Fig. 3
Calculation machine equipment structure does not form the restriction to computer equipment, can include than illustrating more or fewer components or group
Close certain components or different components arrangement.
Each component parts of computer equipment is specifically introduced with reference to Fig. 3：
Memory 301 can be used for storage software program and module, and processor 302 is stored in memory 301 by operation
Software program and module, so as to perform various functions application and data processing.Memory 301 can mainly include storage journey
Sequence area and storage data field, wherein, storing program area can storage program area, the application program (ratio needed at least one function
Such as sound-playing function, image player function) etc.；Storage data field can store data (such as audio data, phone directory etc.)
Deng.In addition, memory 301 can include high-speed random access memory, nonvolatile memory can also be included, for example, at least
One disk memory, flush memory device or other volatile solid-state parts.
Processor 302 is the control centre of computer equipment, by running or performing the software being stored in memory 301
Program and/or module and calling are stored in the data in memory 301, perform various functions and handle data.Optionally,
Processor 302 may include one or more processing units；Preferably, processor 302 can integrate application processor and modulation /demodulation
Processor, wherein, the main processing operation system of application processor, user interface and application program etc., modem processor master
Handle wireless communication.
In embodiments of the present invention, the processor 302 included by the computer equipment can be with the following functions：
Step S1：The video data, the audio data and the lyrics data are encoded into respectively with preset frame rate
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the audio number
Timestamp is carried according to packet and the lyrics data Bao Jun；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio
Data packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is carried from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Take out first frame video data packet, first frame packets of audio data and the first frame lyrics data packet positioned at queue first place；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics
The size of the timestamp of data packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame head
Information is broadcast live transmission to the data packet for being added with frame originating point information, jumps to step S3 up to the number in all queues
According to the equal end of transmission of packet.
In short, the live data multiplexing method of the application, live data include video data, audio data and with the sound
Frequency is according to corresponding lyrics data, and the application is by performing step S1：With preset frame rate respectively by video data, audio data and
Lyrics data is encoded into multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, video data packet,
Packets of audio data and lyrics data Bao Jun carry timestamp, then perform step S2：According to timestamp time order and function sequence,
Multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet are stored to video buffer queue, audio and delayed respectively
It rushes in queue and lyrics buffering queue, then performs step S3：Delay respectively from video buffer queue, Audio Buffer queue and the lyrics
The first frame video data packet, first frame packets of audio data and first frame lyrics data extracted positioned at queue first place is rushed in queue
Packet then performs step S4：Compare first frame video data packet, first frame packets of audio data and first frame lyrics data packet when
Between the size stabbed, the data packet of timestamp minimum is extracted, and the data packet for timestamp minimum adds frame originating point information, to adding
The data packet for having frame originating point information is broadcast live transmission, jump to step S3 until all queues in the equal end of transmission of data packet,
So as to after being broadcast live transmission through the above way, improve the display resolution of the lyrics.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with teaching based on this.As described above, required by constructing this kind of system
Structure be obvious.In addition, the present invention is not also directed to any certain programmed language.It should be understood that it can utilize various
Programming language realizes the content of invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the specification provided in this place, numerous specific details are set forth.It is to be appreciated, however, that the implementation of the present invention
Example can be put into practice without these specific details.In some instances, well known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of each inventive aspect,
Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. required guarantor
Shield the present invention claims the more features of feature than being expressly recited in each claim.More precisely, as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim is in itself
Separate embodiments all as the present invention.
Those skilled in the art, which are appreciated that, to carry out adaptively the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.It can be the module or list in embodiment
Member or component be combined into a module or unit or component and can be divided into addition multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it may be used any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification is (including adjoint power
Profit requirement, abstract and attached drawing) disclosed in each feature can be by providing the alternative features of identical, equivalent or similar purpose come generation
It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments in this include institute in other embodiments
Including certain features rather than other feature, but the combination of the feature of different embodiment means in the scope of the present invention
Within and form different embodiments.For example, in the following claims, embodiment claimed it is arbitrary it
One mode can use in any combination.
The all parts embodiment of the present invention can be with hardware realization or to be run on one or more processor
Software module realize or realized with combination thereof.It will be understood by those of skill in the art that it can use in practice
Microprocessor or digital signal processor (DSP) are realized in gateway according to embodiments of the present invention, proxy server, system
Some or all components some or all functions.The present invention is also implemented as performing side as described herein
The some or all equipment or program of device (for example, computer program and computer program product) of method.It is such
Realizing the program of the present invention can may be stored on the computer-readable medium or can have the shape of one or more signal
Formula.Such signal can be downloaded from internet website to be obtained either providing or with any other shape on carrier signal
Formula provides.
It should be noted that the present invention will be described rather than limits the invention, and ability for above-described embodiment
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference mark between bracket should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real
It is existing.If in the unit claim for listing equipment for drying, several in these devices can be by same hardware branch
To embody.Word, second and the use of third etc. do not indicate that any sequence.These words can be construed to title.
Claims (10)
1. a kind of live data multiplexing method, which is characterized in that the live data include video data, audio data and with institute
The corresponding lyrics data of audio data is stated, the method includes：
Step S1：The video data, the audio data and the lyrics data are encoded by multiframe with preset frame rate respectively
Video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the packets of audio data
Timestamp is carried with the lyrics data Bao Jun；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio data
Packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is extracted from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Positioned at the first frame video data packet of queue first place, first frame packets of audio data and first frame lyrics data packet；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics data
The size of the timestamp of packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame originating point information,
Transmission is broadcast live to the data packet for being added with frame originating point information, jumps to step S3 until the data packet in all queues is equal
End of transmission.
2. live data multiplexing method as described in claim 1, which is characterized in that the video buffer queue, the audio
The queue type of buffering queue and the lyrics buffering queue is First Input First Output.
3. live data multiplexing method as described in claim 1, which is characterized in that the frame originating point information includes frame type, frame
The size for the data volume that timestamp and frame include.
4. live data multiplexing method as described in claim 1, which is characterized in that before step S3, the method is also wrapped
It includes：
The one-frame video data packet positioned at queue first place is extracted from the video buffer queue, and is the frame video data packet
Frame originating point information is added, transmission is broadcast live to the video data packet for being added with frame originating point information.
5. a kind of live data multiplexer, which is characterized in that the live data include video data, audio data and with institute
The corresponding lyrics data of audio data is stated, described device includes：
Coding module, for performing step S1：With preset frame rate respectively by the video data, the audio data and the song
Word data encoding into multi-frame video data packet, multiframe packets of audio data and multiframe lyrics data packet, wherein, the video data
Packet, the packets of audio data and the lyrics data Bao Jun carry timestamp；
Memory module, for performing step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data
Packet, the multiframe packets of audio data and the multiframe lyrics data packet are stored to video buffer queue, Audio Buffer queue and song
In word buffering queue；
Extraction module, for performing step S3：Respectively from the video buffer queue, the Audio Buffer queue and the lyrics
The first frame video data packet positioned at queue first place, first frame packets of audio data and first frame lyrics number are extracted in buffering queue
According to packet；
First transmission module, for performing step S4：Compare the first frame video data packet, the first frame packets of audio data
With the size of the timestamp of the first frame lyrics data packet, the data packet of timestamp minimum is extracted, and minimum for timestamp
Data packet addition frame originating point information, transmission is broadcast live to the data packet for being added with frame originating point information, it is straight to jump to step S3
The equal end of transmission of data packet into all queues.
6. live data multiplexer as claimed in claim 5, which is characterized in that the video buffer queue, the audio
The queue type of buffering queue and the lyrics buffering queue is First Input First Output.
7. live data multiplexer as claimed in claim 5, which is characterized in that the frame originating point information includes frame type, frame
The size for the data volume that timestamp and frame include.
8. live data multiplexer as claimed in claim 5, which is characterized in that described device further includes：
Second transmission module, for extracting the one-frame video data packet positioned at queue first place from the video buffer queue,
And frame originating point information is added for the frame video data packet, transmission is broadcast live to the video data packet for being added with frame originating point information.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
Following steps are realized during row：
Step S1：Video data, audio data and lyrics data are encoded by multi-frame video data packet, more with preset frame rate respectively
Frame packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the packets of audio data and the lyrics data
Bao Jun carries timestamp；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio data
Packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is extracted from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Positioned at the first frame video data packet of queue first place, first frame packets of audio data and first frame lyrics data packet；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics data
The size of the timestamp of packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame originating point information,
Transmission is broadcast live to the data packet for being added with frame originating point information, jumps to step S3 until the data packet in all queues is equal
End of transmission.
10. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor realizes following steps when performing described program：
Step S1：Video data, audio data and lyrics data are encoded by multi-frame video data packet, more with preset frame rate respectively
Frame packets of audio data and multiframe lyrics data packet, wherein, the video data packet, the packets of audio data and the lyrics data
Bao Jun carries timestamp；
Step S2：According to the time order and function sequence of timestamp, respectively by the multi-frame video data packet, the multiframe audio data
Packet and the multiframe lyrics data packet are stored into video buffer queue, Audio Buffer queue and lyrics buffering queue；
Step S3：It is extracted from the video buffer queue, the Audio Buffer queue and the lyrics buffering queue respectively
Positioned at the first frame video data packet of queue first place, first frame packets of audio data and first frame lyrics data packet；
Step S4：Compare the first frame video data packet, the first frame packets of audio data and the first frame lyrics data
The size of the timestamp of packet extracts the data packet of timestamp minimum, and the data packet for timestamp minimum adds frame originating point information,
Transmission is broadcast live to the data packet for being added with frame originating point information, jumps to step S3 until the data packet in all queues is equal
End of transmission.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810018715.8A CN108174242A (en) | 2018-01-09 | 2018-01-09 | Live data multiplexing method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810018715.8A CN108174242A (en) | 2018-01-09 | 2018-01-09 | Live data multiplexing method, device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108174242A true CN108174242A (en) | 2018-06-15 |
Family
ID=62517641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810018715.8A Pending CN108174242A (en) | 2018-01-09 | 2018-01-09 | Live data multiplexing method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108174242A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109348279A (en) * | 2018-09-26 | 2019-02-15 | 广州虎牙信息科技有限公司 | A kind of plug-flow method, apparatus, equipment and storage medium |
| CN112437263A (en) * | 2020-11-12 | 2021-03-02 | 成都麦赛科技有限公司 | Video plug flow method and system |
| CN112751983A (en) * | 2021-04-02 | 2021-05-04 | 湖北亿咖通科技有限公司 | Image time synchronization method and device, electronic equipment and storage medium |
| WO2021217435A1 (en) * | 2020-04-28 | 2021-11-04 | 青岛海信传媒网络技术有限公司 | Streaming media synchronization method and display device |
| CN117221617A (en) * | 2023-09-28 | 2023-12-12 | 杭州星犀科技有限公司 | Live broadcast push flow system, method and computer storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1625161A (en) * | 2003-12-02 | 2005-06-08 | 腾讯科技（深圳）有限公司 | Method and system of information playback in immediate communication |
| US20070223332A1 (en) * | 2004-06-11 | 2007-09-27 | Sony Corporation | Data Processing Device, Data Processing Method, Program, Program Recording Medium, Data Recording Medium, and Data Structure |
| CN101340590A (en) * | 2008-08-14 | 2009-01-07 | 北京创毅视讯科技有限公司 | Multiplex apparatus and multiplex method |
| CN102740064A (en) * | 2012-06-15 | 2012-10-17 | 福建星网视易信息系统有限公司 | Packing method for streaming media transmission in intercom system |
| CN103596037A (en) * | 2013-11-19 | 2014-02-19 | 天津市亚安科技股份有限公司 | Method for controlling video streaming buffering in real time |
| CN105245972A (en) * | 2013-05-15 | 2016-01-13 | 孔涛 | MIMO (Multiple-Input Multiple-Output) type DVB-T (Digital Video Broadcasting-Terrestrial) set top box having subtitle processing function |
| CN105306860A (en) * | 2015-10-12 | 2016-02-03 | 东方网力科技股份有限公司 | Video recording data storage method and device |
-
2018
- 2018-01-09 CN CN201810018715.8A patent/CN108174242A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1625161A (en) * | 2003-12-02 | 2005-06-08 | 腾讯科技（深圳）有限公司 | Method and system of information playback in immediate communication |
| US20070223332A1 (en) * | 2004-06-11 | 2007-09-27 | Sony Corporation | Data Processing Device, Data Processing Method, Program, Program Recording Medium, Data Recording Medium, and Data Structure |
| CN101340590A (en) * | 2008-08-14 | 2009-01-07 | 北京创毅视讯科技有限公司 | Multiplex apparatus and multiplex method |
| CN102740064A (en) * | 2012-06-15 | 2012-10-17 | 福建星网视易信息系统有限公司 | Packing method for streaming media transmission in intercom system |
| CN105245972A (en) * | 2013-05-15 | 2016-01-13 | 孔涛 | MIMO (Multiple-Input Multiple-Output) type DVB-T (Digital Video Broadcasting-Terrestrial) set top box having subtitle processing function |
| CN103596037A (en) * | 2013-11-19 | 2014-02-19 | 天津市亚安科技股份有限公司 | Method for controlling video streaming buffering in real time |
| CN105306860A (en) * | 2015-10-12 | 2016-02-03 | 东方网力科技股份有限公司 | Video recording data storage method and device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109348279A (en) * | 2018-09-26 | 2019-02-15 | 广州虎牙信息科技有限公司 | A kind of plug-flow method, apparatus, equipment and storage medium |
| CN109348279B (en) * | 2018-09-26 | 2021-03-23 | 广州虎牙信息科技有限公司 | Plug flow method, device, equipment and storage medium |
| WO2021217435A1 (en) * | 2020-04-28 | 2021-11-04 | 青岛海信传媒网络技术有限公司 | Streaming media synchronization method and display device |
| CN112437263A (en) * | 2020-11-12 | 2021-03-02 | 成都麦赛科技有限公司 | Video plug flow method and system |
| CN112751983A (en) * | 2021-04-02 | 2021-05-04 | 湖北亿咖通科技有限公司 | Image time synchronization method and device, electronic equipment and storage medium |
| CN112751983B (en) * | 2021-04-02 | 2021-06-18 | 湖北亿咖通科技有限公司 | Image time synchronization method and device, electronic equipment and storage medium |
| CN117221617A (en) * | 2023-09-28 | 2023-12-12 | 杭州星犀科技有限公司 | Live broadcast push flow system, method and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108174242A (en) | Live data multiplexing method, device, storage medium and equipment | |
| CN105052107B (en) | Media content Adaptive Transmission is carried out using quality information | |
| CN103650526B (en) | Playlists for real-time or near real-time streaming | |
| US9203642B2 (en) | Method and system for collecting data on a wireless device | |
| US20170111414A1 (en) | Video playing method and device | |
| CN105721620B (en) | Video information method for pushing and device and video information exhibit method and apparatus | |
| CN106162235A (en) | Method and apparatus for Switch Video stream | |
| US10681431B2 (en) | Real-time interstitial content resolution and trick mode restrictions | |
| CN110784750B (en) | Video playing method and device and computer equipment | |
| CN103583051A (en) | Playlists for real-time or near real-time streaming | |
| CN106131660B (en) | Video location playback method and device | |
| US8824676B2 (en) | Streaming video to cellular phones | |
| CN110495182A (en) | Effective insertion of media item in Media Stream | |
| EP3001692A1 (en) | Streaming media processing method, apparatus and system | |
| CN108259985A (en) | Live audio sound mixing method, device, readable storage medium storing program for executing and equipment | |
| CN103517135B (en) | Method, system and television capable of playing MP4-format video files continuously | |
| CN105228001A (en) | The method and system that a kind of FLV format video is play online | |
| CN108174264A (en) | Synchronous lyrics display methods, system, device, medium and equipment | |
| CN111277869A (en) | Video playing method, device, equipment and storage medium | |
| CN108111872A (en) | A kind of live audio system | |
| CN113794909A (en) | Video streaming system, method, server, device, and storage medium | |
| CN104363511B (en) | A kind of method and system of broadcasting F4v videos online in a mobile device | |
| CN105027572A (en) | Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames | |
| CN112104909A (en) | Interactive video playing method and device, computer equipment and readable storage medium | |
| CN103313091A (en) | Speed-multiplying playing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180615 |