CN107995155A - Video data encoding, decoding, methods of exhibiting, video system and storage medium - Google Patents
Video data encoding, decoding, methods of exhibiting, video system and storage medium Download PDFInfo
- Publication number
- CN107995155A CN107995155A CN201710942414.XA CN201710942414A CN107995155A CN 107995155 A CN107995155 A CN 107995155A CN 201710942414 A CN201710942414 A CN 201710942414A CN 107995155 A CN107995155 A CN 107995155A
- Authority
- CN
- China
- Prior art keywords
- video
- data
- encoded
- display
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present invention provides a kind of video data encoding, decoding, methods of exhibiting, video system and storage medium, and the method for coding video data includes：Video original data is encoded to video channel, supplemental enhancement information passage will be encoded to the relevant display data of video；The display data of the video original data of the video channel and the supplemental enhancement information passage is encoded in video data output file.The present invention by display data by being bundled to self-defined SEI（Supplemental enhancement information）In the data packet of type of message instruction, data packet is encoded into video stream data in encoder-side together with original video data, achieve the purpose that data and video close association, video can be independent of the reliability of the server end of video provider, it still can also can retain original presentation data in video after any propagation, or even after video is transcoded into different video encapsulation forms, also it can ensure the lasting preservation of data, effectively increase user experience.
Description
Technical field
The invention belongs to digital stream technical field of media, is specially one kind more particularly to streaming media video technical field
Video data encoding, decoding, methods of exhibiting, video system and storage medium.
Background technology
H.264 it is a kind of block-oriented Video Codec standard based on motion compensation.By ITU-T Video Coding Experts
Joint video team (JVT, the Joint that group is constituted jointly with ISO/IEC joint working groups (i.e. dynamic image expert group-MPEG)
Video Team) exploitation.It is both ITU-T H.264 and ISO/IEC MPEG-4 advanced video codings (MPEG-
4Part10, Advanced Video Coding, MPEG-4AVC) the 10th part.Therefore, whether MPEG-4AVC, MPEG-
4Part 10, or ISO/IEC 14496-10, can be generally be known as H.264.The final draft of first edition standard in
In May, 2003 completes, and second edition standard was completed in 2005.Up to the present, it has been widely used for multimedia communication, net
Network Streaming Media, in high definition broadcasts digital television system (DVT/ATSC/ISDB).As high-precision video record, compression and
One of the most frequently used form of issue.
User's direct viewing live video/order video on network, or video is downloaded into computing device and is locally broadcast
Viewing is put, can all touch the video file of different-format, such as mp4/ts/flv.These different video formats, it is corresponding
It is different video encapsulation modes.It is that video provider selects to determine according to different business frameworks to select which kind of packaged type
's.For most video providers, based on business and technology consider, under different video encapsulation format, be with
H.264 standard encodes.The video data encoded, according to different business scenarios, by using different encapsulation format, and
Different transport protocols realizes the video-on-demand live service to user.
Order video or live video on network, mostly based on H.264 Video coding, MP4/TS/FLV envelopes
Fill the media file of form.Either live video or order video, at present in face of user can exhibiting data information provide
Mode, is less friendly.For example, user knows sports show race or sportsman's key race data if desired, at present
The mode that can provide a user of video provider have 2 kinds：1st kind be by data by the broadcasting webpage of broadcasting or should
Pushed with third party's module of the page in program to obtain relevant information；2nd kind is that race instructor in broadcasting directly will at Video coding end
Race data make video or picture, and direct shuffling enters in live source video by way of insertion, and this is also that physical culture is straight
Broadcast way most current at present in the industry.But 2 kinds of the above is supplied to the mode with the closely related data message of video of user,
There is different limitations.
1st kind of mode, data are stored in the server end of video provider.User need to such as obtain data, it is necessary to rely on
The module of webpage or application program, the server end by network from video provider, active pull or passive push obtain
Take.This mode is limited to the serviceability and reliability of the server end of video provider.When the server of video provider
When holding unreliable or unavailable, user just can not get data from server end；In addition, being downloaded when video by user becomes
After independent file, the incidence relation of video and server end, which is accused, to be stopped, and user can not just be known to obtain from where to be regarded with this
Frequently relevant any valid data.
2nd kind of mode, data are to generate end in video source, and by artificial editor, textual data are converted into can
It is inserted directly into depending on the video pictures of change, then by the data display of video in source video.Data are a series of insert in this mode
The video pictures broadcast, are widely used in sports show race scene.This mode is succinctly directly perceived, but is regarded because being limited to insertion
The association attributes of frequency, time cannot be too long, can not present at any time, thus it is possible to which the data of displaying will not be very much, and show shape
Formula is once it is determined that i.e. immutable.Meanwhile because its output data by way of intercutting video pictures, this breaks to a certain extent
It is broken the pure property of data source；In addition, because data are video pictures, with certain mandatory, so once intercutting into i.e.
It can not cancel, spectators must receive.
The content of the invention
In view of the foregoing deficiencies of prior art, it is an object of the invention to provide a kind of video data encoding, decoding,
Methods of exhibiting, video system and storage medium, for solving the problems, such as that video display user experience is poor in the prior art.
In order to achieve the above objects and other related objects, the present invention provides a kind of method for coding video data, the video
Data-encoding scheme includes：Video original data is encoded to video channel, benefit will be encoded to the relevant display data of video
Fill enhancement information passage；The display data of the video original data of the video channel and the supplemental enhancement information passage is compiled
Code is into video data output file.
In one embodiment of the invention, supplemental enhancement information passage will be encoded to the relevant display data of video
When, make the display data in the supplemental enhancement information passage different from the video original data holding in the video channel
Step.
In one embodiment of the invention, supplemental enhancement information passage will be encoded to the relevant display data of video
When, make the display data in the supplemental enhancement information passage synchronous with the video data frame holding of the video channel and not
With timestamp different display datas is added in the corresponding supplemental enhancement information passage.
In one embodiment of the invention, the video original data includes net cast data；With the relevant institute of video
Stating display data includes describing data, personnel's description information data, live with the relevant live content of net cast data
The instant associated data of process, with any of the associated interactive data of live content or a variety of.
In one embodiment of the invention, the video output File is H.264 video data output file.
In one embodiment of the invention, the display data is json formatted datas, xml formatted datas or binary system lattice
Formula protocol data.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
Realized when sequence is executed by processor：Video original data is encoded to video channel, will display data coding relevant with video
To supplemental enhancement information passage；By the video original data of the video channel and the displaying number of the supplemental enhancement information passage
According to being encoded in video data output file.
The embodiment of the present invention also provides a kind of video data decoding method, and video data output file is decoded,
The video data decoding method includes：Dock received video data output file to be decoded, separation obtains video channel
Video original data and supplemental enhancement information passage display data；Wherein, the video data output file includes coding
To video channel video original data and be encoded to supplemental enhancement information passage with the relevant display data of video.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
Realized when sequence is executed by processor：Dock received video data output file to be decoded, separation obtains regarding for video channel
The display data of frequency initial data and supplemental enhancement information passage；Wherein, the video data output file includes being encoded to regarding
The video original data of frequency passage and be encoded to supplemental enhancement information passage with the relevant display data of video.
The embodiment of the present invention also provides a kind of video display method, and the video display method includes：In displaying video
During initial data, displaying and the relevant display data of the video original data；Wherein, the video original data encodes in advance
To video channel, supplemental enhancement information passage is encoded in advance with the relevant display data of video.
In one embodiment of the invention, display data displaying synchronous with the video original data or asynchronous exhibition
Show.
In one embodiment of the invention, the video original data includes net cast data；The display data bag
Include and describe data, personnel's description information data, live process with the relevant live content of net cast data and associate immediately
Data, with any of the associated interactive data of live content or a variety of.
In one embodiment of the invention, shown after the display data for separating acquisition is rendered.
In one embodiment of the invention, the display data is shown by the floating layer of display interface, by described in closing
Floating layer exits the display data and shows.
The embodiment of the present invention also provides a kind of video system, including video encoder, Video Decoder, video player
And be connected between the video encoder and the Video Decoder, for the video counts for exporting the video encoder
The communication module of the Video Decoder is transferred to according to output file；Wherein：The video encoder is used for video original number
According to video channel is encoded to, supplemental enhancement information passage will be encoded to the relevant display data of video, by the video channel
Video original data and the display data of the supplemental enhancement information passage be encoded in video data output file；It is described logical
Letter module is used to the video data output file being transferred to the Video Decoder；The Video Decoder is used for receiving
To video data output file decoded, separation obtain video channel video original data and supplemental enhancement information passage
Display data；The video player is used for when playing video original data, and displaying is related to the video original data
Display data.
As described above, the present invention video data encoding, decoding, methods of exhibiting, video system and storage medium, have with
Under
Beneficial effect：
Data packet of the invention by the way that display data to be bundled to the instruction of self-defined SEI (supplemental enhancement information) type of message
It is interior, data packet is encoded into video stream data in encoder-side together with original video data, realizes the fluidisation of display data
Preserve, and achieve the purpose that data and video close association.Because display data is by being stored in video streaming data packet, institute
Can be independent of the reliability of the server end of video provider with video, can also be in video after any propagation, still
Original presentation data can be retained, or even after video is transcoded into different video encapsulation forms, can also ensure the lasting guarantor of data
Deposit, so the present invention effectively increases user experience.
Brief description of the drawings
Fig. 1 is shown as in the video data display method of the present invention the H.264 structure diagram of data flow.
Fig. 2 is shown as flow diagram of the method for coding video data of the present invention in an embodiment.
Fig. 3 is shown as flow diagram of the video data decoding method of the present invention in an embodiment.
Fig. 4 is shown as flow diagram of the video data display method of the present invention in an embodiment.
Fig. 5 is shown as principle schematic of the video system of the present invention in an embodiment.
Component label instructions
100 video encoders
200 Video Decoders
300 video players
400 communication modules
S101~S102 steps
S201 steps
S301 steps
Embodiment
Illustrate embodiments of the present invention below by way of specific instantiation, those skilled in the art can be by this specification
Disclosed content understands other advantages and effect of the present invention easily.The present invention can also pass through in addition different specific realities
The mode of applying is embodied or practiced, the various details in this specification can also be based on different viewpoints with application, without departing from
Various modifications or alterations are carried out under the spirit of the present invention.It should be noted that in the case where there is no conflict, following embodiments and implementation
Feature in example can be mutually combined.
It should be noted that the diagram provided in following embodiments only illustrates the basic structure of the present invention in a schematic way
Think, then only the display component related with the present invention rather than component count, shape and size during according to actual implementation in schema
Draw, kenel, quantity and the ratio of each component can be a kind of random change during its actual implementation, and its assembly layout kenel
It is likely more complexity.
The present embodiment is related to digital stream field of media, it particularly relates to which streaming media video, and streaming media video play
Scene, user interact with video information, and the present embodiment can be applied to live and program request, and the relevant scene of video player
Action.The purpose of the present embodiment is to provide a kind of video data encoding, decoding, methods of exhibiting, video system and storage medium,
For solving the problems, such as that video display user experience is poor in the prior art.Video data of the invention described in detail below is compiled
Code, decoding, methods of exhibiting, the principle and embodiment of video system and storage medium, make those skilled in the art to create
The property made work is appreciated that video data encoding, decoding, methods of exhibiting, video system and the storage medium of the present invention.
The skill of a kind of video data encoding, decoding, methods of exhibiting, video system and storage medium that the present embodiment is proposed
Art principle, based on auxiliary in Video coding H.264 the network abstract layer NAL (Network Abstraction Layer) of standard
Help enhancement information SEI (Supplemental Enhancement Information) expansion.
In H.264/AVC video encoding and decoding standard, coding/decoding system frame has been divided into two aspects：Video coding layer
Face (VCL) and network abstraction aspect (NAL).Wherein, VCL is responsible for effectively representing the content of original video data；And NAL is then responsible for
Format data simultaneously provides header, to ensure transmission that data are adapted on various channels and storage medium.
H.264 the data flow that standard defines is made of NAL unit one by one, and NAL unit is the basic syntax structure of NAL,
It includes the header of byte and a series of byte streams for being known as raw byte sequence payload (RBSP) from VCL.Its
In, different types of original word throttling RBSP has been defined different NAL unit types.As shown in Figure 1.Wherein, NAL unit class
Type is supplemental enhancement information (SEI) for 0x06's.
Supplemental enhancement information (SEI) is a H.264 important technology in standard, main to play supplement and enhancing.
SEI data are not image data informations, simply to the supplement of image data information or video stream data；SEI is nor H.264
Part necessary to image data stream decoding, it is generally used for the effect of control view data synchronization.H.264 standard to SEI into
Go classification, define the different SEI types used for various purposes.It is 0x05 wherein to have a kind of types value, keeps for use
The customized SEI types in family：The SEI message (hereinafter referred to as self-defined SEI type of messages) of customized user data.This implementation
Example is to add related display data into video file using this SEI type.
The present embodiment is by the way that display data is bundled in the data packet of self-defined SEI type of messages instruction, by data packet
Encoded in encoder-side together with original video data into flow data, realize that the fluidisation of display data preserves, and reach data
With the purpose of video close association.Because display data is by being stored in video streaming data packet, video can disobey
Rely in the reliability of the server end of video provider, still can also can retain original presentation in video after any propagation
Data.After even video is transcoded into different video encapsulation forms, it can also ensure the lasting preservation of data.
In the present embodiment, encoder will be encoded into the SEI data packets in video flowing with the relevant display data of video, with
Video data stream is synchronously encapsulated into different video files.After decoder decapsulates video file, from the video text of coding
Decoded in part and isolate video data and with video related information data.
The present embodiment makes data be closely related with video, not easy to lose, and data display mode is controllable variable；
Meanwhile also extend the usage scenario of data.
After the present embodiment can make video depart from server and related pages, important information data can be still shown.And
The displaying form of information is unrelated with video content, can freely be customized at displaying end.For example, sportsman's list of a ball match, and
Corresponding player information, can be converted into byte stream deposit video, such user can pass through in any case
Obtain after video player decoding, then shown on video player video layer, and how to show, can be by video player
Come self-defined.Moreover, these information can be with immediate updating.For example, this competition has goal on the 30th minute, at this time can
When scoring, the information of related sportsman is shown, the instant data message of this race can also be shown.It is various
The displaying of information, can actively or passively show according to the wish of user.In this way, can guarantee that the displaying of data message,
Also improve the usage experience of user.
The video data encoding of the present embodiment, decoding, methods of exhibiting, video system and storage medium are carried out below specific
Explanation.
As shown in Fig. 2, the present invention provides a kind of method for coding video data, the method for coding video data includes following
Step：
Step S101, video channel is encoded to by video original data, will be encoded to benefit with the relevant display data of video
Fill enhancement information passage.
In this present embodiment, video original data is obtained first, and obtaining needs to encode into the display data in video file.
User can at any time directly obtain with the relevant display data of this video original data, wherein, it is described to regard
Frequency initial data includes net cast data and vod data；Include regarding with described with the relevant display data of video
The relevant live content of frequency live data describe data, personnel's description information data, the instant associated data of live process, with it is live
Any of associated interactive data of content is a variety of.
Specifically, the video original data is, for example, football live data, and it is for example sufficient that the live content describes data
Current event introduces data in ball match, including is not limited to race overview, than grading.
Personnel's description information data are, for example, the overall data of both sides troop, detailed data of organizer etc..
The instant associated data of live process is, for example, goal information data, into elements such as sportsman's head portrait/animations and
Score and celebrate the associated datas such as picture, for example, this competition has goal on the 30th minute, at this time can be when scoring, by correlation
The information of sportsman is shown.
The associated interactive data of live content is, for example,：Guess data in real time.Such as carried out when football race
Cheng Zhong, judge, which blows, has penalized a free kick.Data source coding side staff (or football narrator) at this time, you can vertical
I.e. by corresponding software, preprepared guess property data display template is inserted into corresponding real time data, and will be all
Data are added in SEI data channel and encode.
In this present embodiment, will to be encoded to supplemental enhancement information passage with the relevant display data of video specific as follows：
First by optional display data, it is ready to according to different forms.Wherein, the display data is json forms
The standard format such as data, xml formatted datas data or privately owned binary format protocol data.
In this present embodiment, when supplemental enhancement information passage will be encoded to the relevant display data of video, make described
Display data in supplemental enhancement information passage keeps asynchronous with the video original data in the video channel, or makes institute
It is synchronous with the video data frame holding of the video channel and in different time to state the display data in supplemental enhancement information passage
Stab and corresponding add different display datas in the supplemental enhancement information passage.
That is, by the data of different-format by video encoder, the SEI data for being optionally encoded to video flowing are led to
In road.Wherein, it is to continue to keep asynchronous that the data of current SEI passages and video channel data, which may be selected,；Or and video
The video data frame of passage keeps synchronous, and in the selectable above-mentioned different display data of addition of different time stamp.
Step S102, by the video original data of the video channel and the display data of the supplemental enhancement information passage
It is encoded in video data output file.
In this present embodiment, the video output File is H.264 video data output file.I.e. will be upper in encoder-side
The data encoding of data in SEI passages and video channel is stated into H264 video data files.
The present embodiment is the SEI extensible types for utilizing NAL unit in H.264 standard, will be with video phase in encoder-side
The display data of pass is added in a manner of byte stream in the SEI data packets in video file flow data.
For example, it is live it is middle display data is encoded into SEI data channel, be saved in video data equivalent to by display data
Output file.
For example, by the method for the present embodiment be used for live video during when, using the synchronism of SEI data, by exhibition
Registration achievees the purpose that display data is synchronously shown with video content according to being synchronously bundled in SEI data packets, synchronous exhibition displaying
Data can make video content keep synchronous in instant interactive experience with user.
During by the method for during order video, using the continuation of SEI data, complete display data is protected
It is stored in SEI data packets, ensures the complete of data, on the basis of independent of other server resources, makes video band in itself
There is abundant information data.Video can still ensure that the complete of data can illustrative after different modes are propagated.
The data closely related with video are integrated in the number of video flowing by the method for the present embodiment by way of byte stream
According in bag, on the premise of original video frame is not influenced, there is provided selectable data acquisition and exhibition method.Meanwhile because
It is to be stored in by byte mode in the data packet of video flowing for data, data and video are close associations, so can be effective
Guarantee video in data can obtain and show at any time.Meanwhile in video playing terminal, can be according to itself need
Ask, unrestricted choice any-mode comes display data, flexibility and changeability.
The embodiment of the present invention also provides a kind of video encoder, including processor and memory, the memory storage
There is programmed instruction, method for coding video data as described above is realized in the processor operation program instruction.It is above-mentioned to institute
State method for coding video data to be described in detail, details are not described herein.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
Method for coding video data as described above is realized when sequence is executed by processor：Video original data is encoded to video channel,
Supplemental enhancement information passage will be encoded to the relevant display data of video；By the video original data of the video channel and institute
The display data for stating supplemental enhancement information passage is encoded in video data output file.
Above-mentioned that the method for coding video data is described in detail, details are not described herein.
As shown in figure 3, the embodiment of the present invention also provides a kind of video data decoding method, to video counts as described above
Decoded according to the video data output file exported in coding method, the video data decoding method includes：Step S201,
Dock received video data output file to be decoded, separation obtains the video original data and supplement enhancing letter of video channel
Cease the display data of passage.Wherein, the video data output file include be encoded to video channel video original data and
Be encoded to supplemental enhancement information passage with the relevant display data of video.
The video file received is reduced into the video data file of H264 codings by Video Decoder, is isolated SEI and is led to
The data and video channel data in road.By the data for the SEI passages separated, video player displaying is sent to.
The key point of the present embodiment is using the SEI extensible types of NAL unit in H.264 standard, in video encoder
End, will be added in the SEI data packets in video file flow data with the relevant display data of video in a manner of byte stream,
Video Decoder end, then after this SEI decoded packet data is come out, local is stored in, or live shown by video player
Out.By the method, display data being completely effectively stored in video stream file, data will not lose, and by
It is stored in data in video file, so data can be demonstrated independent of any network or other relevant environments.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
Video data decoding method as described above is realized when sequence is executed by processor：Received video data output file is docked to carry out
Decoding, separation obtain the video original data of video channel and the display data of supplemental enhancement information passage；Wherein, the video
Data output file includes being encoded to the video original data of video channel and be encoded to supplemental enhancement information passage and video
Relevant display data.
Above-mentioned that the video data decoding method is described in detail, details are not described herein.
The embodiment of the present invention also provides a kind of Video Decoder, including processor and memory, the memory storage
There is programmed instruction, video data decoding method as described above is realized in the processor operation program instruction, herein not to described
Video data decoding method is repeated.
As shown in figure 4, the embodiment of the present invention also provides a kind of video display method, the video display method is used for such as
The data obtained in the upper video data decoding method are shown, and the video display method includes：Step S301,
When showing video original data, displaying and the relevant display data of the video original data；Wherein, the video original data
Video channel is encoded in advance, supplemental enhancement information passage is encoded in advance with the relevant display data of video.
In this present embodiment, alternatively by display data displaying synchronous with the video original data, i.e., described exhibition
Registration is according to displaying synchronous with the video original data or asynchronous displaying.
Wherein, the video original data includes net cast data；The display data includes and the net cast
The relevant live content of data describes data, personnel's description information data, the instant associated data of live process and live content and closes
Any of interactive data of connection is a variety of.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
Video display method as described above is realized when sequence is executed by processor：When showing video original data, displaying is regarded with described
The relevant display data of frequency initial data；Wherein, the video original data is encoded to video channel in advance, relevant with video
The display data is encoded to supplemental enhancement information passage in advance.
In this present embodiment, shown after the display data for separating acquisition is rendered.The SEI numbers that will be separated
According to, extract original display data, be stored in local, or it is live give video player/page, wash with watercolours on a user interface
Dye is shown.
In this present embodiment, the display data is shown by the floating layer of display interface, is exited by closing the floating layer
The display data is shown.
Using the video display method of our embodiment, whether user can show this when watching video with unrestricted choice
Display data in video.Meanwhile at video player end, can also be according to the attribute of display data, according to different scenes
To select different exhibition strategies.After application this method, because the relevant display data in part has been stored in video, then exist
Video system serving backend, you can stored with saving the relevant resource of this part and video content, and relevant offer service.
Likewise, user can decode in video player decoding end energy random time and get the related exhibition of video
Registration evidence, and optionally it is remained on or off information displaying.Information data is independent in the same way with video data
Storage is hereof.This feature, can be widely used in video distribution field, particularly in terms of the relevant data of competitive sports.
User can directly obtain and the relevant data of this video, such as player information, goal information, race data at any time
Deng.
For example, it is live it is middle by data encoding into SEI data channel, equivalent to storing data into video data file,
After data in live are saved in video file, are supported without server end, still can extract ancestor and carry out race data, and open up
Show to user.After live, because data are saved in video file, program request still may not need server end support,
Race data can be extracted, show user；Data in live, can preserve in SEI data channel, in user's exhibition in real time
Show interface, be equal to real time data refresh function.Data in live, can be associated with Video Key point, in critical event,
Encode the relevant information into critical event.Data encoding is not easy to lose into video file.And with network, the equal nothing of server end
Association.
So this method can be used for live field, including the industries such as physical culture, video display, show field are not limited to, this method can be used for
Streaming Media it is live, it can also be used to the video file commonly distributed, as long as H264 encoding and decoding, can apply.
The embodiment of the present invention also provides a kind of video player, including processor and memory, the memory storage
There is programmed instruction, video display method as described above is realized in the processor operation program instruction.
As shown in figure 5, the embodiment of the present invention also provides a kind of video system, including the decoding of video encoder 100, video
Device 200, video player 300 and it is connected between the video encoder 100 and the Video Decoder 200, for will
The video data output file that the video encoder 100 exports is transferred to the communication module 400 of the Video Decoder 200.
Wherein：The video encoder 100 is used to video original data being encoded to video channel, will be relevant with video
Display data is encoded to supplemental enhancement information passage, by the video original data of the video channel and the supplemental enhancement information
The display data of passage is encoded in video data output file.
The communication module 400 is used to the video data output file being transferred to the Video Decoder.Yu Benshi
Apply in example, the communication module 400 is network communication module or mobile memory medium communication module.The network communication module carries
Downloaded for network direct broadcasting, network program request, network etc., the mobile memory medium communication module provides physical file copy, such as
Including mobile hard disk etc..
The Video Decoder 200 is decoded for docking received video data output file, and separation obtains video
The video original data of passage and the display data of supplemental enhancement information passage.
The video player 300 is used for when playing video original data, and displaying is related to the video original data
Display data.
In the present embodiment, the course of work of the video system is as follows：
Video original data is obtained first, and obtaining needs to encode into the display data in video file.To optionally it show
Data, are ready to according to different forms, and optional form has the standard format such as json/xml data or private herein
Some binary format protocol datas.Video original data is sent to encoder together with display data.By different-format
Data are optionally encoded in the SEI data channel of video flowing by video encoder 100.Wherein, current SEI may be selected to lead to
The data in road are to continue to keep asynchronous or synchronous with the video data frame holding of video channel with video channel data,
And in the above-mentioned different display datas of the selectable addition of different time stamp.By video encoder 100 by above-mentioned SEI passages
Data and video channel data encoding into H264 video data files.The H264 data that coding is completed pass through different-format
Encapsulation, it is optional to include being not limited to the common encapsulation format such as mp4/flv/mkv.H.264 not identical text will be packaged into by video file
The file stream of part form, carries out network transmission/storage.After the completion of encoding and encapsulating, the file of different-format, passes through different propagation
Mode, is output to Video Decoder 200.Wherein, optional circulation way includes network direct broadcasting, network program request, network download, thing
Manage the normal methods such as file copy.The agreement wherein transmitted includes being not limited to the agreements such as common http/ftp/udp, and
The common transport protocol of the live program request such as HLS/RTMP/HTTP-FLV.Video Decoder 200 receives the video file after encapsulation
Afterwards, video file is decapsulated.Video file is reduced into the video data file of H264 codings.Video Decoder 200 will unseal
The H264 video data files obtained after dress, isolate the data and video channel data of SEI passages, the SEI that will be separated
The data of passage, are sent to video player 300 and show, display form can be shown in video player 300UI floating layers.User
The data shown are optionally subjected to shutoff operation, or wait the timing of pending data floating layer to turn itself off.
The video system of the present embodiment can specifically distinguish 2 type of live video and order video, work in practical application
It is similar to make flow, is illustrated below by taking live video as an example.
Under net cast pattern：Live source data is sent to video encoder 100, it is live in instant race data conduct
Display data is also sent to video encoder 100, and the display data is, for example, the full field data of current event, including is not limited to match
Thing score, the overall data of both sides troop, and detailed data of organizer etc.；It can also be sent out herein according to race current time
Raw critical event, insertion and the relevant race information of this event.By race data encoding to SEI data channel, regarded live
Frequency source data encoding is to video data channel.Then race data and live video data are encoded to final transport stream jointly to regard
Frequency file, passes through network transmission video flowing.The Video Decoder 200 of user terminal receives the video flowing that network transmission comes, point
From the data in SEI data channel and video data channel, the data of video data channel are sent to video player 300 and are broadcast
Put, the race data of SEI data channel are saved in local, and parsed.The race that video player 300 will parse
Data are alternatively shown to user interface.Wherein, user alternatively closes race data display interface, Video Decoder 200
Alternatively constantly separate and parse the real-time SEI data in SEI data flows, and preserve renewal and arrive locally, accordingly, video playing
Device 300 alternatively constantly refreshes real time data to user interface.
Extension, this mode can also be applied to common video file distribution.By the relevant data encoding of video to regarding
In the SEI data channel of frequency file, so at specific 300 end of video player also decodable code and corresponding SEI data are exported.
In above net cast, in video encoder 100, by user's display data, according to the real-time shape of race program
Condition, the event related data that will occur in real time, synchronous coding to SEI data channel carry out data encapsulation, reach information and video
The purpose of content synchronization.In Video Decoder 200, you can the synchronous display data by video information leads to from SEI data
Separation is decoded in road, makes video content synchronously be shown on a user interface with display data, improves the real-time interaction experience of user.
In addition, synchronism can be divided into several grades in actual scene：In real time, quasi real time, generally.Lifted individually below
Example explanation.
1) typical real-time scene, it is desirable to data and the relatively accurate displaying of scene.It can be applied to critical short-term thing
Part.The displaying for information of such as scoring.When goal event occurs, at 100 end of video encoder, personnel are immediately by ready goal
Template is taken out, and inserts corresponding data (as goal sportsman numbers), and all data are added in SEI data channel immediately and are compiled
Code.Video Decoder 200 receives and after video data, will encode sportsman's number in data out, triggers what is be locally stored
The elements such as sportsman's head portrait/animation, combine and show abundant goal and celebrate picture, this composite component can pass through video player
300 data update keeps more new change.
2) typical quasi real time scene, it is allowed to which data are not especially accurate with scene relative synchronization.It can be applied to general
The event of generation.Such as during football race progress, judge, which blows, has penalized a free kick.100 end work of video encoder at this time
Make personnel (or football narrator), you can immediately by corresponding software, by preprepared guess property data exhibition
Show that template inserts corresponding real time data, and all data are added in SEI data channel and are encoded；Video flowing after to be encoded
Data reach use Video Decoder 200 after, 300 interface of video player can be according to receiving and decoded SEI data, immediately
Show corresponding guess interface, it is defeated according to 100 end staff of video encoder (or football narrator) on interface
The real-time guess data entered, and guess statistics countdown is started simultaneously at, show and be supplied to user to select.When user determines to guess
After selection, or after countdown, by the upload data module of video player 300, server is upload the data to.This
When 100 end staff of video encoder, operation personnel, and football narrator can see instant data guess system
Meter.Guess data statistics result data can be coded into by final data source code end staff (football narrator) again
In SEI channel datas, and finally it is shown to video player 300.
3) typical general synchronization scenarios, requirement synchronous with scene to data be not relatively high.Such as the renewal of race data.When
During race carries out, because of the data update situation that certain event produces, that is, belong to general synchronization scenarios.Such as call yellow card thing
When part occurs, influence is generated on the red and yellow card data variation of race.Data at this time are equally worked by 100 end of video encoder
Personnel or operation personnel, data are inserted after loading template by corresponding software, are added SEI passages and are encoded.Video Decoder
The requirement of real-time of 200 pairs of this data is not high, and it is poor to allow for the regular hour, after a bit of time occurs for event,
Complete red and yellow card data list list, even historical statistical data are shown in video player 300.
In addition, the form of expression of the similar events in different scenes dimension can also correspond to different scenes.As same
Free kick event, under the scene of guess class, requirement of real-time is of a relatively high；But under the scene of race data display class,
Requirement of real-time is exactly general.Correspondingly, the accuracy and integrity degree to data have different requirements.
In conclusion the present invention is referred to by the way that display data is bundled to self-defined SEI (supplemental enhancement information) type of message
In the data packet shown, data packet is encoded into video stream data in encoder-side together with original video data, realizes displaying
The fluidisation of data preserves, and achievees the purpose that data and video close association.Because display data is by being stored in video flowing
In data packet, so video can be independent of the reliability of the server end of video provider, can also be in video through any
It after propagation, still can retain original presentation data, or even after video is transcoded into different video encapsulation forms, can also ensure number
According to lasting preservation, so the present invention effectively increases user experience.So the present invention effectively overcome it is of the prior art
Various shortcoming and have high industrial utilization.
The above-described embodiments merely illustrate the principles and effects of the present invention, not for the limitation present invention.It is any ripe
Know the personage of this technology all can carry out modifications and changes under the spirit and scope without prejudice to the present invention to above-described embodiment.Cause
This, those of ordinary skill in the art is complete without departing from disclosed spirit and institute under technological thought such as
Into all equivalent modifications or change, should by the present invention claim be covered.
Claims (15)
1. a kind of method for coding video data, it is characterised in that the method for coding video data includes：
Video original data is encoded to video channel, supplemental enhancement information will be encoded to the relevant display data of video and led to
Road；
The display data of the video original data of the video channel and the supplemental enhancement information passage is encoded to video counts
According in output file.
2. method for coding video data according to claim 1, it is characterised in that will be with the relevant display data of video
When being encoded to supplemental enhancement information passage, make in display data and the video channel in the supplemental enhancement information passage
Video original data keeps asynchronous.
3. method for coding video data according to claim 1, it is characterised in that will be with the relevant display data of video
When being encoded to supplemental enhancement information passage, make regarding for display data in the supplemental enhancement information passage and the video channel
Frequency data frame keeps synchronous and adds different display datas in different time stabs the corresponding supplemental enhancement information passage.
4. method for coding video data according to claim 1, it is characterised in that the video original data includes video
Live data；Include describing number with the relevant live content of net cast data with the relevant display data of video
According to, personnel's description information data, the instant associated data of live process, with any of the associated interactive data of live content or
It is a variety of.
5. method for coding video data according to claim 1, it is characterised in that the H.264 video output File is
Video data output file.
6. method for coding video data according to claim 1, it is characterised in that the display data is json form numbers
According to,
Xml formatted datas or binary format protocol data.
7. a kind of computer-readable recording medium, is stored thereon with computer program, it is characterised in that the program is held by processor
Realized during row：
Video original data is encoded to video channel, supplemental enhancement information will be encoded to the relevant display data of video and led to
Road；
The display data of the video original data of the video channel and the supplemental enhancement information passage is encoded to video counts
According in output file.
8. a kind of video data decoding method, it is characterised in that decoded to video data output file, the video data
Coding/decoding method includes：
Dock received video data output file to be decoded, separation obtains the video original data of video channel and supplement increases
The display data of strong information channel；
Wherein, the video data output file includes being encoded to the video original data of video channel and is encoded to supplement enhancing
Information channel with the relevant display data of video.
9. a kind of computer-readable recording medium, is stored thereon with computer program, it is characterised in that the program is held by processor
Realized during row：
Dock received video data output file to be decoded, separation obtains the video original data of video channel and supplement increases
The display data of strong information channel；
Wherein, the video data output file includes being encoded to the video original data of video channel and is encoded to supplement enhancing
Information channel with the relevant display data of video.
A kind of 10. video display method, it is characterised in that the video display method includes：When showing video original data,
Displaying and the relevant display data of the video original data；Wherein, the video original data is encoded to video channel in advance,
Supplemental enhancement information passage is encoded in advance with the relevant display data of video.
11. video display method according to claim 10, it is characterised in that the display data and the video are original
Data synchronously displaying or asynchronous displaying.
12. video display method according to claim 10, it is characterised in that it is straight that the video original data includes video
Multicast data；The display data includes describing data, personnel's description information with the relevant live content of net cast data
Data, the instant associated data of live process, with any of the associated interactive data of live content or a variety of.
13. video display method according to claim 10, it is characterised in that by separate obtain the display data into
Row is shown after rendering.
14. the video display method according to claim 10 or 11, it is characterised in that the display data is by showing boundary
The floating layer in face shows, exiting the display data by closing the floating layer shows.
15. a kind of video system, it is characterised in that including video encoder, Video Decoder, video player and communication mould
Block, is used；Wherein：
The video encoder is used to video original data being encoded to video channel, will display data coding relevant with video
To supplemental enhancement information passage, by the video original data of the video channel and the displaying number of the supplemental enhancement information passage
According to being encoded in video data output file；
The communication module is used to the video data output file being transferred to the Video Decoder；
The Video Decoder is decoded for docking received video data output file, and separation obtains regarding for video channel
The display data of frequency initial data and supplemental enhancement information passage；
The video player is used for when playing video original data, displaying and the relevant displaying number of the video original data
According to.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710942414.XA CN107995155A (en) | 2017-10-11 | 2017-10-11 | Video data encoding, decoding, methods of exhibiting, video system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710942414.XA CN107995155A (en) | 2017-10-11 | 2017-10-11 | Video data encoding, decoding, methods of exhibiting, video system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107995155A true CN107995155A (en) | 2018-05-04 |
Family
ID=62028878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710942414.XA Pending CN107995155A (en) | 2017-10-11 | 2017-10-11 | Video data encoding, decoding, methods of exhibiting, video system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107995155A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108924600A (en) * | 2018-06-28 | 2018-11-30 | 乐蜜有限公司 | Sending and receiving methods, device and the electronic equipment of live data |
| CN110213599A (en) * | 2019-04-16 | 2019-09-06 | 腾讯科技（深圳）有限公司 | A kind of method, equipment and the storage medium of additional information processing |
| CN110233716A (en) * | 2019-05-31 | 2019-09-13 | 北京文香信息技术有限公司 | A kind of communication interaction method, apparatus, storage medium, terminal device and server |
| CN110557649A (en) * | 2019-09-12 | 2019-12-10 | 广州华多网络科技有限公司 | Live broadcast interaction method, live broadcast system, electronic equipment and storage medium |
| CN111163360A (en) * | 2020-01-02 | 2020-05-15 | 腾讯科技（深圳）有限公司 | Video processing method, video processing device, computer-readable storage medium and computer equipment |
| CN111464828A (en) * | 2020-05-14 | 2020-07-28 | 广州酷狗计算机科技有限公司 | Virtual special effect display method, device, terminal and storage medium |
| CN112468822A (en) * | 2020-11-06 | 2021-03-09 | 上海钦文信息科技有限公司 | Multimedia recording and broadcasting course interaction method based on video SEI message |
| CN112788282A (en) * | 2019-11-08 | 2021-05-11 | 株洲中车时代电气股份有限公司 | Video information acquisition method and application thereof |
| CN113206853A (en) * | 2021-05-08 | 2021-08-03 | 杭州当虹科技股份有限公司 | Video correction result storage improvement method |
| CN114125501A (en) * | 2021-10-30 | 2022-03-01 | 杭州当虹科技股份有限公司 | Interactive video generation method and playing method and device thereof |
| CN114710568A (en) * | 2022-04-28 | 2022-07-05 | 中移（杭州）信息技术有限公司 | Audio and video data communication method, equipment and storage medium |
| CN114765691A (en) * | 2021-01-13 | 2022-07-19 | 北京字节跳动网络技术有限公司 | Live video function component loading method, data processing method and equipment |
| CN114786022A (en) * | 2022-02-23 | 2022-07-22 | 南京小灿灿网络科技有限公司 | New media live broadcast and user online interaction method |
| CN115695858A (en) * | 2022-11-08 | 2023-02-03 | 天津萨图芯科技有限公司 | SEI encryption-based virtual film production video master film coding and decoding system, method and platform |
| WO2024098836A1 (en) * | 2022-11-11 | 2024-05-16 | 上海哔哩哔哩科技有限公司 | Video alignment method and apparatus |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1995965A1 (en) * | 2007-05-24 | 2008-11-26 | Apple Inc. | Method and apparatus for video frame marking |
| CN101658034A (en) * | 2007-04-17 | 2010-02-24 | 汤姆森许可贸易公司 | Method to transmit video data in a data stream and associated metadata |
| CN102271249A (en) * | 2005-09-26 | 2011-12-07 | 韩国电子通信研究院 | Method and apparatus for defining and reconstructing rois in scalable video coding |
| CN105052153A (en) * | 2013-04-08 | 2015-11-11 | 高通股份有限公司 | Device and method for scalable coding of video information |
| CN106713909A (en) * | 2016-11-16 | 2017-05-24 | 广州弘度信息科技有限公司 | Video encoding method, device and system |
-
2017
- 2017-10-11 CN CN201710942414.XA patent/CN107995155A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102271249A (en) * | 2005-09-26 | 2011-12-07 | 韩国电子通信研究院 | Method and apparatus for defining and reconstructing rois in scalable video coding |
| CN101658034A (en) * | 2007-04-17 | 2010-02-24 | 汤姆森许可贸易公司 | Method to transmit video data in a data stream and associated metadata |
| EP1995965A1 (en) * | 2007-05-24 | 2008-11-26 | Apple Inc. | Method and apparatus for video frame marking |
| CN105052153A (en) * | 2013-04-08 | 2015-11-11 | 高通股份有限公司 | Device and method for scalable coding of video information |
| CN106713909A (en) * | 2016-11-16 | 2017-05-24 | 广州弘度信息科技有限公司 | Video encoding method, device and system |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108924600A (en) * | 2018-06-28 | 2018-11-30 | 乐蜜有限公司 | Sending and receiving methods, device and the electronic equipment of live data |
| CN110213599A (en) * | 2019-04-16 | 2019-09-06 | 腾讯科技（深圳）有限公司 | A kind of method, equipment and the storage medium of additional information processing |
| CN110233716A (en) * | 2019-05-31 | 2019-09-13 | 北京文香信息技术有限公司 | A kind of communication interaction method, apparatus, storage medium, terminal device and server |
| CN110557649B (en) * | 2019-09-12 | 2021-12-28 | 广州方硅信息技术有限公司 | Live broadcast interaction method, live broadcast system, electronic equipment and storage medium |
| CN110557649A (en) * | 2019-09-12 | 2019-12-10 | 广州华多网络科技有限公司 | Live broadcast interaction method, live broadcast system, electronic equipment and storage medium |
| CN112788282A (en) * | 2019-11-08 | 2021-05-11 | 株洲中车时代电气股份有限公司 | Video information acquisition method and application thereof |
| CN112788282B (en) * | 2019-11-08 | 2022-04-12 | 株洲中车时代电气股份有限公司 | Video information acquisition method |
| CN111163360A (en) * | 2020-01-02 | 2020-05-15 | 腾讯科技（深圳）有限公司 | Video processing method, video processing device, computer-readable storage medium and computer equipment |
| CN111464828A (en) * | 2020-05-14 | 2020-07-28 | 广州酷狗计算机科技有限公司 | Virtual special effect display method, device, terminal and storage medium |
| CN112468822A (en) * | 2020-11-06 | 2021-03-09 | 上海钦文信息科技有限公司 | Multimedia recording and broadcasting course interaction method based on video SEI message |
| CN112468822B (en) * | 2020-11-06 | 2023-02-24 | 学利网络科技(上海)有限公司 | Multimedia recording and broadcasting course interaction method based on video SEI message |
| CN114765691B (en) * | 2021-01-13 | 2024-05-28 | 北京字节跳动网络技术有限公司 | Video live broadcasting functional component loading method, data processing method and device |
| CN114765691A (en) * | 2021-01-13 | 2022-07-19 | 北京字节跳动网络技术有限公司 | Live video function component loading method, data processing method and equipment |
| CN113206853A (en) * | 2021-05-08 | 2021-08-03 | 杭州当虹科技股份有限公司 | Video correction result storage improvement method |
| CN114125501A (en) * | 2021-10-30 | 2022-03-01 | 杭州当虹科技股份有限公司 | Interactive video generation method and playing method and device thereof |
| CN114786022A (en) * | 2022-02-23 | 2022-07-22 | 南京小灿灿网络科技有限公司 | New media live broadcast and user online interaction method |
| CN114710568B (en) * | 2022-04-28 | 2023-12-01 | 中移（杭州）信息技术有限公司 | Audio and video data communication method, device and storage medium |
| CN114710568A (en) * | 2022-04-28 | 2022-07-05 | 中移（杭州）信息技术有限公司 | Audio and video data communication method, equipment and storage medium |
| CN115695858A (en) * | 2022-11-08 | 2023-02-03 | 天津萨图芯科技有限公司 | SEI encryption-based virtual film production video master film coding and decoding system, method and platform |
| WO2024098836A1 (en) * | 2022-11-11 | 2024-05-16 | 上海哔哩哔哩科技有限公司 | Video alignment method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107995155A (en) | Video data encoding, decoding, methods of exhibiting, video system and storage medium | |
| CN103931204B (en) | The network flow of media data | |
| CN101283566B (en) | Rtp payload format for vc-1 | |
| TWI463875B (en) | Flexible sub-stream referencing within a transport data stream | |
| CN102404624B (en) | All-format media player capable of supporting hardware decoding for digital STB (Set Top Box) | |
| BR112019019836A2 (en) | signaling important video information in network video streaming using mime parameters | |
| CN103929657B (en) | Video multiplexing apparatus, video multiplexing method, multiplexed video decoding apparatus, and multiplexed video decoding method | |
| CN109076229A (en) | The region being most interested in picture | |
| CN108370455A (en) | Method and apparatus for making to adapt to display characteristic from the decoded video content of basic flow | |
| CN110447234A (en) | Section type as separator and addressable resource identifier | |
| CN106134146A (en) | Process continuous print multicycle content | |
| US20200092600A1 (en) | Method and apparatus for presenting video information | |
| CN105744295A (en) | Providing sequence data sets for streaming video data | |
| US20210176446A1 (en) | Method and device for transmitting and receiving metadata about plurality of viewpoints | |
| CN109963176A (en) | Video code flow processing method, device, the network equipment and readable storage medium storing program for executing | |
| CN108141614A (en) | High dynamic range and wide colour gamut content in signal transmission transport stream | |
| US11405599B2 (en) | MMT apparatus and MMT method for processing stereoscopic video data | |
| CN106331853A (en) | Multimedia de-packaging method and apparatus | |
| US7216288B2 (en) | Dynamic scene description emulation for playback of audio/visual streams on a scene description based playback system | |
| CN113099282B (en) | Data processing method, device and equipment | |
| CN114363648A (en) | Method, equipment and storage medium for audio and video alignment in mixed flow process of live broadcast system | |
| KR101681835B1 (en) | Method and apparatus for constructing sensory effect media data file, method and apparatus for playing sensory effect media data file and structure of the sensory effect media data file | |
| CN105263021B (en) | A kind of HEVC video encoding/decoding methods based on UVD | |
| CN101977326A (en) | Stunt stream generation method based on MPEG (Moving Picture Experts Group)-2 transmission stream | |
| CN105657448B (en) | A kind of retransmission method, the apparatus and system of encoded video stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180504 |
|
| RJ01 | Rejection of invention patent application after publication |