CN108449605A - Synchronizing information playback method, device, equipment, system and storage medium - Google Patents

Synchronizing information playback method, device, equipment, system and storage medium Download PDF

Info

Publication number
CN108449605A
CN108449605A CN201810205908.4A CN201810205908A CN108449605A CN 108449605 A CN108449605 A CN 108449605A CN 201810205908 A CN201810205908 A CN 201810205908A CN 108449605 A CN108449605 A CN 108449605A
Authority
CN
China
Prior art keywords
video frame
target information
information
mark
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810205908.4A
Other languages
Chinese (zh)
Other versions
CN108449605B (en
Inventor
任金鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Mobile Software Co Ltd
Original Assignee
Beijing Xiaomi Mobile Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Mobile Software Co Ltd filed Critical Beijing Xiaomi Mobile Software Co Ltd
Priority to CN201810205908.4A priority Critical patent/CN108449605B/en
Publication of CN108449605A publication Critical patent/CN108449605A/en
Application granted granted Critical
Publication of CN108449605B publication Critical patent/CN108449605B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835—Generation of protective data, e.g. certificates
    • H04N21/8352—Generation of protective data, e.g. certificates involving content or source identification data, e.g. Unique Material Identifier [UMID]

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A kind of synchronizing information playback method of disclosure offer, device, equipment, system and storage medium, the method includes：Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is the mark of the target information；Based on the mapping relations, described information is identified into corresponding target information video frame corresponding with the video frame identification and is played simultaneously.It can ensure the consistency of audio and video and target information using the embodiment of the present disclosure.

Description

Synchronizing information playback method, device, equipment, system and storage medium
Technical field
This application involves direct seeding technique field more particularly to synchronizing information playback method, device, equipment, system and storages Medium.
Background technology
Internet video live broadcasting can be that the user of viewer end watches the ongoing live sound of main broadcaster's end subscriber by network Live video, such as race, meeting, teaching, operation etc..In live streaming pushes away information scene, main broadcaster is not only played in viewer end The live video stream at end also plays other target informations.For example, in the scene of live streaming answer, viewer end can will include topic Purpose answer frame is illustrated on live video, so that spectators are when host in hearing live video using terminal reports topic, Answer frame can also be seen from terminal screen, and is answered.
However, may cause two paths of data there are certain time-delay due to network etc., so as to cause video frame with it is specific The nonsynchronous situation of information.For example, host, which may also, does not report topic, including the answer frame of topic has shown that in screen. Since live streaming answer is very high to requirement of real-time, answer only is completed in specified Reaction time, this answer just judges effectively. Therefore, report topic is asynchronous with the displaying of answer frame can influence answer Effective judgement.
Invention content
To overcome the problems in correlation technique, present disclose provides synchronizing information playback method, device, equipment, be System and storage medium.
According to the first aspect of the embodiments of the present disclosure, a kind of synchronizing information playback method is provided, the method includes：
Determine that the mapping relations between video frame identification and message identification, the video frame identification are it is expected in live video stream The mark for the video frame being played simultaneously with the target information, described information mark are the marks of the target information；
Live video stream is sent to viewer end by direct broadcast service end, the mapping relations and target information are passed through Communication service end is sent to viewer end, so that viewer end is based on the mapping relations, described information, which is identified corresponding target, to be believed Breath video frame corresponding with the video frame identification plays simultaneously.
In an optional realization method, the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
In an optional realization method, described information issues the time and the determination step of video frame time includes：
It receives when sending instructions under the target information, the time is issued using current time as information, by current video The time of frame in video streaming is as video frame time；
Instruction when sending instructions to start to report the target information under described by specifying objects trigger button to generate.
According to the second aspect of the embodiment of the present disclosure, a kind of synchronizing information playback method is provided, the method includes：
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Based on the mapping relations, described information identifies to corresponding target information is corresponding with the video frame identification to be regarded Frequency frame plays simultaneously.
In an optional realization method, the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
In an optional realization method, the method further includes：
When starting to be played simultaneously corresponding with the video frame identification video frame of the corresponding target information of message identification, opens and fall to count When, stop playing the target information or disabling and the associated operational controls of the target information at the end of countdown；
Wherein, the count down time of countdown is determined based on the difference of standard count down time and terminal network delay, institute It is the network delay between viewer end and communication service end to state terminal network delay.
According to the third aspect of the embodiment of the present disclosure, a kind of synchronizing information playing device is provided, described device includes：
Relationship determination module, the mapping relations being configured to determine that between video frame identification and message identification, the video frame It is identified as the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, described information mark is described The mark of target information；
Information sending module is configured as live video stream being sent to viewer end by direct broadcast service end, be reflected described It penetrates relationship and target information and viewer end is sent to by communication service end, so that viewer end is based on the mapping relations, by institute The corresponding target information of message identification video frame corresponding with the video frame identification is stated to play simultaneously.
In an optional realization method, the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
In an optional realization method, the relationship determination module is specifically configured to：
It receives when sending instructions under the target information, the time is issued using current time as information, by current video The time of frame in video streaming as video frame time, obtains the mapping relations between video frame identification and message identification；
Wherein, finger when sending instructions to start to report the target information under described by specifying objects trigger button to generate It enables.
According to the fourth aspect of the embodiment of the present disclosure, a kind of synchronizing information playing device is provided, described device includes：
Information acquisition module is configured as obtaining live video stream, target information and video frame identification and message identification Between mapping relations, the video frame identification is the video frame for it is expected to be played simultaneously with the target information in live video stream Mark, described information mark are the marks of the target information；
Information playing module, be configured as be based on the mapping relations, by described information identify corresponding target information with The corresponding video frame of the video frame identification plays simultaneously.
In an optional realization method, the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
In an optional realization method, described device further includes countdown module, is configured as：
When starting to be played simultaneously corresponding with the video frame identification video frame of the corresponding target information of message identification, opens and fall to count When, stop playing the target information or disabling and the associated operational controls of the target information at the end of countdown；
Wherein, the count down time of countdown is determined based on the difference of standard count down time and terminal network delay, institute It is the network delay between viewer end and communication service end to state terminal network delay.
According to a fifth aspect of the embodiments of the present disclosure, a kind of live broadcast system is provided, including main broadcaster end, direct broadcast service end, logical Telecommunications services end, viewer end；
Main broadcaster end determines the mapping relations between video frame identification and message identification, and live video stream is passed through direct broadcast service End is sent to viewer end, and the mapping relations and target information are sent to viewer end, the video by communication service end Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Viewer end obtains live video stream, target information and mapping relations, the mapping relations is based on, by described information Corresponding target information video frame corresponding with the video frame identification is identified to play simultaneously.
According to the 6th of the embodiment of the present disclosure the aspect, a kind of electronic equipment is provided, including：
Processor；
Memory for storing processor-executable instruction；
Wherein, the processor is configured as：
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Based on the mapping relations, described information identifies to corresponding target information is corresponding with the video frame identification to be regarded Frequency frame plays simultaneously.
According to the 7th of the embodiment of the present disclosure the aspect, a kind of computer readable storage medium is provided, calculating is stored thereon with Machine program, when which is executed by processor the step of realization any of the above-described the method.
The technical scheme provided by this disclosed embodiment can include the following benefits：
The embodiment of the present disclosure determines the mapping relations between video frame identification and message identification at main broadcaster end, since viewer end can To obtain the mapping relations between video frame identification and message identification, therefore can be asked according to mapping relations by message identification is corresponding Topic topic video frame corresponding with video frame identification plays simultaneously, to ensure the consistency of audio and video and target information, improves User experience.
It should be understood that above general description and following detailed description is only exemplary and explanatory, not The disclosure can be limited.
Description of the drawings
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure Example, and together with specification for explaining the principles of this disclosure.
Fig. 1 is a kind of application scenarios schematic diagram of realizing live streaming of the disclosure shown according to an exemplary embodiment.
Fig. 2A is a kind of live broadcast system schematic diagram of the disclosure shown according to an exemplary embodiment.
Fig. 2 B are a kind of flow charts of synchronizing information playback method of the disclosure shown according to an exemplary embodiment.
Fig. 2 C are a kind of live streaming answer interface schematic diagram of the disclosure shown according to an exemplary embodiment.
Fig. 3 is a kind of application scenario diagram of synchronizing information playback method of the disclosure shown according to an exemplary embodiment.
Fig. 4 A or Fig. 4 B are the stream of another synchronizing information playback method of the disclosure shown according to an exemplary embodiment Cheng Tu.
Fig. 5 is a kind of block diagram of synchronizing information play system of the disclosure shown according to an exemplary embodiment.
Any one of Fig. 6 to 8 is a kind of frame of synchronizing information playing device of the disclosure shown according to an exemplary embodiment Figure.
Fig. 9 is a kind of block diagram of the device that for synchronizing information plays of the disclosure shown according to an exemplary embodiment.
Specific implementation mode
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended The example of the consistent device and method of some aspects be described in detail in claims, the disclosure.
It is the purpose only merely for description specific embodiment in the term that the disclosure uses, is not intended to be limiting the disclosure. The "an" of singulative used in disclosure and the accompanying claims book, " described " and "the" are also intended to including majority Form, unless context clearly shows that other meanings.It is also understood that term "and/or" used herein refers to and wraps Containing one or more associated list items purposes, any or all may be combined.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the disclosure A little information should not necessarily be limited by these terms.These terms are only used for same type of information being distinguished from each other out.For example, not departing from In the case of disclosure range, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ... When " or " in response to determination ".
Internet video live broadcasting can be that the user of viewer end watches the ongoing live sound of main broadcaster's end subscriber by network Live video, such as race, meeting, teaching, operation etc..In certain live scenes, main broadcaster end is not only played in viewer end Live video stream, also play other target informations.For example, in the scene of live streaming answer, target information may include problem Topic.It is that user (spectators) within a specified time logs in direct broadcasting room that question-answering mode, which is broadcast live, in the case where host guides into answering on line Topic, answering questions specified quantity (such as 12) topic can rush to summit bonus, carve up the bonus of each issue of setting.In viewer end, can will wrap Answer frame containing topic is illustrated on live video, so that host reports topic to spectators in hearing live video using terminal When, answer frame can be seen from terminal screen, and answer.For another example, it is rushed to purchase in scene in live streaming, target information can wrap Include goods for sale information.Be broadcast live panic buying pattern can be that user within a specified time logs in direct broadcasting room, host guide under into It is rushed to purchase on line.In viewer end, goods for sale information can be illustrated on live video, spectators can hear in live video Host reports merchandise news, while can select the commodity of preference, and carry out the operation that places an order/buy with operation display.
It is understood that the adaptable scene of the embodiment of the present disclosure：During live streaming, viewer end can receive two ways According to a circuit-switched data can be live video stream, and another circuit-switched data can be target information.It is closed with target information in live video stream The video frame of connection need to be played simultaneously with target information.Wherein, target information exists with an at least frame video frame in live video stream Incidence relation.For example, incidence relation can the presence of the video frame for reporting target information in live video stream.In addition, target is believed Breath can be problem topic, goods for sale information, the data with reproduction time limitation such as wait for recall info, not arrange one by one herein It lifts.
As shown in Figure 1, Fig. 1, which is a kind of shown according to an exemplary embodiment of the disclosure, realizes that the application scenarios of live streaming show It is intended to.In this scenario, it may include main broadcaster end 110, server-side 120, the first viewer end 131, the second viewer end 132.It can be with Understand, which is illustrated by taking two viewer ends as an example, in fact, viewer end can be one or more.It is main It can be video flowing originating end to broadcast end, and viewer end can be video flowing receiving terminal.In certain scenes, main broadcaster end is properly termed as presiding over End, viewer end are properly termed as user terminal.Main broadcaster end, viewer end are (such as：First viewer end and the second viewer end) can be mounted Software on an electronic device can also be terminal device.Main broadcaster end can call the sides such as camera recorded video, shooting photo Formula makes live data, is then sent to server-side by network.Viewer end and main broadcaster end can be by the interactive messages of input (such as：Barrage message) it is sent to server-side, realize that each end in same direct broadcasting room can show interactive message.And it is directed to target Information is often sent to server-side by main broadcaster end, and each viewer end is issued by server-side.Server-side is for providing internet The background service of live streaming, such as the correspondence at main broadcaster end and viewer end is preserved, and it is broadcast live the distribution of data, and it is interactive The distribution of message and the distribution of target information etc..
Wherein, the server-side can be the general designation of multiple servers equipment, can also be that at least one be mounted on services The general designation of software in device equipment.In one example, can live video stream distribution and number be realized by the same server-side According to stream distribution.In another example, in order to distinguish different business and improve the promptness of information processing, server-side can be with Including direct broadcast service end and communication service end etc., direct broadcast service end can be used for being broadcast live the distribution of video flowing, communication service Hold the distribution for carrying out data, the especially distribution of target information.It is understood that according to actual demand, server-side is also It may include barrage server-side, present server-side etc., do not repeat one by one herein.
It can be able to be that hand-held electronics is set with the electronic equipment of runs software, electronic equipment that the electronic equipment, which can be various, Standby or other electronic equipments.For example, it may be cellular phone, media player or other handheld portable devices, such as watch The slightly smaller portable device such as equipment or other wearable or compact apparatus, can also be PDA (Personal Digital Assistant, personal digital assistant), tablet computer, notebook computer, desktop computer, television set, collection At the computer or other electronics etc. in computer display.
The embodiment of the present disclosure is illustrated with one of which live broadcast system.As shown in Figure 2 A, Fig. 2A is disclosure root A kind of live broadcast system schematic diagram shown according to an exemplary embodiment.As shown in Figure 2 B, Fig. 2 B are that the disclosure is exemplary according to one Implement a kind of flow chart of the synchronizing information playback method exemplified.The method is applied to live broadcast system, and live broadcast system can be with Including main broadcaster end 210, direct broadcast service end 220, communication service end 230, viewer end 240.The embodiment includes live streaming with server-side Carry out example for server-side and communication service end, video stream distribution be broadcast live by direct broadcast service end, by communication service end into Row data distribution.It is understood that in other examples, direct broadcast service end and communication service end can also be same service End.The method includes：
The step of main broadcaster end 210 executes may include step 201 to 203：
In step 201, the mapping relations between video frame identification and message identification are determined.
In step 202, live video stream is sent to viewer end by direct broadcast service end.
In step 203, the mapping relations and target information are sent to viewer end by communication service end.
Wherein, the video frame identification is the video frame for it is expected to play simultaneously with the target information in live video stream Mark, when it is multiframe it is expected the video frame played simultaneously with target information in live video stream, the video described in the present embodiment Frame identification can be the mark of the first frame associated video frame in multiframe associated video frame, to play first frame associated video frame When, start simultaneously at broadcasting target information.Video frame identification can also be the set of the mark of all associated video frames, or first frame The mark of associated video frame and the mark of tail frame associated video frame play simultaneously when so as to during playing associated video frame Target information.Described information mark is the mark of the target information.
The step of viewer end 240 executes may include step 204 and 205：
In step 204, live video stream, target information and mapping relations are obtained；
In step 205, the mapping relations are based on, described information is identified into corresponding target information and the video frame It identifies corresponding video frame while playing.
Wherein, server-side is not shown in fig. 2b.Main broadcaster end can call camera recorded video, shooting photo, also may be used Directly to obtain audio, video data etc. from audio-video collection end, and make acquisition live data.In one example, in order to reach Audio and video synchronize, and can carry out audio data and image data after live streaming end obtains audio data and image data Synthesis, and ensure that audio data is synchronous with image data, lead to picture and the nonsynchronous situation of audio to avoid transmission process.
It, can be with for the scene that need to be played simultaneously with target information with the associated video frame of target information in live video stream It is that user within a specified time logs in direct broadcasting room, into operating on line under the guiding of host.Wherein, in live video stream with The associated video frame of target information is properly termed as associated video frame.Pass through picture of the collection terminal shooting comprising host and acquisition Respective audio issues target information, so that viewer end can receive live video fluxion when host reports target information According to, and target information can be received, and while playing live video stream, play target information.
The target information be with the related information of live video stream, divide target information by type, target information can To be character information, image information, audio-frequency information, video information etc..By division of teaching contents target information, target information can be asked Inscribe topic, goods for sale information etc..During live streaming, the user (such as host) at main broadcaster end can report the target information. So-called report target information, if target information is character information or image information, it can read target letter to report target information The particular content of breath can also be to read the instruction information for being used to indicate target information；If target information is character information, image Information, audio-frequency information or video information, it can play target information to report target information, at the same time it can also to target information It is introduced.Wherein, broadcasting can be displaying, can also be to broadcast.By taking target information is problem topic as an example, in an example In, main broadcaster's end subscriber can report the particular content of problem topic, such as：Which state famous " piss child " bronze statue is located at Family, or even answer choice can also be reported, such as：One, Dutch；Two, Belgian；Three, Irish.In another example, main broadcaster End subscriber can only report the topic quantity of problem topic, such as：Look at the first topic.
In order to realize that target information is played with associated video frame synchronization in live video stream, the embodiment of the present disclosure determines video The mapping relations of frame identification and message identification.
Wherein, video frame identification can be intended to indicate that the mark of associated video frame, associated video frame can be that live streaming regards Frequency it is expected the video frame being played simultaneously with target information in flowing.If live video stream includes multiframe associated video frame, this reality The mark that the video frame identification described in example can be the first frame associated video frame in multiframe associated video frame is applied, to play head When frame associated video frame, broadcasting target information is started simultaneously at.In one example, video frame identification can be associated video frame Label information, for example, title, the information such as number, ID.And in practical applications, the program of viewer end, which is often determined with the time, to be regarded Frequency frame, in order to reduce the change to viewer end program, in another example, video frame identification can be that main broadcaster end sends target The time of current video frame (video frame currently generated) is properly termed as video in order to which same other times distinguish when information Frame time.Video frame time can be the time of current video frame in video streaming.Since target information often needs at main broadcaster end It is issued with then can just be triggered, i.e., when main broadcaster end sends target information, positive generation and the associated video frame of target information, Therefore, the video frame currently generated be with the related video frame of target information, need to viewer end simultaneously play the video frame and Target information.
Wherein, message identification is the information for identifying target information.In one example, message identification can be target The label information of information, for example, title, the information such as number, ID.In another example, message identification can be main broadcaster end hair Time when target information is sent, in order to be distinguished with other times, information is properly termed as and issues the time.
The method of determination that time and video frame time are issued for information can be in an optional realization method When sending instructions under receiving target information, the time is issued using current time as information, by current video frame in video flowing In time as video frame time.
Wherein, it is the instruction for sending target information to send instructions under.Under the condition that sends instructions, can be rectify in main broadcaster/it is pre- It generates associated video frame and triggers the instruction of generation under the pre- opportunity for playing target information.Sending instructions under for example, can be open The instruction that triggering generates when beginning to report target information.The present embodiment, which will start to report under target information is generated as triggering, to send instructions Condition, may be implemented viewer end by target information and report target information when video frame synchronization play.
In one example, sending instructions under can be given birth to by specified objects trigger button when starting to report the target information At instruction.Button can be physical button or touch virtual button.For example, when host starts to report target information, such as： When host reads " first topic be ... ", from the button on staff's pointing device with host, realize triggering life At sending instructions under target information.As it can be seen that by the transmission opportunity to send instructions under manual control, it is easy to implement.
In another example, sending instructions under can be that recognize live video frame include preset keyword or pre- to equipment If crucial audio, then trigger and send instructions under generation.For example, when host starts to report target information, such as：Host reads " first Topic is ... " when, main broadcaster end identifies the default crucial voice of phonetic acquisition, then triggers to generate and send instructions under target information.As it can be seen that By speech recognition or image recognition, it can be generated down and be sent instructions with automatic trigger, save cost of labor.
Send instructions to be triggered by other conditions under it is understood that and generate, does not repeat one by one herein.
The mapping relations and target information are sent to viewer end by communication service end about main broadcaster end, in main broadcaster End issues data flow by communication channel, and communication channel can be fetched realization by long-chain.Long link：The link of terminal and server-side, It can be with bi-directional transfer of data.The link of usually TCP or UDP, after establishing the link, from server-side to user equipment downlink propelling data, Usually heartbeat packet is sent by terminal uplink to server-side.If can't detect heartbeat packet or do not return, server-side can break chain It connects, client can be linked again.
In the embodiment of the present disclosure, main broadcaster end can send target information to viewer end, and target information can be comprising specific The information of content.In order to improve the safety of target information, it can send to add the content of target information to send target information Information after close.As it can be seen that encrypted target information is sent to viewer end by communication service end, both ensure target information Promptness, and can ensure the safety of target information particular content.
It is understood that the embodiment of the present disclosure by viewer end send target information for carry out example, other can In the realization method of choosing, main broadcaster end can send the ID of target information to viewer end.In one example, it can will include target The database of information is preloaded in viewer end.When viewer end receives the ID of target information, acquisition and ID from pre-stored data library Corresponding target information.
However, there is the risk cracked by people in the database for being stored in viewer end, in order to avoid this risk, at another In example, main broadcaster end, can be according to target information ID from the business service for being stored with target information after obtaining target information ID Target information corresponding with ID is obtained in device, to improve the safety of target information.
After viewer end obtains live video stream, target information and mapping relations, mapping relations can be based on by the letter The corresponding target information of breath mark video frame corresponding with the video frame identification plays simultaneously.Wherein, play include displaying with It broadcasts, for example, displaying target information, broadcasts video data stream.In practical applications, since the transmission speed of data flow often compares The transmission speed of video flowing is fast, therefore, viewer end receive target information time may earlier than video frame associated with it, because This does not show immediately after receiving target information, and is stored in local.Pair based on message identification and video frame identification It should be related to, when playing video frame corresponding with video frame identification, play the corresponding target information of corresponding message identification.Example Such as, the correspondence that time and video frame time are issued according to information, when player plays to the corresponding video of video frame time When frame, corresponding problem topic is shown immediately, to ensure picture, audio, the consistency of target information.
Wherein, target information is played about in viewer end, it, can be by target information exhibition if target information is display information Show on the picture corresponding to live video stream.When display, target information can be directly displayed, target information can also be carried out It is shown again after processing, to provide the control that can be operated by spectators.Can will include to ask so that target information is problem topic as an example The answer frame of topic topic is shown on video frame picture.In one example, include not only item content in answer frame, also carry For answer input control, so that spectators in answer input control by inputting answer.In another example, it answers to improve Efficiency, each option all can be trigger control, when spectators click or touch option, the option that is triggered is as spectators' submission Answer.As shown in Figure 2 C, Fig. 2 C are a kind of live streaming answer interface schematic diagram of the disclosure shown according to an exemplary embodiment. In the schematic diagram, each option all can be trigger control, when the user clicks when one of option, the option that is triggered The answer that (Belgium) submits as spectators.
In an optional realization method, the reproduction time of target information has time restriction, can pass through setting The mode of countdown determines the reproduction time of target information.Specifically, from starting to play target information, countdown is opened, It can stop playing the target information or disabling and the associated operational controls of the target information at the end of timing, to realize Spectators are forbidden to continue to operate at the end of countdown.
Wherein, the count down time of countdown can be the preset set time, for example, countdown can be 10s. As it can be seen that by directly setting count down time, it is easy to implement.
However, in practical application, since different spectators' end equipments are because of reasons such as networks, lead to the receiving time of target information It is different.And by the fixed count down time of setting often since receiving and timing showing target information, so as to occur Following situations：Since distinct device receives target information there may be the time difference, utilize the sight of the slow equipment of receiving time Crowd, can be by the target information in the fast equipment of viewing receiving time, to know target information in advance.In order to avoid above-mentioned Situation, the count down time of countdown is based on the difference determination of standard count down time and terminal network delay, the terminal network Network delay can be the network delay between viewer end and communication service end.
In one example, terminal network delay can based on communication service end initiate clock synchronization request sending time with And viewer end receives the receiving time determination of the clock synchronization request.It is taken for example, communication service end is periodically sent to communication service end Clock synchronization request with sending time, in receiving time ask based on clock synchronization so as to viewer end and request entrained by sending time Difference, determine terminal network be delayed.
In another example, terminal network delay can also send the sending time of target information based on communication service end And the receiving time of viewer end reception target information determines, when to realize transmission target information every time, can determine network Delay, realizes the real-time update of network delay.
In order to remind the progress of countdown, count down time can be reminded.The mode of prompting can be that voice carries It wakes up, can also be display alarm.When display alarm, it can show count down time number, the modes table such as progress bar can also be used Show.Such as shown in Fig. 2 C, countdown progress is shown with bar shaped progress bar in figure.It is understood that progress bar can be bar shaped, The shapes such as circular ring-type, are not limited herein.
Operating result can be back to server-side by viewer end.For example, operating result can be back to communication service end Or other business service ends.Server-side can be for statistical analysis to operating result, and analysis result is fed back to viewer end and straight Broadcast end.By taking target information is problem topic as an example, server-side can count the selection percentage of each option, and then by statistical result Feed back to viewer end and main broadcaster end.
Various technical characteristics in embodiment of above can be arbitrarily combined, as long as the combination between feature is not present Conflict or contradiction, but as space is limited, it is not described one by one, therefore the various technical characteristics in the above embodiment is arbitrary It is combined the range for also belonging to this disclosure.It is illustrated below with one of which embodiment：
As shown in figure 3, Fig. 3 is a kind of answering for synchronizing information playback method of the disclosure shown according to an exemplary embodiment With scene graph, the method is applied to live streaming answering system, and live streaming answering system may include main broadcaster end 310, direct broadcast service end 320, communication service end 330, viewer end 340.Viewer end may include the first viewer end and N viewer ends, and N is more than 1 Integer.
The audio, video data that main broadcaster end can be acquired based on collection terminal generates live data streams, is reported in problem topic When, the time (i.e. topic issues the time) is issued using current time as information, the time of current video frame in video streaming is made For video frame time, the mapping relations that video frame time issues the time with topic are obtained, and live video stream is taken by being broadcast live Business end is sent to viewer end, and mapping relations and problem topic are sent to viewer end by communication service end.
Viewer end is sent to by communication service end about by problem topic, it in one example, can be directly by problem Topic is sent to communication service end, and problem topic is sent to viewer end by communication service end again.In another example, communication clothes Business end can prestore problematic topic, or even can also store the answer of problematic topic, then main broadcaster end can be by problem The topic mark (such as ID) of topic is sent to communication service end, and communication service end identifies the problem of determination topic according to topic, and will Determining problem topic is sent to viewer end.Further, problem topic can also be encrypted for communication service end, and will add Problem topic after close is sent to viewer end, to improve the safety of problem topic.
Viewer end obtains live video stream, problem topic and mapping relations, can be issued topic according to mapping relations Problem topic video frame corresponding with video frame time corresponding to time plays simultaneously.It, can be with for example, after receiving problem topic Problem topic is stored in local.The mapping relations that the time is issued based on video frame time and topic, when playing with video frame Between corresponding video frame when, show that corresponding topic issues problem topic corresponding to time on the screen, to ensure picture, sound Frequently, the consistency of topic.
Further, the reproduction time of target information can be determined in such a way that countdown is set.Such as, from starting to broadcast It puts problem topic to rise, opens countdown, can stop playing the target information or disabling and the target at the end of countdown The operational controls of information association forbid spectators to continue to operate to realize at the end of countdown.The count down time of countdown can To be determined based on standard count down time and the difference of terminal network delay, the terminal network delay can be viewer end and lead to Network delay between telecommunications services end.
After communication service terminates the answer for receiving viewer end, the validity of answer can be judged.Reaction time can To be the preset judgement time, for example, it may be 10s.Judge that the time is identical as standard count down time.In an example In, consider that communication takes, answer time-out time allows that there are tolerance values.When judging the validity of answer, it can will ask The difference for inscribing topic sending time and answer receiving time, with standard count down time and tolerance value and value be compared, root The validity of answer is determined according to comparison result, can delete invalid answer.Tolerance value can take system based on historical communication The time value obtained is counted, such as adds 1s on the basis of original Reaction time, the answer received in 11s is considered effective Answer.Tolerance value can also be to issue problem topic object time based on communication service end and issue the time of first frame associated video frame Difference obtain.
It is understood that the present embodiment is existed with sending mapping relations, problem topic and carrying out interpretation of result with statistics For same server-side (communication service end) is realized, in other embodiments, sending mapping relations and problem topic can be by Communication service end is completed, and interpretation of result can be completed with statistics by answer server-side (business service end), no longer be arranged one by one herein It lifts.
As seen from the above-described embodiment, since record has video frame time to issue the mapping relations of time, Ke Yigen with topic Topic is issued problem topic video frame corresponding with video frame time corresponding to time according to mapping relations to play simultaneously, to protect Picture, audio, the consistency of topic are demonstrate,proved, user experience is improved, while introducing fair timing strategy, accomplishes that fair answer is fallen Timing.
Next, being illustrated from the angle at main broadcaster end to the embodiment of the present disclosure.
As shown in Figure 4 A, Fig. 4 A are another synchronizing information playback method of the disclosure shown according to an exemplary embodiment Flow chart, this method can be used in main broadcaster end, includes the following steps：
In step 401, determine that the mapping relations between video frame identification and message identification, the video frame identification are live streaming The mark for the video frame being played simultaneously with the target information it is expected in video flowing, described information mark is the target information Mark；
In step 402, live video stream is sent to viewer end by direct broadcast service end, by the mapping relations and Target information is sent to viewer end by communication service end, so that described information is identified corresponding target information and institute by viewer end It states the corresponding video frame of video frame identification while playing.
Wherein, the video frame identification is the video frame for it is expected to play simultaneously with the target information in live video stream Mark.In one example, if it is expected that the video frame played simultaneously with target information is multiframe in live video stream, this implementation Video frame identification described in example can be the mark of the first frame associated video frame in multiframe associated video frame, to play first frame When associated video frame, broadcasting target information is started simultaneously at.In another example, video frame identification can also be that institute relevant regards The set of the mark of frequency frame, or the mark of first frame associated video frame and tail frame associated video frame mark, to play When during associated video frame, while playing target information.Described information mark is the mark of the target information.
The target information be with the related information of live video stream, divide target information by type, target information can To be character information, image information, audio-frequency information, video information etc..By division of teaching contents target information, target information can be asked Inscribe topic, goods for sale information etc..It is properly termed as associated video frame with the associated video frame of target information in live video stream, depending on Frequency frame identification can be intended to indicate that the mark of associated video frame.In an optional realization method, video frame identification can be with The time of current video frame (video frame currently generated) when sending target information for main broadcaster end, in order to which same other times carry out area Point, it is properly termed as video frame time.Video frame time can be the time of current video frame in video streaming.Due to target information It often needs to be issued with then can just be triggered at main broadcaster end, i.e., when main broadcaster end sends target information, positive generate is believed with target Cease associated video frame, therefore, the video frame currently generated be with the related video frame of target information, need to viewer end simultaneously Play the video frame and target information.In an optional realization method, message identification can be that main broadcaster end sends target letter Time when breath is properly termed as information and issues the time to be distinguished with other times.
Live video stream can be sent to viewer end by main broadcaster end by direct broadcast service end, and mapping relations and target are believed Breath is sent to viewer end by communication service end, after obtaining live video stream, target information and mapping relations so as to viewer end, Described information can be identified into corresponding target information video frame corresponding with the video frame identification simultaneously based on mapping relations It plays.
It is understood that the embodiment of the present disclosure is identical as embodiment the relevant technologies shown in Fig. 2 B, do not go to live in the household of one's in-laws on getting married one by one herein It states.
As seen from the above-described embodiment, the mapping relations between video frame identification and message identification, Ke Yiti are determined by main broadcaster end The accuracy of high mapping relations, and after mapping relations are sent to viewer end, viewer end can be according to mapping relations by information mark Know corresponding problem topic video frame corresponding with video frame identification to play simultaneously, to ensure picture, audio, target information Consistency improves user experience.
Next, being illustrated from the angle of viewer end to the embodiment of the present disclosure.
As shown in Figure 4 B, Fig. 4 B are another synchronizing information playback method of the disclosure shown according to an exemplary embodiment Flow chart, this method can be used in viewer end, includes the following steps：
In step 403, the mapping between acquisition live video stream, target information and video frame identification and message identification is closed System, the video frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, described Message identification is the mark of the target information；
In step 404, the mapping relations are based on, described information is identified into corresponding target information and the video frame It identifies corresponding video frame while playing.
Wherein, the video frame identification is the video frame for it is expected to play simultaneously with the target information in live video stream Mark.In one example, if it is expected that the video frame played simultaneously with target information is multiframe in live video stream, this implementation Video frame identification described in example can be the mark of the first frame associated video frame in multiframe associated video frame, to play first frame When associated video frame, broadcasting target information is started simultaneously at.In another example, video frame identification can also be that institute relevant regards The set of the mark of frequency frame, or the mark of first frame associated video frame and tail frame associated video frame mark, to play When during associated video frame, while playing target information.Described information mark is the mark of the target information.
The target information be with the related information of live video stream, divide target information by type, target information can To be character information, image information, audio-frequency information, video information etc..By division of teaching contents target information, target information can be asked Inscribe topic, goods for sale information etc..It is properly termed as associated video frame with the associated video frame of target information in live video stream, depending on Frequency frame identification can be intended to indicate that the mark of associated video frame.In an optional realization method, video frame identification can be with The time of current video frame (video frame currently generated) when sending target information for main broadcaster end, in order to which same other times carry out area Point, it is properly termed as video frame time.Video frame time can be the time of current video frame in video streaming.Due to target information It often needs to be issued with then can just be triggered at main broadcaster end, i.e., when main broadcaster end sends target information, positive generate is believed with target Cease associated video frame, therefore, the video frame currently generated be with the related video frame of target information, need to viewer end simultaneously Play the video frame and target information.In an optional realization method, message identification can be that main broadcaster end sends target letter Time when breath is properly termed as information and issues the time to be distinguished with other times.
About the acquisition pattern of target information, viewer end can receive the target information of server-side transmission, can also basis The ID for the target information that communication service end is sent obtains the corresponding target informations of ID from local or other server-sides, does not do herein Limitation.
After viewer end obtains live video stream, target information and mapping relations, mapping relations can be based on by the letter The corresponding target information of breath mark video frame corresponding with the video frame identification plays simultaneously.Wherein, play include displaying with It broadcasts, for example, displaying target information, broadcasts video data stream.In practical applications, since the transmission speed of data flow often compares The transmission speed of video flowing is fast, therefore, viewer end receive target information time may earlier than video frame associated with it, because This does not show immediately after receiving target information, and is stored in local.Pair based on message identification and video frame identification It should be related to, when playing video frame corresponding with video frame identification, play the corresponding target information of corresponding message identification, to Ensure picture, audio, the consistency of target information.
Wherein, target information is played about in viewer end, it, can be by target information exhibition if target information is display information Show on the picture corresponding to live video stream.When display, target information can be directly displayed, target information can also be carried out It is shown again after processing, to provide the control that can be operated by spectators.
In an optional realization method, the reproduction time of target information has time restriction, can pass through setting The mode of countdown determines the reproduction time of target information.Specifically, from starting to play target information, countdown is opened, It can stop playing the target information or disabling and the associated operational controls of the target information at the end of timing, to realize Spectators are forbidden to continue to operate at the end of countdown.
Wherein, the count down time of countdown can be the preset set time, for example, countdown can be 10s. As it can be seen that by directly setting count down time, it is easy to implement.
However, in practical application, since different spectators' end equipments are because of reasons such as networks, lead to the receiving time of target information It is different.And by the fixed count down time of setting often since receiving and timing showing target information, so as to occur Following situations：Since distinct device receives target information there may be the time difference, utilize the sight of the slow equipment of receiving time Crowd, can be by the target information in the fast equipment of viewing receiving time, to know target information in advance.In order to avoid above-mentioned Situation, the count down time of countdown is based on the difference determination of standard count down time and terminal network delay, the terminal network Network delay can be the network delay between viewer end and communication service end.
In one example, terminal network delay can based on communication service end initiate clock synchronization request sending time with And viewer end receives the receiving time determination of the clock synchronization request.It is taken for example, communication service end is periodically sent to communication service end Clock synchronization request with sending time, in receiving time ask based on clock synchronization so as to viewer end and request entrained by sending time Difference, determine terminal network be delayed.
In another example, terminal network delay can also send the sending time of target information based on communication service end And the receiving time of viewer end reception target information determines, when to realize transmission target information every time, can determine network Delay, realizes the real-time update of network delay.
Operating result can be back to server-side by viewer end.For example, operating result can be back to communication service end Or other business service ends.Server-side can be for statistical analysis to operating result, and analysis result is fed back to viewer end and straight Broadcast end.By taking target information is problem topic as an example, server-side can count the selection percentage of each option, and then by statistical result Feed back to viewer end and main broadcaster end.
It is understood that the embodiment of the present disclosure is identical as embodiment the relevant technologies shown in Fig. 2 B, do not go to live in the household of one's in-laws on getting married one by one herein It states.
As seen from the above-described embodiment, since viewer end can obtain the mapping relations between video frame identification and message identification, Therefore problem topic corresponding to message identification video frame corresponding with video frame identification can simultaneously be played according to mapping relations, To ensure picture, audio, the consistency of target information, user experience is improved.
It is corresponding with the embodiment of aforementioned information synchronous broadcast method, the disclosure additionally provide synchronizing information playing device, The embodiment of equipment, system and storage medium that device is applied.
As shown in figure 5, Fig. 5 is a kind of frame of synchronizing information play system of the disclosure shown according to an exemplary embodiment Figure is used for live broadcast system, and live broadcast system includes main broadcaster end, direct broadcast service end, communication service end, viewer end, the system comprises Relationship determination module 510 and information sending module 520 set on main broadcaster end, and the information acquisition module set on viewer end 530 and information playing module 540：
Relationship determination module 510, the mapping relations being configured to determine that between video frame identification and message identification；
Information sending module 520 is configured as live video stream being sent to viewer end by direct broadcast service end, will be described Mapping relations and target information are sent to viewer end by communication service end, and the video frame identification is live video stream mid-term Hope that the mark for the video frame being played simultaneously with the target information, described information mark are the marks of the target information；
Information acquisition module 530 is configured as obtaining live video stream, target information and mapping relations；
Information playing module 540 is configured as being based on the mapping relations, described information is identified corresponding target information Video frame corresponding with the video frame identification plays simultaneously.
As shown in fig. 6, Fig. 6 is a kind of frame of synchronizing information playing device of the disclosure shown according to an exemplary embodiment Figure, described device include：
Relationship determination module 610, the mapping relations being configured to determine that between video frame identification and message identification, the video Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Information sending module 620 is configured as live video stream being sent to viewer end by direct broadcast service end, will be described Mapping relations and target information are sent to viewer end by communication service end, so that viewer end is based on the mapping relations, it will The corresponding target information of described information mark video frame corresponding with the video frame identification plays simultaneously.It is optional real at one In existing mode, the target information includes problem topic.
In an optional realization method, described information is identified as under information when main broadcaster end sends the target information Send out the time.
In an optional realization method, the video frame identification is that forward sight is worked as in main broadcaster end when sending the target information The video frame time of frequency frame.
In an optional realization method, the relationship determination module 610 is specifically configured to：
It receives when sending instructions under the target information, the time is issued using current time as information, by current video The time of frame in video streaming as video frame time, obtains the mapping relations between video frame identification and message identification；
Wherein, finger when sending instructions to start to report the target information under described by specifying objects trigger button to generate It enables.
As shown in fig. 7, Fig. 7 is another synchronizing information playing device of the disclosure shown according to an exemplary embodiment Block diagram, described device include：
Information acquisition module 710 is configured as obtaining live video stream, target information and video frame identification and information mark Mapping relations between knowledge, the video frame identification are the video frame for it is expected to be played simultaneously with the target information in live video stream Mark, described information mark be the target information mark；
Information playing module 720 is configured as being based on the mapping relations, described information is identified corresponding target information Video frame corresponding with the video frame identification plays simultaneously.
In an optional realization method, the target information includes problem topic.
In an optional realization method, described information is identified as under information when main broadcaster end sends the target information Send out the time.
In an optional realization method, the video frame identification is that forward sight is worked as in main broadcaster end when sending the target information The video frame time of frequency frame.
As shown in figure 8, Fig. 8 is another synchronizing information playing device of the disclosure shown according to an exemplary embodiment Block diagram, for the embodiment on the basis of aforementioned embodiment illustrated in fig. 7, described device further includes countdown module 730, is configured For：
When starting to be played simultaneously corresponding with the video frame identification video frame of the corresponding target information of message identification, opens and fall to count When, stop playing the target information or disabling and the associated operational controls of the target information at the end of countdown；
Wherein, the count down time of countdown is determined based on the difference of standard count down time and terminal network delay, institute It is the network delay between viewer end and communication service end to state terminal network delay.
Correspondingly, the disclosure also provides a kind of live broadcast system, including main broadcaster end, direct broadcast service end, communication service end, spectators End；
Main broadcaster end determines the mapping relations between video frame identification and message identification, and live video stream is passed through direct broadcast service End is sent to viewer end, and the mapping relations and target information are sent to viewer end, the video by communication service end Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Viewer end obtains live video stream, target information and mapping relations, the mapping relations is based on, by described information Corresponding target information video frame corresponding with the video frame identification is identified to play simultaneously.
Correspondingly, the disclosure also provides a kind of electronic equipment, the equipment includes processor；It can for storing processor The memory executed instruction；Wherein, the processor is configured as：
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；
Based on the mapping relations, described information identifies to corresponding target information is corresponding with the video frame identification to be regarded Frequency frame plays simultaneously.
Correspondingly, the disclosure also provides a kind of computer readable storage medium, it is stored thereon with computer program, the program The step of any of the above-described the method is realized when being executed by processor.
It (includes but not limited to disk that the disclosure, which can be used in the storage medium that one or more wherein includes program code, Memory, CD-ROM, optical memory etc.) on the form of computer program product implemented.Computer-usable storage medium packet Permanent and non-permanent, removable and non-removable media is included, information storage is can be accomplished by any method or technique.Letter Breath can be computer-readable instruction, data structure, the module of program or other data.The example packet of the storage medium of computer It includes but is not limited to：Phase transition internal memory (PRAM), static RAM (SRAM), dynamic random access memory (DRAM), Other kinds of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), fast flash memory bank or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM), digital versatile disc (DVD) or other optical storages, magnetic tape cassette, tape magnetic disk storage or other magnetic storage apparatus or any other non-biography Defeated medium can be used for storage and can be accessed by a computing device information.
The specific details of the realization process of the function of modules and effect, which are shown in, in above-mentioned apparatus corresponds to step in the above method Realization process, details are not described herein.
For device embodiments, since it corresponds essentially to embodiment of the method, so related place is referring to method reality Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separating component The module of explanation may or may not be physically separated, and the component shown as module can be or can also It is not physical module, you can be located at a place, or may be distributed on multiple network modules.It can be according to actual It needs that some or all of module therein is selected to realize the purpose of disclosure scheme.Those of ordinary skill in the art are not paying In the case of going out creative work, you can to understand and implement.
As shown in figure 9, Fig. 9 is a kind of dress that for synchronizing information plays of the disclosure shown according to an exemplary embodiment The block diagram set.The device 900 can be mobile phone, computer, digital broadcast terminal, messaging devices, game console, Tablet device, Medical Devices, body-building equipment, the terminals such as personal digital assistant.
With reference to Fig. 9, device 900 may include following one or more components：Processing component 902, memory 904, power supply Component 906, multimedia component 909, audio component 910, the interface 912 of input/output (I/O), sensor module 914, and Communication component 916.
The integrated operation of 902 usual control device 900 of processing component, such as with display, call, data communication, phase Machine operates and record operates associated operation.Processing component 902 may include that one or more processors 920 refer to execute It enables, to perform all or part of the steps of the methods described above.In addition, processing component 902 may include one or more modules, just Interaction between processing component 902 and other assemblies.For example, processing component 902 may include multi-media module, it is more to facilitate Interaction between media component 909 and processing component 902.
Memory 904 is configured as storing various types of data to support the operation in device 900.These data are shown Example includes instruction for any application program or method that are operated on device 900, contact data, and telephone book data disappears Breath, picture, video etc..Memory 904 can be by any kind of volatibility or non-volatile memory device or their group It closes and realizes, such as static RAM (SRAM), electrically erasable programmable read-only memory (EEPROM) is erasable to compile Journey read-only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash Device, disk or CD.
Power supply module 906 provides electric power for the various assemblies of device 900.Power supply module 906 may include power management system System, one or more power supplys and other generated with for device 900, management and the associated component of distribution electric power.
Multimedia component 909 is included in the screen of one output interface of offer between described device 900 and user.One In a little embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touch panel, screen Curtain may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touch sensings Device is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or sliding action Boundary, but also detect duration and pressure associated with the touch or slide operation.In some embodiments, more matchmakers Body component 909 includes a front camera and/or rear camera.When device 900 is in operation mode, such as screening-mode or When video mode, front camera and/or rear camera can receive external multi-medium data.Each front camera and Rear camera can be a fixed optical lens system or have focusing and optical zoom capabilities.
Audio component 910 is configured as output and/or input audio signal.For example, audio component 910 includes a Mike Wind (MIC), when device 900 is in operation mode, when such as call model, logging mode and speech recognition mode, microphone by with It is set to reception external audio signal.The received audio signal can be further stored in memory 904 or via communication set Part 916 is sent.In some embodiments, audio component 910 further includes a loud speaker, is used for exports audio signal.
I/O interfaces 912 provide interface between processing component 902 and peripheral interface module, and above-mentioned peripheral interface module can To be keyboard, click wheel, button etc..These buttons may include but be not limited to：Home button, volume button, start button and lock Determine button.
Sensor module 914 includes one or more sensors, and the state for providing various aspects for device 900 is commented Estimate.For example, sensor module 914 can detect the state that opens/closes of device 900, and the relative positioning of component, for example, it is described Component is the display and keypad of device 900, and sensor module 914 can be with a group in detection device 900 or device 900 The position change of part, the existence or non-existence that user contacts with device 900,900 orientation of device or acceleration/deceleration and device 900 Temperature change.Sensor module 914 may include proximity sensor, be configured to examine without any physical contact Survey presence of nearby objects.Sensor module 914 can also include that optical sensor is used for such as CMOS or ccd image sensor It is used in imaging applications.In some embodiments, which can also include acceleration transducer, and gyroscope passes Sensor, Magnetic Sensor, pressure sensor or temperature sensor.
Communication component 916 is configured to facilitate the communication of wired or wireless way between device 900 and other equipment.Device 900 can access the wireless network based on communication standard, such as WiFi, 2G or 3G or combination thereof.In an exemplary implementation In example, communication component 916 receives broadcast singal or broadcast related information from external broadcasting management system via broadcast channel. In one exemplary embodiment, the communication component 916 further includes near-field communication (NFC) module, to promote short range communication.Example Such as, NFC module can be based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology and other technologies are realized.
In the exemplary embodiment, device 900 can be believed by one or more application application-specific integrated circuit (ASIC), number Number processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are realized, for executing the above method.
In the exemplary embodiment, it includes the non-transitorycomputer readable storage medium instructed, example to additionally provide a kind of Such as include the memory 904 of instruction, above-metioned instruction can be executed by the processor 920 of device 900 to complete the above method.For example, The non-transitorycomputer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, floppy disk With optical data storage devices etc..
Wherein, when the instruction in the storage medium is executed by the processor so that device 900 is able to carry out one kind Synchronizing information playback method, including：
Determine that the mapping relations between video frame identification and message identification, the video frame identification are it is expected in live video stream The mark for the video frame being played simultaneously with the target information, described information mark are the marks of the target information；It will live streaming Video flowing is sent to viewer end by direct broadcast service end, and the mapping relations and target information are sent by communication service end Described information is identified into corresponding target information and the video frame to viewer end so that viewer end is based on the mapping relations It identifies corresponding video frame while playing.
Alternatively,
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video Frame identification is the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream, and described information mark is institute State the mark of target information；Based on the mapping relations, described information is identified into corresponding target information and the video frame mark Know corresponding video frame while playing.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure Its embodiment.The disclosure is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or Person's adaptive change follows the general principles of this disclosure and includes the undocumented common knowledge in the art of the disclosure Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by following Claim is pointed out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the accompanying claims.
The foregoing is merely the preferred embodiments of the disclosure, not limiting the disclosure, all essences in the disclosure With within principle, any modification, equivalent substitution, improvement and etc. done should be included within the scope of the disclosure protection god.

Claims (15)

1. a kind of synchronizing information playback method, which is characterized in that the method includes：
Determine that the mapping relations between video frame identification and message identification, the video frame identification are expectation and institute in live video stream The mark for the video frame that target information is played simultaneously is stated, described information mark is the mark of the target information；
Live video stream is sent to viewer end by direct broadcast service end, the mapping relations and target information are passed through into communication Server-side is sent to viewer end so that viewer end be based on the mapping relations, by described information identify corresponding target information with The corresponding video frame of the video frame identification plays simultaneously.
2. according to the method described in claim 1, it is characterized in that,
The target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
3. according to the method described in claim 2, it is characterized in that, described information issues the determination of time and video frame time Step includes：
It receives when sending instructions under the target information, issues the time using current time as information, current video frame is existed Time in video flowing is as video frame time；
Instruction when sending instructions to start to report the target information under described by specifying objects trigger button to generate.
4. a kind of synchronizing information playback method, which is characterized in that the method includes：
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video frame mark Know the mark of the video frame it is expected to be played simultaneously with the target information in live video stream, described information mark is the mesh Mark the mark of information；
Based on the mapping relations, described information is identified into corresponding target information video frame corresponding with the video frame identification It plays simultaneously.
5. according to the method described in claim 4, it is characterized in that, the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
6. according to the method described in claim 4, it is characterized in that, the method further includes：
When starting to be played simultaneously corresponding with the video frame identification video frame of the corresponding target information of message identification, countdown is opened, Stop playing the target information or disabling and the associated operational controls of the target information at the end of countdown；
Wherein, the count down time of countdown is determined based on the difference of standard count down time and terminal network delay, the end End network delay is the network delay between viewer end and communication service end.
7. a kind of synchronizing information playing device, which is characterized in that described device includes：
Relationship determination module, the mapping relations being configured to determine that between video frame identification and message identification, the video frame identification It is expected the mark for the video frame being played simultaneously with the target information in live video stream, described information mark is the target The mark of information；
Information sending module is configured as live video stream being sent to viewer end by direct broadcast service end, and the mapping is closed System and target information are sent to viewer end by communication service end, so that viewer end is based on the mapping relations, by the letter The corresponding target information of breath mark video frame corresponding with the video frame identification plays simultaneously.
8. device according to claim 7, which is characterized in that
The target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
9. device according to claim 8, which is characterized in that the relationship determination module is specifically configured to：
It receives when sending instructions under the target information, issues the time using current time as information, current video frame is existed Time in video flowing obtains the mapping relations between video frame identification and message identification as video frame time；
Wherein, instruction when sending instructions to start to report the target information under described by specifying objects trigger button to generate.
10. a kind of synchronizing information playing device, which is characterized in that described device includes：
Information acquisition module is configured as obtaining between live video stream, target information and video frame identification and message identification Mapping relations, the video frame identification are the mark for the video frame for it is expected to be played simultaneously with the target information in live video stream Know, described information mark is the mark of the target information；
Information playing module, be configured as be based on the mapping relations, by described information identify corresponding target information with it is described The corresponding video frame of video frame identification plays simultaneously.
11. device according to claim 10, which is characterized in that the target information includes problem topic；And/or
The information that described information is identified as when main broadcaster end sends the target information issues the time；And/or
The video frame identification is the video frame time of current video frame when main broadcaster end sends the target information.
12. device according to claim 10, which is characterized in that described device further includes countdown module, is configured as：
When starting to be played simultaneously corresponding with the video frame identification video frame of the corresponding target information of message identification, countdown is opened, Stop playing the target information or disabling and the associated operational controls of the target information at the end of countdown；
Wherein, the count down time of countdown is determined based on the difference of standard count down time and terminal network delay, the end End network delay is the network delay between viewer end and communication service end.
13. a kind of live broadcast system, which is characterized in that including main broadcaster end, direct broadcast service end, communication service end, viewer end；
Main broadcaster end determines the mapping relations between video frame identification and message identification, and live video stream is sent out by direct broadcast service end It send to viewer end, the mapping relations and target information is sent to viewer end, the video frame mark by communication service end Know the mark of the video frame it is expected to be played simultaneously with the target information in live video stream, described information mark is the mesh Mark the mark of information；
Viewer end obtains live video stream, target information and mapping relations, is based on the mapping relations, described information is identified Corresponding target information video frame corresponding with the video frame identification plays simultaneously.
14. a kind of electronic equipment, which is characterized in that including：
Processor；
Memory for storing processor-executable instruction；
Wherein, the processor is configured as：
Mapping relations between acquisition live video stream, target information and video frame identification and message identification, the video frame mark Know the mark of the video frame it is expected to be played simultaneously with the target information in live video stream, described information mark is the mesh Mark the mark of information；
Based on the mapping relations, described information is identified into corresponding target information video frame corresponding with the video frame identification It plays simultaneously.
15. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor The step of any one of claim 1 to 6 the method is realized when execution.
CN201810205908.4A 2018-03-13 2018-03-13 Information synchronous playing method, device, equipment, system and storage medium Active CN108449605B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810205908.4A CN108449605B (en) 2018-03-13 2018-03-13 Information synchronous playing method, device, equipment, system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810205908.4A CN108449605B (en) 2018-03-13 2018-03-13 Information synchronous playing method, device, equipment, system and storage medium

Publications (2)

Publication Number Publication Date
CN108449605A true CN108449605A (en) 2018-08-24
CN108449605B CN108449605B (en) 2020-10-30

Family

ID=63194207

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810205908.4A Active CN108449605B (en) 2018-03-13 2018-03-13 Information synchronous playing method, device, equipment, system and storage medium

Country Status (1)

Country Link
CN (1) CN108449605B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109714622A (en) * 2018-11-15 2019-05-03 北京奇艺世纪科技有限公司 A kind of video data handling procedure, device and electronic equipment
CN109819300A (en) * 2019-01-03 2019-05-28 北京潘达互娱科技有限公司 A kind of message content methods of exhibiting, sending method, device, terminal and system
CN110602260A (en) * 2019-11-12 2019-12-20 南京创维信息技术研究院有限公司 Method and device for synchronous scene playing of intelligent equipment based on AIOT technology
CN110650353A (en) * 2019-09-25 2020-01-03 广州华多网络科技有限公司 Multi-person continuous-wheat mixed drawing method and device, storage medium and electronic equipment
CN112118488A (en) * 2019-06-20 2020-12-22 京东方科技集团股份有限公司 Live broadcast method, electronic equipment and live broadcast system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7721308B2 (en) * 2005-07-01 2010-05-18 Microsoft Corproation Synchronization aspects of interactive multimedia presentation management
US8843959B2 (en) * 2007-09-19 2014-09-23 Orlando McMaster Generating synchronized interactive link maps linking tracked video objects to other multimedia content in real-time
CN104811774A (en) * 2015-04-29 2015-07-29 北京中传数广技术有限公司 Live television precise advertisement and information delivery method and system
CN106488291A (en) * 2016-11-17 2017-03-08 百度在线网络技术（北京）有限公司 The method and apparatus of simultaneous display file in net cast
CN106534944A (en) * 2016-11-30 2017-03-22 北京锤子数码科技有限公司 Video display method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7721308B2 (en) * 2005-07-01 2010-05-18 Microsoft Corproation Synchronization aspects of interactive multimedia presentation management
US8843959B2 (en) * 2007-09-19 2014-09-23 Orlando McMaster Generating synchronized interactive link maps linking tracked video objects to other multimedia content in real-time
CN104811774A (en) * 2015-04-29 2015-07-29 北京中传数广技术有限公司 Live television precise advertisement and information delivery method and system
CN106488291A (en) * 2016-11-17 2017-03-08 百度在线网络技术（北京）有限公司 The method and apparatus of simultaneous display file in net cast
CN106534944A (en) * 2016-11-30 2017-03-22 北京锤子数码科技有限公司 Video display method and device

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109714622A (en) * 2018-11-15 2019-05-03 北京奇艺世纪科技有限公司 A kind of video data handling procedure, device and electronic equipment
CN109714622B (en) * 2018-11-15 2021-04-16 北京奇艺世纪科技有限公司 Video data processing method and device and electronic equipment
CN109819300A (en) * 2019-01-03 2019-05-28 北京潘达互娱科技有限公司 A kind of message content methods of exhibiting, sending method, device, terminal and system
CN112118488A (en) * 2019-06-20 2020-12-22 京东方科技集团股份有限公司 Live broadcast method, electronic equipment and live broadcast system
CN110650353A (en) * 2019-09-25 2020-01-03 广州华多网络科技有限公司 Multi-person continuous-wheat mixed drawing method and device, storage medium and electronic equipment
CN110650353B (en) * 2019-09-25 2020-12-04 广州华多网络科技有限公司 Multi-person continuous-wheat mixed drawing method and device, storage medium and electronic equipment
CN110602260A (en) * 2019-11-12 2019-12-20 南京创维信息技术研究院有限公司 Method and device for synchronous scene playing of intelligent equipment based on AIOT technology

Also Published As

Publication number Publication date
CN108449605B (en) 2020-10-30

Similar Documents

Publication Publication Date Title
CN108260016A (en) Processing method, device, equipment, system and storage medium is broadcast live
CN108449605A (en) Synchronizing information playback method, device, equipment, system and storage medium
CN112218103B (en) Live broadcast room interaction method and device, electronic equipment and storage medium
US20210281909A1 (en) Method and apparatus for sharing video, and storage medium
CN104834435B (en) The playing method and device of audio commentary
WO2022121557A1 (en) Live streaming interaction method, apparatus and device, and medium
CN106921560A (en) Voice communication method, apparatus and system
CN109168034A (en) Merchandise news display methods, device, electronic equipment and readable storage medium storing program for executing
KR20160022286A (en) Method and apparatus for sharing video information
CN104104990B (en) Adjust the method and device of subtitle in video
CN110602394A (en) Video shooting method and device and electronic equipment
CN109831690A (en) It has been shown that, sending method, device, terminal and the server of barrage message
CN109151565B (en) Method and device for playing voice, electronic equipment and storage medium
CN113382274B (en) Data processing method and device, electronic equipment and storage medium
CN109309843A (en) Video distribution method, terminal and server
CN113065008A (en) Information recommendation method and device, electronic equipment and storage medium
CN112153407A (en) Live broadcast room data interaction method, related device and equipment
CN105635846B (en) Apparatus control method and device
CN109168062A (en) Methods of exhibiting, device, terminal device and the storage medium of video playing
CN113573092B (en) Live broadcast data processing method and device, electronic equipment and storage medium
EP4114010A2 (en) Method and apparatus for information interaction in live broadcast room
CN110392304A (en) A kind of video display method, apparatus, electronic equipment and storage medium
US20220137756A1 (en) Method for displaying interactive content, electronic device, and storage medium
CN110475134A (en) A kind of comment content display method, device, electronic equipment and storage medium
CN111859196A (en) Information recommendation method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant