CN110166788A - Synchronizing information playback method, device and storage medium - Google Patents

Synchronizing information playback method, device and storage medium Download PDF

Info

Publication number
CN110166788A
CN110166788A CN201810868257.7A CN201810868257A CN110166788A CN 110166788 A CN110166788 A CN 110166788A CN 201810868257 A CN201810868257 A CN 201810868257A CN 110166788 A CN110166788 A CN 110166788A
Authority
CN
China
Prior art keywords
synchronized
broadcast information
time
timestamp
video frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810868257.7A
Other languages
Chinese (zh)
Other versions
CN110166788B (en
Inventor
彭勇
陈爱平
王威威
李帅
魏亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201810868257.7A priority Critical patent/CN110166788B/en
Publication of CN110166788A publication Critical patent/CN110166788A/en
Application granted granted Critical
Publication of CN110166788B publication Critical patent/CN110166788B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242Synchronization processes, e.g. processing of PCR [Program Clock References]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8547Content authoring involving timestamps for synchronizing content

Abstract

The embodiment of the invention discloses a kind of synchronizing information playback method, device and storage mediums, the synchronizing information playback method includes: the video data for receiving first server and sending, and the video data is played out, the video data includes the video frame for carrying timestamp;When the broadcast information to be synchronized that the needs for receiving second server transmission are played with the video data synchronization, the timestamp of the broadcast information to be synchronized is extracted;Extract the timestamp of newest video frame, wherein the newest video frame is nearest apart from current time in the video frame being currently received and carries the video frame of timestamp;Obtain the time difference between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized;The play time of the broadcast information to be synchronized is determined according to the time difference, and the broadcast information to be synchronized is played according to the play time.The program can promote the broadcasting synchronism of video Yu broadcast information to be synchronized.

Description

Synchronizing information playback method, device and storage medium
Technical field
The present invention relates to fields of communication technology, and in particular to a kind of synchronizing information playback method, device and storage medium.
Background technique
With the development of internet and the continuous expansion of live streaming user, country's live streaming enter the period of full bloom, and in recent years The problem of tending towards stability, industry faces customer churn be broadcast live.
Major live streaming platform is successively proposed various living broadcast interactive playing methods to solve the problems, such as customer churn, but also sudden and violent More significant problem is revealed, since video flowing need to be distributed by video flowing the distribution server, there are certain time delay, has caused straight It broadcasts video and interactive information broadcasting is asynchronous.
For example, the living broadcast interactive playing method of answer with lottery, the appearance of topic and presides over population and broadcast that topic is inconsistent, and topic is Out, there are no mouths to broadcast by host, although influencing less on answer, but it is synchronous with topic not can guarantee video, sternly Weight reduces user experience.
Summary of the invention
The embodiment of the invention provides a kind of synchronizing information playback method, device and storage medium, can be promoted video with The broadcasting synchronism of information.
The embodiment of the invention provides a kind of synchronizing information playback methods, comprising:
The video data that first server is sent is received, and the video data is played out, the video data packet Include the video frame for carrying timestamp;
When the broadcast information to be synchronized that the needs for receiving second server transmission are played with the video data synchronization, Extract the timestamp of the broadcast information to be synchronized;
Extract the timestamp of newest video frame, wherein the newest video frame is distance in the video frame being currently received Current time is nearest and carries the video frame of timestamp;
Obtain the time difference between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized;
The play time of the broadcast information to be synchronized is determined according to the time difference, and is played according to the play time The broadcast information to be synchronized.
The embodiment of the invention provides a kind of synchronizing information playing devices, comprising:
Video reception unit for receiving the video data of first server transmission, and is broadcast the video data It puts, the video data includes the video frame for carrying timestamp;
First extraction unit, for what is played when the needs for receiving second server transmission with the video data synchronization When broadcast information to be synchronized, the timestamp of the broadcast information to be synchronized is extracted;
Second extraction unit, for extracting the timestamp of newest video frame, wherein the newest video frame is current receives To video frame in apart from current time recently and carry timestamp video frame;
Time difference acquiring unit, for obtaining the timestamp of the newest video frame and the time of the broadcast information to be synchronized Time difference between stamp;
Information playing unit, for determining the play time of the broadcast information to be synchronized, and root according to the time difference The broadcast information to be synchronized is played according to the play time.
The embodiment of the invention also provides a kind of storage medium, the storage equipment is for storing a plurality of instruction, the finger It enables and is suitable for being loaded as processor and executing synchronizing information playback method provided by the embodiment of the present invention.
The video data that the embodiment of the present invention is sent using first server is received, and the video data is broadcast It puts, the video data includes the video frame for carrying timestamp;When the needs and the video for receiving second server transmission When the broadcast information to be synchronized that data are played simultaneously, the timestamp of the broadcast information to be synchronized is extracted;Extract newest video frame Timestamp, wherein the newest video frame be in the video frame being currently received apart from current time recently and carry the time The video frame of stamp;Obtain the time between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized Difference;The play time of the broadcast information to be synchronized is determined according to the time difference, and according to play time broadcasting Broadcast information to be synchronized.The program is by separately transmitting video data and broadcast information to be synchronized, and according in video data Video frame time stamp the timestamp of broadcast information to be synchronized between time difference determine the broadcasting of broadcast information to be synchronized when Between, broadcast information to be synchronized is played further according to play time, improves the broadcasting synchronism of video Yu broadcast information to be synchronized.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.
Fig. 1 a is the schematic diagram of a scenario of synchronizing information play system provided in an embodiment of the present invention.
Fig. 1 b is another schematic diagram of a scenario of synchronizing information play system provided in an embodiment of the present invention.
Fig. 1 c is the flow diagram of synchronizing information playback method provided in an embodiment of the present invention.
Fig. 2 is the flow diagram of another synchronizing information playback method provided in an embodiment of the present invention.
Fig. 3 a is the structural schematic diagram of synchronizing information playing device provided in an embodiment of the present invention.
Fig. 3 b is another structural schematic diagram of synchronizing information playing device provided in an embodiment of the present invention.
Fig. 4 is the structural schematic diagram of another synchronizing information playing device provided in an embodiment of the present invention.
Fig. 5 is the structural schematic diagram of server provided in an embodiment of the present invention.
Specific embodiment
Schema is please referred to, wherein identical component symbol represents identical component, the principle of the application is to implement one It is illustrated in computing environment appropriate.The following description be based on illustrated by the application specific embodiment, should not be by It is considered as limitation the application other specific embodiments not detailed herein.
In the following description, the specific embodiment of the application will refer to the step as performed by one or multi-section computer And symbol illustrates, unless otherwise stating clearly.Therefore, these steps and operation will have to mention for several times is executed by computer, this paper institute The computer execution of finger includes by representing with the computer processing unit of the electronic signal of the data in a structuring pattern Operation.This operation is converted at the data or the position being maintained in the memory system of the computer, reconfigurable Or in addition change the running of the computer in mode known to the tester of this field.The maintained data structure of the data For the provider location of the memory, there is the specific feature as defined in the data format.But the application principle is with above-mentioned text Word illustrates that be not represented as a kind of limitation, this field tester will appreciate that plurality of step and behaviour as described below Also it may be implemented in hardware.
Term as used herein " module " can regard the software object to execute in the arithmetic system as.It is as described herein Different components, module, engine and service can be regarded as the objective for implementation in the arithmetic system.And device as described herein and side Method can be implemented in the form of software, can also be implemented on hardware certainly, within the application protection scope.
Term " first ", " second " and " third " in the application etc. are for distinguishing different objects, rather than for retouching State particular order.In addition, term " includes " and " having " and their any deformations, it is intended that cover and non-exclusive include. Such as contain series of steps or module process, method, system, product or equipment be not limited to listed step or Module, but some embodiments further include the steps that not listing or module or some embodiments further include for these processes, Method, product or equipment intrinsic other steps or module.
Referenced herein " embodiment " is it is meant that a particular feature, structure, or characteristic described can wrap in conjunction with the embodiments It is contained at least one embodiment of the application.Each position in the description occur the phrase might not each mean it is identical Embodiment, nor the independent or alternative embodiment with other embodiments mutual exclusion.Those skilled in the art explicitly and Implicitly understand, embodiment described herein can be combined with other embodiments.
The embodiment of the invention provides a kind of synchronizing information play system, which may include that the embodiment of the present invention provides First information simultaneously playing device and the second synchronizing information playing device.
For example, the user that internet video live broadcasting can be viewer end watches that live streaming end subscriber is ongoing existing by network Field audio-video is live, such as race, meeting, teaching, operation etc..In certain live scenes, master is not only played in viewer end The video data at end is broadcast, other broadcast informations to be synchronized are also played.For example, in the scene of live streaming answer, broadcasting letter to be synchronized Breath may include problem topic.Live streaming question-answering mode can be user and within a specified time log in direct broadcasting room, guide in host Answer on lower carry out line, answering questions specified quantity (such as 12) topic can rush to summit bonus, carve up the bonus of each issue of setting.In spectators End can play the answer frame comprising topic on live video, so that spectators are hearing master in live video using terminal When holding people's casting topic, answer frame can be seen from terminal screen, and answer.For another example, in live streaming panic buying scene, to It may include goods for sale information that information, which is played simultaneously,.Live streaming panic buying mode can be user and within a specified time log in live streaming Between, it carries out rushing to purchase on line under host's guidance.In viewer end, goods for sale information can be played on live video, be seen Crowd can hear that host broadcasts merchandise news in live video, while can be selected the commodity of preference with operation display, be gone forward side by side The capable operation that places an order/buy.
It is understood that the adaptable scene of the embodiment of the present disclosure: during live streaming, viewer end can receive two numbers According to a circuit-switched data can be live video data, and another circuit-switched data can be broadcast information to be synchronized.In live video data with The associated video frame of broadcast information to be synchronized need to be played simultaneously with broadcast information to be synchronized.Wherein, broadcast information to be synchronized and straight Broadcasting an at least frame video frame in video data, there are incidence relations.Exist for example, incidence relation can be in live video data Broadcast the video frame of broadcast information to be synchronized.In addition, broadcast information to be synchronized can be problem topic, answer statistical result, to Sell merchandise news, to data such as recall infos, it is numerous to list herein.
In one embodiment, live video data distribution and broadcast information to be synchronized can be realized by the same server Distribution.As described in Fig. 1 a, Fig. 1 a is the schematic diagram of a scenario of synchronizing information play system provided in an embodiment of the present invention.In the scene In, it may include main broadcaster end, server and viewer end, first information simultaneously playing device is integrated in viewer end, the second information Simultaneously playing device is integrated in server.It is understood that the scene is illustrated by taking a viewer end as an example, in fact, Viewer end can be one or more.
Main broadcaster end can be video data originating end, and user terminal can be video data receiving end, in certain scenes, main broadcaster End is properly termed as presiding over end, and viewer end is properly termed as user terminal.Main broadcaster end and viewer end, which can be, to be mounted on an electronic device Software, be also possible to terminal device.Main broadcaster end can call the methods of camera recorded video, shooting photo production live streaming number According to being then sent to server by network;Viewer end and main broadcaster end can be by the interactive messages (such as: barrage message) of input It is sent to server, interactive message can be played at each end of same direct broadcasting room by realizing.And it is directed to broadcast information to be synchronized, it is past Toward being to be sent to server by main broadcaster end, each viewer end is issued by server.Server is used to provide internet live streaming Background service, such as the corresponding relationship at main broadcaster end and viewer end is saved, and be broadcast live distribution and the interactive message of data Distribution and the distribution of broadcast information to be synchronized etc..In one embodiment, each straight in order to be adapted to the video data received Platform, various different agreements are broadcast, the video data received can be transcoded into server and support such as PTMP, HLS, FLV Format draws for corresponding viewer end to flow.
In another embodiment, in order to distinguish different business and improve the timeliness of information processing, server can be with Including Video Distribution Server and live streaming platform service server etc., Video Distribution Server can be used for carrying out point of video flowing Hair, live streaming platform service server are used to carry out the distribution of data, the distribution of broadcast information especially to be synchronized.It is understood that It is that according to actual needs, server can also include that the present for distributing the barrage server of barrage, for distributing present takes Business device etc., will not repeat them here.
For example, Fig. 1 b is another scene signal of synchronizing information play system provided in an embodiment of the present invention with reference to Fig. 1 b Figure.It in this scenario, may include main broadcaster end, first server (Video Distribution Server), second server (live streaming platform industry Business server) and viewer end, first information simultaneously playing device be integrated in viewer end, the second synchronizing information playing device is integrated in In first server and second server.
When needing to send video data to first server, main broadcaster end is collected video frame according to the frame per second of video Corresponding timestamp is added, is retransmited to first server, then is sent video data to viewer end by first server;Work as touching When sending out broadcast information to be synchronized and sending instruction, main broadcaster end is that broadcast information to be synchronized adds corresponding timestamp, is retransmited to the Two servers, then the broadcast information to be synchronized for carrying timestamp is sent to viewer end by second server.Wherein, to the second clothes When business device issues broadcast information to be synchronized, the video frame currently generated is determined as associated video frame.Due to be synchronized in transmission When broadcast information, positive generation and the associated video frame of broadcast information to be synchronized, that is, being at this time broadcast information to be synchronized addition Timestamp is corresponding with timestamp added by the video frame at this time for acquisition.Therefore, the video frame currently generated be with to same Walk the related video frame of broadcast information.Therefore, when viewer end synchronizes operation, video frame can be compared by viewer end The timestamp of timestamp and broadcast information to be synchronized, when the time difference between the two reaching minimum threshold, in the broadcasting of video Broadcast information to be synchronized is played in picture, to improve the broadcasting synchronism of associated video frame Yu broadcast information to be synchronized.
It is illustrated individually below.
The present embodiment will be described from the angle of first information simultaneously playing device, which can be with It is integrated in viewer end, as illustrated in figure 1 c, the detailed process of the method for the present embodiment can be such that
101, the video data that first server is sent is received, and the video data is played out, the video counts According to the video frame for including carrying timestamp.
Wherein, video data can be with main broadcaster end by calling camera recorded video, shooting photo, can also be directly from sound Video acquisition end obtains audio, video data etc., and makes acquisition video data.In one embodiment, in order to reach audio and video It is synchronous, audio data and image data can be synthesized, and guarantees with main broadcaster end after obtaining audio data and image data Audio data is synchronous with image data, leads to picture and the nonsynchronous situation of audio to avoid transmission process.
Wherein, video data is made of multiple video frames, and timestamp is added according to the frame per second of video, than Such as, a timestamp is added every two frames.
Wherein it is possible to can be broadcast to video frame using the mode that receiving while plays when receiving video frame It puts.
102, when the needs for receiving second server transmission are believed with the broadcasting to be synchronized that the video data synchronization plays When breath, the timestamp of the broadcast information to be synchronized is extracted.
Wherein, broadcast information to be synchronized carries corresponding timestamp, the timestamp and view of the broadcast information to be synchronized Frequency timestamp entrained by an at least frame associated video frame in is corresponding.
Specifically, broadcasting letter to be synchronized can be extracted when receiving the broadcast information to be synchronized of second server transmission The timestamp of broadcast information to be synchronized can be compared by the timestamp of breath so as to subsequent with the timestamp of video frame, thus It can be treated according to comparison result and information is played simultaneously play out.
Wherein, broadcast information to be synchronized can be that there are the letters of incidence relation with an at least frame video frame in video data Breath.For example, incidence relation can be the video frame for existing in video data and broadcasting broadcast information to be synchronized.It is divided by type to same Broadcast information is walked, broadcast information to be synchronized can be character information, image information, audio-frequency information, video information etc..It is drawn by content Divide broadcast information to be synchronized, broadcast information to be synchronized can be problem topic, answer statistical result, goods for sale information etc..
Such as during live streaming, host can broadcast the broadcast information to be synchronized.So-called casting broadcasting letter to be synchronized Breath is broadcasted broadcast information to be synchronized and be can be and read to be synchronized broadcast if broadcast information to be synchronized is character information or image information The particular content for putting information is also possible to read the instruction information for being used to indicate broadcast information to be synchronized;If broadcasting letter to be synchronized Breath is character information, image information, audio-frequency information or video information, broadcasts broadcast information to be synchronized and can be and plays to be synchronized broadcast Information is put, information is played simultaneously is introduced at the same time it can also treat.Wherein, it plays and is not limited to playing audio-video, can also wrap Include broadcast information.By taking broadcast information to be synchronized is problem topic as an example, in one example, host can broadcast problem topic Particular content, such as: what animal Chinese national treasure is, or even can also broadcast answer choice, such as: A, kangaroo;B, panda; C, penguin.
103, the timestamp of newest video frame is extracted, wherein the newest video frame is in the video frame being currently received Nearest apart from current time and carrying timestamp video frame.
Wherein, newest video frame is nearest apart from current time in currently received video frame, and carries timestamp Video frame.For example, current time is 10:00,5 video frames are had received before this, receiving time is respectively 9:51,9: 53,9:55,9:57,9:59, wherein the video frame that only 9:53 and 9:57 are received carries timestamp, then, newest video frame It is then this video frame that 9:57 is received.
When needing to extract the timestamp of newest video frame, can from the video frame being currently received selected distance it is current Time is nearest, and carries the video frame of timestamp as newest video frame, then extract the timestamp of newest video frame, so as to The timestamp of broadcast information to be synchronized is compared, and information is played simultaneously plays out so as to be treated according to comparison result.
104, the time between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized is obtained Difference.
Wherein, since broadcast information to be synchronized is by signalling, and the real-time of signalling compares video data The real-time of transmission is high.Therefore video frame and broadcast information to be synchronized that synchronization is sent, broadcast information to be synchronized can arrive first It reaches, therefore the video frame and broadcast information to be synchronized that synchronization reaches, the timestamp of broadcast information to be synchronized is than video frame Timestamp is big, i.e., the timestamp of broadcast information to be synchronized is greater than the timestamp of newest video frame.
For example, timestamp is added according to sending instant.For example, 9:51 simultaneously have sent information to be played and be associated with regard Frequency frame, it is to be synchronized since the real-time of the information to be played by signalling is higher than the real-time of video data transmission Broadcast information can first be reached than associated video frame.Therefore, the video frame reached with broadcast information synchronization to be synchronized is not to close Join video frame, i.e. the timestamp of the video frame of synchronization arrival can be less than the timestamp of broadcast information to be synchronized, i.e., currently arrive The timestamp of the video frame reached is all less than the timestamp of broadcast information to be synchronized.
Specifically, the timestamp of broadcast information to be synchronized and the timestamp of newest video frame can be subtracted each other, obtain to same Walk the time difference between the timestamp of broadcast information and the timestamp of newest video frame.
105, the play time of the broadcast information to be synchronized is determined according to the time difference, and according to the play time Play the broadcast information to be synchronized.
Wherein, since the video frame that the broadcast information to be synchronized that synchronization is sent can be sent than synchronization first reaches, Therefore, the key for being aligned associated video frame and broadcast information to be synchronized is exactly that broadcast information to be synchronized reaches the view to be associated such as later Frequency frame for a period of time, guarantees that the time difference between the two is sufficiently small.It therefore, can be to be synchronized to play by setting minimum threshold Broadcast information, but the timestamp that video data transmission Caton may be caused to make video frame due to network difference can not timely update, Cause the time difference of the two timestamp infinitely great, so need to be arranged a max-thresholds, when the time difference being greater than max-thresholds, Broadcast information to be synchronized only waits the duration of max-thresholds.
Specifically, the time difference can be compared with the first preset duration and the second preset duration, tied further according to comparing Fruit determines the play time of broadcast information to be synchronized, then treats information is played simultaneously plays out further according to play time.
Wherein, the first preset duration is max-thresholds, and the second preset duration is minimum threshold, the first preset duration and second Preset duration can preview data according to existing net and obtain.
In practical applications, such as in live streaming answer scene, user can be forbidden to submit before host announces answer Answer, it is contemplated that the fairness of topic time of occurrence between network and user needs that an answer deadline is arranged on backstage, Provide user's residue Reaction time.After topic arrival, need with from the background to remaining Reaction time subtract and need stand by tone The time of audio video synchronization obtains the playing duration of topic.Due to network problem, it is possible that topic is reached than later, video Data transmission card or cutout, if topic just plays out after being simply waiting max-thresholds at this time, it is possible that user Remaining Reaction time is only left few time, such as 3s or even 1s, and its influences user experience.Therefore, in user's answer Between it is synchronous between, need preferentially to guarantee the Reaction time of user.In the specific implementation process, data can be previewed according to existing net It obtains averagely each user and answers one of topic object time, averagely each user is answered into one of topic object time and is set as leaving user's for Minimum Reaction time, then the real waiting time is that the remaining Reaction time that backstage is given subtracts the minimum Reaction time for leaving user for. Wherein, the remaining Reaction time given from the background starts counting after receiving topic from viewer end, and the remaining Reaction time given from the background can In the light of actual conditions to set.
Therefore, when the time difference being greater than max-thresholds, need to pay the utmost attention to broadcast information to be synchronized in the broadcasting of viewer end Duration.At this point, the waiting time of broadcast information to be synchronized is not max-thresholds.Wherein, the waiting of broadcast information to be synchronized at this time Shi Changwei maximum waiting time, it can be subtracted by the default playing duration given from the background and preset minimum playing duration and obtain.
Specifically, comparison result can be divided into three kinds of situations:
When the time difference being more than or equal to the first preset duration, belonging to Network Abnormal leads to video data transmission Caton, at this point, Broadcast information to be synchronized plays out after directly waiting maximum waiting time, play minimum playing duration, namely wait maximum etc. Current time after duration is the play time of broadcast information to be synchronized.
When the time difference being less than or equal to the second preset duration, belong to Network Abnormal cause broadcast information to be synchronized reach compared with Evening, at this point, broadcast information to be synchronized is not to wait for directly playing normal play duration namely current time is broadcast information to be synchronized Play time.
When the time difference is less than the first preset duration and is greater than the second preset duration, belongs to normal condition, can set at this time Timer is set, timing waits maximum waiting time.In waiting process, if having received the new video frame for carrying timestamp, The timestamp that new video frame can then be extracted is updated the timestamp of newest video frame, and then updates broadcast information to be synchronized Timestamp and newest video frame timestamp between time difference, and reset timer, timing waits current need The duration of waiting.When the timestamp of newest video frame has update, the step is repeated, until the timestamp of broadcast information to be synchronized Time difference between the timestamp of newest video frame is fixed less than before the update of time stamp of the second preset duration or newest video frame When the time until, terminate the cyclic process.
Wherein, currently need the duration waited can by from the background to default playing duration subtract and preset minimum playing duration The duration that broadcast information to be synchronized has currently waited is subtracted again, i.e., maximum waiting time has subtracted broadcast information to be synchronized currently etc. To duration obtain, when the update of time stamp of newest video frame, currently the waiting time waited is needed also to update therewith.Its In, default playing duration presets minimum playing duration and normal play duration and can preview data by existing net and obtain, can also be with It is set according to actual conditions.
That is, step " determining the play time of the broadcast information to be synchronized according to the time difference " may include:
When the time difference being not less than the first preset duration, waited according to the first waiting time, and waiting At when determine current time be the broadcast information to be synchronized play time;
When the time difference being not more than the second preset duration, determine that current time is broadcasting for the broadcast information to be synchronized Put the time, wherein second preset duration is less than first preset duration.
Step " play time of the broadcast information to be synchronized is determined according to the time difference " can also include:
When the time difference being less than first preset duration and being greater than second preset duration, waited according to first Duration is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, the timestamp based on the new video frame is updated the time difference, and according to it is updated when Between difference determine the play time of the broadcast information to be synchronized;
If it is not, then the current time after first waiting time will be waited to be determined as broadcasting for the broadcast information to be synchronized Put the time.
Wherein, the current time after waiting for the first waiting time refers to the time after the first waiting time of waiting, for example, The first waiting time for needing to wait is 10 seconds, and the time started waiting is 9:40:45 (when/point/second), then waiting first etc. Current time after duration is then 9:40:55.The first waiting time at this time is maximum waiting time, can by will after It default playing duration that platform is given and presets minimum playing duration and subtracts each other to obtain.
For example, default playing duration is 20 seconds, presetting minimum playing duration is 7 seconds, then need to wait at this time first Waiting time is then -7 seconds 20 seconds=13 seconds.
It in one embodiment, can be according to new after having received the new video frame for carrying timestamp in waiting process The timestamp of video frame is updated the timestamp of newest video frame, then the timestamp again based on broadcast information to be synchronized with The timestamp of newest video frame is updated the time difference after update.Namely step " the timestamp pair based on the new video frame The time difference is updated " may include:
It is updated according to timestamp of the timestamp of the new video frame to the newest video frame;
Obtain the new time between the timestamp of the broadcast information to be synchronized and the timestamp of newest video frame after update Difference;
The time difference is updated according to the new time difference.
Specifically, the timestamp of new video frame can be replaced to the timestamp of original newest video frame, that is, after updating most The timestamp of new video frame is the timestamp of new video frame.Then, then by the timestamp and new video frame of broadcast information to be synchronized Timestamp subtract each other, obtain the new time difference, the new time difference replaced to the original time difference again later.
Wherein, after updating the time difference between the timestamp of broadcast information to be synchronized and the timestamp of newest video frame, The updated time difference can be compared with the first preset duration, the second preset duration, then be determined according to comparison result The play time that broadcast information to be synchronized is played simultaneously.
It, can be directly true by current time when the updated time difference being not more than the second preset duration in one embodiment It is set to the play time of broadcast information to be synchronized.When the updated time difference is between the first preset duration and the second preset duration When, the duration for currently needing to wait is obtained, is then waited, carries the new of timestamp if having received in waiting process Video frame can then be updated the time difference according to the timestamp of new video frame.That is, step is " according to the updated time Difference determines the target video frame needed with the broadcast information to be synchronized " may include:
When the updated time difference being not more than second preset duration, determine that current time is the broadcasting to be synchronized The play time of information;
When the updated time difference being less than first preset duration and being greater than second preset duration, second is obtained Waiting time;
First waiting time is updated according to second waiting time, and returns to execution and is waited according to first The step of duration is waited.
Specifically, the second waiting time can be replaced for the first waiting time, after update after obtaining for the second waiting time The first waiting time be not maximum waiting time.
Wherein, the second waiting time can be subtracted by the default playing duration given from the background presets minimum playing duration and subtracts again The duration for going broadcast information to be synchronized currently to wait obtains, and is that broadcasting letter to be synchronized can be subtracted by maximum waiting time The duration that breath has currently waited obtains.
Wherein, the duration that broadcast information to be synchronized has currently waited can for broadcast information to be synchronized from reach it is current The duration of waiting.For example, broadcast information to be synchronized has waited 10 seconds from current time is reached, then broadcast information to be synchronized The duration currently waited is then 10 seconds.For example, current waiting time is 10 seconds, maximum waiting time is 20 seconds, then second Waiting time is then -10 seconds 20 seconds=10 seconds.
In one embodiment, when broadcast information to be synchronized is display information, broadcast information to be synchronized can be played On the picture of target video frame.When display, broadcast information to be synchronized can be directly displayed, can also by broadcast information to be synchronized into It is shown again after row processing, to provide the control that can be operated by spectators.It, can be with by taking broadcast information to be synchronized is problem topic as an example Answer frame comprising problem topic is shown on video frame picture.In one embodiment, not only comprising in topic in answer frame Hold, also furnish an answer input control, so that spectators are by inputting answer in answer input control.In another embodiment, in order to Raising is answered efficiency, each option all can be trigger control, when spectators click or touch option, the option conduct that is triggered The answer that spectators submit.
The video data that the present embodiment is sent using first server is received, and the video data is played out, institute Stating video data includes the video frame for carrying timestamp;When the needs and the video data that receive second server transmission are same When walking the broadcast information to be synchronized played, the timestamp of the broadcast information to be synchronized is extracted, the time of newest video frame is extracted Stamp, wherein the newest video frame is nearest apart from current time in the video frame being currently received and carries the view of timestamp Frequency frame;Obtain the time difference between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized;According to The time difference determines the play time of the broadcast information to be synchronized, and plays described to be synchronized broadcast according to the play time Put information.The program is by separately transmitting video data and broadcast information to be synchronized, and according to the video frame in video data Time difference between timestamp and the timestamp of broadcast information to be synchronized determines the play time of broadcast information to be synchronized, further according to Play time plays broadcast information to be synchronized, improves the broadcasting synchronism of video Yu broadcast information to be synchronized.
For better implementation above method, the present invention also provides another synchronizing information playback methods, with the second letter Breath simultaneously playing device is integrated in for an entity device realization.
The present embodiment will be described from the angle of the second synchronizing information playing device, and with reference to Fig. 2, which is played The detailed process of method can be such that
201, video data is obtained, the video data includes multiple video frames.
Wherein, video data can be with main broadcaster end by calling camera recorded video, shooting photo, can also be directly from sound Video acquisition end obtains audio, video data etc., and makes acquisition video data.In one embodiment, in order to reach audio and video It is synchronous, audio data and image data can be synthesized, and guarantees with main broadcaster end after obtaining audio data and image data Audio data is synchronous with image data, leads to picture and the nonsynchronous situation of audio to avoid transmission process.
202, it when needing to send the video data, retransmits after adding timestamp for the video frame to first service Device.
Specifically, can be the video frame in video data according to the frame per second of video when receiving video transmission instruction Corresponding timestamp is added, for example, adding a timestamp every 2 frames.Then by the video data encoding after addition timestamp Plug-flow is to first server.Wherein, timestamp can be added according to current time.
203, the broadcast information to be synchronized for needing to play with the video data synchronization is obtained.
Wherein, broadcast information to be synchronized can be that there are the letters of incidence relation with an at least frame video frame in video data Breath.For example, incidence relation can be the video frame for existing in video data and broadcasting broadcast information to be synchronized.It is divided by type to same Broadcast information is walked, broadcast information to be synchronized can be character information, image information, audio-frequency information, video information etc..It is drawn by content Divide broadcast information to be synchronized, broadcast information to be synchronized can be problem topic, answer statistical result, goods for sale information etc..
Such as during live streaming, host can broadcast the broadcast information to be synchronized.So-called casting broadcasting letter to be synchronized Breath is broadcasted broadcast information to be synchronized and be can be and read to be synchronized broadcast if broadcast information to be synchronized is character information or image information The particular content for putting information is also possible to read the instruction information for being used to indicate broadcast information to be synchronized;If broadcasting letter to be synchronized Breath is character information, image information, audio-frequency information or video information, broadcasts broadcast information to be synchronized and can be and plays to be synchronized broadcast Information is put, information is played simultaneously is introduced at the same time it can also treat.Wherein, it plays and is not limited to playing audio-video, can also wrap Include broadcast information.By taking broadcast information to be synchronized is problem topic as an example, in one example, host can broadcast problem topic Particular content, such as: what animal Chinese national treasure is, or even can also broadcast answer choice, such as: A, kangaroo;B, panda; C, penguin.
204, it when needing to send the broadcast information to be synchronized, is added after timestamp again for the broadcast information to be synchronized It is sent to second server.
Specifically, can be when the audio-video currently acquired is associated with broadcast information to be synchronized generation, triggering is executed to same Send instructions under step broadcast information.For example, by picture of the collection terminal shooting comprising host and acquisition respective audio, in master When holding people's casting broadcast information to be synchronized, broadcast information to be synchronized is issued, so that viewer end can receive video data and receive To broadcast information to be synchronized, and while playing main video frame, broadcast information to be synchronized is played.
Wherein it is possible to which the video frame currently generated is determined as when issuing broadcast information to be synchronized to second server Associated video frame.Due to when sending broadcast information to be synchronized, it is positive generate with the associated video frame of broadcast information to be synchronized, That is, at this time for broadcast information to be synchronized addition timestamp be at this time acquire video frame added by timestamp it is corresponding. Therefore, the video frame currently generated is and the related video frame of broadcast information to be synchronized.Therefore, behaviour is synchronized in viewer end When making, the timestamp of video frame and the timestamp of broadcast information to be synchronized can be compared by viewer end, when the time between the two When difference reaches minimum threshold, broadcast information to be synchronized is played in the broadcasting pictures of video, with improve associated video frame with to same Walk the broadcasting synchronism of broadcast information.
Wherein, sending instructions under is the instruction for sending broadcast information to be synchronized.Under the condition that sends instructions, can be just/pre- produce Raw associated video frame and the instruction that generation is triggered under the opportunity for playing broadcast information to be synchronized in advance.Sending instructions under for example, can be with It is the instruction that triggering generates when starting to broadcast broadcast information to be synchronized.The present embodiment will start to broadcast broadcast information to be synchronized as The condition that sends instructions under triggering generates may be implemented broadcast information to be synchronized and casting broadcast information to be synchronized in viewer end When video frame synchronization play.
In one embodiment, sending instructions under can be when starting to broadcast the broadcast information to be synchronized by specifying objects trigger The instruction that key generates.Key can be physical button or touch virtual key.For example, host starts to broadcast to be synchronized broadcast When putting information, such as: when host reads " first topic be ... ", from pressing on staff's pointing device with host Key realizes that triggering generates and sends instructions under broadcast information to be synchronized.As it can be seen that by the transmission opportunity to send instructions under manual control, It is easy to accomplish.
In one embodiment, it includes preset keyword or default in live video frame that sending instructions under, which can be equipment and recognize, Crucial audio then triggers and sends instructions under generation.For example, such as: host reads when host starts to broadcast broadcast information to be synchronized When " first topic be ... ", instructor in broadcasting end identifies the default crucial voice of phonetic acquisition, then triggers and generate under broadcast information to be synchronized Send instructions.As it can be seen that can generate down and send instructions with automatic trigger by speech recognition or image recognition, cost of labor is saved.
Send instructions to be triggered by other conditions under it is understood that and generate, will not repeat them here.
From the foregoing, it will be observed that the present embodiment is by obtaining video data;It is the video when needing to send the video data It retransmits after video frame addition timestamp in data to first server;It obtains and needs to play with the video data synchronization Broadcast information to be synchronized;When needing to send the broadcast information to be synchronized, the time is added for the broadcast information to be synchronized It retransmits after stamp to second server.So that terminal can timestamp by comparing video frame and broadcast information to be synchronized Timestamp determines the play time of broadcast information to be synchronized, and it is synchronous with the broadcasting of broadcast information to be synchronized to improve associated video frame Property.
For better implementation above method, the embodiment of the invention provides a kind of synchronizing information playing devices, such as Fig. 3 a It is shown, the synchronizing information playing device include: video reception unit 301, the first extraction unit 302, the second extraction unit 303, Duration acquiring unit 304 and information playing unit 305 are as follows:
Video reception unit 301 is carried out for receiving the video data of first server transmission, and to the video data It plays, the video data includes the video frame for carrying timestamp;
First extraction unit 302, for being broadcast when the needs for receiving second server transmission with the video data synchronization When the broadcast information to be synchronized put, the timestamp of the broadcast information to be synchronized is extracted;
Second extraction unit 303, for extracting the timestamp of newest video frame, wherein the newest video frame is current Nearest apart from current time and carrying timestamp video frame in the video frame received;
Time difference acquiring unit 304, for obtain the newest video frame timestamp and the broadcast information to be synchronized Time difference between timestamp;
Information playing unit 305, for determining the play time of the broadcast information to be synchronized according to the time difference, and The broadcast information to be synchronized is played according to the play time.
In some embodiments, with reference to Fig. 3 b, the information playing unit 305 may include:
First plays subelement 3051, for when the time difference being not less than the first preset duration, acquisition to need to wait Target duration, and when the current time after the target duration will be waited to be determined as the broadcasting of the broadcast information to be synchronized Between, the broadcast information to be synchronized is played according to the play time;
Second determines subelement 3052, for current for determining when the time difference being not more than the second preset duration Time is the play time of the broadcast information to be synchronized, and plays the broadcast information to be synchronized according to the play time, Wherein, second preset duration is less than first preset duration.
In some embodiments, the information playing unit 305 can also include that third plays subelement 3053, be used for:
When the time difference being less than first preset duration and being greater than second preset duration, acquisition needs to wait Target duration;
It is waited according to the target duration, and detects whether to receive the new view for carrying timestamp in waiting process Frequency frame;
If so, the timestamp based on the new video frame is updated the time difference, and according to it is updated when Between difference determine the play time of the broadcast information to be synchronized, the broadcast information to be synchronized is played according to the play time.
In some embodiments, the broadcasting subelement 3053, specifically can be used for:
When the time difference being less than first preset duration and being greater than second preset duration, waited according to first Duration is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, being updated according to timestamp of the timestamp of the new video frame to the newest video frame;
Obtain the new time between the timestamp of the broadcast information to be synchronized and the timestamp of newest video frame after update Difference;
The time difference is updated according to the new time difference;
The play time of the broadcast information to be synchronized is determined according to the updated time difference;
If it is not, then determining that current time is the broadcasting of the broadcast information to be synchronized after waiting first waiting time Time;
The broadcast information to be synchronized is played according to the play time.
Synchronizing information playing device provided in this embodiment receives what first server was sent by video reception unit 301 Video data, and the video data is played out, the video data includes the video frame for carrying timestamp;First extracts Unit 302 when receive second server transmission needs with the video data synchronization play broadcast information to be synchronized when, Extract the timestamp of the broadcast information to be synchronized;Second extraction unit 303 extracts the timestamp of newest video frame, wherein institute Newest video frame is stated to be nearest apart from current time in the video frame being currently received and carry the video frame of timestamp;The time difference Acquiring unit 304 obtains the time between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized Difference;Information playing unit 305 determines the play time of the broadcast information to be synchronized according to the time difference, and is broadcast according to described Put the time broadcasting broadcast information to be synchronized.The program by the way that video data and broadcast information to be synchronized are separately transmitted, and It is determined according to the time difference between the timestamp of the video frame in video data and the timestamp of broadcast information to be synchronized to be synchronized The play time of broadcast information improves the broadcasting synchronism of video Yu broadcast information to be synchronized.
For better implementation above method, the embodiment of the invention also provides another synchronizing information playing devices, such as Shown in Fig. 4, which may include: video acquisition unit 401, video transmission unit 402, acquisition of information list Member 403 and information transmitting unit 404 are as follows:
Video acquisition unit 401, for obtaining video data, the video data includes multiple video frames;
Video transmission unit 402, for when needing to send the video data, being after the video frame adds timestamp It retransmits to first server;
Information acquisition unit 403, for obtaining the broadcast information to be synchronized for needing to play with the video data synchronization;
Information transmitting unit 404, for being believed for the broadcasting to be synchronized when needing to send the broadcast information to be synchronized It retransmits after breath addition timestamp to second server.
Synchronizing information playing device provided in this embodiment obtains video data, the view by video acquisition unit 401 Frequency is according to including multiple video frames;Video transmission unit 402 is added when needing to send the video data for the video frame It retransmits after timestamp to first server;Information acquisition unit 403, which obtains, to be needed and video data synchronization broadcasting Broadcast information to be synchronized;Information transmitting unit 404 is the broadcasting to be synchronized when needing to send the broadcast information to be synchronized It retransmits after information addition timestamp to second server.The program is by being video frame addition pair while acquiring plug-flow The timestamp answered, when send instructions under broadcast information to be synchronized be triggered when, add corresponding timestamp for broadcast information to be synchronized. When can determine the broadcasting of broadcast information to be synchronized by comparing the timestamp of video frame and broadcast information to be synchronized so as to terminal Between, to promote the broadcasting synchronism of associated video frame Yu broadcast information to be synchronized.
It should be noted that synchronizing information playing device provided by the above embodiment is when carrying out information broadcasting, only more than The division progress of each functional module is stated for example, can according to need and in practical application by above-mentioned function distribution by difference Functional module complete, i.e., the internal structure of equipment is divided into different functional modules, with complete it is described above whole or Person's partial function.In addition, synchronizing information playing device provided by the above embodiment and synchronizing information playback method belong to same structure Think, specific implementation process is detailed in embodiment of the method, and details are not described herein again.
The embodiment of the present invention also provides a kind of server, as shown in figure 5, it illustrates take involved in the embodiment of the present invention The structural schematic diagram of business device, specifically:
The server may include one or processor 501, one or more meters of more than one processing core Memory 502, radio frequency (Radio Frequency, RF) circuit 503, power supply 505, input unit of calculation machine readable storage medium storing program for executing The components such as 505 and display unit 506.It will be understood by those skilled in the art that the not structure of server architecture shown in Fig. 5 The restriction of pairs of server may include perhaps combining certain components or different portions than illustrating more or fewer components Part arrangement.Wherein:
Processor 501 is the control centre of the server, utilizes each of various interfaces and the entire server of connection Part by running or execute the software program and/or module that are stored in memory 502, and calls and is stored in memory Data in 502, the various functions and processing data of execute server, to carry out integral monitoring to server.Optionally, locate Managing device 501 may include one or more processing cores;Preferably, processor 501 can integrate application processor and modulatedemodulate is mediated Manage device, wherein the main processing operation system of application processor, user interface and application program etc., modem processor is main Processing wireless communication.It is understood that above-mentioned modem processor can not also be integrated into processor 501.
Memory 502 can be used for storing software program and module, and processor 501 is stored in memory 502 by operation Software program and module, thereby executing various function application and data processing.Memory 502 can mainly include storage journey Sequence area and storage data area, wherein storing program area can the (ratio of application program needed for storage program area, at least one function Such as sound-playing function, image player function) etc.;Storage data area, which can be stored, uses created data according to server Deng.In addition, memory 502 may include high-speed random access memory, it can also include nonvolatile memory, for example, at least One disk memory, flush memory device or other volatile solid-state parts.Correspondingly, memory 502 can also include Memory Controller, to provide access of the processor 501 to memory 502.
During RF circuit 503 can be used for receiving and sending messages, signal is sended and received, and particularly, the downlink of base station is believed After breath receives, one or the processing of more than one processor 501 are transferred to;In addition, the data for being related to uplink are sent to base station.It is logical Often, RF circuit 503 includes but is not limited to antenna, at least one amplifier, tuner, one or more oscillators, user identity Module (SIM) card, transceiver, coupler, low-noise amplifier (LNA, Low Noise Amplifier), duplexer etc..This Outside, RF circuit 503 can also be communicated with network and other equipment by wireless communication.The wireless communication can be used any logical Beacon standard or agreement, including but not limited to global system for mobile communications (GSM, Global System of Mobile Communication), general packet radio service (GPRS, General Packet Radio Service), CDMA (CDMA, Code Division Multiple Access), wideband code division multiple access (WCDMA, Wideband Code Division Multiple Access), long term evolution (LTE, Long Term Evolution), Email, short message clothes Be engaged in (SMS, Short Messaging Service) etc..
Server further includes the power supply 504 (such as battery) powered to all parts, it is preferred that power supply 504 can pass through Power-supply management system and processor 501 are logically contiguous, to realize management charging, electric discharge, Yi Jigong by power-supply management system The functions such as consumption management.Power supply 504 can also include one or more direct current or AC power source, recharging system, power supply The random components such as fault detection circuit, power adapter or inverter, power supply status indicator.
The server may also include input unit 505, which can be used for receiving the number or character letter of input Breath, and generation keyboard related with user setting and function control, mouse, operating stick, optics or trackball signal are defeated Enter.Specifically, in a specific embodiment, input unit 505 may include touch sensitive surface and other input equipments.
The server may also include display unit 506, the display unit 506 can be used for showing information input by user or Be supplied to the information of user and the various graphical user interface of server, these graphical user interface can by figure, text, Icon, video and any combination thereof are constituted.
Although being not shown, server can also including bluetooth module etc. other modules, details are not described herein.Specifically in this reality It applies in example, the processor 501 in server can be according to following instruction, by the process pair of one or more application program The executable file answered is loaded into memory 502, and the application journey being stored in memory 502 is run by processor 501 Sequence, thus realize various functions, it is as follows:
The video data that first server is sent is received, and the video data is played out, the video data packet Include the video frame for carrying timestamp;
When the broadcast information to be synchronized that the needs for receiving second server transmission are played with the video data synchronization, Extract the timestamp of the broadcast information to be synchronized;
Extract the timestamp of newest video frame, wherein the newest video frame is distance in the video frame being currently received Current time is nearest and carries the video frame of timestamp;
Obtain the time difference between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized;
The play time of the broadcast information to be synchronized is determined according to the time difference, and is played according to the play time The broadcast information to be synchronized;
In one embodiment, when determining the play time of the broadcast information to be synchronized according to the time difference, processor 501 Specifically for executing following steps:
When the time difference being not less than the first preset duration, waited according to the first waiting time, and waiting It is the play time of the broadcast information to be synchronized at rear determining current time;
When the time difference being not more than the second preset duration, determine that current time is broadcasting for the broadcast information to be synchronized Put the time, wherein second preset duration is less than first preset duration.
In one embodiment, when determining the play time of the broadcast information to be synchronized according to the time difference, processor 501 Specifically for executing following steps:
When the time difference being less than first preset duration and being greater than second preset duration, waited according to first Duration is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, the timestamp based on the new video frame is updated the time difference, and according to it is updated when Between difference determine the play time of the broadcast information to be synchronized;
If it is not, then determining that current time is the broadcasting of the broadcast information to be synchronized after waiting first waiting time Time.
In one embodiment, when determining the play time of the broadcast information to be synchronized according to the updated time difference, processing Device 501 is specifically used for executing following steps:
When the updated time difference being not more than second preset duration, determine that current time is the broadcasting to be synchronized The play time of information;
When time difference after updated is less than first preset duration and is greater than second preset duration, the is obtained Two waiting times;
First waiting time is updated according to second waiting time, and returns to execution and is waited according to first The step of duration is waited.
In one embodiment, when the timestamp based on the new video frame is updated the time difference, processor 501 has Body is for executing following steps:
It is updated according to timestamp of the timestamp of the new video frame to the newest video frame;
Obtain the new time between the timestamp of the broadcast information to be synchronized and the timestamp of newest video frame after update Difference;
The time difference is updated according to the new time difference.
Server provided by the present embodiment receives the video data that first server is sent, and to the video data It plays out, the video data includes the video frame for carrying timestamp;When the needs and institute for receiving second server transmission When stating the broadcast information to be synchronized of video data synchronization broadcasting, the timestamp of the broadcast information to be synchronized is extracted;It extracts newest The timestamp of video, wherein the newest video frame is nearest apart from current time in the video frame being currently received and carries The video frame of timestamp;Obtain between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized when Between it is poor;The play time of the broadcast information to be synchronized is determined according to the time difference, and institute is played according to the play time State broadcast information to be synchronized.The program is by separately transmitting video data and broadcast information to be synchronized, and according to video data In video frame time stamp the timestamp of broadcast information to be synchronized between time difference determine the broadcast information to be synchronized Play time improves the broadcasting synchronism of video Yu broadcast information to be synchronized.
The embodiment of the present application also provides a kind of storage equipment, and the storage equipment is stored with computer program, when the meter When calculation machine program is run on computers, so that the computer executes the synchronizing information broadcasting side in any of the above-described embodiment Method, such as: the video data that first server is sent is received, and the video data is played out, the video data packet Include the video frame for carrying timestamp;When receive second server transmission needs with the video data synchronization play to same When walking broadcast information, the timestamp of the broadcast information to be synchronized is extracted;Extract the timestamp of newest video, wherein it is described most New video frame is nearest apart from current time in the video frame being currently received and carries the video frame of timestamp;Described in acquisition Time difference between the timestamp of newest video frame and the timestamp of the broadcast information to be synchronized;It is determined according to the time difference The play time of the broadcast information to be synchronized, and the broadcast information to be synchronized is played according to the play time.
In the embodiment of the present application, storage equipment can be magnetic disk, CD, read-only memory (Read Only Memory, ROM) or random access memory (Random Access Memory, RAM) etc..
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, reference can be made to the related descriptions of other embodiments.
It should be noted that for the synchronizing information playback method of the embodiment of the present application, the common decision-maker in this field It is understood that realize all or part of the process of the synchronizing information playback method of the embodiment of the present application, being can be by computer journey Sequence is completed to control relevant hardware, and the computer program can be stored in a computer-readable storage medium, such as deposit Storage executes in the memory of electronic equipment, and by least one processor in the electronic equipment, can wrap in the process of implementation Include the process of the embodiment such as synchronizing information playback method.Wherein, the storage medium can be magnetic disk, CD, read-only storage Device, random access memory etc..
For the synchronizing information playing device of the embodiment of the present application, each functional module be can integrate in a processing core In piece, it is also possible to modules and physically exists alone, can also be integrated in two or more modules in a module.On It states integrated module both and can take the form of hardware realization, can also be realized in the form of software function module.The collection If at module realized in the form of software function module and when sold or used as an independent product, also can store In one computer-readable storage medium, the storage medium is for example read-only memory, disk or CD etc..
A kind of synchronizing information playback method, terminal and storage medium provided by the embodiment of the present application have been carried out in detail above Thin to introduce, specific examples are used herein to illustrate the principle and implementation manner of the present application, and above embodiments are said It is bright to be merely used to help understand the present processes and its core concept;Meanwhile for those skilled in the art, according to this Shen Thought please, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not manage Solution is the limitation to the application.

Claims (10)

1. a kind of synchronizing information playback method characterized by comprising
The video data that first server is sent is received, and the video data is played out, the video data includes taking Video frame with timestamp;
When the broadcast information to be synchronized that the needs for receiving second server transmission are played with the video data synchronization, extract The timestamp of the broadcast information to be synchronized;
Extract the timestamp of newest video frame, wherein the newest video frame is that distance is current in the video frame being currently received Time is nearest and carries the video frame of timestamp;
Obtain the time difference between the timestamp of the newest video frame and the timestamp of the broadcast information to be synchronized;
The play time of the broadcast information to be synchronized is determined according to the time difference, and according to play time broadcasting Broadcast information to be synchronized.
2. synchronizing information playback method as described in claim 1, which is characterized in that described according to time difference determination The play time of broadcast information to be synchronized, comprising:
When the time difference being not less than the first preset duration, waited according to the first waiting time, and when waiting for Determine that current time is the play time of the broadcast information to be synchronized;
When the time difference being not more than the second preset duration, when determining that current time is the broadcasting of the broadcast information to be synchronized Between, wherein second preset duration is less than first preset duration.
3. synchronizing information playback method as described in claim 1, which is characterized in that described according to time difference determination The play time of broadcast information to be synchronized, further includes:
When the time difference being less than first preset duration and being greater than second preset duration, according to the first waiting time It is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, the timestamp based on the new video frame is updated the time difference, and according to the updated time difference Determine the play time of the broadcast information to be synchronized;
If it is not, then when determining that current time is the broadcasting of the broadcast information to be synchronized after waiting first waiting time Between.
4. synchronizing information playback method as claimed in claim 3, which is characterized in that described to be determined according to the updated time difference The play time of the broadcast information to be synchronized, comprising:
When the updated time difference being not more than second preset duration, determine that current time is the broadcast information to be synchronized Play time;
When the updated time difference being less than first preset duration and being greater than second preset duration, obtains second and wait Duration;
First waiting time is updated according to second waiting time, and returns to execution according to the first waiting time The step of being waited.
5. synchronizing information playback method as claimed in claim 3, which is characterized in that the time based on the new video frame Stamp is updated the time difference, comprising:
It is updated according to timestamp of the timestamp of the new video frame to the newest video frame;
Obtain the new time difference between the timestamp of the broadcast information to be synchronized and the timestamp of newest video frame after update;
The time difference is updated according to the new time difference.
6. a kind of synchronizing information playing device characterized by comprising
Video reception unit for receiving the video data of first server transmission, and plays out the video data, institute Stating video data includes the video frame for carrying timestamp;
First extraction unit, for when receive second server transmission needs with the video data synchronization play to same When walking broadcast information, the timestamp of the broadcast information to be synchronized is extracted;
Second extraction unit, for extracting the timestamp of newest video frame, wherein the newest video frame is currently received Nearest apart from current time and carrying timestamp video frame in video frame;
Time difference acquiring unit, for obtain the newest video frame timestamp and the broadcast information to be synchronized timestamp it Between time difference;
Information playing unit, for determining the play time of the broadcast information to be synchronized according to the time difference, and according to institute It states play time and plays the broadcast information to be synchronized.
7. synchronizing information playing device as claimed in claim 6, which is characterized in that the information playing unit, comprising:
First plays subelement, for being carried out according to the first waiting time when the time difference being not less than the first preset duration It waits, and determines that current time is the play time of the broadcast information to be synchronized when waiting for, when according to the broadcasting Between play the broadcast information to be synchronized;
Second plays subelement, for when the time difference being not more than the second preset duration, determine current time be it is described to The play time of information is played simultaneously, and the broadcast information to be synchronized is played according to the play time, wherein described second Preset duration is less than first preset duration.
8. synchronizing information playing device as claimed in claim 6, which is characterized in that the information playing unit further includes Three play subelement, are used for:
When the time difference being less than first preset duration and being greater than second preset duration, according to the first waiting time It is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, the timestamp based on the new video frame is updated the time difference, and according to the updated time difference Determine the play time of the broadcast information to be synchronized;
If it is not, then when determining that current time is the broadcasting of the broadcast information to be synchronized after waiting first waiting time Between;
The broadcast information to be synchronized is played according to the play time.
9. synchronizing information playing device as claimed in claim 8, which is characterized in that the third plays subelement, specific to use In:
When the time difference being less than first preset duration and being greater than second preset duration, according to the first waiting time It is waited, and detects whether to receive the new video frame for carrying timestamp in waiting process;
If so, being updated according to timestamp of the timestamp of the new video frame to the newest video frame;
Obtain the new time difference between the timestamp of the broadcast information to be synchronized and the timestamp of newest video frame after update;
The time difference is updated according to the new time difference;
The play time of the broadcast information to be synchronized is determined according to the updated time difference;
If it is not, then when determining that current time is the broadcasting of the broadcast information to be synchronized after waiting first waiting time Between;
The broadcast information to be synchronized is played according to the play time.
10. a kind of storage medium, which is characterized in that the storage equipment for storing a plurality of instruction, described instruction be suitable for by Reason device loads and executes step as described in any one in claim 1-5.
CN201810868257.7A 2018-08-02 2018-08-02 Information synchronous playing method, device and storage medium Active CN110166788B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810868257.7A CN110166788B (en) 2018-08-02 2018-08-02 Information synchronous playing method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810868257.7A CN110166788B (en) 2018-08-02 2018-08-02 Information synchronous playing method, device and storage medium

Publications (2)

Publication Number Publication Date
CN110166788A true CN110166788A (en) 2019-08-23
CN110166788B CN110166788B (en) 2021-11-02

Family

ID=67645168

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810868257.7A Active CN110166788B (en) 2018-08-02 2018-08-02 Information synchronous playing method, device and storage medium

Country Status (1)

Country Link
CN (1) CN110166788B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110636321A (en) * 2019-09-30 2019-12-31 北京达佳互联信息技术有限公司 Data processing method, device, system, mobile terminal and storage medium
CN110933449A (en) * 2019-12-20 2020-03-27 北京奇艺世纪科技有限公司 Method, system and device for synchronizing external data and video pictures
CN111601136A (en) * 2020-05-11 2020-08-28 腾讯科技(深圳)有限公司 Video data processing method and device, computer equipment and storage medium
CN112380266A (en) * 2020-10-16 2021-02-19 广州市百果园网络科技有限公司 Message data processing method, device, equipment and storage medium
CN112839235A (en) * 2020-12-30 2021-05-25 北京达佳互联信息技术有限公司 Display method, comment sending method, video frame pushing method and related equipment
CN113312119A (en) * 2021-06-04 2021-08-27 广州博冠信息科技有限公司 Information synchronization method and device, computer readable storage medium and electronic equipment
CN114339328A (en) * 2021-12-31 2022-04-12 杭州当虹科技股份有限公司 Method and system for realizing timestamp synchronization service
CN114900727A (en) * 2022-05-11 2022-08-12 上海哔哩哔哩科技有限公司 Video stream processing method and device
CN115278292A (en) * 2022-06-30 2022-11-01 北京爱奇艺科技有限公司 Video reasoning information display method and device and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1859565A (en) * 2005-05-01 2006-11-08 腾讯科技(深圳)有限公司 Method for broadcastin stream media caption and its stream media player
CN105323655A (en) * 2015-10-10 2016-02-10 上海慧体网络科技有限公司 Method for synchronizing videos/score according to time stamps on mobile terminal
US20170134831A1 (en) * 2011-04-21 2017-05-11 Shah Talukder Flow Controlled Based Synchronized Playback of Recorded Media
CN108260024A (en) * 2018-04-12 2018-07-06 腾讯科技(深圳)有限公司 Live data processing method, device, computer equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1859565A (en) * 2005-05-01 2006-11-08 腾讯科技(深圳)有限公司 Method for broadcastin stream media caption and its stream media player
US20170134831A1 (en) * 2011-04-21 2017-05-11 Shah Talukder Flow Controlled Based Synchronized Playback of Recorded Media
CN105323655A (en) * 2015-10-10 2016-02-10 上海慧体网络科技有限公司 Method for synchronizing videos/score according to time stamps on mobile terminal
CN108260024A (en) * 2018-04-12 2018-07-06 腾讯科技(深圳)有限公司 Live data processing method, device, computer equipment and storage medium

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110636321A (en) * 2019-09-30 2019-12-31 北京达佳互联信息技术有限公司 Data processing method, device, system, mobile terminal and storage medium
CN110933449A (en) * 2019-12-20 2020-03-27 北京奇艺世纪科技有限公司 Method, system and device for synchronizing external data and video pictures
CN111601136A (en) * 2020-05-11 2020-08-28 腾讯科技(深圳)有限公司 Video data processing method and device, computer equipment and storage medium
CN111601136B (en) * 2020-05-11 2021-03-26 腾讯科技(深圳)有限公司 Video data processing method and device, computer equipment and storage medium
CN112380266A (en) * 2020-10-16 2021-02-19 广州市百果园网络科技有限公司 Message data processing method, device, equipment and storage medium
CN112839235A (en) * 2020-12-30 2021-05-25 北京达佳互联信息技术有限公司 Display method, comment sending method, video frame pushing method and related equipment
CN113312119A (en) * 2021-06-04 2021-08-27 广州博冠信息科技有限公司 Information synchronization method and device, computer readable storage medium and electronic equipment
CN113312119B (en) * 2021-06-04 2024-03-15 广州博冠信息科技有限公司 Information synchronization method and device, computer readable storage medium and electronic equipment
CN114339328A (en) * 2021-12-31 2022-04-12 杭州当虹科技股份有限公司 Method and system for realizing timestamp synchronization service
CN114339328B (en) * 2021-12-31 2023-09-01 杭州当虹科技股份有限公司 Method and system for realizing time stamp synchronous service
CN114900727A (en) * 2022-05-11 2022-08-12 上海哔哩哔哩科技有限公司 Video stream processing method and device
CN115278292A (en) * 2022-06-30 2022-11-01 北京爱奇艺科技有限公司 Video reasoning information display method and device and electronic equipment
CN115278292B (en) * 2022-06-30 2023-12-05 北京爱奇艺科技有限公司 Video reasoning information display method and device and electronic equipment

Also Published As

Publication number Publication date
CN110166788B (en) 2021-11-02

Similar Documents

Publication Publication Date Title
CN110166788A (en) Synchronizing information playback method, device and storage medium
CN106792188B (en) Data processing method, device and system for live broadcast page and storage medium
CN105872830B (en) Interactive approach and device in direct broadcast band
CN110418151B (en) Bullet screen information sending and processing method, device, equipment and medium in live game
CN108243151B (en) Automatic animation playing method and device, client and video live broadcast system
CN105933739B (en) Program interaction system, method, client and background server
CN108123945B (en) Distribution method, system and the intelligent terminal of random data
CN104519124B (en) A kind of distribution method and device of virtual resource
CN109525851A (en) Live broadcasting method, device and storage medium
CN111314721B (en) Method, device, equipment and medium for determining abnormal live broadcast
CN109685538B (en) Resource acquisition information processing method and device and electronic equipment
CN110225412B (en) Video interaction method, device and storage medium
CN109921976A (en) A kind of communication control method based on group, device and storage medium
CN108260016A (en) Processing method, device, equipment, system and storage medium is broadcast live
CN107645682B (en) The method and system being broadcast live
CN111314714B (en) Game live broadcast method and device
CN108449605B (en) Information synchronous playing method, device, equipment, system and storage medium
CN109587509A (en) Live-broadcast control method, device, computer readable storage medium and terminal
CN110830813B (en) Video switching method and device, electronic equipment and storage medium
CN107333163A (en) A kind of method for processing video frequency and device, a kind of terminal and storage medium
CN112672179A (en) Method, device and equipment for live game
CN108900855A (en) live content recording method and device
CN112367530A (en) Video processing method and device, electronic equipment and storage medium
CN111569436A (en) Processing method, device and equipment based on interaction in live broadcast fighting
CN114286156A (en) Live broadcast interaction method and device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant