CN111405298A - Android end-to-end live broadcast method based on KCP protocol - Google Patents
Android end-to-end live broadcast method based on KCP protocol Download PDFInfo
- Publication number
- CN111405298A CN111405298A CN202010095724.4A CN202010095724A CN111405298A CN 111405298 A CN111405298 A CN 111405298A CN 202010095724 A CN202010095724 A CN 202010095724A CN 111405298 A CN111405298 A CN 111405298A
- Authority
- CN
- China
- Prior art keywords
- data
- android
- kcp
- playing
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2347—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving video stream encryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
- H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/835—Generation of protective data, e.g. certificates
- H04N21/8358—Generation of protective data, e.g. certificates involving watermark
Abstract
The invention discloses an Android end-to-end live broadcast method based on a KCP protocol, which comprises the following steps: 101, an Android plug-in terminal collects live audio and video data to preview the local computer and copies the live audio and video data to the local computer; 102Android plug-flow terminals transcode copied data; 103, the Android streaming end sends data to an Android playing end through KCP; 104 receiving the data by the Android playing terminal through KCP; 105, calculating the received data by the Android playing end; and 106, playing the data by the Android playing end. According to the method, the Android stream pushing end is mainly used for copying and transcoding the collected audio and video data, the Android stream pushing end is used for sending the audio and video data through a KCP (KCP protocol), and the Android playing end is used for decompressing, calculating and playing the received data, so that live broadcasting can still maintain the real-time performance and the fluency of watching under the condition of poor network, and the live broadcasting experience of a user is optimized.
Description
Technical Field
The invention belongs to the technical field of video transmission, and particularly relates to an Android end-to-end live broadcast method based on a KCP protocol.
Background
The traditional Android end-to-end live broadcast method generally adopts RTMP and H L S protocols for propagation, the live broadcast delay of RTMP can be maintained at 2S, but when the network is unstable, the RTMP is based on TCP, the RTMP fully utilizes the attribute of network bandwidth to enable a receiving end to not receive data on time in a live broadcast scheme, so live broadcast delay is caused, and the H L S protocol has the design principle that M3U8 files and TS Playlist must be buffered by 4-10S of time, so that the live broadcast delay of H L S is certainly more than 4S.
The KCP protocol is a protocol that has a transmission effect of reducing the average delay by 30-40% and the maximum delay by three times at the cost of wasting 10-20% of bandwidth compared with TCP, and can transmit data faster in a scene with a poor network state. The KCP is only used in the field of game research and development due to huge flow consumption. Under the condition of poor network state, the data can be more efficiently transmitted due to high flow consumption of KCP, so that the method is applied to Android' end-to-end video live broadcast, the Android playing end can be guaranteed to receive the video data of the Android streaming end on time under the condition of poor network state, and the real-time performance and the fluency of live broadcast are improved. The invention is based on the scene that the network state is very poor.
Disclosure of Invention
The present invention is directed to solving the problems of the prior art. The Android end-to-end live broadcast method enables live broadcast to still maintain watching instantaneity and fluency under the condition of poor network and optimizes live broadcast experience of users based on the KCP protocol. The technical scheme of the invention is as follows:
an Android end-to-end live broadcast method based on a KCP protocol comprises the following steps:
the Android streaming terminal collects live audio and video data, previews the live audio and video data and copies the live audio and video data to the Android streaming terminal;
102, transcoding the copied data by the Android plug-flow end;
103, the Android streaming end sends data to an Android playing end through a KCP (fast reliable transport protocol);
104, receiving data through KCP by the Android playing terminal;
105, calculating the received data by the Android playing end by adopting a frame tracing algorithm;
and 106, playing the data by the Android playing end.
Further, in the step 101, the Android stream pushing end collects live audio and video data to preview the local computer and copies the live audio and video data to the local computer, and the specific steps are as follows:
1011. live broadcast audio and video data acquisition: opening a front camera or a rear camera of the mobile phone, setting the width and the height of data to be obtained, and then circularly obtaining video data Image0 in a callback function of the mobile phone camera, wherein the Image0 obtained each time is a picture generated by the camera at the current moment;
1012. copying live audio and video data: copying each Image0, and sequentially storing copied data images 1 in a blocking Queue 0;
1013. previewing live audio and video data: images 0 are sequentially displayed on the display screen of the mobile phone.
Further, the specific steps of transcoding the copied data by the Android stream pushing end in step 102 are as follows:
1021. watermarking the copy data of the live audio and video: sequentially taking data a0 from Queue0 and storing a0 by an array a0[ i0] [ j0], wherein i0 represents the width of the picture and is the unit of pixel; j0 represents the height of the picture in pixels;
the current date and time are generated into a picture and stored by an array a1[ i1] [ j1], wherein i1 represents the width of the watermark in pixels; j1 denotes the height of the watermark in pixels, the element value in a1 that denotes the date and time is 1, and the other element values are 0;
if the watermark needs to be added at a position which is x pixel distance away from the left boundary of the a0 picture and y pixel distance away from the upper boundary of the a0 picture, a2[ i1] [ j1] of a matrix which starts from a0[ x ] [ y ] and ends from a0[ x + i1] [ y + j1] is found from a0, a1[ a ] [ b ] is compared with a2[ a ] [ b ], and when the element value of a1[ a ] [ b ] is 1, the element value of a2[ a ] [ b ] is set as c, and the c is the corresponding value of the watermark color to be added in the current coding format, wherein a ∈ {0, i1-1}, b ∈ {0, j1-1 };
1022. transcoding live audio and video copy data: the format of the watermarked a0 is NV21, after a0 needs to be converted into NV12, a0 is encoded into a1 according to H264, and finally, the Android push stream end stores a1 in a blocking Queue1 to wait for being sent;
1023. the user can selectively store the pushed video data in the Android push streaming terminal locally: copying the a1, and removing SPS and PPS data, namely packaging the data into MP4 data and storing the MP4 data in a designated mobile phone storage position.
Further, the step 103 of sending the data to the Android player by the Android streaming client through KCP specifically includes:
1031. creating and initializing KCP objects: creating and initializing a KCP Object-KCP, and setting a transmission callback function F0 for the Object-KCP, wherein F0 is used for calling a UDP receiving function F1 at the bottom layer;
1032. circularly calling a KCP Update function KCP-Update: calling a KCP-Update once every 10ms, and processing data receiving logic and data sending logic in the KCP-Update;
the data receiving logic in the KCP-Update is as follows: the application layer transmits the received data D0 to F0 in a KCP-Update through a KCP entry function KCP-Input through F1, D0 is analyzed and secondarily packaged into D1 by F0 and then transmitted to a KCP receiving function KCP-Recv, and the application layer can obtain processed D1 by calling the KCP-Recv;
the data transmission logic in the KCP-Update is as follows: the application layer sends data D2 needing to be sent to KCP-Update through a KCP sending function KCP-Send, the KCP-Update divides and numbers D2 into pieces and packages the pieces into D3, and then calls a UDP sending function F2 through a KCP outlet function KCP-Out to Send Out D3;
1033, a method for calling KCP-Update by the Android plug-flow end is as follows: the Android streaming end sends a video data packet in Queue1 by using data sending logic in KCP-Update, receives acknowledgement packet reply ACK sent by the Android playing end by using data receiving logic in KCP-Update and judges whether retransmission is needed according to the ACK.
Further, the step 104 of receiving data by the Android playing terminal through KCP specifically includes:
the method for calling the KCP-Update by the Android playing end comprises the following steps: the Android playing end sends a confirmation packet reply ACK of the video data packet by using the data sending logic in the KCP-Update, receives the video data packet sent by the Android streaming end by using the data receiving logic in the KCP-Update, and judges whether packet loss is needed or not according to the number of the packet, and the Android playing end stores the data packet needing to be played into a blocking Queue Queue 2.
Further, the step 105 of calculating the received data by the Android playing end specifically includes:
1051, the Android play end decodes the received data: video data needing to be played is taken out from Queue2, the video data is H264 data, the video data can be displayed after being decoded into NV12 data, and the decoded data is placed in a blocking Queue 3;
1052, performing frame pursuit operation on the received video data by the Android playing end: suppose that a packet is received by the playing end within a time unit t. When the network is unstable, the playing end may not receive the data packets from the z-th time unit, but receive n data packets in the z + n time units, and at this time, if the player sequentially fetches the packets from the Queue2 to decode and play, the live broadcast picture will be delayed, because the packet that should be played in the z-th time unit is originally received only in the z + n time units, the Android playing end will start to chase the frame so that the player plays the data corresponding to the current time, and the specific frame tracking algorithm is as follows: and discarding the first n-1 packets of the n data packets received by the z + n time units, and directly decoding and playing the nth data packet.
Further, the step 106 of playing the received data by the Android playing end specifically includes:
1061, rendering the received data by the Android play end: video data are taken out from the Queue 3and drawn on a display screen of an Android playing end;
1062. the user can select to pause playing operation and continue playing operation on the Android playing end: when the user chooses to pause the play, the player will first stop playing data to Queue1, i.e. no more data packets are stored, then empty Queue1, and then Queue 2. When the user selects to continue playing after pausing, the player resumes storing, decoding and rendering each received packet.
The invention has the following advantages and beneficial effects:
the innovation point of the invention is mainly the step 103and the step 104. The KCP is only used in the field of game research and development due to huge flow consumption. Under the condition of poor network state, the data can be more efficiently transmitted due to high flow consumption of KCP, so that the method is applied to Android' end-to-end video live broadcast, the Android playing end can be guaranteed to receive the video data of the Android streaming end on time under the condition of poor network state, and the real-time performance and the fluency of live broadcast are improved.
Drawings
Fig. 1 is a flowchart of an Android end-to-end live broadcast method based on a KCP protocol according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described in detail and clearly with reference to the accompanying drawings. The described embodiments are only some of the embodiments of the present invention.
The technical scheme for solving the technical problems is as follows:
referring to fig. 1, fig. 1 is a flowchart of an Android end-to-end live broadcast method based on a KCP protocol according to an embodiment of the present invention, which specifically includes:
101, an Android plug-in terminal collects live audio and video data to preview the local computer and copies the live audio and video data to the local computer, and the method specifically comprises the following steps:
1011. live broadcast audio and video data acquisition: opening a front camera or a rear camera of the mobile phone, setting the width and the height of the acquired data, and then circularly acquiring video data Image0 in a callback function of the mobile phone camera, wherein the Image0 acquired each time is a picture generated by the camera at the current moment.
1012. Copying live audio and video data: each Image0 is copied, and the copied data Image1 is stored in the block Queue0 in sequence.
1013. Previewing live audio and video data: images 0 are sequentially displayed on the display screen of the mobile phone.
102, the Android plug-flow end transcodes the copied data, and specifically comprises the following steps:
1021. watermarking the copy data of the live audio and video: sequentially taking data a0 from Queue0 and storing a0 by an array a0[ i0] [ j0], wherein i0 represents the width of the picture and is the unit of pixel; j0 represents the height of the picture in pixels.
The current date and time are generated into a picture and stored by an array a1[ i1] [ j1], wherein i1 represents the width of the watermark in pixels; j1 represents the height of the watermark in pixels. The element value of a1 indicating the date and time is 1, and the other element values are 0.
If the watermark needs to be added at a position which is x pixel distance away from the left boundary of the a0 picture and y pixel distance away from the upper boundary of the a0 picture, a2[ i1] [ j1] of a matrix which starts from a0[ x ] [ y ] and ends from a0[ x + i1] [ y + j1] is found from a0, a1[ a ] [ b ] is compared with a2[ a ] [ b ], and when the element value of a1[ a ] [ b ] is 1, the element value of a2[ a ] [ b ] is set as c (c is the corresponding value of the watermark color to be added in the current coding format), wherein a ∈ {0, i1-1}, b ∈ {0, j1-1 };
1022. transcoding live audio and video copy data: the format of the watermarked a0 is NV21, after a0 needs to be converted into NV12, a0 is encoded into a1 according to H264. Finally, the Android push flow end will save a1 to the blocking Queue1, waiting to be sent.
1023. The user can select to store the pushed video data in the Android push streaming terminal locally: copying the a1, and removing SPS and PPS data, namely packaging the data into MP4 data and storing the MP4 data in a designated mobile phone storage position.
103.103 the Android streaming end sends data to the Android playing end through KCP, and the method specifically comprises the following steps:
1031. creating and initializing KCP objects: creates and initializes a KCP Object-KCP and sets a transmission callback function F0 for the Object-KCP. The function of F0 is to call the underlying UDP reception function F1.
1032. Circularly calling a KCP Update function KCP-Update: the KCP-Update is called once every 10ms, and data receiving logic and data transmitting logic are processed in the KCP-Update.
The data receiving logic in the KCP-Update is as follows: the application layer transmits the received data D0 to F0 in the KCP-Update through a KCP entry function KCP-Input through F1, D0 is analyzed and secondarily packaged into D1 by F0 and then transmitted to a KCP receiving function KCP-Recv, and the application layer can obtain the processed D1 by calling the KCP-Recv.
The data transmission logic in the KCP-Update is as follows: the application layer sends data D2 needing to be sent to KCP-Update through a KCP sending function KCP-Send, the KCP-Update divides and numbers D2 into pieces and packages the pieces into D3, and then the application layer calls a UDP sending function F2 through a KCP outlet function KCP-Out to Send Out D3.
1033, a method for calling KCP-Update by the Android plug-flow end is as follows: the Android streaming end sends a video data packet in Queue1 by using data sending logic in KCP-Update, receives acknowledgement packet reply ACK sent by the Android playing end by using data receiving logic in KCP-Update and judges whether retransmission is needed according to the ACK.
104, receiving data through KCP by the Android playing end, specifically as follows: the method for calling the KCP-Update by the Android playing end comprises the following steps: and the Android playing end sends an acknowledgement packet ACK (acknowledgement) to the video data packet by using the data sending logic in the KCP-Update, receives the video data packet sent by the Android streaming end by using the data receiving logic in the KCP-Update, and judges whether packet loss is required according to the number of the packet. The Android player will store the data packets to be played into the congestion Queue 2. 1041. Definition of nodes of the similarity graph: each sample in the partial label dataset is considered a node in the similarity graph.
And 105, calculating the received data by the Android playing end, specifically as follows:
1051, the Android play end decodes the received data: and taking out video data to be played from Queue2, wherein the data is H264 data and can be displayed after being decoded into NV12 data. The decoded data is placed in a block Queue 3.
1052, performing frame pursuit operation on the received video data by the Android playing end: suppose that a packet is received by the playing end within a time unit t. When the network is unstable, the playing end may not receive the data packets from the z-th time unit, but receives n data packets in the z + n time units, and at this time, if the player sequentially fetches the packets from the Queue2 for decoding and playing, the live broadcast frame will have a delay, because the packets that should be played in the z-th time unit are received in the z + n time units. At this time, in order to ensure real-time performance of the live video, the Android playing end starts frame tracking so that the player plays data corresponding to the current time. The specific frame tracking algorithm is as follows: and discarding the first n-1 packets of the n data packets received by the z + n time units, and directly decoding and playing the nth data packet.
106, the Android playing end plays the received data, specifically as follows:
1061, rendering the received data by the Android play end: and taking out the video data from the Queue 3and drawing the video data on a display screen of the Android player.
1062. The user can select to pause playing operation and continue playing operation on the Android playing end: when the user chooses to pause the play, the player will first stop playing data to Queue1, i.e. no more data packets are stored, then empty Queue1, and then Queue 2. When the user selects to continue playing after pausing, the player resumes storing, decoding and rendering each received packet.
The above examples are to be construed as merely illustrative and not limitative of the remainder of the disclosure. After reading the description of the invention, the skilled person can make various changes or modifications to the invention, and these equivalent changes and modifications also fall into the scope of the invention defined by the claims.
Claims (7)
1.An Android end-to-end live broadcast method based on a KCP protocol is characterized by comprising the following steps:
the Android streaming terminal collects live audio and video data, previews the live audio and video data and copies the live audio and video data to the Android streaming terminal;
102, transcoding the copied data by the Android plug-flow end;
103, the Android streaming end sends data to an Android playing end through a KCP fast and reliable transmission protocol;
104, receiving data through KCP by the Android playing terminal;
105, calculating the received data by the Android playing end by adopting a frame tracing algorithm;
and 106, playing the data by the Android playing end.
2. The Android end-to-end live broadcast method based on the KCP protocol as claimed in claim 1, wherein in step 101, the Android stream pushing end collects live audio and video data to preview and copy the live audio and video data to the machine, and the specific steps are as follows:
1011. live broadcast audio and video data acquisition: opening a front camera or a rear camera of the mobile phone, setting the width and the height of data to be obtained, and then circularly obtaining video data Image0 in a callback function of the mobile phone camera, wherein the Image0 obtained each time is a picture generated by the camera at the current moment;
1012. copying live audio and video data: copying each Image0, and sequentially storing copied data images 1 in a blocking Queue 0;
1013. previewing live audio and video data: images 0 are sequentially displayed on the display screen of the mobile phone.
3. The Android end-to-end live broadcast method based on the KCP protocol as claimed in claim 2, wherein the step 102 of transcoding the copied data by the Android stream push end specifically comprises the steps of:
1021. watermarking the copy data of the live audio and video: sequentially taking data a0 from Queue0 and storing a0 by an array a0[ i0] [ j0], wherein i0 represents the width of the picture and is the unit of pixel; j0 represents the height of the picture in pixels;
the current date and time are generated into a picture and stored by an array a1[ i1] [ j1], wherein i1 represents the width of the watermark in pixels; j1 denotes the height of the watermark in pixels, the element value in a1 that denotes the date and time is 1, and the other element values are 0;
if the watermark needs to be added at the position which is x pixel distance away from the left boundary of the a0 picture and y pixel distance away from the upper boundary of the a0 picture, a matrix a2[ i1] [ j1] starting from a0[ x ] [ y ] and ending from a0[ x + i1] [ y + j1] is found from a0, a1[ a ] [ b ] is compared with a2[ a ] [ b ], when the element value of the a1[ a ] [ b ] is 1, the element value of a2[ a ] [ b ] is set as c, and the c is the corresponding value of the watermark color to be added in the current coding format, wherein,
a∈{0，i1-1}，b∈{0，j1-1}；
1022. transcoding live audio and video copy data: the format of the watermarked a0 is NV21, after a0 needs to be converted into NV12, a0 is encoded into a1 according to H264, and finally, the Android push stream end stores a1 in a blocking Queue1 to wait for being sent;
1023. the user can selectively store the pushed video data in the Android push streaming terminal locally: copying the a1, and removing SPS and PPS data, namely packaging the data into MP4 data and storing the MP4 data in a designated mobile phone storage position.
4. The Android end-to-end live broadcast method based on the KCP protocol as claimed in claim 3, wherein the step 103 of the Android streaming end sending data to the Android playing end through KCP comprises the specific steps of:
1031. creating and initializing KCP objects: creating and initializing a KCP Object-KCP, and setting a transmission callback function F0 for the Object-KCP, wherein F0 is used for calling a UDP receiving function F1 at the bottom layer;
1032. circularly calling a KCP Update function KCP-Update: calling a KCP-Update once every 10ms, and processing data receiving logic and data sending logic in the KCP-Update;
the data receiving logic in the KCP-Update is as follows: the application layer transmits the received data D0 to F0 in a KCP-Update through a KCP entry function KCP-Input through F1, D0 is analyzed and secondarily packaged into D1 by F0 and then transmitted to a KCP receiving function KCP-Recv, and the application layer can obtain processed D1 by calling the KCP-Recv;
the data transmission logic in the KCP-Update is as follows: the application layer sends data D2 needing to be sent to KCP-Update through a KCP sending function KCP-Send, the KCP-Update divides and numbers D2 into pieces and packages the pieces into D3, and then calls a UDP sending function F2 through a KCP outlet function KCP-Out to Send Out D3;
1033, a method for calling KCP-Update by the Android plug-flow end is as follows: the Android streaming end sends a video data packet in Queue1 by using data sending logic in KCP-Update, receives acknowledgement packet reply ACK sent by the Android playing end by using data receiving logic in KCP-Update and judges whether retransmission is needed according to the ACK.
5. The Android end-to-end live broadcast method based on the KCP protocol as claimed in claim 4, wherein the step 104 of receiving data by the Android playing terminal through KCP comprises the specific steps of:
the method for calling the KCP-Update by the Android playing end comprises the following steps: the Android playing end sends a confirmation packet reply ACK of the video data packet by using the data sending logic in the KCP-Update, receives the video data packet sent by the Android streaming end by using the data receiving logic in the KCP-Update, and judges whether packet loss is needed or not according to the number of the packet, and the Android playing end stores the data packet needing to be played into a blocking Queue Queue 2.
6. The Android end-to-end live broadcast method based on the KCP protocol of claim 5, wherein the step 105 of calculating the received data by the Android play end specifically comprises the steps of:
1051, the Android play end decodes the received data: video data needing to be played is taken out from Queue2, the video data is H264 data, the video data can be displayed after being decoded into NV12 data, and the decoded data is placed in a blocking Queue 3;
1052, performing frame pursuit operation on the received video data by the Android playing end: suppose that a packet is received by the playing end within a time unit t. When the network is unstable, the playing end may not receive the data packets from the z-th time unit, but receive n data packets in the z + n time units, and at this time, if the player sequentially fetches the packets from the Queue2 to decode and play, the live broadcast picture will be delayed, because the packet that should be played in the z-th time unit is originally received only in the z + n time units, the Android playing end will start to chase the frame so that the player plays the data corresponding to the current time, and the specific frame tracking algorithm is as follows: and discarding the first n-1 packets of the n data packets received by the z + n time units, and directly decoding and playing the nth data packet.
7. The Android end-to-end live broadcast method based on the KCP protocol as claimed in claim 6, wherein the step 106 of playing the received data by the Android playing end specifically comprises the steps of:
1061, rendering the received data by the Android play end: video data are taken out from the Queue 3and drawn on a display screen of an Android playing end;
1062. the user can select to pause playing operation and continue playing operation on the Android playing end: when the user chooses to pause the playback, the player will first stop playing data to Queue1, i.e. no more data packets are stored, then empty Queue 1and then Queue2, and when the user chooses to continue the playback after pausing, the player resumes storing, decoding and rendering each received packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095724.4A CN111405298A (en) | 2020-02-17 | 2020-02-17 | Android end-to-end live broadcast method based on KCP protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010095724.4A CN111405298A (en) | 2020-02-17 | 2020-02-17 | Android end-to-end live broadcast method based on KCP protocol |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111405298A true CN111405298A (en) | 2020-07-10 |
Family
ID=71428462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010095724.4A Pending CN111405298A (en) | 2020-02-17 | 2020-02-17 | Android end-to-end live broadcast method based on KCP protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111405298A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106385594A (en) * | 2016-09-18 | 2017-02-08 | 深圳市青柠互动科技开发有限公司 | Method for optimizing video live broadcast services |
| CN107277558A (en) * | 2017-06-19 | 2017-10-20 | 网宿科技股份有限公司 | A kind of player client for realizing that live video is synchronous, system and method |
| CN107846621A (en) * | 2017-09-28 | 2018-03-27 | 北京京东尚科信息技术有限公司 | Delay process method, system and electronic equipment |
| CN108055472A (en) * | 2017-12-21 | 2018-05-18 | 长沙全度影像科技有限公司 | A kind of real time panoramic live broadcast system and method |
| CN108184136A (en) * | 2018-01-16 | 2018-06-19 | 北京三体云联科技有限公司 | A kind of video collaborates method and device |
| US20180248969A1 (en) * | 2015-10-23 | 2018-08-30 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and apparatus for secure content caching and delivery |
| CN108540855A (en) * | 2018-04-18 | 2018-09-14 | 王健 | A kind of adaptive low delay streaming media playing software suitable under network direct broadcasting scene |
| CN108966008A (en) * | 2018-08-02 | 2018-12-07 | 腾讯科技（深圳）有限公司 | Live video back method and device |
| CN109005194A (en) * | 2018-09-04 | 2018-12-14 | 厦门安胜网络科技有限公司 | Portless shadow communication means and computer storage medium based on KCP agreement |
| CN109151594A (en) * | 2018-09-27 | 2019-01-04 | 广州虎牙信息科技有限公司 | Direct playing and playback video broadcasting method, device and electronic equipment |
| CN109327716A (en) * | 2018-10-31 | 2019-02-12 | 北京达佳互联信息技术有限公司 | Delay control method, delay control device and computer readable storage medium |
| CN109474365A (en) * | 2018-12-29 | 2019-03-15 | 深圳市柠檬互动科技有限公司 | A kind of frame synchronization UDP network synchronization method |
| CN110049347A (en) * | 2019-04-11 | 2019-07-23 | 广州虎牙信息科技有限公司 | In method, system, terminal and the device of live streaming interface configurations image |
-
2020
- 2020-02-17 CN CN202010095724.4A patent/CN111405298A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180248969A1 (en) * | 2015-10-23 | 2018-08-30 | Telefonaktiebolaget Lm Ericsson (Publ) | Method and apparatus for secure content caching and delivery |
| CN106385594A (en) * | 2016-09-18 | 2017-02-08 | 深圳市青柠互动科技开发有限公司 | Method for optimizing video live broadcast services |
| CN107277558A (en) * | 2017-06-19 | 2017-10-20 | 网宿科技股份有限公司 | A kind of player client for realizing that live video is synchronous, system and method |
| CN107846621A (en) * | 2017-09-28 | 2018-03-27 | 北京京东尚科信息技术有限公司 | Delay process method, system and electronic equipment |
| CN108055472A (en) * | 2017-12-21 | 2018-05-18 | 长沙全度影像科技有限公司 | A kind of real time panoramic live broadcast system and method |
| CN108184136A (en) * | 2018-01-16 | 2018-06-19 | 北京三体云联科技有限公司 | A kind of video collaborates method and device |
| CN108540855A (en) * | 2018-04-18 | 2018-09-14 | 王健 | A kind of adaptive low delay streaming media playing software suitable under network direct broadcasting scene |
| CN108966008A (en) * | 2018-08-02 | 2018-12-07 | 腾讯科技（深圳）有限公司 | Live video back method and device |
| CN109005194A (en) * | 2018-09-04 | 2018-12-14 | 厦门安胜网络科技有限公司 | Portless shadow communication means and computer storage medium based on KCP agreement |
| CN109151594A (en) * | 2018-09-27 | 2019-01-04 | 广州虎牙信息科技有限公司 | Direct playing and playback video broadcasting method, device and electronic equipment |
| CN109327716A (en) * | 2018-10-31 | 2019-02-12 | 北京达佳互联信息技术有限公司 | Delay control method, delay control device and computer readable storage medium |
| CN109474365A (en) * | 2018-12-29 | 2019-03-15 | 深圳市柠檬互动科技有限公司 | A kind of frame synchronization UDP network synchronization method |
| CN110049347A (en) * | 2019-04-11 | 2019-07-23 | 广州虎牙信息科技有限公司 | In method, system, terminal and the device of live streaming interface configurations image |
Non-Patent Citations (2)
| Title |
|---|
| 王倩: "基于 Android端到端的实时无线视频传输系统", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 腾讯WETEST: "可靠UDP，KCP协议快在哪？", 《HTTPS://WWW.CNBLOGS.COM/WETEST/P/9190786.HTML》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11470405B2 (en) | Network video streaming with trick play based on separate trick play files | |
| US10720188B2 (en) | Systems and methods of thumbnail generation | |
| US10368075B2 (en) | Clip generation based on multiple encodings of a media stream | |
| US9350780B2 (en) | Insertion of graphic overlays into a stream | |
| US9332048B2 (en) | Key frame detection and synchronization | |
| US20140359678A1 (en) | Device video streaming with trick play based on separate trick play files | |
| US11128897B2 (en) | Method for initiating a transmission of a streaming content delivered to a client device and access point for implementing this method | |
| CN1893364A (en) | Milestone synchronization in broadcast multimedia streams | |
| US10862940B1 (en) | Low latency live video on a communication session | |
| CN111294612B (en) | Multimedia data processing method, system and storage medium | |
| WO2018014523A1 (en) | Media data acquisition method and apparatus | |
| SG183571A1 (en) | Movie file download device and method | |
| CN110602522B (en) | Multi-path real-time live webRTC stream synthesis method | |
| CN112616065A (en) | Screen image initiating method and device, computer equipment, readable storage medium and screen image presenting system | |
| KR100848309B1 (en) | Apparaus and method of providing internet TV brodacasting service using fast buffering switch | |
| CN111405298A (en) | Android end-to-end live broadcast method based on KCP protocol | |
| CN110769326B (en) | Method and device for loading video slice file and playing video file | |
| CN113508601A (en) | Client and method for managing a streaming session of multimedia content at a client | |
| WO2022100742A1 (en) | Video encoding and video playback method, apparatus and system | |
| CN116112739A (en) | Picture splitting screen protection method and device based on active frame loss and storage medium | |
| CN115665117A (en) | Webpage-side video stream playing method | |
| CN114189686A (en) | Video encoding method, apparatus, device, and computer-readable storage medium | |
| WO2009146341A1 (en) | Method and apparatus for improving channel acquisition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200710 |
|
| RJ01 | Rejection of invention patent application after publication |