WO2016049976A1 - 一种智能家居中流媒体数据无缝连接实现方法及系统 - Google Patents

一种智能家居中流媒体数据无缝连接实现方法及系统 Download PDF

Info

Publication number
WO2016049976A1
WO2016049976A1 PCT/CN2014/092604 CN2014092604W WO2016049976A1 WO 2016049976 A1 WO2016049976 A1 WO 2016049976A1 CN 2014092604 W CN2014092604 W CN 2014092604W WO 2016049976 A1 WO2016049976 A1 WO 2016049976A1
Authority
WO
WIPO (PCT)
Prior art keywords
streaming media
client
data
media data
server
Prior art date
Application number
PCT/CN2014/092604
Other languages
English (en)
French (fr)
Inventor
江跃龙
Original Assignee
宇龙计算机通信科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 宇龙计算机通信科技(深圳)有限公司 filed Critical 宇龙计算机通信科技(深圳)有限公司
Priority to US15/513,625 priority Critical patent/US20170302988A1/en
Priority to EP14903087.6A priority patent/EP3203754A1/en
Publication of WO2016049976A1 publication Critical patent/WO2016049976A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43078—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen for seamlessly watching content streams when changing device, e.g. when watching the same program sequentially on a TV and then on a tablet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/64—Addressing
    • H04N21/6405—Multicasting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/64—Addressing
    • H04N21/6408—Unicasting

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a method and system for seamlessly connecting streaming media data in a smart home.
  • Mobile streaming media technology is a method of compressing continuous video/audio information and putting it on a streaming media server, allowing users to download and watch while downloading through the mobile terminal, without waiting for the entire compressed file to be downloaded to their own machine. Watch.
  • the technology first creates a buffer on the computer, and pre-downloads a piece of data as a buffer before playing. When the actual connection speed of the network is less than the speed of playing the data, the player will use the small buffer.
  • the information to avoid the interruption of playback, also enables the playback quality to be maintained, which involves the collection, compression, storage, transmission and network communication of streaming media data.
  • the streaming media service has high requirements on real-time performance.
  • the streaming media jam, delay, and video quality jitter are often caused.
  • adversely affect the user's viewing experience For example, when the user watches a program on the living room television, when the user goes to the bedroom, the user needs to watch the program viewed on the living room television on the mobile phone.
  • the user can only reload the program on the mobile phone, and cannot perform the fast resume. And seamless fast switching. Therefore, for platforms that span different systems, solving the problem of users receiving video services is extremely important for seamless switching of streaming media.
  • the object of the present invention is to provide a method and system for seamlessly connecting streaming media data in a smart home, which can be realized when playing streaming media data between two terminals that are isolated from each other.
  • One terminal resumes the content of another terminal, enabling seamless and fast switching, making streaming media data easier to transfer between different smart terminals.
  • the present invention provides a method for seamlessly connecting streaming media data in a smart home, including:
  • the streaming media server caches the streaming media data between the current frame and the latest streaming media data frame in the streaming media file, where the latest streaming media data frame is the latest received streaming media data frame from the network, and the current frame is the first frame. a first streaming media data frame corresponding to a current playback progress when the client plays the streaming media file;
  • the second client sends a second streaming media data frame corresponding to the current playback progress when the streaming media file is played to the streaming media server.
  • the step of the streaming media server transmitting the streaming media data starting from the current frame to the second client requesting the streaming media file includes:
  • the step of the second client sending the second streaming media data frame corresponding to the current playing progress when the streaming media file is played to the streaming media server includes:
  • the second client sends a second streaming media data frame corresponding to the current playback progress when the streaming media file is played to the streaming media server in a unicast manner.
  • the method further comprises:
  • the streaming media server creates and maintains a media playing information database, where the media playing information database stores the association relationship between the media information and the time node information, and the streaming media data frame corresponding to the current playing progress;
  • the second client After the step of the second client sending the second streaming media data frame corresponding to the current playback progress to the streaming media file in the unicast mode, the second client further includes:
  • the streaming media server updates an association relationship between the media information in the media playing information library, the time node information, and a streaming media data frame corresponding to the current playing progress.
  • the method further comprises:
  • the streaming media server confirms the pre-selected streaming media file according to the recorded behavior record information of the user, and pre-decodes and/or decompresses the pre-selected streaming media file;
  • the streaming media server After receiving the pre-selected streaming media file selection information sent by the first client or the second client, the streaming media server sends the pre-decoded and/or decompressed streaming media file data to the The first client or the second client.
  • the method further comprises:
  • the invention also provides a seamless connection implementation system for streaming media data in a smart home, comprising a streaming media server, a first client and a second client;
  • the streaming server includes:
  • a cache module configured to cache the streaming media data between the current frame and the latest streaming media data frame in the streaming media file, where the latest streaming media data frame is the latest received streaming media data frame from the network, where the current frame is The first streaming media data frame corresponding to the current progress when the first client plays the streaming media file;
  • a data transmission module configured to send, to the second client that requests the streaming media file, the streaming media data that starts from the current frame
  • the second client is configured to send, to the streaming media server, a second streaming media data frame corresponding to a current playback progress when the streaming media file is played.
  • the data transmission module is further configured to send a multicast command to the second client, and send the flow starting from the current frame to the second client in a unicast or multicast manner.
  • the second client is further configured to send, to the streaming media server, the second streaming media data frame corresponding to the current playback progress when the streaming media file is played, in a unicast manner.
  • the streaming media server further includes a database management module for creating and maintaining a media play information database, wherein the media play information database stores media information corresponding to time node information and the current play progress. Correlation of streaming media data frames;
  • the database management module is further configured to update an association relationship between the media information in the media play information library, the time node information, and a streaming media data frame corresponding to the current play progress.
  • the streaming media server further includes a user behavior information processing module and a decoding/decompression module.
  • the streaming media server is further configured to receive user interface selection information and input information sent by the first client or the second client;
  • the user behavior information processing module determines a pre-selected streaming media file according to the recorded behavior record information of the user, and the decoding/decompression module pre-decodes and/or decompresses the data of the pre-selected streaming media file;
  • the data transmission module sends the pre-decoded and/or decompressed streaming media file data. To the first client or the second client.
  • the streaming media server is further configured to receive difference data of an audio data processing delay and a video data processing delay sent by the first client or the second client, and The audio data of the video data in advance of the same time stamp is sent to the first client or the second client, and the time of early transmission of the video data is determined according to the difference data.
  • the streaming media server of the present invention caches data between the current frame and the latest streaming media data frame in the streaming media file.
  • the streaming media server receives the download request of the user for the same streaming media file by the second client, the streaming media server will The streaming media data of the current frame is sent to the second client, and the second client sends the second streaming media data frame corresponding to the current playback progress to the streaming server for saving.
  • the streaming media server downloads and buffers the streaming media file requested by the first client from the network, and the streaming media server transmits the streaming media file to the first client, where the first client sends the streaming media file to the current progress.
  • Corresponding first streaming media data frame is sent to the streaming media server for saving, and the streaming media server buffers streaming media data from the first streaming media data frame to the newly downloaded streaming media data frame; when the first client pauses or stops playing The streaming media server saves the first streaming media data frame corresponding to the playback progress when the first client pauses or stops playing, and caches the first streaming media data frame corresponding to the playback progress from the pause or stop playing to the latest download.
  • the streaming media data begins with the first streaming media data frame. Therefore, the present invention can realize the content of continuing to broadcast another terminal at one terminal when the two terminals that are isolated from each other play streaming media data. Enable seamless and fast switching, making streaming media data easier to transfer between different smart terminals.
  • FIG. 1 is a block diagram of a seamless connection implementation system for streaming media data in a smart home according to the present invention
  • FIG. 2 is a flow chart of predicting streaming media data by a streaming media server according to an embodiment of the present invention
  • FIG. 3 is a flow chart of predicting streaming media data by a streaming media server according to another embodiment of the present invention.
  • FIG. 4 is a flow chart of a method for implementing seamless connection of streaming media data in a smart home according to the present invention.
  • the present invention provides a seamless connection implementation system 100 for streaming media data in a smart home, including a streaming media server 10 and a first client 20 and a second client 30.
  • the streaming server 10 can be a smart device such as a smart router or a smart box.
  • the first client 20 and the second client 30 can be smart devices such as a smart TV, a smart box, a smart phone or a tablet.
  • the system 100 is a software unit, a hardware unit or a combination of hardware and software built in a smart device.
  • the streaming server 10 includes a cache module 11 and a data transmission module 12.
  • the cache module 11 is configured to cache the streaming media data between the latest streaming media data frame in the streaming media file and the current frame.
  • the latest streaming media data frame is the latest received streaming media data frame from the network, and the latest streaming media data.
  • the frame is a frame between the current frame and the last frame (end frame) of the streaming media file, and the latest streaming media data frame may be determined according to the user's selection, and the current frame is the first client 20 playing stream.
  • the first streaming media data frame corresponding to the current progress of the media file.
  • the streaming media file is typically stored in a network node of the content sharing network, and the streaming media server 10 downloads the streaming media file from the network node over the network upon receiving the on-demand request. Before the streaming file download is completed, the streaming server 10 continues to receive streaming media data from the network, and the latest streaming media data frame is continuously updated as the data is received.
  • the cache module 11 buffers the streaming media data between the I frame newly received from the network in the streaming media file and the current frame.
  • the code stream structure defined by the existing video coding standard is a layered structure, including an image group, an image frame, a macroblock (MacroBlock, MB) and the like.
  • an I frame I frame, intra-coded frame
  • a P frame P frame, forward predictive coding frame
  • B frame B frame, bidirectionally predictively interpolated coded frame.
  • I frame as a "starting frame", its encoding does not need to refer to other images and only use information on the airspace, and can be used as a reference point for random access and a breakpoint for error diffusion; P frames will be sufficiently lower than the image sequence
  • the time-redundant information of the previously encoded frame is used to compress the encoded image of the transmitted data amount, also called the predicted frame; the B frame considers both the encoded frame in front of the source image sequence and the time redundancy between the encoded frames following the source image sequence.
  • the remaining information is used to compress the encoded image of the amount of transmitted data, also called a bi-predictive frame.
  • the cache module 11 buffers the latest I frame received from the network in the streaming media file to ensure better cached image quality and save space.
  • the data transmission module 12 is configured to send the streaming media data starting from the current frame to the second client 30 requesting the streaming media file.
  • the data transmission module 12 is further configured to send a multicast command to the second client 30 and send the streaming media data starting from the current frame to the second client in a unicast or multicast manner.
  • the data transmission module 12 sends the same multicast command to all clients requesting the same streaming media file and sends the streaming media data to the client in a unicast or the same multicast manner.
  • the second client 30 is configured to send, to the streaming media server 10, a second streaming media data frame corresponding to the current playback progress when the client 20 plays the streaming media file.
  • the second client 30 sends the streaming media data frame corresponding to the current playback progress when the streaming media file is played to the streaming media server 10 in a unicast manner.
  • the streaming media server 10 updates the saved first streaming media data frame to the second streaming media data frame when receiving the second streaming media data frame, so that other clients (including the first client 10) request the same stream.
  • the streaming media data is started from the second streaming media data frame.
  • the streaming media server 10 of the present invention caches the latest streaming media data frame in the streaming media file to the data between the current frame of the streaming media file played by the user at the first client 20, and the streaming media server 10 receives the user at the second client 30.
  • the streaming media server 10 sends the streaming media data starting from the current frame to the second client 30, and the second client 30 corresponds to the current playback progress when the user plays the streaming media file.
  • the second-stream media data frame is sent to the streaming media server 10 for storage, so that other clients (including the first client 10) can send streaming media data from the second streaming media data frame to other clients when requesting the same streaming media file.
  • the streaming media server 10 downloads and buffers the streaming media file requested by the first client 20 from the network, and the streaming media server 10 transmits the streaming media file to the first client 20, the first client.
  • the first streaming media data frame corresponding to the current progress when the user plays the streaming media file is sent to the streaming server 10 for storage, and the streaming media server 10 buffers the flow from the first streaming media data frame to the newly downloaded streaming media data frame.
  • the second client 30 transmits the streaming media data starting from the first streaming media data frame corresponding to the playback progress when the playback was last paused or stopped. Therefore, the present invention can realize the content of continuing to broadcast another terminal in one terminal when the two terminals are isolated from each other, and realize seamless and fast switching, so that the streaming media data is more easily transferred between different intelligent terminals. .
  • the streaming media data of the smart home multi-level intelligent device mainly uses the RTSP (Real Time Streaming Protocol) to control the media stream transmission to control the transmission of real-time data.
  • RTSP will serve as an extensible framework for real-time data, such as audio and video, controlled and continuous broadcast of multiple data transmission connections, for selecting transmission channels, such as UDP (User Datagram Protocol), multicast UDP and TCP. (Transmission Control Protocol). Interleaving the RTSP control information and the media data stream, controlling multiple data transmission connections, selecting a transmission channel, providing a path and providing a method for the RTP (Realtime Transport Protocol) transmission mechanism.
  • RTP Realtime Transport Protocol
  • the streaming media server 10 further includes a database management module 13 for creating and maintaining a media play information database, where the media play information database stores media information corresponding to time node information and current play progress. The association of media data frames.
  • the database management module 13 is further configured to update the association relationship between the media information in the media playing information database and the time node information and the streaming media data frame corresponding to the current playing progress.
  • the time node information and the streaming media data frame corresponding to the current playback progress are interleaved by the first client 20 and/or the second client 30 with the RTSP control information and the media data stream and transmitted to the streaming server through RTP/RTCP. 10.
  • the streaming media data when the streaming media data is transmitted, the media file is transmitted in time segmentation, and is loaded from the time node during the user's continuous broadcast.
  • the present invention saves the time node information and the specific streaming media data frame when the user plays by creating a media playing information library, so that the streaming media server 10 can locate the streaming media data frame corresponding to the user's last playback progress.
  • the streaming media server 10 Since the streaming media server 10 records the last playback data and the last playback time, when the user resumes the broadcast, the streaming server 10 calls the seekto, obtains the last play position through the getCurrentPosition method, and acquires the last play time node and the play data. And playback position.
  • the streaming media server 10 sends the same multicast command to all clients requesting the same streaming media data through the network layer; the streaming media server 10 simultaneously transmits the streaming media starting from the current frame to all clients in a unicast or the same multicast manner. data.
  • the other client will make a judgment when playing next time. If it is played for the first time, it will be played directly from the beginning. If it has already been played, it will continue to play from the last recorded time node, and it can be played from the recorded point. Streaming media playback between different devices is achieved without pulling the progress bar to a certain location.
  • the streaming server 10 further includes a user behavior information processing module 14 and a decoding/decompression module 15.
  • the streaming server 10 is further configured to receive user interface selection information sent by the first client 20 or the second client 30. Entering information; the user behavior information processing module 14 determines a pre-selected streaming media file according to the recorded behavior record information of the user, and the decoding/decompression module 15 pre-decodes and/or decompresses the data of the pre-selected streaming media file; the streaming media server 10 receives After the pre-selected streaming media file selection information sent by the first client 20 or the second client 30, the pre-decoded and/or decompressed streaming media file data is transmitted to the client 20 through the data transmission module 12. Thereby, the streaming media file can be decoded, decompressed and displayed earlier.
  • the client receives user input and identifies a program that the user may select, the marked program being a program selected based on user input, or a program based on the user input expected to be selected.
  • the program is selected (determined by the user behavior information processing module 13).
  • the user's input can be used to evoke a guide to the program, user navigation on the program guide, keyboard input to the remote control unit, and the like.
  • the streaming server 10 pre-decodes and/or decompresses the indicated program so that if the program is later selected by other clients, it can be decompressed and displayed earlier. During the early decoding and/or decompression of the streaming server 10, the client continues to receive, decompress and display the current program until the information needed to receive and decode the new program is obtained.
  • decoding is performed at the physical layer receiver processing or channel decoding, and decompression is performed at a higher layer receiver processing or resource decoding, and after obtaining the overhead information, the client decompresses its current program.
  • the streaming data keywords (play time node, play data, program start point, current play point, program end point) are packaged data streams.
  • the current and new programs are transmitted in a layered code, and for early decoding, the streaming server 10 receives user input and identifies a program that the user may select through the user behavior information processing module 13.
  • FIG. 2 is a flowchart of predicting streaming media data by a streaming media server according to an embodiment of the present invention, where the process includes:
  • step S201 the streaming media server receives the information input by the user on the client interface. For example, the user inputs "China on the tip of the tongue" on the client interface.
  • Step S202 The streaming media server lists the first-class media file list for the user to select according to the input information.
  • the streaming server 10 lists a plurality of streaming media files whose keywords are "China on the tip of the tongue" for the user to select.
  • Step S203 When receiving the information of the selected file by the user, the streaming media server determines that the streaming media file is a pre-selected streaming media file according to the user behavior information, and decodes and decompresses the pre-selected streaming media file in advance.
  • the streaming media server 10 considers that the streaming media file is to be selected by the user, and the streaming media file is decoded and decompressed in advance.
  • Step S204 When the streaming server receives the selected pre-selected streaming media file sent by the client, the streaming media server sends the decompressed and decompressed streaming media data to the client.
  • the streaming media server when the user selects the streaming media file highlighted by the (motor), the streaming media server sends the pre-decoded and decompressed streaming media data to the client. Thereby, the streaming media file can be decoded, decompressed and displayed earlier.
  • FIG. 3 is a flowchart of predicting streaming media data by a streaming media server according to another embodiment of the present invention, where the process includes:
  • Step S301 the streaming media server receives the program ticket request sent by the client. For example, the user selects a program list on the client interface, or the user uses the remote control motor program button.
  • step S302 the streaming media server sends a program list containing the program list to the client for selection.
  • the streaming server 10 eg, a smart router
  • Step S303 the streaming media server determines, according to the user behavior record information, that the user is to select a program, and performs decoding and decompression in advance on the channel or program to be selected.
  • the streaming media server 10 determines, according to the user's habit information, that the user will select "Hunan Satellite TV", and then decodes and decompresses the streaming media file corresponding to Hunan Satellite TV.
  • Step S304 The streaming media server sends the pre-decoded and decompressed streaming media data to the client when receiving the program that the client selects to decode and decompress in advance.
  • the streaming server 10 will decode and decompress the data in advance to the Smart box or smart TV. Thereby, the streaming media file can be decoded, decompressed and displayed earlier.
  • the streaming media server 10 is further configured to receive the difference data of the audio data processing delay and the video data processing delay sent by the first client 20 or the second client 30, and advance the video data at the same time.
  • the stamped audio data is sent to the first client 20 or the second client 30, and the time of early transmission of the video data is determined based on the difference data.
  • the audio and video data packets of the streaming media files are separately transmitted in the network transmission, and the streaming media files such as compressed audio and video are decoded and played and viewed by the client using the player and related hardware and software. Audio and video decoding and transmission speed are inconsistent (general audio is faster than video), resulting in unsynchronized audio and video after decoding.
  • the frames of the audio and video are compared.
  • the timestamp determines the time when the audio is sent to the speaker and the video is sent to the LCD. This can minimize the time deviation caused by the device during the transmission and decoding process, but the streaming media data transmitted to the client cannot be played earlier.
  • the data will be buffered on the client side.
  • the present invention transmits the audio data of the video data ahead of the same time stamp to the client by receiving the difference data of the audio data processing delay and the video data processing delay sent by the client, so that the video data is decoded at a speed and Although the transmission speed is slower than the audio data, because it is sent in advance, the audio and video synchronization or small buffering after decoding by the client is realized, and the proper time alignment of the video and the audio is obtained.
  • This allows the smart router or smart TV and smart box device to be displayed earlier in the program change, providing a faster response to program changes.
  • the application scenario of the present invention is as follows:
  • the present invention also provides a method for implementing seamless connection of streaming media data in a smart home, which is implemented by the system shown in FIG. 1, the method includes:
  • Step S401 The streaming media server caches the streaming media data between the latest streaming media data frame and the current frame in the streaming media file, and the latest streaming media data frame is the latest received streaming media data frame from the network, and the current frame is the first client.
  • the first streaming media data frame corresponding to the current playback progress when the streaming media file is played. This step is implemented by the cache module 11.
  • Step S402 the streaming media server sends the streaming media data starting from the current frame to the second client requesting the streaming media file.
  • This step is implemented by the data transmission module 12.
  • the step further includes: the streaming media server sending a multicast command to the second client, and sending the streaming media data starting from the current frame to the second client in a unicast or multicast manner.
  • Step S403 The second client sends, to the streaming media server, a second streaming media data frame corresponding to the current playback progress when the streaming media file is played.
  • This step is implemented by the second client 30.
  • the step further includes: sending, by the second client, the second streaming media data frame corresponding to the current playing progress when the streaming media file is played to the streaming media server in a unicast manner.
  • the streaming media server updates the saved first streaming media data frame to the second streaming media data frame when receiving the second streaming media data frame, so that other clients (including the first client) request the same streaming media file.
  • the streaming data is started from the second streaming media data frame to other clients.
  • the method further includes: the streaming media server creates and maintains a media playing information database, wherein the media playing information database stores the association relationship between the media information and the time node information, and the streaming media data frame corresponding to the current playing progress;
  • the step of the second streaming media data frame corresponding to the current playback progress when the client sends the streaming media file to the streaming server in a unicast manner further includes: the streaming media server updates the media information and the time node information in the media playing information database, The association relationship of the streaming media data frames corresponding to the current playback progress.
  • the method further includes: the streaming media server receives user interface selection information and input information sent by the first client or the second client; and the streaming media server confirms the pre-selected streaming media file according to the recorded behavior record information of the user. And pre-decoding and/or decompressing the pre-selected streaming media file; the streaming media server receives the pre-selected streaming media file selection information sent by the first client or the second client Thereafter, the pre-decoded and/or decompressed streaming media file data is sent to the first client or the second client.
  • the method further includes: the streaming media server receiving the difference data of the audio data processing delay and the video data processing delay sent by the first client or the second client; and the streaming media server to the video data
  • the audio data ahead of the same time stamp is sent to the first client or the second client, and the time of early transmission of the video data is determined according to the difference data.
  • the present invention solves the problem that a seamless connection between streaming media data services between various smart terminals has not been realized in a conventional smart home.
  • the smart router also has information that the user previously (according to the user's usage habits) and can use this information to predict the user's future choices.
  • the intelligent routing can perform early decoding of the related program before the user selects the program. It solves the function of realizing streaming media continuous broadcasting on different devices in the smart home TV and box device; this can bring a wonderful experience to the user, thereby satisfying the user's operating habits and real-time needs, and improving the user experience. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Transfer Between Computers (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明适用于通信技术领域，提供了种一种智能家居中流媒体数据无缝连接实现方法及系统，该方法包括：流媒体服务器缓存流媒体文件中当前帧到最新流媒体数据帧之间的流媒体数据，最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，当前帧为第一客户端播放流媒体文件时当前播放进度对应的第一流媒体数据帧；流媒体服务器向请求流媒体文件的第二客户端发送从当前帧开始的流媒体数据；第二客户端向流媒体服务器发送播放流媒体文件时当前播放进度对应的第二流媒体数据帧。本发明能够实现在相互隔离的两个终端播放流媒体数据时，实现在一个终端续播另一个终端的内容，实现无缝快速切换，让流媒体数据更容易在不同智能终端之间转移。

Description

一种智能家居中流媒体数据无缝连接实现方法及系统 技术领域
本发明涉及通信技术领域，尤其涉及一种智能家居中流媒体数据无缝连接实现方法及系统。
背景技术
移动流媒体技术是一种将连续的视频/音频信息经过压缩处理后放到流媒体服务器上，可让用户通过移动终端一边下载一边收看，而不需要等整个压缩文件下载到自己机器后才可以观看。该技术先在计算机上创造一个缓冲区，于播放前预先下载一段资料作为缓冲，于网路实际连线速度小于播放所耗用资料的速度时，播放程序就会取用这一小段缓冲区内的资料，避免播放的中断，也使得播放品质得以维持，它涉及到流媒体数据的采集、压缩、存储、传输和网络通信等多项技术。
目前，在智能家居中，相互隔离的两个智能终端播放流媒体数据时还不能实现续播和无缝快速切换，不能让流媒体数据在不同智能终端之间快速的转移。流媒体业务对实时性要求较高，当从一个移动终端切换到另一个移动终端并且继续流媒体业务时，由于切换延迟较大，常常造成流媒体播放卡壳、延迟、视频质量抖动剧烈等不良情况，给用户的收看体验带来不良影响。例如，当用户在客厅电视上观看一节目，当用户到卧室后需要在手机上观看之前客厅电视上观看的节目，现有技术中用户只能重新在手机上加载该节目，无法进行快速续播和无缝快速切换。因此对于跨越不同系统的平台，解决好用户接收视频服务的问题对于流媒体无缝切换极为重要。
综上可知，现有技术在实际使用上显然存在不便与缺陷，所以有必要加以改进。
发明内容
针对上述的缺陷，本发明的目的在于提供一种智能家居中流媒体数据无缝连接实现方法及系统，其能在相互隔离的两个终端播放流媒体数据时，实现在 一个终端续播另一个终端的内容，实现无缝快速切换，让流媒体数据更容易在不同智能终端之间转移。
为了实现上述目的，本发明提供一种智能家居中流媒体数据无缝连接实现方法，包括：
流媒体服务器缓存流媒体文件中当前帧到最新流媒体数据帧之间的流媒体数据，所述最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，所述当前帧为第一客户端播放所述流媒体文件时当前播放进度对应的第一流媒体数据帧；
所述流媒体服务器向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据；
所述第二客户端向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
根据本发明的方法，所述流媒体服务器向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据的步骤包括：
所述流媒体服务器向所述第二客户端发送组播命令以及以单播或组播的方式向所述第二客户端发送从所述当前帧开始的流媒体数据；
所述第二客户端向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧的步骤包括：
所述第二客户端以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
根据本发明的方法，所述方法还包括：
所述流媒体服务器创建并维护一媒体播放信息数据库，所述媒体播放信息数据库存储有媒体信息与时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系；
所述第二客户端以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧的步骤之后还包括：
所述流媒体服务器更新所述媒体播放信息库中的所述媒体信息与所述时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系。
根据本发明的方法，所述方法还包括：
所述流媒体服务器接收所述第一客户端或所述第二客户端发来的用户界面 选择信息和输入信息；
所述流媒体服务器根据记录的用户的行为记录信息确认预选流媒体文件，并对该预选流媒体文件进行预先解码和/或解压；
所述流媒体服务器接收到所述第一客户端或所述第二客户端发来的预选流媒体文件选定信息后，将所述预先解码和/或解压的流媒体文件数据发送至所述第一客户端或所述第二客户端。
根据本发明的方法，所述方法还包括：
所述流媒体服务器接收所述第一客户端或所述第二客户端发来的音频数据处理时延和视频数据处理时延的差值数据；
所述流媒体服务器对所述视频数据提前于同一时间戳的所述音频数据发送至所述第一客户端或所述第二客户端，所述视频数据的提前发送的时间根据所述差值数据进行确定。
本发明还提供一种智能家居中流媒体数据无缝连接实现系统，包括流媒体服务器、第一客户端以及第二客户端；
所述流媒体服务器包括：
缓存模块，用于缓存流媒体文件中当前帧到最新流媒体数据帧之间的流媒体数据，所述最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，所述当前帧为第一客户端播放流媒体文件时当前进度对应的第一流媒体数据帧；
数据传输模块，用于向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据；
所述第二客户端用于向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
根据本发明的系统，所述数据传输模块还用于向所述第二客户端发送组播命令以及以单播或组播的方式向所述第二客户端发送从所述当前帧开始的流媒体数据；
所述第二客户端还用于以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
根据本发明的系统，所述流媒体服务器还包括一数据库管理模块，用于创建并维护一媒体播放信息数据库，所述媒体播放信息数据库存储有媒体信息与时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系；
所述数据库管理模块还用于更新所述媒体播放信息库中的所述媒体信息与所述时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系。
根据本发明的系统，所述流媒体服务器还包括用户行为信息处理模块及解码/解压模块，
所述流媒体服务器还用于接收所述第一客户端或所述第二客户端发来的用户界面选择信息和输入信息；
所述用户行为信息处理模块根据记录的用户的行为记录信息确定预选流媒体文件，解码/解压模块对所述预选流媒体文件的数据进行预先解码和/或解压；
所述流媒体服务器接收到所述第一客户端或所述第二客户端发来的预选流媒体文件选定信息后，数据传输模块将所述预先解码和/或解压的流媒体文件数据发送至所述第一客户端或所述第二客户端。
根据本发明的系统，所述流媒体服务器还用于接收所述第一客户端或所述第二客户端发来的音频数据处理时延和视频数据处理时延的差值数据，且对所述视频数据提前于同一时间戳的所述音频数据发送至所述第一客户端或所述第二客户端，所述视频数据的提前发送的时间根据所述差值数据进行确定。
本发明流媒体服务器缓存流媒体文件中当前帧到最新流媒体数据帧之间的数据，当流媒体服务器接收到用户在第二客户端对于同一流媒体文件的下载请求时，流媒体服务器将从当前帧开始的流媒体数据发送至第二客户端，而第二客户端将用户播放流媒体文件时当前播放进度对应的第二流媒体数据帧发送至流媒体服务器保存。具体的，流媒体服务器从网络下载并缓冲第一客户端请求的流媒体文件，并且流媒体服务器将该流媒体文件传送至第一客户端，第一客户端将用户播放流媒体文件时当前进度对应的第一流媒体数据帧发送至流媒体服务器保存，并且该流媒体服务器缓存从第一流媒体数据帧到最新下载的流媒体数据帧之间的流媒体数据；当第一客户端暂停或停止播放时，流媒体服务器保存有第一客户端暂停或停止播放时的播放进度对应的第一流媒体数据帧，并缓存了从该暂停或停止播放时的播放进度对应的第一流媒体数据帧到最新下载的流媒体数据帧之间的流媒体数据；当用户从第二客户端再次播放相同的流媒体文件时，流媒体服务器向该第二客户端传输从上次暂停或停止播放时的播放进度对应的第一流媒体数据帧开始的流媒体数据。从而，本发明能够实现在相互隔离的两个终端播放流媒体数据时，实现在一个终端续播另一个终端的内容， 实现无缝快速切换，让流媒体数据更容易在不同智能终端之间转移。
附图说明
图1是本发明一种智能家居中流媒体数据无缝连接实现系统的框图；
图2是本发明一种实施例中流媒体服务器对流媒体数据预测的流程图；
图3是本发明另一实施例中流媒体服务器对流媒体数据预测的流程图；
图4是本发明一种智能家居中流媒体数据无缝连接实现方法的流程图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
如图1所示，本发明提供一种智能家居中流媒体数据无缝连接实现系统100，包括流媒体服务器10以及第一客户端20、第二客户端30。该流媒体服务器10可以是智能路由器或智能盒子等智能设备，该第一客户端20和第二客户端30可以是智能电视、智能盒子、智能手机或平板电脑等智能设备。该系统100是内置于智能设备的软件单元，硬件单元或软硬件结合单元。流媒体服务器10包括缓存模块11及数据传输模块12。
缓存模块11，用于缓存流媒体文件中最新流媒体数据帧到当前帧之间的流媒体数据，最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，所述最新流媒体数据帧是位于当前帧到所述流媒体文件的最后一帧(终点帧)之间的某一帧，可以根据用户的选择确定所述最新流媒体数据帧，当前帧为第一客户端20播放流媒体文件时当前进度对应的第一流媒体数据帧。流媒体文件通常保存在内容分享网络的网络节点中，流媒体服务器10在接收到点播请求时通过网络从网络节点下载流媒体文件。在流媒体文件下载完成之前，流媒体服务器10持续从网络接收流媒体数据，该最新流媒体数据帧随着数据的接收不断更新。
优选的，该缓存模块11缓存流媒体文件中从网络最新接收的I帧到当前帧之间的流媒体数据。具体的，现有视频编码标准所定义的码流结构是一种分层的结构，包括图像组、图像帧、宏块(MacroBlock，MB)等层。通常在一个图像组中包括I帧(I frame，帧内编码帧)、P帧(P frame，前向预测编码帧)、 B帧(B frame，双向预测内插编码帧)。I帧作为“起始帧”，它的编码不需要参考其他图像而仅利用空域上的信息，并可以作为随机访问的参考点以及错误扩散的隔断点；P帧通过充分将低于图像序列中前面已编码帧的时间冗余信息来压缩传输数据量的编码图像，也叫预测帧；B帧既考虑与源图像序列前面已编码帧，也顾及源图像序列后面已编码帧之间的时间冗余信息来压缩传输数据量的编码图像，也叫双向预测帧。由于B帧的图像质量取决于其前一帧和后一帧，因此在流媒体的传输过程中，如果一个GOP图像组内所有帧未传输完整，那么该图像组内已传输完成的帧图像质量将会较差。在本发明中，缓存模块11缓存流媒体文件中从网络接收到的最新I帧能够保证缓存的图像质量较好，并且节省空间。
数据传输模块12，用于向请求流媒体文件的第二客户端30发送从当前帧开始的流媒体数据。优选地，数据传输模块12还用于向第二客户端30发送组播命令以及以单播或组播的方式向第二客户端发送从当前帧开始的流媒体数据。具体的，数据传输模块12向所有请求同一流媒体文件的客户端发送同一组播命令以及以单播或同一组播的方式向客户端发送流媒体数据。
第二客户端30用于向流媒体服务器10发送客户端20播放流媒体文件时当前播放进度对应的第二流媒体数据帧。优选地，第二客户端30以单播方式向流媒体服务器10发送播放流媒体文件时当前播放进度对应的流媒体数据帧。并且流媒体服务器10在接收到该第二流媒体数据帧时，将其保存的第一流媒体数据帧更新为第二流媒体数据帧，便于其他客户端(包括第一客户端10)请求同一流媒体文件时向其他客户端发送从该第二流媒体数据帧开始流媒体数据。
本发明的流媒体服务器10缓存流媒体文件中最新流媒体数据帧到用户在第一客户端20播放流媒体文件当前帧之间的数据，当流媒体服务器10接收到用户在第二客户端30对于同一流媒体文件的下载请求时，流媒体服务器10将从当前帧开始的流媒体数据发送至第二客户端30，而第二客户端30将用户播放流媒体文件时当前播放进度对应的第二流媒体数据帧发送至流媒体服务器10保存，便于其他客户端(包括第一客户端10)请求同一流媒体文件时向其他客户端发送从该第二流媒体数据帧开始流媒体数据。
具体的，流媒体服务器10从网络下载并缓冲第一客户端20请求的流媒体文件，并且流媒体服务器10将该流媒体文件传送至第一客户端20，第一客户端 20将用户播放流媒体文件时当前进度对应的第一流媒体数据帧发送至流媒体服务器10保存，并且该流媒体服务器10缓存从第一流媒体数据帧到最新下载的流媒体数据帧之间的流媒体数据；当第一客户端20暂停或停止播放时，流媒体服务器10保存有第一客户端20暂停或停止播放时的播放进度对应的第一流媒体数据帧，并缓存了从该暂停或停止播放时的播放进度对应的第一流媒体数据帧到最新下载的流媒体数据帧之间的流媒体数据；当用户从第二客户端30再次请求播放相同的流媒体文件时，流媒体服务器10向该第二客户端30传输从上次暂停或停止播放时的播放进度对应的第一流媒体数据帧开始的流媒体数据。从而，本发明能够实现在相互隔离的两个终端播放流媒体数据时，实现在一个终端续播另一个终端的内容，实现无缝快速切换，让流媒体数据更容易在不同智能终端之间转移。
智能家居多级智能设备的流媒体数据，主要采用RTSP(Real Time Streaming Protocol，实时流传输协议)建立的控制媒体流传输控制实时数据的发送。RTSP将作为一可扩展框架，使实时数据，如音视频的受控、续播多个数据发送连接，为选择发送通道，如UDP(User Datagram Protocol，用户数据报协议)、组播UDP与TCP(Transmission Control Protocol，传输控制协议)。把RTSP控制信息和媒体数据流交织在一起传送，将控制多个数据发送连接，选择发送通道，提供途径并为RTP(Realtime Transport Protocol，实时传输协议)上发送机制提供方法。
优选的是，流媒体服务器10还包括一数据库管理模块13，数据库管理模块13用于创建并维护一媒体播放信息数据库，媒体播放信息数据库存储有媒体信息与时间节点信息、当前播放进度对应的流媒体数据帧的关联关系。数据库管理模块13还用于更新媒体播放信息库中的媒体信息与时间节点信息、当前播放进度对应的流媒体数据帧的关联关系。该时间节点信息以及当前播放进度对应的流媒体数据帧由第一客户端20和/或第二客户端30将其与RTSP控制信息和媒体数据流交织在一起通过RTP/RTCP传送至流媒体服务器10。在现有技术中，流媒体数据进行传输时是按照时间分段传输媒体文件，在用户续播的过程中是从时间节点开始加载。本发明通过创建媒体播放信息库，保存用户播放时的时间节点信息以及具体的流媒体数据帧，这样流媒体服务器10就能定位到用户上次播放进度对应的流媒体数据帧。
由于流媒体服务器10记录保存上次的播放数据、上次播放的时间，在用户续播时，流媒体服务器10调用seekto，通过getCurrentPosition方法获取上次播放位置，获取上一次播放时间节点、播放数据和播放位置。流媒体服务器10向所有请求同一流媒体数据的客户端通过网络层发送同一组播命令；流媒体服务器10同时以单播或同一组播的方式向所有的客户端发送从当前帧开始的流媒体数据。其他客户端下次播放时做一个判断，若第一次播放的话就直接从头播放，若已经播放了就从上次记录的时间节点去继续播放，即可从记录的点开始播放了，也就实现了不同设备之间实现流媒体续播，无需重新拉进度条到某个位置。
优选的是，流媒体服务器10还包括用户行为信息处理模块14及解码/解压模块15，流媒体服务器10还用于接收第一客户端20或第二客户端30发来的用户界面选择信息和输入信息；用户行为信息处理模块14根据记录的用户的行为记录信息确定预选流媒体文件，解码/解压模块15对该预选流媒体文件的数据进行预先解码和/或解压；流媒体服务器10接收到第一客户端20或第二客户端30发来的预选流媒体文件选定信息后，通过数据传输模块12将预先解码和/或解压的流媒体文件数据发送至客户端20。从而能使该流媒体文件能较早的解码、解压和显示。
在一个实施例中，客户端接收用户输入，并且标示用户可能选择的一个节目，该标示的节目可是一个根据用户输入选择的节目，或是一个基于该用户输入的预料将要被选择的节目该标示的节目被选择时(通过用户行为信息处理模块13进行确定)。用户的输入可用于唤起节目的指南、在节目指南上的用户导航、远程控制单元的键盘输入等等。流媒体服务器10对标示的节目进行提前解码和/或解压，使得若该节目后来被其它客户端选定后，它能被较早地解压和显示。在流媒体服务器10提前解码和/或解压的过程中，客户端继续接收、解压缩并且显示当前的节目直到获得接收和解码该新的节目所需信息。
在流媒体处理的情况中，解码在物理层接收机处理或信道解码，而解压缩的是在高层接收机处理或资源解码，获得开销信息以后，客户端对其当前的节目进行解压缩。在压缩的视频流之前，对流媒体的关键字(播放时间节点、播放数据、节目起始点、当前播放点、节目结束点)进行打包数据流。在本发明中，当前的和新的节目是用分层编码发送，对于提前解码，流媒体服务器10接收用户输入并且通过用户行为信息处理模块13识别用户可能选择的一节目。
图2是本发明一种实施例中流媒体服务器对流媒体数据预测的流程图，该流程包括：
步骤S201，流媒体服务器接收到用户在客户端界面输入的信息。例如用户在客户端界面上输入“舌尖上的中国”。
步骤S202，流媒体服务器根据该输入信息列出一流媒体文件列表供用户选择。续上例，流媒体服务器10列出多个关键字为“舌尖上的中国”的流媒体文件供用户选择。
步骤S203，流媒体服务器接收到用户选择某一文件的信息时，根据用户行为信息确定该流媒体文件为预选流媒体文件，并对预选流媒体文件提前解码、解压。续上例，用户将光标移动到某一流媒体文件时，该流媒体文件高亮显示，流媒体服务器10认为用户将要选择该流媒体文件，对该流媒体文件提前解码、解压。
步骤S204，流媒体服务器接收到客户端发来的用户选定预选流媒体文件时，将提前解码、解压的流媒体数据发送至客户端。续上例，用户选定(电机)高亮显示的流媒体文件时，流媒体服务器将该提前解码、解压的流媒体数据发送至客户端。从而能使该流媒体文件能较早的解码、解压和显示。
图3是本发明另一实施例中流媒体服务器对流媒体数据预测的流程图，该流程包括：
步骤S301，流媒体服务器接收客户端发来的节目单请求。例如：用户在客户端界面上选择节目单，或者用户采用遥控器电机节目单按钮。
步骤S302，流媒体服务器发送包含了节目列表的节目单至客户端供选择。续上例，流媒体服务器10(例如智能路由器)将频道列表发送至智能盒子或智能电视。
步骤S303，流媒体服务器根据用户行为记录信息判断用户将要选择节目，并对将要选择的频道或节目进行提前解码、解压。续上例，流媒体服务器10根据用户习惯信息判断用户将要选择“湖南卫视”，则对湖南卫视对应的流媒体文件进行解码、解压。
步骤S304，流媒体服务器接收到客户端选定提前解码、解压的节目时，将该提前解码、解压的流媒体数据发送至客户端。续上例，当用户在智能盒子或智能电视上选定“湖南卫视”时，流媒体服务器10将提前解码、解压的发送至 智能盒子或智能电视。从而能使该流媒体文件能较早的解码、解压和显示。
优选的是，流媒体服务器10还用于接收第一客户端20或第二客户端30发来的音频数据处理时延和视频数据处理时延的差值数据，且对视频数据提前于同一时间戳的音频数据发送至第一客户端20或第二客户端30，该视频数据的提前发送的时间根据差值数据进行确定。流媒体文件的音视频数据包在网络传输中是分开传输的，而在客户端利用播放器以及相关的硬件、软件对压缩的音视频等流媒体文件解码后进行播放和观看。音视频解码和传输速度是不一致(一般音频快于视频)的，导致解码后的音频和视频不同步，在现有技术中，在音视频数据完成相应的解码后，通过比较音视频各帧的时间戳来决定音频送扬声器和视频送LCD显示的时间，这样可以将音、视频分别在传输解码过程中由于设备产生的时间偏差减到最小，但传输至客户端的流媒体数据不能较早的播放，数据会在客户端缓冲。因此，本发明通过接收客户端发来的音频数据处理时延和视频数据处理时延的差值数据，对视频数据提前于同一时间戳的音频数据发送至客户端，这样视频数据虽然解码速度和传输速度虽然慢于音频数据，但由于其提前发送，从而实现客户端解码后的音视频同步或者很小的缓冲，获得视频和音频的合适的时间对齐。这使得该智能路由器或智能电视以及智能盒子设备能在节目变更中较早地显示，从而提供了对节目变更的更快的响应。
本发明的应用场景如下：
1、用户将智能电视、智能路由器、电视盒子、手机或iPad等智能设备连接在一起，用户在户外通过手机收看电影，然后在家里的iPad上用户可继续收看(续播)之前某个时段的电影节目，无需用户手动重新拉进度条到某个时段的电影节目，即可在电影视频节目在不同智能终端之间实现了续播(流媒体数据无缝切换)。
2、用户将智能电视、智能路由器、电视盒子、手机或iPad等智能设备连接在一起，在客厅内手机上收看电影或节目，然后在房间内上用户可打开电视或电视盒子继续收看(续播)之前某个时段的电影或电视节目，无需用户手动重新拉进度条到某个时段的电影节目，即可在电影视频节目在不同智能终端之间实现了续播(流媒体数据无缝切换)。
3、用户将智能电视、智能路由器、电视盒子、手机或iPad等智能设备连接在一起，用户在室内通过智能电视收看电影或节目，该智能路由器或智能电视 盒子会根据用户的收看节目习惯或收看内容的频率，来预测用户当前需要看哪些节目，并且智能路由器、电视盒子设备会根据这些习惯和频率来提前对节目下载或加载，这样方便用户在需要收看时候，可直接收看，减小缓冲。
如图4所示，本发明还提供一种智能家居中流媒体数据无缝连接实现方法，其通过如图1所示的系统实现，该方法包括：
步骤S401，流媒体服务器缓存流媒体文件中最新流媒体数据帧到当前帧之间的流媒体数据，最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，当前帧为第一客户端播放流媒体文件时当前播放进度对应的第一流媒体数据帧。本步骤通过缓存模块11实现。
步骤S402，流媒体服务器向请求流媒体文件的第二客户端发送从当前帧开始的流媒体数据。本步骤通过数据传输模块12实现。优选的，本步骤还包括：所述流媒体服务器向第二客户端发送组播命令以及以单播或组播的方式向第二客户端发送从当前帧开始的流媒体数据。
步骤S403，第二客户端向流媒体服务器发送播放流媒体文件时当前播放进度对应的第二流媒体数据帧。本步骤通过第二客户端30实现。优选的，本步骤还包括：第二客户端以单播方式向流媒体服务器发送播放流媒体文件时当前播放进度对应的第二流媒体数据帧。并且流媒体服务器在接收到该第二流媒体数据帧时，将其保存的第一流媒体数据帧更新为第二流媒体数据帧，便于其他客户端(包括第一客户端)请求同一流媒体文件时向其他客户端发送从该第二流媒体数据帧开始流媒体数据。
优选的是，所述方法还包括：流媒体服务器创建并维护一媒体播放信息数据库，媒体播放信息数据库存储有媒体信息与时间节点信息、当前播放进度对应的流媒体数据帧的关联关系；第二客户端以单播方式向流媒体服务器发送播放流媒体文件时当前播放进度对应的第二流媒体数据帧的步骤之后还包括：流媒体服务器更新媒体播放信息库中的媒体信息与时间节点信息、当前播放进度对应的流媒体数据帧的关联关系。
优选的是，所述方法还包括：流媒体服务器接收第一客户端或第二客户端发来的用户界面选择信息和输入信息；流媒体服务器根据记录的用户的行为记录信息确认预选流媒体文件，并对预选流媒体文件进行预先解码和/或解压；流媒体服务器接收到第一客户端或第二客户端发来的预选流媒体文件选定信息 后，将预先解码和/或解压的流媒体文件数据发送至第一客户端或第二客户端。
优选的是，所述方法还包括：流媒体服务器接收第一客户端或第二客户端发来的音频数据处理时延和视频数据处理时延的差值数据；流媒体服务器对所述视频数据提前于同一时间戳的音频数据发送至第一客户端或第二客户端，视频数据的提前发送的时间根据差值数据进行确定。
综上所述，本发明解决了在传统智能家居中还没有实现通过各个智能终端之间进行流媒体数据业务无缝连接的方法。在智能路由器同样具有用户先前(根据用户使用习惯)动作的信息，并且可利用这些信息来预测用户将来的选择。为了获得节目之间更快的切换，在用户选择节目之前，该智能路由(流媒体服务器)可对该相关节目执行提前解码。解决了在智能家居的电视和盒子设备这样实现在不同设备上可实现流媒体续播的功能；这样可给用户带来的美好体验，从而满足用户的操作习惯和实时需求，提高用户的体验感受。
当然，本发明还可有其它多种实施例，在不背离本发明精神及其实质的情况下，熟悉本领域的技术人员当可根据本发明作出各种相应的改变和变形，但这些相应的改变和变形都应属于本发明所附的权利要求的保护范围。

Claims (10)

  1. 一种智能家居中流媒体数据无缝连接实现方法，其特征在于，包括：
    流媒体服务器缓存流媒体文件中当前帧到最新流媒体数据帧之间的流媒体数据，所述最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，所述当前帧为第一客户端播放所述流媒体文件时当前播放进度对应的第一流媒体数据帧；
    所述流媒体服务器向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据；
    所述第二客户端向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
  2. 根据权利要求1所述的方法，其特征在于，所述流媒体服务器向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据的步骤包括：
    所述流媒体服务器向所述第二客户端发送组播命令以及以单播或组播的方式向所述第二客户端发送从所述当前帧开始的流媒体数据；
    所述第二客户端向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧的步骤包括：
    所述第二客户端以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
  3. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述流媒体服务器创建并维护一媒体播放信息数据库，所述媒体播放信息数据库存储有媒体信息与时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系；
    所述第二客户端以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧的步骤之后还包括：
    所述流媒体服务器更新所述媒体播放信息库中的所述媒体信息与所述时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述流媒体服务器接收所述第一客户端或所述第二客户端发来的用户界面选择信息和输入信息；
    所述流媒体服务器根据记录的用户的行为记录信息确认预选流媒体文件， 并对所述预选流媒体文件进行预先解码和/或解压；
    所述流媒体服务器接收到所述第一客户端或所述第二客户端发来的预选流媒体文件选定信息后，将所述预先解码和/或解压的流媒体文件数据发送至所述第一客户端或所述第二客户端。
  5. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述流媒体服务器接收所述第一客户端或所述第二客户端发来的音频数据处理时延和视频数据处理时延的差值数据；
    所述流媒体服务器对所述视频数据提前于同一时间戳的所述音频数据发送至所述第一客户端或所述第二客户端，所述视频数据的提前发送的时间根据所述差值数据进行确定。
  6. 一种智能家居中流媒体数据无缝连接实现系统，其特征在于，包括流媒体服务器、第一客户端以及第二客户端；
    所述流媒体服务器包括：
    缓存模块，用于缓存流媒体文件中当前帧到最新流媒体数据帧之间的流媒体数据，所述最新流媒体数据帧为从网络中最新接收到的流媒体数据帧，所述当前帧为第一客户端播放流媒体文件时当前进度对应的第一流媒体数据帧；
    数据传输模块，用于向请求所述流媒体文件的第二客户端发送从所述当前帧开始的流媒体数据；
    所述第二客户端用于向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
  7. 根据权利要求6所述的系统，其特征在于，所述数据传输模块还用于向所述第二客户端发送组播命令以及以单播或组播的方式向所述第二客户端发送从所述当前帧开始的流媒体数据；
    所述第二客户端还用于以单播方式向所述流媒体服务器发送播放所述流媒体文件时当前播放进度对应的第二流媒体数据帧。
  8. 根据权利要求6所述的系统，其特征在于，所述流媒体服务器还包括一数据库管理模块，用于创建并维护一媒体播放信息数据库，所述媒体播放信息数据库存储有媒体信息与时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系；
    所述数据库管理模块还用于更新所述媒体播放信息库中的所述媒体信息与 所述时间节点信息、所述当前播放进度对应的流媒体数据帧的关联关系。
  9. 根据权利要求6所述的系统，其特征在于，所述流媒体服务器还包括用户行为信息处理模块及解码/解压模块，
    所述流媒体服务器还用于接收所述第一客户端或所述第二客户端发来的用户界面选择信息和输入信息；
    所述用户行为信息处理模块根据记录的用户的行为记录信息确定预选流媒体文件，解码/解压模块对所述预选流媒体文件的数据进行预先解码和/或解压；
    所述流媒体服务器接收到所述第一客户端或所述第二客户端发来的预选流媒体文件选定信息后，数据传输模块将所述预先解码和/或解压的流媒体文件数据发送至所述第一客户端或所述第二客户端。
  10. 根据权利要求6所述的系统，其特征在于，所述流媒体服务器还用于接收所述第一客户端或所述第二客户端发来的音频数据处理时延和视频数据处理时延的差值数据，且对所述视频数据提前于同一时间戳的所述音频数据发送至所述第一客户端或所述第二客户端，所述视频数据的提前发送的时间根据所述差值数据进行确定。
PCT/CN2014/092604 2014-09-29 2014-11-29 一种智能家居中流媒体数据无缝连接实现方法及系统 WO2016049976A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/513,625 US20170302988A1 (en) 2014-09-29 2014-11-29 Method And System For Realizing Streaming Media Data Seamlessly Connecting In Intelligent Home
EP14903087.6A EP3203754A1 (en) 2014-09-29 2014-11-29 Method and system for realizing streaming media data seamlessly connecting in intelligent home

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410513217.2A CN105516791A (zh) 2014-09-29 2014-09-29 一种智能家居中流媒体数据无缝连接实现方法及系统
CN201410513217.2 2014-09-29

Publications (1)

Publication Number Publication Date
WO2016049976A1 true WO2016049976A1 (zh) 2016-04-07

Family

ID=55629377

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/092604 WO2016049976A1 (zh) 2014-09-29 2014-11-29 一种智能家居中流媒体数据无缝连接实现方法及系统

Country Status (4)

Country Link
US (1) US20170302988A1 (zh)
EP (1) EP3203754A1 (zh)
CN (1) CN105516791A (zh)
WO (1) WO2016049976A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3469790A4 (en) * 2016-09-07 2019-04-17 Samsung Electronics Co., Ltd. DISPLAY APPARATUS AND CORRESPONDING CONTROL METHOD
CN110769280A (zh) * 2019-10-23 2020-02-07 北京地平线机器人技术研发有限公司 一种接续播放文件的方法及装置
CN112286110A (zh) * 2020-11-04 2021-01-29 上海美家美沪装饰科技有限公司 一种智慧型家居自适应交互控制系统

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10127908B1 (en) 2016-11-11 2018-11-13 Amazon Technologies, Inc. Connected accessory for a voice-controlled device
US10958966B2 (en) * 2017-03-31 2021-03-23 Gracenote, Inc. Synchronizing streaming media content across devices
US10366692B1 (en) 2017-05-15 2019-07-30 Amazon Technologies, Inc. Accessory for a voice-controlled device
CN108471547B (zh) * 2018-03-27 2021-01-19 上海麦克风文化传媒有限公司 媒体播放切换方法和服务器装置
CN109168078B (zh) * 2018-09-10 2022-07-12 苏宁智能终端有限公司 一种视频清晰度切换方法及装置
CN115665100A (zh) 2018-10-09 2023-01-31 华为技术有限公司 语音切换方法、电子设备及系统
CN110071919A (zh) * 2019-04-22 2019-07-30 南京邮电大学 一种基于嵌入式流媒体技术的多媒体传输方法
CN112019858B (zh) * 2019-05-31 2022-11-29 腾讯科技（深圳）有限公司 一种视频播放方法、装置、计算机设备和存储介质
CN113132782A (zh) * 2020-01-16 2021-07-16 百度在线网络技术（北京）有限公司 一种流媒体传输的方法、流媒体播放设备及电子设备
CN114173193A (zh) * 2020-09-10 2022-03-11 华为技术有限公司 一种多媒体流的播放方法和电子设备
CN112738551A (zh) * 2021-02-02 2021-04-30 成都云帆数联科技有限公司 一种视频流畅播放方法及装置
CN112887774A (zh) * 2021-02-05 2021-06-01 冠捷显示科技（厦门）有限公司 一种无缝播放内容的方法及装置
CN115580602A (zh) * 2021-06-30 2023-01-06 星络家居云物联科技有限公司 流媒体推送方法、移动终端及计算机可读存储介质
CN113852853B (zh) * 2021-09-18 2024-05-28 青岛海信宽带多媒体技术有限公司 一种机顶盒及连续播放节目方法
JP2023166230A (ja) * 2022-05-09 2023-11-21 ソニーグループ株式会社 通信装置、情報処理装置、通信方法、及び情報処理方法
CN115604404A (zh) * 2022-10-19 2023-01-13 深圳看到科技有限公司（Cn） 用于多路视频流的切换方法、切换装置及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103096178A (zh) * 2011-11-02 2013-05-08 腾讯科技（深圳）有限公司 播放流媒体文件的方法和装置
CN103259938A (zh) * 2013-06-03 2013-08-21 贝壳网际（北京）安全技术有限公司 一种视频播放的处理方法及系统
WO2014000198A1 (zh) * 2012-06-28 2014-01-03 华为技术有限公司 多屏流媒体播放方法及流媒体服务器

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101997969A (zh) * 2009-08-13 2011-03-30 索尼爱立信移动通讯有限公司 图片声音注释添加方法和装置以及包括该装置的移动终端
CN103139278A (zh) * 2011-12-05 2013-06-05 北京网康科技有限公司 一种网络资源预取并缓存加速的方法及其装置
CN102510541B (zh) * 2011-12-30 2014-12-17 Tcl数码科技（深圳）有限责任公司 多屏互动的音视频内容切换方法及媒体播放器

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103096178A (zh) * 2011-11-02 2013-05-08 腾讯科技（深圳）有限公司 播放流媒体文件的方法和装置
WO2014000198A1 (zh) * 2012-06-28 2014-01-03 华为技术有限公司 多屏流媒体播放方法及流媒体服务器
CN103259938A (zh) * 2013-06-03 2013-08-21 贝壳网际（北京）安全技术有限公司 一种视频播放的处理方法及系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3469790A4 (en) * 2016-09-07 2019-04-17 Samsung Electronics Co., Ltd. DISPLAY APPARATUS AND CORRESPONDING CONTROL METHOD
CN110769280A (zh) * 2019-10-23 2020-02-07 北京地平线机器人技术研发有限公司 一种接续播放文件的方法及装置
CN112286110A (zh) * 2020-11-04 2021-01-29 上海美家美沪装饰科技有限公司 一种智慧型家居自适应交互控制系统

Also Published As

Publication number Publication date
US20170302988A1 (en) 2017-10-19
CN105516791A (zh) 2016-04-20
EP3203754A1 (en) 2017-08-09

Similar Documents

Publication Publication Date Title
WO2016049976A1 (zh) 一种智能家居中流媒体数据无缝连接实现方法及系统
US10368119B2 (en) Method, device and computer program product for outputting a transport stream
US10063656B2 (en) Server-side scheduling for media transmissions
KR20170074866A (ko) 수신 장치, 송신 장치, 및 데이터 처리 방법
CN107819809B (zh) 对内容进行同步操作的方法及装置
JP4735697B2 (ja) 電子機器、コンテンツ再生方法及びプログラム
US11128897B2 (en) Method for initiating a transmission of a streaming content delivered to a client device and access point for implementing this method
US8930442B2 (en) Apparatus and method for playing media content data
WO2021143360A1 (zh) 资源传输方法及计算机设备
KR20020045571A (ko) 데이터 재생방법, 데이터 수신단말 및 데이터 수신방법
US20240031451A1 (en) Pushing content in time windows based on different positions
KR20220031120A (ko) 비디오 플레이백에서 예측-기반 드롭된 프레임 처리 로직
US10237195B1 (en) IP video playback
US8151299B2 (en) Method and receiving unit for fast change of services
KR100851634B1 (ko) 라이브 멀티미디어 스트림을 풀방식으로 스트리밍하는 방법및 시스템
KR100880569B1 (ko) 모바일 아이피티브이의 이피지 환경에서 브이오디 컨텐츠의전송방법
KR20190032671A (ko) 실시간 iptv 방송에서의 채널 전환 시스템
KR20160074310A (ko) 하이브리드망에서의 스트리밍 방법 및 그 장치
KR20140085696A (ko) 미디어 시스템 및 이를 제어하는 방법
JP2014072652A (ja) 映像コンテンツ配信装置
KR20100068780A (ko) 스트리밍 서비스에서 프리 디코더 버퍼의 오버플로우 방지 방법 및 장치
JP2014072650A (ja) 映像コンテンツ配信装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14903087

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2014903087

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014903087

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 15513625

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE