US20190075342A1 - Codec techniques for fast switching - Google Patents
Codec techniques for fast switching Download PDFInfo
- Publication number
- US20190075342A1 US20190075342A1 US16/180,115 US201816180115A US2019075342A1 US 20190075342 A1 US20190075342 A1 US 20190075342A1 US 201816180115 A US201816180115 A US 201816180115A US 2019075342 A1 US2019075342 A1 US 2019075342A1
- Authority
- US
- United States
- Prior art keywords
- channel
- video
- stream
- version
- transition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
- H04N21/234372—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution for performing aspect ratio conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2381—Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
- H04N21/4383—Accessing a communication channel
- H04N21/4384—Accessing a communication channel involving operations to reduce the access time, e.g. fast-tuning for reducing channel switching latency
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
Abstract
Description
- This disclosure relates generally to the field of video streaming techniques. More specifically, this disclosure relates to reducing codec-related latency in channel switching for video streaming.
- Portable devices increasingly are being utilized to watch traditional programs (e.g., movies, news, and live broadcasts). The programs are encoded and transmitted over wireless or wired networks to distribution devices (e.g., cable boxes) and/or portable devices (e.g., smart phones, laptops or tablets). The programs are typically encoded and sent over the networks in coded data set parsed into chunks (e.g., “transmission units”).
- The chunks may include one or more reference frames and a plurality of frames that are encoded based on the reference frames. The chunks may include an index frame (I-frames) that is encoded without using information from any other frames, predictive frames (P-frames) that express differences from the content present in the index frame, and bidirectional frames (B-frames) that express differences from the index frame and the predictive frames that occur before and after the bidirectional frame. The frames in the chunks may occur in an order that is different from the order in which the content represented by the frames in chunks should be presented.
- To display the video, the device receiving the chunks will buffer the chunk and decode the frames in the chunk for display. The decoding may include converting each frame into the original frame and arranging the frames in the correct order. While the user may observe a delay at the beginning of a video segment due to the buffering and decoding, because the chunks are received and decoded continuously one right after another, the delay due to the buffering and decoding is not observed during regular playback.
- However, when a user makes a request to transition to a video source in a new channel, there is a delay associated with switching to the new video source. The contributing factors to the delay may include, network latency, buildup of bit stream buffer, decoding latency and display latency. During this delay the user may observe a gap in the video display. To reduce the gap some methods continue to display the content of the original video source until the video content of the new chunk is ready to be displayed. Other methods, to reduce the buffer and/or decoding latency, transition at a time when the index frame of the chunk is encountered in the new video source. However, these methods do not transition at the time that the user makes the request. In addition, when using these methods on live streaming content, the delay causes content of the new video source to be missed.
- A similar delay may also be observed when transitioning between low and high resolution sources of the same video content. When transitioning between the same content, existing methods skip or replay a portion of the content due to the delay.
- Accordingly, there is a need for techniques to reduce the delay when transitioning between multiple channels during video streaming.
- So that features of the present invention can be understood, a number of drawings are described below. It is to be noted, however, that the appended drawings illustrate only particular embodiments of the invention and are therefore not to be considered limiting of its scope, for the invention may encompass other equally effective embodiments.
-
FIG. 1 is a simplified block diagram of a video distribution system suitable for use with the present disclosure. -
FIG. 2 illustrates an exemplary display of a client device receiving video content according to an embodiment of the present disclosure. -
FIG. 3 illustrates a method to transition directly from a first channel to a second channel according to an embodiment of the present disclosure. -
FIG. 4 illustrates a method to transition from a first channel to a second channel using an intermediate channel according to an embodiment of the present disclosure -
FIG. 5 illustrates a method to transition from a first channel to a second channel using an intermediate channel according to another embodiment of the present disclosure. -
FIG. 6 illustrates a method to transition from a first channel to a second channel using reference frames within chunks of the data stream according to an embodiment of the present disclosure. -
FIG. 7 illustrates a method to transition from a first channel to a second channel, which is one of a plurality of offset channels according to an embodiment of the present disclosure. -
FIG. 8 illustrates a method to transition from a first channel to a second channel via a staggered channel according to an embodiment of the present disclosure. -
FIG. 9 illustrates a playback method using multiple scalable channels according to an embodiment of the present disclosure. -
FIG. 10 illustrates amethod 1000 to provide different aspect ratios of video content according to an embodiment of the present disclosure. -
FIG. 11 is a simplified block diagram of a system having an integrated coding server and distribution server for use with the present disclosure. - Embodiments of the present invention provide techniques and devices for transitioning between multiple sequences of coded video data. A video streaming method may include receiving transmission units from a first sequence of coded video data and decoding the transmission units from the first sequence. In response to a request to transition to a second sequence of coded video data, the method may receive transmission units from the second sequence of coded video data and decode the transmission units from the second sequence. Decoding the transmission units from the second sequence may include buffering a selected transmission unit from the second sequence that includes a frame corresponding to the request to transition, decoding reference frames of the selected transmission unit up to the frame that corresponds to the request to transition, without decoding the non-reference frames, and decoding all of the frames following the frame that corresponds to the request.
- Embodiments of the present invention for transitioning between multiple sequences of coded video data may include transitioning to the requested sequence via one or more intermediate sequences of coded video data. The transition to the requested sequence via the intermediate sequences can reduce the transition delay.
-
FIG. 1 is a simplified block diagram of avideo distribution system 100 suitable for use with the present invention. Thesystem 100 may include adistribution server system 110 and a plurality ofclient devices communication network 130. Thedistribution server 110 may receive a variety of video content items (e.g., movies, television shows and other motion picture content) from aprogram source 112 and/or astorage system 114 for download by theclient device distribution server 110 to one ormore client devices - The
distribution server 110 may process the video content and provide several copies of the video content items as coded video streams, each of which are coded at a respective bit rate, frame rate and/or frame size. The video streams may be parsed into a plurality of chunks (e.g., transmission units or GOPs, but is not so limited) of the coded video representing video content of a predetermined duration (e.g., 5 or 10 seconds). The example ofFIG. 1 illustrates three different video programs A, B and C that are available for transmission to theclient devices distribution server 110 to provide the program using different bit rate, frame rate and/or frame size or video items that are offset in time. For example, the content of program A may be provided in channels 1.0, 1.1 and 1.2, each corresponding to an encoded video with different parameters. - The
distribution server 110 may provide theclient device manifest file 150. Based on this information, theclient device network 130. The selection of the program and/or channel may be made by the user or by the client device in response to a user command or changing condition (e.g., changes in network bandwidth or processor availability of the device). When a request for a new program and/or channel is issued, theclient device distribution server 110 may determine how to transition from the current channel being transmitted to theclient 120 to the requested channel. In making the determination theclient device distribution server 110 may select which channels and chunks of the channels should be sent to theclient device - Transitioning from one channel to another channel may include utilizing one or more intermediate channels having different bit rate, frame rate and/or frame size or channels that are offset in time. For example, as shown in
FIG. 1 , if a request is received to transition from channel 1.0 to channel 2.0, the transitioning may include transmitting at least portions of intermediate channel 1.1 and channel 1.2 in thechannel stream 140. Utilizing the additional channels for the transition may reduce a gap in the programming during the transition from channel 1.0 to channel 2.0. - As shown in
FIG. 1 ,multiple channel streams client device stream 140 may provide video content that is different from the video content ofchannel stream 142. Channelstream 142 may provide video content as a preview using a low video quality of the available program that can be provided with a high quality. - The
manifest file 150 may store an index of the available programs and channels, associate the coded video streams with the programs, and correlate chunks of each coded video stream with corresponding chunks of the other video streams. Themanifest file 150 may include details of each available channel and a web address to the channels or for each chunk in the channel. Themanifest file 150 may be updated to maintain an updated index of the available programs and channels. Themanifest file 150 may be provided to theclient device - Embodiments of the present invention are directed to transitioning from one video source to another video source in response to a request from the
client device client device - The transition methods may be utilized by one or more of the distribution server, and the
client devices distribution server 110 may use one or more of the transitioning methods to decide which channels should be sent to theclient device distribution server 110 may perform the transitioning methods and re-encode the video that includes the transition from one channel to another channel. The data may be re-encoded with one chunk including the transition and the data from the two channels. Alternatively, the data may be re-encoded with one chunk including data up to the transition request and the next chunk including data after the transition request. - The
client device client device distribution server 110 that includes the channels that theclient device client device 120 may receive all of the channels over thenetwork 130 and function as a local distribution device for theclient device 122. -
FIG. 2 illustrates anexemplary display 210 of aclient device 200 receiving video content according to an embodiment of the present invention. Thedisplay 210 may include one ormore display windows 220, 230, 240 to accommodate different video content. Thedisplay 210 may also includelabels Display window 220 may display a first video content A that is provided with high quality. Display windows 230 and 240 may display a second video content B and a third video content C, respectively, with, for example, a lower quality. For example, display windows 230 and 240 may display a lower quality video that are provided as a preview of video content that can be received with higher quality if user makes such a selection. - The video content displayed in
display windows 220, 230, 240 may be received from one or more distribution servers. Each of the video content displayed indisplay windows 220, 230, 240 or referenced bylabels display 210 may correspond to a live broadcast of a show being captured with different cameras, each providing a different perspective of the live broadcast. - A user of the
client device 200 may make selections as to which video content to display in thedisplay windows 220, 230, 240 of thedisplay 210. For example, the user may make a selection on theclient device 200 to display in thedisplay widow 220, one of the video content currently being provided as a preview in display window 230 or 240. In response to the selection, the distribution server may receive two requests. A first request to transition from a low resolution to a high resolution of the video content being provided in display windows 230 or 240 and a second request to transition from a high resolution to a low resolution of the video content being provided in thedisplay window 220. - In response to the requests, the distribution server may change the channel from which the chunks are provided to each of the channel streams sent to the
client device 200. To reduce the switching latency, while maintaining good quality during the transition, the distribution server may utilize one or more transition methods discussed below. The transition method from one channel to another channel may utilize one or more intermediate channels to reduce the latency at theclient 200. Using the chunks of different channels theclient device 200 may buffer and decode the chunks of each channel according to transition method discussed below to reduce the buffering and/or decoding latency. -
FIG. 3 illustrates a method to transition directly from a first channel to a second channel according to an embodiment of the present invention.FIG. 3 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to channel B, parsed into chunks CH B.1-B.N. Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request. - As shown in
FIG. 3 the request to transition from channel A to channel B may be received when a synchronization frame of channel B does not align with the request. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by buffering and decoding channel B up to a destination frame that is approximately aligned with the request. The client may already be receiving channel B at the time of the request or, alternatively, may issue the request to receive channel B in response to the request. - To display the content of channel B at approximately the time that the request is received, the decoder receiving channel B may determine which chunk (e.g., chunk CH B.2) is aligned with the request, buffer the chunk CH B.2 and rapidly decode the data in the chunk up to the transition point using the synchronization frame. Buffering the chunk may include buffing only a portion of the chunk up to the frame corresponding to the request. The speed of the decoding can be increased by skipping the decoding of the frames up to the frame that is aligned with the transition point and/or request.
- In one embodiment, only the reference frames of the chunk up to the destination frame may be decoded and all of the frames may be decoded following the destination frame. The reference frame may include all of the frames that are needed to decode the frame that corresponds to the request and frames following this frame. The reference frames that are decoded may include all of the I-frames and/or P-frames.
- The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence. The synchronization frame SF may be coded as an intra-coded frame (colloquially, an “I frame”). For example, if the video sequence is coded according to the H.265 coding protocol, the synchronization frame may be coded as an Instantaneous Decoder Refresh frame (IDR frame). Other coding protocols may provide other definitions of synchronization frames.
- While the synchronization frames SF are shown at the beginning of each chunk in
FIG. 3 , the synchronization frames may be located at other locations within the chunk. In addition, the synchronization frames SF across the different channels (e.g., A and B) may be aligned. In other embodiments, the synchronization frames SF in one channel may be offset from the synchronization frames SF in the other channels by a predetermined or random number of frames. - Channel A and channel B may correspond to video having different content or may include the same video content but provided at a different variants of bit rate, frame rate and/or frame size. For example, channel A may be a low resolution video of a live broadcast that is provided to a user as a preview and channel B may be a high resolution video of the same live broadcast that is provided to the user in response to a request to watch the live broadcast. In another example, channel A may be a live broadcast of a show being captured with a first camera and channel B may be a live broadcast of a show being captured with a second camera.
- The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B. The playback of channels A and B, is shown in
FIG. 3 . - In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device a list of the available channels and chunks associated with those channels. Based on the information in the manifest file and the details of the request to transition, the client device may select the chunk of channel B that corresponds to the request to transition and request that the selected chunk be sent to the client device. The client device may buffer the selected chunk and select the destination frame in the received chunk that corresponds to the request to transition. The destination frame and the frames following the destination frame may be decoded and displayed on the client device. The destination frame may be decoded by only decoding the preceding frames that are needed to decode the destination frame. Thus only the reference frame preceding the destination frame may be decoded and all of the frames following the destination frames may be decoded.
-
FIG. 4 illustrates a method to transition from a first channel to a second channel using an intermediate channel according to an embodiment of the present invention.FIG. 4 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to channel B, parsed into chunks CH B.1-B.N. The transitioning from channel A to channel B may include using an intermediate channel B′, parsed into chunks CH B′.1-B′.N. The intermediate channel B′ may be provided at a quality that is lower than the quality channel B. The lower quality allows for the content of intermediate channel B′ to be buffered and decoded faster than the content of channel B. - Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request. The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence.
- Channel A and channel B may include video having different content or may include the same video content but provided at a different variants of bit rate, frame rate and/or frame size. Channel B′ may include video content that corresponds to the video content of channel B but with a lower quality (e.g., lower bit rate, frame rate and/or frame size).
- As shown in
FIG. 4 , the request to transition from channel A to channel B may be received when the synchronization frame SF of channel B does not align with the request to transition. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by transitioning first to an intermediate channel B′ and then transitioning to channel B at the next synchronization frame SF of channel B. - The lower quality (e.g., lower bit rate, frame rate and/or frame size) of the intermediate channel B′ allows the decoder to receive and decode the video content faster than the decoder would otherwise decode the higher quality data of channel B. Thus, using the intermediate channel allows for the decoding latency to be reduced and to provide a faster transition to the content of channel B. The decoder may decode the content of intermediate channel B′ up until the next synchronization frame SF of channel B. At the synchronization frame SF of channel B, the decoder can immediately switch to channel B and decode the content of channel B without having to buffer any preceding frames of the higher quality data. While switching to the lower quality intermediate channel B′ may provide lower quality video for a short period of time, the transition can be performed at the time of the request without significant delay.
- While the synchronization frame SF of the intermediate channel B′ is shown aligned with the synchronization frame SF of channel B, the synchronization frames may be offset from each other. In one embodiment, if the request is made at the time of the synchronization frame of channel B, the transition can be made directly to channel B without using the intermediate channel B′. One or more of the other transitioning embodiment discussed in this disclosure may be used to transition from channel A to channel B′.
- The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B. As shown in
FIG. 4 , the channel stream can provide data to transition from content of channel A to content of channel B, via channel B′, approximately at the time of the request. While the channel stream will include a chunk with content of channel B that is at a lower bit rate, frame rate and/or frame size, for a short period of time, the content of the channel stream can be switched at the time of the request without a significant delay. While the whole chunk of the respective channels may be sent to the client device, only a portion of the data in the chunk can be decoded to facilitate the transition. The playback of channels A and B, is shown inFIG. 4 . - In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device a list of the available channels and chunks associated with those channels. Based on the information in the manifest file and the details of the request to transition, the client device may decide whether the destination frame in channel B aligns with one of the synchronization frames in channel B. If the destination frame in channel B aligns with a synchronization frame in channel B, then the chunks starting with the chunk containing the aligned synchronization frame can be received and decoded by the client device.
- If the destination frame in channel B does not align with a synchronization frame in channel B, the client device may determine whether the transition time may be reduced by using one or more of the intermediate channels. If the transition time cannot be reduced, then the client device may request chunks from only channel B to be sent to and decoded by the client device. However, if the transition time can be reduced, the client device may select a chunk of an intermediate channel that corresponds to the request to transition and request that the selected chunk be sent to the client device. The client device may buffer the selected chunk and select the frame in the chunk that corresponds to the destination frame. The selected chunk and following chunks in the intermediate channel may be decoded and displayed until a synchronization frame is encountered in channel B. Once the synchronization frame in channel B is encountered, the chunks of channel B may be received, decoded and displayed by the client device.
-
FIG. 5 illustrates a method to transition from a first channel to a second channel using an intermediate channel according to another embodiment of the present invention.FIG. 5 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to channel B, parsed into chunks CH B.1-B.N. The transitioning from channel A to channel B may include using an intermediate channel B′, parsed into chunks CH B′.1-B′.N. The intermediate channel B′ may include chunks of data that are shorter than the chunks in channel B. For example, the chunks in channel B may be provided using a default chunk size (e.g., 10 seconds) and the chunks in the intermediate channel B′ may be provided using a smaller chunk size than the default value (e.g., 3 seconds). The shorter chunks in the intermediate channel B′ provide for a higher frequency of the synchronization frames to be provided in the intermediate channel B′ as compared to the frequency of synchronization frames in channel B. - Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request. The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence.
- Channel A and channel B may include video having different content or may include the same video content but provided at a different variants of bit rate, frame rate and/or frame size. Channel B′ may include video content that corresponds to the video content of channel B but encoded with chunks that are shorter in duration. In one embodiment, the quality of the video content in channels B and B′ may be similar but because smaller chunks are used to encode channel B′ the bit rate of channel B′ may be higher than the bit rate of channel B. In one embodiment, the bit rate of channel B and channel B′ may be similar but the quality may be lower in channel B′ to compensate for the smaller chunks. The default chunk size may be set by the encoder depending on the video content, available network bandwidth, desired size of bit stream, parameters of the client device, or other encoder settings. The chunks of the intermediate channel may made smaller even though a higher number of bits will be used or the quality of the video will be reduced.
- As shown in
FIG. 5 the request to transition from channel A to channel B may be received when the synchronization frame SF of channel B does not align with the request to transition. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by transitioning first to an intermediate channel B′ and then transitioning to channel B at the next synchronization frame SF of channel B. - The higher frequency of the synchronization frames SF in the intermediate channel B′ provides for synchronization frames that are potentially closer to the request then the synchronization frames of channel B. Thus, when the request for the transition is made, transition from channel A to intermediate channel B′ can be processed faster than the transition from channel A to channel B. After transitioning to intermediate channel B′, the content of intermediate channel B′ may be provided and decoded until the next synchronization frame SF of channel B is encountered, at which point the transition can be made from intermediate channel B′ to channel B without significant delay.
- Transitioning from channel A to intermediate channel B′ may include waiting until the next synchronization frame SF is encountered. Because of the high frequency of the synchronization frames, the delay between the channel change request and the next synchronization frame in the intermediate channel B′ is smaller as compared to the delay associated with switching to channel B. In an alternative embodiment, transitioning from channel A to intermediate channel B′ may include finding the closest preceding synchronization frame SF in the intermediate channel B′, and buffering and decoding intermediate channel B′ up to the frame that is aligned with the request (e.g., according to the embodiment shown in
FIG. 3 ). - In one embodiment, when the request to transition from channel A to channel B is received, a determination can be made as to which channel between intermediate channel B′ and channel B includes the closest synchronization frame SF following request or preceding the request. If channel B has the closest synchronization frame SF or the synchronization frames SF of both channels are found in the same location, then the transition can be made directly to channel B without using intermediate channel B′. However, if the closest synchronization frame SF is found in the intermediate channel B′ then the transition from channel A to channel B may include, first transitioning to the intermediate channel B′ and then, transitioning to the channel B at the next synchronization frame SF of channel B. In one embodiment, a plurality of intermediate channels may be included with each providing chunks of different durations that are smaller than the default chunk duration.
- The synchronization frames SF of channel B and channel B′ do not have to be aligned. The synchronization frames of channel B and channel B′ may be offset from each other by a predetermined length. In another embodiment, the length of channel B′ may be set such that the synchronization frames SF of channel B and channel B′ are offset from each other. Having the synchronization frames SF of channel B and channel B′ not aligned increases the chances that a synchronization frame will be aligned with the request to transition to channel B.
- The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B. As shown in
FIG. 5 , the channel stream can provide data transition from the content of channel A to content of channel B approximately at the time of the request. While the channel stream may include a chunk with content of channel B that is at a lower quality or which required more bandwidth for a short period of time, the content of the channel stream can be switched at the time of the request without a significant delay. - In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device a list of the available channels and chunks associated with those channels. Based on the information in the manifest file and the details of the request to transition, the client device may decide whether the destination frame in channel B aligns with one of the synchronization frames in channel B. If the destination frame in channel B aligns with a synchronization frame in channel B, then the chunks starting with the chunk containing the aligned synchronization frame can be received and decoded by the client device.
- If the destination frame in channel B does not align with a synchronization frame in channel B, the client device may determine whether the transition time may be reduced by using one or more of the intermediate channels having varying duration of chunks. If the transition time cannot be reduced, then the client device may request chunks from only channel B to be sent to and decoded by the client device. However, if the transition time can be reduced, the client device may select a chunk of an intermediate channel that corresponds to the request to transition and request that the selected chunk be sent to the client device. The client device may buffer the selected chunk and select the frame in the chunk that corresponds to the destination frame. The selected chunk and following chunks in the intermediate channel may be decoded and displayed until a synchronization frame is encountered in channel B. Once the synchronization frame in channel B is encountered, the chunks of channel B may be received, decoded and displayed by the client device.
-
FIG. 6 illustrates a method to transition from a first channel to a second channel using additional reference fames provided within chunks of the data stream according to an embodiment of the present invention.FIG. 6 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to channel B, parsed into chunks CH B.1-B.N. The transitioning from channel A to channel B may include using a number of reference frames RF associated with the chunk of channel B. The reference frames RF provide additional reference frames within the chunk that can be quickly decoded to allow for quick decoding of channel B data following the request. The additional reference frames may be frames that are not provided within the chunk using default settings of the encoder. As shown inFIG. 6 , the reference frames RF may be provided as part of chunk to which the transition is made. In another embodiment, the reference frames RF′, which correspond to the reference frames RF in chunk of channel B, may be provided as a side channel. - Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request. The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence. At the time of the request, a destination channel may be determined in channel B that corresponds to the request to transition.
- One or more chunks of channel B provided by the distribution server may include additional reference frames (e.g., P-frames) distributed within the chunk. The additional reference frames may be part of channel B or may be provided in a separate channel. While with default settings the encoder may limit the number of reference frames that are provided within a particular chunk to limit the size of the chunk, the additional reference frames may be included in a chunk to improve the transition to channel B. The reference frames may be frames that are provided in addition to the synchronization frame (e.g., I-frame) of the respective chunk. The reference frames may use data from one or more previous frames to be decoded. In one embodiment, the reference frames may need only the data of the synchronization frame to be decoded. Thus, any one of the reference frame can be quickly decoded without needing every preceding frame or more than one preceding frame. The additional reference frames also do not need any frames following the reference frame to be decoded.
- As shown in
FIG. 6 the request to transition from channel A to channel B may be received when the synchronization frame SF of channel B does not align with the request to transition. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by utilizing the available additional reference frames within the chunk that correspond to the request. In response to the request to transition, a chunk with the additional reference frames or a side channel with the additional reference frames may be sent to the client device. Using the additional reference frames, the closest additional reference frame to the request within the chunk can be determined. The closest reference frame may be the closest reference frame preceding the request or the closest reference frame following the request. - The selected reference frame (e.g., reference frame FR5 in
FIG. 6 ) may be decoded and used as the starting frame to provide the content of channel B. Alternatively, the selected reference frame may be used as a reference to decode another nearby frame that corresponds to the destination frame. Because the selected reference frame does not need every single preceding frame or any frames following the selected reference frame to be decoded, the selected reference frame can be decoded by only using the frames needed for the decoding. For example, if the selected reference frame only needs the synchronization frame SF of the chunk to be decoded, only the synchronization frame SF and the selected reference frame need to be buffered and decoded to start providing the content of channel B. All of the frames following the selected reference frames can be decoded using the information in the selected reference frame. - The use of additional reference frames provides more transitioning locations to be provided within the chunk without using the more costly synchronization frames SF (e.g., I-Frames). The additional reference frames may be provided at a predetermined interval (e.g., every half a second) within the chunk. The bitstream may be indexed in a manifest file, so that all the bits of the chunk do not have to be buffered to decode the reference frame corresponding to the request to transition. For example, pointers to the reference frames may be provided in the manifest file, so that the decoder can locate the reference frame that is closest to the request.
- The reference frames may be provided in one or more channels by the distribution server. The additional reference frames may be provided in the channels that are provided as a preview to the user. In another embodiment, the additional reference frames may be provided in response to a request and/or only in the chunks that correspond to the request.
- The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B. As shown, because chunk CH B.1 corresponds to the request to transition, additional channels may be provided with chunk CH B.1.
- In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device the available channels and chunks associated with those channels. Based on the information in the manifest file and the details of the request to transition, the client device may decide whether the destination frame in channel B aligns with one of the synchronization frames in channel B. If the destination frame in channel B aligns with a synchronization frame in channel B, then the chunks starting with the chunk containing the aligned synchronization frame can be received and decoded by the client device.
- If the destination frame in channel B does not align with a synchronization frame in channel B, the client device may determine whether the transition time may be reduced by using additional reference frames associated with the chunk corresponding to the request to transition. If the transition time cannot be reduced, then the client device may request chunks from channel B to be sent to and decoded by the client device. However, if the transition time can be reduced, the client device may request that one or more additional reference frames are provided as part of the chunk corresponding to the request to transition or as part of a side channel. The client device may request that only the synchronization frame and the reference frame corresponding to the destination frame be sent to the client device, based on the information in the manifest file.
- The client device may buffer and decode the synchronization frame and the reference frames. The decoded reference frame may be used to provide the destination frame and continue the decoding of frames following the destination frame. The decoded frames may be displayed by the client device.
-
FIG. 7 illustrates a method to transition from a first channel to a second channel, which is one of a plurality of offset channels, according to an embodiment of the present invention.FIG. 7 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to one of a plurality of offset channels B, B′, B″ and B′″. Each of the offset channels may be parsed into chunks CH B.1-B.N. The transitioning from channel A to one of the offset channels may include determining which of the offset channels includes a synchronization frame that is most closely aligned with the request and transitioning from channel A to the channel with the closest synchronization frame. - Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request to transition. Additional channels B′, B″ and B′″ with the same content as channel B may be provided but with the chunks and/or the synchronization frames that are offset. The distribution server may encode the channels to provide the channels with offset synchronization frames. The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence. While the synchronization frames SF in channel B and additional channels B′, B″ and B′″ may be offset, the content at a given time that is available in each of the channels may be the same.
- Channel A and channel B may include video having different content or may include the same video content but provided at a different variants of bit rate, frame rate and/or frame size. As shown in
FIG. 7 , the request to transition from channel A to channel B is received when the synchronization frame SF of channel B does not align with the request to transition. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by transitioning to one of the offset channels B′, B″, or B′″ that has a closer synchronization frame SF to the request. - When the request to transition is received at the client device, the device may determine the channel from the available offset channels that includes the closest synchronization frame SF. The closest synchronization frame may be the closest synchronization frame before the request or the closest synchronization frame SF after the request. The client device may process the transition from channel A to the content of channel B by using the closest synchronization frame SF as the first frame of channel B content. Because the synchronization frame SF does not need any other frames to be buffered and decoded, the content of channel B can be displayed on the client device as soon as the synchronization frame SF is decoded.
- The distribution server can provide all of the offset channels to the client device. In one embodiment, channel B and the additional channels B′, B″ and B′″ are all provided with a same quality. In another embodiment, the offset channels B′, B″ and B″ may be channels having a lower quality than the quality of channel B. Once the synchronization frame of the channel B having a higher quality is encountered a transition could be made to channel B from the offset channels B′, B″ and B′″.
- In another embodiment, in response to a request form the client device, the distribution server may provide only the offset channel that includes the closest synchronization frame SF to the client device. The information about the offset channels and the location of the synchronization frames may be provided to the client device via a manifest file. Thus, using the information in the manifest file, the client device may determine which channel should be sent to the client device and include the requested channel in the request to the distribution server.
- In one embodiment, the offset channels may be mixed with channels that have chunks of different lengths to provide a better distribution of available synchronization frames. While the synchronization frames are shown at the beginning of the chunks in
FIG. 7 , the synchronization frames may be located at other locations within the chunks. - The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B.
- In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device a list of the available channels and chunks associated with those channels. The manifest file may include information about a plurality of channels that include content of channel B and have synchronization frames that are offset from each other. Based on the information in the manifest file and the details of the request to transition, the client device may determine which chunk from the available channels can be received and decoded faster. The client device may select the channel that includes the closest synchronization frame to the request to transition. The client device may send a request to the distribution server to send the selected channel and chunk from the channel corresponding to the request. The received chunk may be decoded and displayed by the client device.
-
FIG. 8 illustrates a method to transition from a first channel to a second channel via a staggered channel according to another embodiment of the present invention.FIG. 8 illustrates transitioning from channel A, parsed into chunks CH A.1-A.N, to channel B, parsed into chunks CH B.1-B.N, via one of staggered channels B′, B″ and B′. Transitioning from channel A to channel B may include first transitioning to one of the staggered channels B′, B″ and B′″ and to channel B when the synchronization frame SF is encountered in channel B. - Channel A may represent the channel currently being sent by a distribution server to the client device and channel B may represent the channel to which the distribution server and/or the client device will transition in response to a request. The first frame of each chunk may be coded as a synchronization frame SF that can be decoded without reference to any previously-coded frame of the video sequence. The content at a given time that is available in each of the channels B, B′, B″ and B′ may be the same.
- While additional channels B′, B″ and B′″ are provided with the same content as channel B at a given time, they may include only partial portions of the content provided in the respective chunk of channel B. Each additional channel B′, B″ and B′ may have a predetermined portion of the corresponding chunk not included in the channel. For example, the first portion of each chunk in the additional channels B′, B″ and B′″ may have a different predetermined portion (e.g., based on number of frames or a predetermined duration) not provided in the chunks. In the embodiment with the synchronization frames SF provided at the beginning of each chunk, the synchronization frames SF will be offset from each other due to the different starting points and lengths of the chunks.
- Channel A and channel B may include video having different content or may include the same video content but provided at a different variants of bit rate, frame rate and/or frame size. As shown in
FIG. 8 , the request to transition from channel A to channel B is received when the synchronization frame SF of channel B does not align with the request to transition. To avoid waiting until the next synchronization frame SF of channel B to transition to channel B, the transition may be initiated immediately by transitioning to one of the staggered channels B′, B″, or B′″ that has a closer synchronization frame SF to the request. - When the request to transition is received at the client device, the client device may determine the intermediate channel (e.g., channel B″) from the staggered channels that includes the closest synchronization frame SF. The closest synchronization frame SF may be the closest synchronization frame SF before the request or the closest synchronization frame SF after the request. The client device may process the transition from channel A to the content of channel B by using the closest synchronization frame SF of the intermediate channel (e.g., channel B″) as the first frame of channel B content. Because the synchronization frame SF does not need any other frames to be buffered and decoded, the content of channel B can be displayed on the client device as soon as the synchronization frame SF of the intermediate channel is decoded. Once the synchronization frame SF of channel B is encountered (e.g., at the end of the chunk in channel B″) the transition can be made to the synchronization frame SF of channel B from the intermediate channel. As shown in
FIG. 8 , the end of each chunk in the additional channels may be aligned with the synchronization frames of channel B. - The distribution server can provide all of the staggered channels to the client device. The staggered channels B′, B″ and B′ may be channels having a lower quality than channel B. The content of channels B′, B″ or B′″ may be provided in lower quality than channel B until the synchronization SF of channel B is encountered. Once the synchronization frame of the channel B is encountered, a transition could be made to channel B.
- In another embodiment, in response to a request form the client device, the distribution server may provide only the staggered channel that includes the closest synchronization frame SF to the client device. The information about the staggered channels and the location of the synchronization frames may be provided to the client device via a manifest file. Thus, using the information in the manifest file, the client device may determine which channel should be sent to the client device and include the requested channel in the request to the distribution server.
- The channel stream may represent the data transmitted over the network and received by the display device in response to the request to transition from channel A to channel B.
- In response to the request to transition from channel A to channel B, the client device may issue a request to receive the manifest file from the distribution server. The received manifest file may provide the client device a list of the available channels and chunks associated with those channels. The manifest file may include a list of intermediate channels that include chunks with only portions of the frames of corresponding chunks of channel B. Based on the information in the manifest file and the details of the request to transition, the client device may decide whether the destination frame in channel B aligns with one of the synchronization frames in channel B. If the destination frame in channel B aligns with a synchronization frame in channel B, then the chunks starting with the chunk containing the aligned synchronization frame can be received and decoded by the client device.
- If the destination frame in channel B does not align with a synchronization frame in channel B, the client device may determine whether the transition time may be reduced by using one or more of the intermediate channels having varying portions of the corresponding chunks of channel B. If the transition time cannot be reduced, then the client device may request chunks from only channel B to be sent to and decoded by the client device. However, if the transition time can be reduced, the client device may select the channel from the intermediate channels which includes a synchronization frame that is closest to the request to transition. A chunk from the selected channel that includes the closest synchronization frame may be selected and requested to be sent to the client device.
- The client device may buffer and decode the selected chunk. The selected chunk and following chunks in the intermediate channel may be decoded and displayed until a synchronization frame is encountered in channel B. Once the synchronization frame in channel B is encountered, the chunks of channel B may be received, decoded and displayed by the client device.
-
FIG. 9 illustrates a playback method using multiple scalable channels according to an embodiment of the present invention.FIG. 9 illustrates channel A, parsed into chunks CH A.1-A.N, and a plurality of channels B-D providing reference frames RF at a plurality of rates and/or at different instances of time. Channels B-D include reference frames (e.g., synchronization frames or I-Frames) which may be used to support fast forward (ffwd) and/or rewind (rwd) functions on a player of the content in channel A. Multiple channels B-D may be combined to provide different speeds of the ffwd and/or rwd functions. The ffwd and rwd functions using the I-Frames may be implanted by the HTTP Live Streaming (HLS) protocol, but is not so limited. - Channels B-D may provide reference frames at different rates. For example, channels B and C may include reference frames at one-eighth frames per second (i.e., one frame every eight seconds). Channel D may include reference frames at one-half frames per second (i.e., one frame every two seconds). The channels B-D may also include reference frames which are offset from each other. For example, as shown in
FIG. 9 , while channel B and C have reference frames provided at the same rate, the channels may be offset from each other. All of the channels B-D may have reference frames that are offset from the reference frames in the other channels. - Controlling the reference frames which are provided from the one or more of the channels B-D may provide the needed reference frames for a specific ffwd and/or rwd function. For example, the reference frames from all of the channels B-D may be used for slow ffwd and/or rwd operations (e.g., 2X). Because the reference frames in all of the channels B-D are offset from each other, the different channels may be combined into a single stream having a high frequency of reference frames. For faster ffwd and/or rwd operations a fewer number of reference frames may be used. To increase the speed of the ffwd and/or rwd operations, the reference frames from one or more of the channels may be removed. For example, only channels B and C may be used to provide ffwd and/or rwd operation with a faster speed (e.g., 4X). Similarly, only one of channel B and C may be used to provide ffwd and/or rwd operation for the fastest speed (e.g., 8X).
- The distribution server may provide the one or more of the channels B-D based on a command received from the client device or the mobile device. The distribution server or the client device may make a decision as to which channels should be pulled without pulling channels that are not needed for the particular operation. The distribution server or the client device may select the number of reference frames needed for the particular ffwd and/or rwd operation and determine the channels that should be sent to the client device or the mobile device to perform the requested operation. Once the ffwd and/or rwd operation is finished, one or more of the above discussed transitioning methods may be used to transition back to the channel (e.g., channel A) providing the original content. The information about the available channels may be provided to the client device in a manifest file. The client device may request the channels needed for the particular operation based on the information provided in the manifest file.
- The above discussed transitioning methods may also be applied to transitioning between channels that provide video content with different aspect ratios. Traditionally, when video content is received with an aspect ratio that is different from the aspect ratio of the display, the video content is either displayed with black areas on some edges of the video content or the video content is zoomed and cropped at the edges. The cropping is generally performed after the video content is received and decoded. With such approaches the cropped video content waists bits that are not displayed on the display device.
- Multiple channels may be used to provide the same video content with different aspect ratios. For example, the video content of the first channel may provide video using a first aspect ratio (e.g., 16:9) and the second channel may provide the same video content with a second aspect ratio (e.g., 4:3). While the first aspect ratio may correspond to the aspect ratio of the original video source, the second aspect ratio may correspond to a zoomed and cropped video of the original video source.
- Because the second channel does not include the cropped area, lower number of bits may be used to encode the center of the video content in the second channel. Alternatively, the same number of bits may be used to encode the second channel as the first channel but with a higher quality as compared to the quality of the first channel.
- In another embodiment, a first channel may provide the center content of the video and additional channels may provide the side content of the video content for various aspect ratios. One or more of the additional channels may be added to the center of the video content to provide different aspect ratios. The center content may correspond to the 4:3 aspect ratio and the additional channels may provide side content go generate additional aspect ratios. For example, the first channel may provide the center content, the second channel may provide the side content for 16:9 aspect ratio and the third channel may provide the side content for a 2.4:1 aspect ratio. When the 16:9 aspect ratio is requested, both the first channel with the center content and the second channel may be combined to provide the video content with the 16:9 aspect ratio. When the 2.4:1 aspect ratio is requested, both the first channel with the center content and the third channel may be combined to provide the video content with the 2.4:1 aspect ratio. When only the center content can be displayed on the device (e.g., for 4:3 aspect ratio), only the first channel can be sent to the display device. In one embodiment, the center content and the channels with the side content may overlap to provide better prediction and for blending.
- The distribution server or the client device may perform the processing of the original video content to provide the multiple channels that may be used to display the different aspect ratios at the display device. For example, the distribution server or the client device may perform the cropping and encoding of the second channel. The channels that are used for the different aspect ratios may be provided to the display device (e.g., client device or mobile device) in response to a request by the user or automatic operation of the device. When such a request is made, one or more of the above discussed method may be used to transition from the first channel to another channel. In response to the request, the distribution server or the client device may send multiple channels to the display device to provide the requested aspect ratio. The request may be issued by the user, for example, when the user maximizes or minimizes the display of the video. The request may be automatically made by the device, for example, when the user rotates the device and the video is flipped between being displayed horizontally and vertically.
- The information about the available channels for the various aspect ratios may be provided to the client device via a manifest file. The client device may request the channels needed for the particular aspect ratio based on the information provided in the manifest file.
-
FIG. 10 illustrates amethod 1000 to provide different aspect ratios of video content according to an embodiment of the present disclosure. Themethod 1000 may include receiving video content with a first aspect ratio and, in response to a request to provide video with a second aspect ratio, providing video content with the second aspect ratio. The video content with the second aspect ratio may be provided by using existing video content with the second aspect ratio or by obtaining side streams that can be combined with the already received video content to provide the second aspect ratio. - The
method 1000 may include receiving coded video with a first aspect ratio (box 1010). The coded video with a first aspect ratio may be decoded (box 1012) and displayed (box 1014) on a display device. In response to a request to receive the same video content with a second aspect ratio (box 1016), a determination may be made whether a coded video data is available with the second aspect ratio (box 1018). If the coded video data is available with the second aspect ratio, then the video with the second aspect ratio can be received (box 1020) and decoded (box 1022). The decoded video stream can be displayed on a display device using the second aspect ratio (box 1024). - If the coded video data with second aspect ratio is not available, then a determination can be made as to whether additional channels can be received and combined to provide the second aspect ratio (box 1026). If the second aspect ratio cannot be obtained with the additional channels, then the first aspect ratio can still be displayed on the display device (box 1014).
- However, if the content in the additional channels can be combined with the video content having the first aspect ratio, then the channels with the additional content (e.g., side streams) can be received (box 1028) and decoded (box 1030). The decoded content can be combined with the content of the video having the first aspect ratio (box 1032). If there is overlap between the video content then blending may also be performed on the video content (box 1032). The combined video content may be displayed with the second aspect ratio on a display device (box 1024). One or more of the above discussed transitioning methods may be used to transition from the video with the first aspect ratio to the video with the second aspect ratio.
- In another embodiment, the above discussed methods to transition from a first channel to a second channel may include slowing down the playback of the first and/or second channel to provide more time for the content of the second channel to be prepared for playback. Slowing down the playback of the second channel can help to provide a smooth transition to the second channel. The playback of the second channel may be slowed down such that it is not noticeable to the user. The playback of the second channel may be slowed down at the time of the request. In another embodiment, the playback of the first or second channel, or any intermediate channel, may be slowed down when it is determined that more time is needed to buffer and decode the data of the second channel. The determination may be made by comparing the expected or calculated transition time to a predetermined time. If the expected transition time exceeds the predetermined time, the playback speed of the one or more channels may be adjusted. In another embodiment both the playback of the first channel and the playback of the second channel may be slowed down to provide more time to buffer and decode the content of the second channel. These methods may be applied to any of the above discussed methods of transitioning from a first channel to a second channel.
- In another embodiment, the above discussed methods to transition from a first channel to a second channel may include adjusting the duration of the transition (e.g., animated transition) to control the speed with which the transition is made from a first channel to a second channel. The transition may include an animated transition that includes, for example, one or more effect. The effects may include providing blinds, blur, checker, cross fade, curtain, slide, push, wipe, zoom in, and zoom out. The animated transition may include an animation effect during the transition between a smaller resolution stream and a larger resolution stream. The transition may be slowed down to provide more time for the content of the second channel to be prepared for playback. Slowing down the transition of the second channel can help to provide a smooth transition to the second channel. The transition to the second channel may be slowed down such that it is not noticeable to the user. The transition may be slowed down when it is determined that more time is needed to buffer and decode the data of the second channel. The determination may be made by comparing the expected or calculated transition time to a predetermined time. If the expected transition time exceeds the predetermined time, the duration of the animated transition may be adjusted. These methods may be applied to any of the above discussed methods of transitioning from a first channel to a second channel.
- One or more of the above discussed transition methods may be combined to transition from one channel to another channel.
-
FIG. 11 is a simplified block diagram of asystem 1100 having anintegrated coding server 1110 anddistribution server 1150 for use with the present invention. Thecoding server 1110 may includebuffer storage 1115, apreprocessor 1120, acoding engine 1125, aparameter selector 1130, aquality estimator 1135 and a target bit-rate estimator 1140. Thedistribution server 1150 may include asegmenter 1155 andstorage 1160. - The
coding service 1110 may receive one or more videos and provide different versions of coded video to thedistribution server 1150 of each received video. Thedistribution server 1150 may use the different versions of the coded video to improve the transition from one video channel to another video channel. The various versions of the coded video may include different size, frame rate, timing, chunk size and/or key frame location of a common video sequence. The various versions may include video content with different aspect ratios or different portions of video content provided in separate channels. - The
buffer 1115 may store input video, typically from a camera or a storage device, for processing by other components of thecoding server 1110. For example, thepreprocessor 1120 may alter a size and/or a frame rate of the video sequence. Thepreprocessor 1120 may apply size, frame rate and/or timing modifications to video that is read from thebuffer 1115. Thepreprocessor 1120 may perform different size, frame rate and/or timing modifications on a common video sequence as thecoding server 1110 generates its various coded versions of the sequence. - The
coding engine 1125 may apply data compression operations to the video sequence from thepreprocessor 1120. Thecoding engine 1125 may operate according to any of the common video coding protocols including the MPEG, H.263, H.264 and HEVC (H.265) families of coding standards. Thecoding engine 1125 may apply coding parameters to different elements of the video sequence, including, for example, a coding mode selection and quantizer parameter selection. The coding mode selection may select a coding mode to be applied to a frame being coded, for example, whether to code an input frame as an I-frame, P-frame or B-frame. The quantizer parameter selection may select a quantization parameter to apply to transform coefficients (or other data) to truncate these data elements prior to entropy coding and transmission. - A
parameter selector 1130 may generate parameter data to thepreprocessor 1120 and/orcoding engine 1125 to govern their operation. Theparameter selector 1130, for example, may cause thepreprocessor 1120 to alter the size, frame rate and/or timing of data output to thecoding engine 1125. Theparameter selector 1130 may impose coding modes and/or quantization parameters to thecoding engine 1125. Theparameter selector 1130 may select the coding parameters based on bit rate estimates received from the target bit-rate estimator 1140 and/or based on complexity estimates of the source video. - A
quality estimator 1135 may estimate quality of coded video data output by the coding engine. Thequality estimator 1135 may output digital data representing a quantitative estimate of the quality of the coded video data. - A target bit-
rate estimator 1140 may generate bit-rate estimates for segments of video based on the data rates to be supported by thedistribution server 1150. - During operation, the target bit-
rate estimator 1140 may apportion a bit rate to the video sequence and determine a refresh rate based on data rate and chunk size estimates provided by thedistribution server 1150. In response to the bit rate selected by the target bit-rate estimator 1140 and based on analysis of the video sequence itself, theparameter selector 1130 may select operational parameters for thepreprocessor 1120 and/orcoding engine 1125. For example, theparameter selector 1130 may cause thepreprocessor 1120 to adjust the frame size (or resolution) of the video sequence. Theparameter selector 1130 also may select coding modes and quantization parameters to frames within the video sequence. Thecoding engine 1125 may process the input video by motion compensation predictive techniques and output coded video data representing the input video sequence. - The
quality estimator 1135 may evaluate the coded video data and estimate the quality of the video sequence coded according to the selected parameters. Thequality estimator 1135 may determine whether the quality of the coding meets predetermined qualitative thresholds associated with the bit rate set by thedistribution server 1150. If thequality estimator 1135 determines that the coding meets the thresholds, thequality estimator 1135 may validate the coding. By contrast, if thequality estimator 1135 determines that the coding does not meet sufficient quality thresholds associated with the target bit rate, thequality estimator 1135 may revise the coding parameters applied by theparameter selector 1130 and may cause thepreprocessor 1120 andcoding engine 1125 to repeat operation on the source video. - Once the
parameter selector 1130 selects a set of processing and coding parameters that satisfy quality metrics established by thequality estimator 1135, thecoding server 1110 may advance to the next bit rate supported by thedistribution server 1150. Again, theparameter selector 1130 andquality estimator 1135 may operate recursively, selecting parameters, applying them in preprocessing operations and coding, estimating quality of the coded video data obtained thereby and revising parameters until the quality requirements are met. - In some applications, the modules described hereinabove may be provided as elements of an integrated software system, in which the blocks may be provided as separate elements of a computer program. Some embodiments may be implemented, for example, using a non-transitory computer-readable storage medium or article which may store an instruction or a set of instructions that, if executed by a processor, may cause the processor to perform a method in accordance with the disclosed embodiments. Other applications of the present invention may be embodied as a hybrid system of dedicated hardware and software components.
- The exemplary methods and computer program instructions may be embodied on a non-transitory machine readable storage medium. In addition, a server or database server may include machine readable media configured to store machine executable program instructions. The features of the embodiments of the present invention may be implemented in hardware, software, firmware, or a combination thereof and utilized in systems, subsystems, components or subcomponents thereof. The “machine readable storage media” may include any medium that can store information. Examples of a machine readable storage medium include electronic circuits, semiconductor memory device, ROM, flash memory, erasable ROM (EROM), floppy diskette, CD-ROM, optical disk, hard disk, fiber optic medium, or any electromagnetic or optical storage device.
- It will be appreciated that in the development of any actual implementation (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system and business related constraints), and that these goals will vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the digital video capture, processing and distribution field having the benefit of this disclosure.
- Although the processes illustrated and described herein include series of steps, it will be appreciated that the different embodiments of the present disclosure are not limited by the illustrated ordering of steps, as some steps may occur in different orders, some concurrently with other steps apart from that shown and described herein. In addition, not all illustrated steps may be required to implement a methodology in accordance with the present invention. Moreover, it will be appreciated that the processes may be implemented in association with the apparatus and systems illustrated and described herein as well as in association with other systems not illustrated.
- It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the above described embodiments may be used in combination with each other. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
Claims (23)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/180,115 US20190075342A1 (en) | 2013-03-13 | 2018-11-05 | Codec techniques for fast switching |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/800,905 US9900629B2 (en) | 2013-03-13 | 2013-03-13 | Codec techniques for fast switching with intermediate sequence |
US15/845,287 US10638169B2 (en) | 2013-03-13 | 2017-12-18 | Codec techniquest for fast switching without a synchronization frame |
US16/180,115 US20190075342A1 (en) | 2013-03-13 | 2018-11-05 | Codec techniques for fast switching |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/845,287 Division US10638169B2 (en) | 2013-03-13 | 2017-12-18 | Codec techniquest for fast switching without a synchronization frame |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190075342A1 true US20190075342A1 (en) | 2019-03-07 |
Family
ID=50382546
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/800,905 Active 2035-01-10 US9900629B2 (en) | 2013-03-13 | 2013-03-13 | Codec techniques for fast switching with intermediate sequence |
US15/845,287 Active US10638169B2 (en) | 2013-03-13 | 2017-12-18 | Codec techniquest for fast switching without a synchronization frame |
US16/180,115 Abandoned US20190075342A1 (en) | 2013-03-13 | 2018-11-05 | Codec techniques for fast switching |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/800,905 Active 2035-01-10 US9900629B2 (en) | 2013-03-13 | 2013-03-13 | Codec techniques for fast switching with intermediate sequence |
US15/845,287 Active US10638169B2 (en) | 2013-03-13 | 2017-12-18 | Codec techniquest for fast switching without a synchronization frame |
Country Status (6)
Country | Link |
---|---|
US (3) | US9900629B2 (en) |
EP (1) | EP2954681B1 (en) |
KR (1) | KR102218385B1 (en) |
CN (2) | CN105144727B (en) |
TW (1) | TWI596933B (en) |
WO (1) | WO2014163867A2 (en) |
Families Citing this family (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20140111859A (en) * | 2013-03-12 | 2014-09-22 | 삼성전자주식회사 | Method and device for sharing content |
EP3025483B1 (en) | 2013-07-25 | 2022-09-21 | Convida Wireless, LLC | End-to-end m2m service layer sessions |
US9900362B2 (en) * | 2014-02-11 | 2018-02-20 | Kiswe Mobile Inc. | Methods and apparatus for reducing latency shift in switching between distinct content streams |
US10409781B2 (en) * | 2015-04-29 | 2019-09-10 | Box, Inc. | Multi-regime caching in a virtual file system for cloud-based shared content |
KR102186742B1 (en) * | 2015-06-25 | 2020-12-04 | 인더스마트 주식회사 | Multi image processing apparatus and multi image processing method of android system for high speed and short distance interface |
CA3014370A1 (en) * | 2016-02-16 | 2017-08-24 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Device, system and method for adaptive streaming |
WO2017196670A1 (en) | 2016-05-13 | 2017-11-16 | Vid Scale, Inc. | Bit depth remapping based on viewing parameters |
CN109644089B (en) | 2016-06-15 | 2022-04-26 | 康维达无线有限责任公司 | Unlicensed uplink transmission for new radio |
EP4336850A3 (en) | 2016-07-08 | 2024-04-17 | InterDigital Madison Patent Holdings, SAS | Systems and methods for region-of-interest tone remapping |
US20180091812A1 (en) * | 2016-09-23 | 2018-03-29 | Apple Inc. | Video compression system providing selection of deblocking filters parameters based on bit-depth of video data |
US10932276B2 (en) | 2016-11-03 | 2021-02-23 | Convida Wireless, Llc | Frame structure in NR |
CN106791870B (en) * | 2016-11-30 | 2019-11-05 | 华为技术有限公司 | A kind of method for video coding, video encoding/decoding method and relevant device |
EP3583780B1 (en) | 2017-02-17 | 2023-04-05 | InterDigital Madison Patent Holdings, SAS | Systems and methods for selective object-of-interest zooming in streaming video |
EP3593536A1 (en) | 2017-03-07 | 2020-01-15 | PCMS Holdings, Inc. | Tailored video streaming for multi-device presentations |
CN106961625B (en) * | 2017-03-13 | 2020-02-21 | 华为技术有限公司 | Channel switching method and device |
CN106937141A (en) * | 2017-03-24 | 2017-07-07 | 北京奇艺世纪科技有限公司 | A kind of bitstreams switching method and device |
EP3393129A1 (en) | 2017-04-21 | 2018-10-24 | Alcatel-Lucent España, S.A. | Multimedia content delivery with reduced delay |
US10523820B2 (en) * | 2017-06-02 | 2019-12-31 | Apple Inc. | High-quality audio/visual conferencing |
GB2563387B (en) * | 2017-06-09 | 2020-04-15 | Sony Interactive Entertainment Inc | Image processing device and system |
US10873775B2 (en) * | 2017-06-12 | 2020-12-22 | Netflix, Inc. | Staggered key frame video encoding |
US11470131B2 (en) | 2017-07-07 | 2022-10-11 | Box, Inc. | User device processing of information from a network-accessible collaboration system |
JP7037642B2 (en) * | 2017-09-08 | 2022-03-16 | Line株式会社 | Video quality control |
US10791366B2 (en) * | 2017-11-30 | 2020-09-29 | Hulu, LLC | Fast channel change in a video delivery network |
CN108307248B (en) * | 2018-02-01 | 2019-10-29 | 腾讯科技(深圳)有限公司 | Video broadcasting method, calculates equipment and storage medium at device |
WO2019210152A1 (en) | 2018-04-26 | 2019-10-31 | Phenix Real Time Solutions, Inc. | Adaptive bit-rate methods for live broadcasting |
EP3858023A1 (en) | 2018-09-27 | 2021-08-04 | Convida Wireless, Llc | Sub-band operations in unlicensed spectrums of new radio |
KR102654716B1 (en) * | 2019-02-11 | 2024-04-04 | 한화비전 주식회사 | Method and Apparatus for playing back video in accordance with requested video playback time |
US10735516B1 (en) | 2019-02-15 | 2020-08-04 | Signiant Inc. | Cloud-based authority to enhance point-to-point data transfer with machine learning |
CN110267096A (en) * | 2019-06-21 | 2019-09-20 | 北京达佳互联信息技术有限公司 | Video broadcasting method, device, electronic equipment and storage medium |
US11323730B2 (en) | 2019-09-05 | 2022-05-03 | Apple Inc. | Temporally-overlapped video encoding, video decoding and video rendering techniques therefor |
GB202015670D0 (en) * | 2020-10-02 | 2020-11-18 | Communication Security Group Inc | End-to-end encrypted group communications |
CN115734026A (en) * | 2021-08-31 | 2023-03-03 | 北京字跳网络技术有限公司 | Video processing method, device and system |
KR20230034064A (en) * | 2021-09-02 | 2023-03-09 | 삼성전자주식회사 | Display apparatus and operating method thereof |
US11806078B1 (en) | 2022-05-01 | 2023-11-07 | Globe Biomedical, Inc. | Tear meniscus detection and evaluation system |
CN115802074B (en) * | 2022-11-10 | 2024-03-29 | 中国联合网络通信集团有限公司 | Multipath video transmission method, device, equipment and medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110109810A1 (en) * | 2008-07-28 | 2011-05-12 | John Qiang Li | Method an apparatus for fast channel change using a scalable video coding (svc) stream |
US20110314130A1 (en) * | 2010-04-08 | 2011-12-22 | Nery Strasman | Managing streaming bandwidth for multiple clients |
US20120076204A1 (en) * | 2010-09-23 | 2012-03-29 | Qualcomm Incorporated | Method and apparatus for scalable multimedia broadcast using a multi-carrier communication system |
US20150373355A1 (en) * | 2003-06-16 | 2015-12-24 | Thomson Licensing | Decoding method and apparatus enabling fast channel change of compressed video |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6415326B1 (en) * | 1998-09-15 | 2002-07-02 | Microsoft Corporation | Timeline correlation between multiple timeline-altered media streams |
US6909743B1 (en) * | 1999-04-14 | 2005-06-21 | Sarnoff Corporation | Method for generating and processing transition streams |
FI114527B (en) * | 2002-01-23 | 2004-10-29 | Nokia Corp | Grouping of picture frames in video encoding |
US20040160974A1 (en) | 2003-02-13 | 2004-08-19 | Read Christopher Jensen | Method and system for rapid channel change within a transport stream |
US7603689B2 (en) * | 2003-06-13 | 2009-10-13 | Microsoft Corporation | Fast start-up for digital video streams |
US8249113B2 (en) | 2004-03-19 | 2012-08-21 | Broadlogic Network Technologies, Inc. | Method and system for providing faster channel switching in a digital broadcast system |
US20070174880A1 (en) * | 2005-07-05 | 2007-07-26 | Optibase Ltd. | Method, apparatus, and system of fast channel hopping between encoded video streams |
US8340098B2 (en) * | 2005-12-07 | 2012-12-25 | General Instrument Corporation | Method and apparatus for delivering compressed video to subscriber terminals |
US8330866B2 (en) * | 2006-02-21 | 2012-12-11 | Qualcomm Incorporated | Multi-program viewing in a wireless apparatus |
US7965771B2 (en) | 2006-02-27 | 2011-06-21 | Cisco Technology, Inc. | Method and apparatus for immediate display of multicast IPTV over a bandwidth constrained network |
BRPI0621545A2 (en) | 2006-04-18 | 2012-04-17 | Thomson Licensing | method for reducing channel change times in a digital video player |
US8335873B2 (en) | 2006-09-14 | 2012-12-18 | Opentv, Inc. | Method and systems for data transmission |
WO2008041896A1 (en) * | 2006-10-02 | 2008-04-10 | Telefonaktiebolaget Lm Ericsson (Publ) | Multimedia management |
US20100118941A1 (en) | 2008-04-28 | 2010-05-13 | Nds Limited | Frame accurate switching |
KR101178853B1 (en) | 2008-06-03 | 2012-09-03 | 알카텔-루센트 유에스에이 인코포레이티드 | Method and apparatus for reducing channel change response times for internet protocol television |
WO2010023504A1 (en) * | 2008-08-26 | 2010-03-04 | Csir | A method of switching from a first encoded video stream to a second encoded video stream |
US20100118938A1 (en) * | 2008-11-12 | 2010-05-13 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Encoder and method for generating a stream of data |
CN101753973B (en) * | 2008-12-12 | 2013-01-02 | 华为技术有限公司 | Channel switching method, device and system |
WO2010069427A1 (en) | 2008-12-19 | 2010-06-24 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method and encoder for providing a tune- in stream for an encoded video stream and method and decoder for tuning into an encoded video stream |
KR101777347B1 (en) * | 2009-11-13 | 2017-09-11 | 삼성전자주식회사 | Method and apparatus for adaptive streaming based on segmentation |
JP5996439B2 (en) | 2010-02-19 | 2016-09-21 | テレフオンアクチーボラゲット エルエム エリクソン(パブル) | Method and apparatus for representation switching in HTTP streaming |
US8930442B2 (en) * | 2010-03-08 | 2015-01-06 | Samsung Electronics Co., Ltd. | Apparatus and method for playing media content data |
EP2395754A1 (en) | 2010-06-14 | 2011-12-14 | Thomson Licensing | Receiver and method at the receiver for enabling channel change with a single decoder |
US8918533B2 (en) * | 2010-07-13 | 2014-12-23 | Qualcomm Incorporated | Video switching for streaming video data |
EP2485472A1 (en) * | 2011-02-04 | 2012-08-08 | Thomson Licensing | Fast channel change companion stream solution with bandwidth optimization |
US9025941B2 (en) * | 2011-02-10 | 2015-05-05 | Panasonic intellectual property Management co., Ltd | Data creation device and playback device for video picture in video stream |
CA2825117A1 (en) * | 2011-02-17 | 2012-08-23 | Panasonic Corporation | Video encoding device, video encoding method, video encoding program, video playback device, video playback method, and video playback program |
KR101804516B1 (en) * | 2011-08-31 | 2017-12-07 | 삼성전자주식회사 | Broadcast receiving device and method |
PL2798816T3 (en) * | 2011-12-29 | 2016-11-30 | Network-initiated content streaming control |
-
2013
- 2013-03-13 US US13/800,905 patent/US9900629B2/en active Active
-
2014
- 2014-02-25 CN CN201480013866.4A patent/CN105144727B/en active Active
- 2014-02-25 WO PCT/US2014/018285 patent/WO2014163867A2/en active Application Filing
- 2014-02-25 CN CN201911132024.1A patent/CN110996126A/en active Pending
- 2014-02-25 EP EP14712806.0A patent/EP2954681B1/en active Active
- 2014-02-25 KR KR1020157025084A patent/KR102218385B1/en active IP Right Grant
- 2014-03-12 TW TW103108827A patent/TWI596933B/en active
-
2017
- 2017-12-18 US US15/845,287 patent/US10638169B2/en active Active
-
2018
- 2018-11-05 US US16/180,115 patent/US20190075342A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150373355A1 (en) * | 2003-06-16 | 2015-12-24 | Thomson Licensing | Decoding method and apparatus enabling fast channel change of compressed video |
US20110109810A1 (en) * | 2008-07-28 | 2011-05-12 | John Qiang Li | Method an apparatus for fast channel change using a scalable video coding (svc) stream |
US20110314130A1 (en) * | 2010-04-08 | 2011-12-22 | Nery Strasman | Managing streaming bandwidth for multiple clients |
US20120076204A1 (en) * | 2010-09-23 | 2012-03-29 | Qualcomm Incorporated | Method and apparatus for scalable multimedia broadcast using a multi-carrier communication system |
Also Published As
Publication number | Publication date |
---|---|
US10638169B2 (en) | 2020-04-28 |
US20180109824A1 (en) | 2018-04-19 |
KR102218385B1 (en) | 2021-02-22 |
US20140269932A1 (en) | 2014-09-18 |
WO2014163867A2 (en) | 2014-10-09 |
TWI596933B (en) | 2017-08-21 |
EP2954681A2 (en) | 2015-12-16 |
CN105144727A (en) | 2015-12-09 |
TW201448575A (en) | 2014-12-16 |
US9900629B2 (en) | 2018-02-20 |
EP2954681B1 (en) | 2019-11-13 |
CN105144727B (en) | 2019-11-19 |
KR20150126860A (en) | 2015-11-13 |
WO2014163867A3 (en) | 2015-01-08 |
CN110996126A (en) | 2020-04-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10638169B2 (en) | Codec techniquest for fast switching without a synchronization frame | |
US9992499B2 (en) | Adaptive streaming techniques | |
US10009628B2 (en) | Tuning video compression for high frame rate and variable frame rate capture | |
CN107690073B (en) | Video live broadcast method and video live broadcast server | |
KR101737325B1 (en) | Method and apparatus for reducing decreasing of qualitly of experience in a multimedia system | |
US9756369B2 (en) | Method and apparatus for streaming media data segments of different lengths wherein the segment of different length comprising data not belonging to the actual segment and beginning with key frames or containing key frames only | |
EP3008911B1 (en) | Trick play in digital video streaming | |
JP2017522767A (en) | Random access in video bitstream | |
US20160360244A1 (en) | Live consecutive ad insertion | |
CN109792547B (en) | Method for transmitting video content from server to client device | |
US20200267423A1 (en) | Method for initiating a transmission of a streaming content delivered to a client device and access point for implementing this method | |
US11563962B2 (en) | Seamless content encoding and transmission | |
EP3386194A1 (en) | Method of delivery audiovisual content and corresponding device | |
US10349105B2 (en) | Channel change processing using stored content | |
De Wachter et al. | Optimizing channel switching for digital television through low quality streams | |
EP2733953A1 (en) | Content compression system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |