Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
  • H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
  • H04N19/895—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/164—Feedback from the receiver or from the transmission channel
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
  • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
  • H04N19/58—Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one

Definitions

• Predictive coding of video data can improve coding efficiency.
• predictive coding can cause "drift" when some video data is lost in transmission (such as by not arriving at all or arriving too late).
• “Drift” refers to the propagation of errors from missing data in subsequent frames. For example, when a first video frame (sometimes referred to as a picture) is lost, a second frame that follows the first frame may be coded using prediction that references that first frame. Accordingly, the decoding computer system may be unable to correctly decode that second frame. A third frame may be coded using prediction that references that second frame, and so forth.
• intra-coded frames may be inserted in the bitstream to combat this drift problem.
• intra-coded frames may be periodically inserted in the bitstream.
• a coding computer system may dynamically insert an intra-coded frame when the encoding computer system is informed that data from the bitstream has been lost.
• the disclosure relates to dynamically inserting synchronization predicted video frames.
• dynamically inserted synchronization video frames are video frames that are inserted dynamically and avoid having predictions that rely on specified data, such as lost data. Because these dynamically inserted frames can be predictively coded with reference to previous frames, the frames may be more efficient than comparable intra-coded frames. However, the synchronization predicted video frames can allow for synchronization to cut off drift by avoiding predictions that reference lost data.
• the tools and techniques can include an encoding computer system encoding and sending a video bitstream over a computer network to a decoding computer system.
• the bitstream can follow a regular prediction structure when the encoding computer system is not notified of lost data from the bitstream.
• the encoding computer system can receive a notification of lost data in the bitstream.
• the lost data can include at least a portion of a reference frame of the bitstream.
• the encoding computer system can respond to the notification by dynamically encoding a
• the encoding computer system can insert the synchronization predicted frame in the bitstream in a position where the regular prediction structure would have dictated inserting a different predicted frame with a prediction that would have referenced the lost data according to the regular prediction structure.
• FIG. 1 is a block diagram of a suitable computing environment in which one or more of the described embodiments may be implemented.
• FIG. 2 is a schematic diagram of a video transmission environment.
• FIG. 3 is a schematic diagram of an example of a regular prediction structure with periodic key frames (top) and a corresponding prediction structure with dynamic key frames (bottom).
• FIG. 4 is a schematic diagram of an example of a regular prediction structure without periodic key frames (top) and a corresponding prediction structure with dynamic long term predicted key frames (bottom).
• FIG. 5 is a schematic diagram of an example of a regular prediction structure without predicted key frames (top) and a corresponding prediction structure with a dynamic anchor predicted frame (bottom).
• FIG. 6 is a flowchart of a technique for dynamic insertion of
• Fig. 7 is a flowchart of another technique for dynamic insertion of synchronization predicted video frames.
• Fig. 8 is a flowchart of yet another technique for dynamic insertion of synchronization predicted video frames.
• Embodiments described herein are directed to techniques and tools for improved encoding of video bitstreams when a coding computer system is informed that data from a bitstream has been lost. Such improvements may result from the use of various techniques and tools separately or in combination.
• Such techniques and tools may include dynamically inserting different types of synchronization predicted video frames for different types of regular prediction structures. For example, in a bitstream with periodic key frames, which have predictions that are limited to referring to other key frames, a key frame can be dynamically inserted when a coding computer system is notified of lost data from the bitstream. As another example, in a bitstream without periodic key frames but that allows long term reference key frames, a long term reference key frame can be dynamically inserted when a coding computer system is notified of lost data from the bitstream.
• Long term reference key frames are key frames that are kept in an active frame window (the window of frames that are to be kept in a decoder frame buffer) for longer than regular frames.
• a long term reference key frame may be kept in the active frame window until a coding computer system sends an explicit notification to remove that key frame from the active frame window.
• the base layer of a frame in a bitstream with a base layer and an enhancement layer, can be coded using prediction that references a previous frame's base layer but does not reference the previous frame's enhancement layer, and the enhancement layer of a frame can be coded using prediction that references a previous frame's enhancement layer but does not reference the previous frame's base layer.
• an anchor frame is a frame where the base layer is predictively coded with prediction that references a previous frame's base layer and an enhancement layer is intra- coded so that the enhancement layer only references other layers (e.g., the base layer) within the reference frame.
• the dynamic insertion of synchronization predicted video frames can allow for synchronization to cut off drift, while preserving some efficiency by using some predictive coding where the prediction references data from previous frames.
• Techniques described herein may be used with one or more of the systems described herein and/or with one or more other systems.
• the various procedures described herein may be implemented with hardware or software, or a combination of both.
• dedicated hardware implementations such as application specific integrated circuits, programmable logic arrays and other hardware devices, can be constructed to implement at least a portion of one or more of the techniques described herein.
• Applications that may include the apparatus and systems of various embodiments can broadly include a variety of electronic and computer systems. Techniques may be implemented using two or more specific interconnected hardware modules or devices with related control and data signals that can be communicated between and through the modules, or as portions of an application-specific integrated circuit.
• implementations can include distributed processing, component/object distributed processing, and parallel processing.
• virtual computer system processing can be constructed to implement one or more of the techniques or functionality, as described herein.
• FIG. 1 illustrates a generalized example of a suitable computing
• one or more such computing environments can be used as a coding computer environment, a decoding computer environment, and/or a server that facilitates transmission of a video bitstream between the coding computer system and one or more decoding computer systems.
• a coding computer environment e.g., a graphics processing unit (GPU)
• a decoding computer environment e.g., a graphics processing unit (GPU)
• a server e.g., a server that facilitates transmission of a video bitstream between the coding computer system and one or more decoding computer systems.
• various different general purpose or special purpose computing system configurations can be used.
• Examples of well-known computing system configurations that may be suitable for use with the tools and techniques described herein include, but are not limited to, server farms and server clusters, personal computers, server computers, hand-held or laptop devices, slate devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
• the computing environment (100) is not intended to suggest any limitation as to scope of use or functionality of the invention, as the present invention may be implemented in diverse general-purpose or special-purpose computing environments.
• the computing environment (100) includes at least one processing unit or processor (1 10) and memory (120).
• the processing unit (1 10) executes computer-executable instructions and may be a real or a virtual processor. In a multi- processing system, multiple processing units execute computer-executable instructions to increase processing power.
• the memory (120) may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory), or some combination of the two.
• the memory (120) stores software (180) implementing dynamic insertion of synchronization predicted video frames.
• FIG. 1 Although the various blocks of Fig. 1 are shown with lines for the sake of clarity, in reality, delineating various components is not so clear and, metaphorically, the lines of Fig. 1 and the other figures discussed below would more accurately be grey and blurred.
• a presentation component such as a display device to be an I/O component.
• processors have memory. The inventors hereof recognize that such is the nature of the art and reiterate that the diagram of Fig. 1 is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present invention. Distinction is not made between such categories as "workstation,” “server,” “laptop,” “handheld device,” etc., as all are contemplated within the scope of Fig. 1 and reference to "computer,” “computing environment,” or “computing device.”
• a computing environment (100) may have additional features.
• the computing environment (100) includes storage (140), one or more input devices (150), one or more output devices (160), and one or more communication connections (170).
• An interconnection mechanism such as a bus, controller, or network interconnects the components of the computing environment (100).
• operating system software provides an operating environment for other software executing in the computing environment (100), and coordinates activities of the components of the computing environment (100).
• the storage (140) may be removable or non-removable, and may include computer-readable storage media such as magnetic disks, magnetic tapes or cassettes, CD- ROMs, CD-RWs, DVDs, or any other medium which can be used to store information and which can be accessed within the computing environment (100).
• the storage (140) stores instructions for the software (180).
• the input device(s) (150) may be a touch input device such as a keyboard, mouse, pen, or trackball; a voice input device; a scanning device; a network adapter; a CD/DVD reader; or another device that provides input to the computing environment (100).
• the output device(s) (160) may be a display, printer, speaker, CD/DVD-writer, network adapter, or another device that provides output from the computing environment (100).
• the communication connection(s) (170) enable communication over a communication medium to another computing entity.
• the computing environment (100) may operate in a networked environment using logical connections to one or more remote computing devices, such as a personal computer, a server, a router, a network PC, a peer device or another common network node.
• the communication medium conveys information such as data or computer-executable instructions or requests in a modulated data signal.
• a modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
• communication media include wired or wireless techniques implemented with an electrical, optical, RF, infrared, acoustic, or other carrier.
• Computer-readable storage media are any available storage media that can be accessed within a computing environment, but the term computer-readable storage media does not refer to propagated signals per se.
• computer-readable storage media include memory (120), storage (140), and combinations of the above.
• program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
• the functionality of the program modules may be combined or split between program modules as desired in various embodiments.
• Computer-executable instructions for program modules may be executed within a local or distributed computing environment. In a distributed computing environment, program modules may be located in both local and remote computer storage media.
• Fig. 2 is a schematic diagram of a video transmission environment (200) in conjunction with which one or more of the described embodiments may be implemented.
• the environment (200) can include a coding computer system (210).
• the coding computer system (210) can code video data according to a standard or protocol and transmit the video data in a bitstream (220) to one or more decoding computer systems (230).
• This can be done using any of various different coding standards and techniques.
• a scalable video standard may be used.
• An example of such a scalable video coding standard is H.264 SVC (scalable video coding), as described in Recommendation ITU-T H.264 (06/2011), although the techniques described herein could be used with other standards.
• the bitstream (220) can be transmitted over a computer network (240), which may include a global computer network (e.g., the Internet) and/or one or more other networks (e.g., an extranet, local area network, wide area network, etc.).
• a transmission server 250
• the transmission server (250) may act as an intermediary between the coding computer system (210) and the decoding computer system(s) (230).
• the decoding computer system(s) (210) may encode their own bitstreams and send them out to other computer systems such as the coding computer system (210), and the coding computer system (210) may receive and decode video bitstreams.
• the computer systems (210 and 230) may be participating in a real time audiovisual event, such as a videoconference, where video bitstreams transmitted between multiple computer systems.
• a real time audiovisual event such as a videoconference
• video bitstreams transmitted between multiple computer systems may be lost during transmission to the decoding computer system(s) (230).
• the lost data may be delayed so that the data arrives at the decoding computer system(s) too late to be used, or the data may never arrive at the decoding computer system(s). Either way, the data can be considered to be lost.
• the decoding computer system(s) (230) and/or the transmission server (250) may send one or more loss notices (260) to the coding computer system (210) to identify the lost data (e.g., identifying which frames and/or frame layers included all or part of the lost data).
• the loss notices (260) may be sent in transmissions using the same protocol as the bitstream (220), or in one or more out-of-band communications.
• the coding computer system (210) can code and insert in the bitstream (220) a synchronization predicted video frame (270) that is coded using prediction, but where the prediction does not directly or indirectly reference the data identified in the loss notice (260) (the prediction does not reference the data identified in the loss notice (260), does not reference data that itself references the data identified in the loss notice (260), etc.).
• a video frame (270) can be used to synchronize the decoding computer system(s) (230) with the coding computer system (210), which may have been out of synchronization due to the lost data.
• the video frame (270) can cut off drift that may have occurred due to the lost data.
• the synchronization predicted video frame (270) may be the next frame after a frame that includes the lost data, or it may be some later frame.
• the coding computer system (210) may not receive the loss notice (260) until the coding computer system (210) has already coded and sent one or more subsequent frames in the bitstream (220).
• the decoding computer system (230) may take measures to avoid or decrease the adverse effects from the reliance on lost data, such as by dropping or concealing those intervening frames.
• the coding computer system (210) may use different techniques and/or different types of inserted synchronization predicted video frames (270) to allow for synchronization.
• performance may be improved by analyzing the location of the loss (such as by receiving a notice of data loss and a location (e.g., which frame and/or which layer) of the data loss) and inserting appropriate synchronization information based on the inter-layer dependency and predictive coding structure.
• location of the loss such as by receiving a notice of data loss and a location (e.g., which frame and/or which layer) of the data loss
• dynamic synchronization video frame insertion will be discussed with reference to some predictive coding structures from H.264 SVC as an example, although the tools and techniques can be applied to other standards as well.
• each frame can include a base layer (302) and an enhancement layer (304) (or multiple enhancement layers, not shown), with the predictions' references being illustrated with arrows between the layers in the same frame and different frames.
• the video can be scaled by adding or omitting enhancement layers from a frame.
• an enhancement layer may add additional quality features (e.g., smaller quantization step size) and/or higher spatial resolution.
• the prediction structure (300) may be a prediction structure for a H.264 medium grain scalability (MGS) bitstream with periodic key frames.
• MMS medium grain scalability
• the illustrated regular prediction structure (300) shows frames numbered 0 to 10 in sequential order, and a similar order is shown for frames in the other prediction structures of Figs. 3-5.
• the regular prediction structure (300) can start with an instantaneous decoding refresh (IDR) type key frame (310) (frame 0), which is an intra-coded key frame.
• IDR instantaneous decoding refresh
• a key frame as used herein, is a frame that is limited to having no inter-frame predictions or only having inter-frame predictions that reference other key frames.
• the IDR-type key frame (310) can have intra- frame prediction, with an enhancement layer (304) (such as a quality enhancement layer) being coded with a prediction that references (directly or indirectly) the base layer of that frame and possibly lower enhancement layers of that frame. Additionally, and IDR-type key frame (310) can signal that subsequent frames should not include prediction references to frames prior to the IDR-type key frame (310).
• the IDR-type key frame (310) can be followed by regular predicted frames (330) (frames 1, 2, 3, 4, 6, 7, 8, and 9).
• the regular predicted frames (330) can each include a base layer (302) and an enhancement layer (304).
• Each base layer (302) of a regular predicted frame (330) can be coded with a prediction that references a highest enhancement layer of the previous frame.
• Each enhancement layer (304) of a regular predicted frame (330) can be coded with a prediction that also references the highest enhancement layer of the previous frame, and that references the base layer (302) and/or one or more lower enhancement layers of that same frame.
• the regular prediction structure (300) of Fig. 3 can include periodic insertion of predicted key frames (320) (frames 5 and 10).
• Each predicted key frame (320) can include a prediction that references other key frames (320 and/or 310), but does not reference regular predicted frames. Accordingly, a predicted key frame (320) can allow for synchronization between the coding computer system and decoding computer system(s) as long as previous key frames have not been lost. Accordingly, additional protections against loss may be used for the predicted key frames (320), as compared to protections used for the regular predicted frames (330). If a key frame (310 or 320) is lost, then the coding computer system can insert an ID -type key frame (320) upon being notified of that loss.
• the prediction structure (350) can be the same as the regular prediction structure (300) discussed above, except when the coding computer system receives a notification of lost data (360), which is illustrated with the layers including lost data being blacked out in Figs. 3-5.
• lost data 360
• the lost data (360) is in the enhancement layer (304), but the base layer (302) was not lost.
• the next regular predicted frame (330) (frame 7) would have had a prediction that referenced the lost enhancement layer (i.e., the enhancement layer with the lost data) if the regular prediction structure (300) were followed.
• the coding computer system can receive a notice of the lost data and can code and insert a predicted key frame (320) as frame 7.
• That predicted key frame (320) can include a prediction that references the previous key frame (frame 5), but that does not reference the regular predicted frame (330) with the lost data (360) (because predicted key frames (320) are limited to not referencing such regular predicted frames (330)).
• the predicted key frame (320) inserted as frame 7 can act as a synchronization predicted video frame to cut off drift that may have occurred from the lost data (360) in the enhancement layer (304) of frame 6.
• the coding computer system can receive a notification of the lost data (360) in frame 9, and can respond by coding and inserting a predicted key frame (320) as frame 10 in the bitstream, so that frame 10 can act as a synchronization predicted video frame, as with the predicted key frame (320) coded and inserted as frame 7.
• the predicted key frames (320) use some inter- frame prediction, they may be coded more efficiently than would intra-coded frames, such as IDR-type key frames, and they can still allow for synchronization and cutting off of drift from lost data.
• FIG. 4 another example of the use of dynamic insertion of synchronization predicted video frames will be discussed.
• MCS medium grain scalability
• a regular prediction structure (400) without periodic key frames is illustrated at the top of Fig. 4.
• the regular prediction structure (400) can start with a regular IDR-type key frame (410) as shown, or alternatively with an LIDR-type key frame (412) (see the bottom of Fig. 4).
• the LIDR-type key frame (412) can be similar to the IDR-type key frame (410) discussed above, except that that the long-term frame can be kept in the active frame window longer than would be done with a frame that is not a long term frame (e.g., a regular IDR-type key frame).
• the IDR frame (410) can be followed by regular predicted frames (430), which can be the same as the regular predicted frames (330) discussed above.
• the frames (410 and 430) can each have a base layer (402) and one or more enhancement layers (404), as discussed above.
• FIG. 4 illustrates a prediction structure (450) with an LIDR- type key frame (412) at the beginning and later with dynamic predicted key frames.
• the enhancement layer (404) of frame 6 and the base layer (402) of frame 9 include lost data (460).
• the coding computer system can respond to each notice of lost data by coding and inserting in the bitstream a long term predicted key frame (420).
• Such a frame can be kept longer than regular frames, as discussed above with reference to the LIDR-type key frame (412). Accordingly, the latest key frame can be retained in the decoding buffer, at least until another key frame is coded and sent.
• the long term predicted key frames (420) can include prediction that can reference other key frames, but that is limited to not referencing regular predicted frames (430).
• each long-term predicted key frame (420) can act as a synchronization predicted frame to allow synchronization of the coding and decoding computer systems after data is lost, which can cut off resulting drift.
• each regular predicted frame (530) can have a base layer (502) that includes a prediction that references a base layer (502) of a previous frame, rather than referencing an enhancement layer (504) of the previous frame.
• the regular prediction structure (500) may be structured according to a coarse grain scalable (CGS) bitstream or a spatial scalable bitstream under the H.264 SVC standard.
• CGS coarse grain scalable
• the bottom of Fig. 5 illustrates a prediction structure (550) with a dynamic anchor predicted frame.
• the enhancement layer (504) of frame 6 and the base layer (502) of frame 9 include lost data (560).
• the coding computer system can code and insert an anchor predicted frame (520) as frame 7.
• the anchor predicted frame can include a base layer (502) with a prediction that references the base layer (502) of frame 6.
• the base layer (502) of the anchor predicted frame (520) can be coded as in the regular predicted frames (520), while the enhancement layer (504) of frame 7 can include a prediction that is limited to only including intra-frame references.
• the anchor predicted frame can include inter-frame prediction referencing the base layer (502) of the previous frame to improve coding efficiency, but can avoid inter- frame prediction referencing the enhancement layer (504) of frame 6 that includes lost data (560).
• the coding computer system can code and insert an ID -type key frame (510) as frame 10 to cut off drift from the lost data (560) in frame 9.
• each technique may be performed in a computer system that includes at least one processor and memory including instructions stored thereon that when executed by at least one processor cause at least one processor to perform the technique (memory stores instructions (e.g., object code), and when processor(s) execute(s) those instructions, processor(s) perform(s) the technique).
• memory stores instructions (e.g., object code), and when processor(s) execute(s) those instructions, processor(s) perform(s) the technique).
• one or more computer-readable storage media may have computer-executable instructions embodied thereon that, when executed by at least one processor, cause at least one processor to perform the technique.
• the technique can include an encoding system encoding (605) and sending (610) a video bitstream over a computer network to a decoding computer system.
• the video bitstream can follow a regular prediction structure when the encoding computer system is not notified of lost data from the bitstream.
• the encoding computer system can receive (620) a notification of lost data in the bitstream.
• the lost data can include at least a portion of a reference frame of the bitstream.
• the encoding computer system can dynamically encode (630) a synchronization predicted frame with a prediction that references one or more other previously-sent frames in the bitstream without referencing the lost data.
• the encoding computer system can insert (640) the synchronization predicted frame in the bitstream in a position where the regular prediction structure would have dictated inserting a different predicted frame with a prediction that would have referenced the lost data according to the regular prediction structure.
• the synchronization predicted frame can include a predicted key frame whose prediction is limited to referencing one or more other key frames prior to the reference frame.
• the prediction of the predicted key frame can reference one or more intra-coded key frames (e.g. IDR frame(s)) and/or one or more predicted key frames.
• the synchronization predicted frame can be a long term predicted key frame whose prediction is limited to referencing one or more other key frames prior to the lost data.
• the long term predicted key frame can reference one or more other long term key frames.
• the different predicted frame that would have been coded and sent under the regular structure may have been a frame that would have referenced an enhancement layer of the reference frame.
• the synchronization predicted frame may be a key frame whose prediction references one or more key frames prior to the lost data.
• the lost data may include at least a portion of the enhancement layer of the reference frame and/or at least a portion of a base layer of the reference frame.
• the base layer can be referenced by prediction of the enhancement layer.
• the lost data can include at least a portion of a lost enhancement layer
• the synchronization predicted frame can include an enhancement layer that references a base layer of the synchronization predicted frame.
• a prediction of the enhancement layer of the synchronization predicted frame can avoid referencing the lost enhancement layer.
• an enhancement layer of the different predicted frame may have been a frame that would have referenced the enhancement layer with at least a portion of the lost data.
• a prediction of the base layer of the synchronization predicted frame may reference the base layer of a frame that includes at least a portion of the lost data.
• the technique can include encoding (705) and sending (710) a video bitstream over a computer network to a decoding computer system.
• the video bitstream can follow a regular prediction structure when the encoding system is not notified of lost data from the bitstream.
• a notification of lost data in the bitstream can be received (720), where the lost data can include at least a portion of a reference frame of the bitstream.
• the regular prediction structure can be dynamically modified (730) by encoding and inserting in the bitstream a synchronization predicted frame having a prediction that does not reference the lost data.
• Inserting the synchronization predicted frame can include inserting the synchronization predicted frame in the bitstream in a position where the regular prediction structure would have dictated inserting a different predicted frame with a prediction that would have referenced the lost data according to the regular prediction structure.
• the different frame can be a frame that would have referenced an enhancement layer of the reference frame.
• the enhancement layer of the reference frame may be a quality enhancement layer or a spatial enhancement layer.
• the lost data may include at least a portion of the enhancement layer, and the prediction of the synchronization predicted frame may reference a base layer below the enhancement layer without referencing the enhancement layer.
• the synchronization predicted frame may include a key frame whose prediction references one or more key frames prior to the lost data in the bitstream.
• the technique can include an encoding computer system encoding (805) and sending (810) a real-time video bitstream over a computer network via a transmission server to a decoding computer system.
• the video bitstream can follow a regular prediction structure between frames when the encoding computer system is not notified of lost data from the bitstream.
• the encoding computer system can receive (820) a notification of lost data in the bitstream.
• the lost data can include at least a portion of a reference frame of the bitstream.
• the encoding computer system can dynamically encode (830) a predicted key frame whose prediction references one or more other key frames in the bitstream that are prior to the reference frame and does not reference the reference frame.
• the predicted key frame can be inserted (840) in the bitstream in a position where the regular prediction structure would have dictated inserting a different predicted frame with a prediction that would have referenced the lost data according to the regular prediction structure.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=49002849

Family Applications (1)

Country Status (6)

Families Citing this family (8)

Citations (5)

Family Cites Families (14)

• 2012
  • 2012-02-29 US US13/407,759 patent/US20130223524A1/en not_active Abandoned
• 2013
  • 2013-02-13 WO PCT/US2013/025800 patent/WO2013130263A1/en not_active Ceased
  • 2013-02-13 CN CN201380011737.7A patent/CN104137543B/zh active Active
  • 2013-02-13 KR KR1020147024169A patent/KR20140136438A/ko not_active Ceased
  • 2013-02-13 EP EP13754476.3A patent/EP2798845B1/en active Active
  • 2013-02-13 JP JP2014559911A patent/JP6191886B2/ja active Active
• 2016
  • 2016-09-07 US US15/259,012 patent/US20160381396A1/en not_active Abandoned

Patent Citations (5)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events