EP0882358A2 - Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique - Google Patents

Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique

Info

Publication number
EP0882358A2
EP0882358A2 EP97908727A EP97908727A EP0882358A2 EP 0882358 A2 EP0882358 A2 EP 0882358A2 EP 97908727 A EP97908727 A EP 97908727A EP 97908727 A EP97908727 A EP 97908727A EP 0882358 A2 EP0882358 A2 EP 0882358A2
Authority
EP
European Patent Office
Prior art keywords
frame
frames
computer
decoder
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP97908727A
Other languages
German (de)
English (en)
Inventor
John A. Viii Toebes
Douglas J. Walker
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SAS Institute Inc
Original Assignee
SAS Institute Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US08/801,254 external-priority patent/US5959690A/en
Application filed by SAS Institute Inc filed Critical SAS Institute Inc
Publication of EP0882358A2 publication Critical patent/EP0882358A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234318—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/102—Programmed access in sequence to addressed parts of tracks of operating record carriers
    • G11B27/105—Programmed access in sequence to addressed parts of tracks of operating record carriers of operating discs
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G11B27/32—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
    • G11B27/327—Table of contents
    • G11B27/329—Table of contents on a disc [VTOC]
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/78—Television signal recording using magnetic recording
    • H04N5/782—Television signal recording using magnetic recording on tape
    • H04N5/783—Adaptations for reproducing at a rate different from the recording rate
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00—Record carriers by type
    • G11B2220/90—Tape-like record carriers
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/022—Electronic editing of analogue information signals, e.g. audio or video signals
    • G11B27/024—Electronic editing of analogue information signals, e.g. audio or video signals on tapes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction

Definitions

  • the present invention relates to the field of digital motion video and more particularly to a system and techniques for altering and decompressing digital motion video signals in a manner which allows efficient reverse play of the motion video as well as efficient, frame-level access and play of the motion video stream for creation of other special video effects.
  • the system and techniques are compatible with the MPEG-1 standard adopted by the International Standards Organization ' s (ISO's) Moving Picture Experts Group (MPEG), however the invention taught herein may also be applied to other video coding algorithms which share some of the features of the MPEG algorithm, such as Intel Corporation ' s IndeoTM and Indeo Video Interactive algorithms, the Fractal Codec algorithm from Iterated Systems of Atlanta. Georgia. MVI from Sirius Publishing of Scottsdale. Arizona. Cinepak from Radius of Sunnyvale. California and the Smacker 2.0 algorithm from RAD Software of Salt Lake City, Utah.
  • the present invention relates generally to the field of digital video and more specifically to the coding and compression of analog video signals into digital video and the decoding and decompression of the digital bitstream into a displayable video signal.
  • Digital video compression is used in a variety of applications where video images are displayed in a system where available bandwidth is limited, such as video telephone, digital television and interactive multimedia using such digital storage technology as CD-ROM. digital audio tape and magnetic disk. Such applications require digital video coding, or video compression to achieve the necessary high data transfer rates over relatively low bandwidth channels.
  • This invention relates most specifically to those digital video applications where the user interacts with the system in ways which can modify the video display, such as in interactive computer games or other interactive multimedia applications.
  • digital video systems such as MPEG video players in personal computers or video game machines would benefit from use of the apparatus and methods of the present invention to allow more efficient and realistic navigation through a video world, creation of special effects, frame specific search and access to a video stream and reverse playback of a video stream.
  • the ISO ' s MPEG-1 algorithm is designed to yield a true TV-like image with compression ratios around 180: 1 at data rates low enough for use in storage applications with data transfer rates at or below 1.5 Mb/s (megabits/sec), comparable to those used on CD-ROM drives on personal computers. While the algorithm is designed for such data rates, it is usable at higher data rates. The inventor routinely uses data rates of 2 to 2.5 Mb/s.
  • MPEG-1 is designed to work with images having a one-fourth of broadcast-quality resolution: 352 by 240 pels. This is approximately the quality of a picture presented by standard VHS video cassettes.
  • An MPEG-1 stream may consist of 0 to 16 separate video streams, 0 to 32 separate audio streams, any of which may be in stereo, and possibly other customized streams carrying user-specified information and padding bytes.
  • the various streams are multiplexed into a single MPEG composite stream called a "system stream. " This invention relates to the manipulation of an MPEG-1 video stream. It also relates to ways of de-multiplexing the system stream to create an actual or virtual non-multiplexed, valid MPEG stream.
  • the further aspects of the MPEG video standard including the other data streams which comprise the MPEG system stream, are well known in the art, are extensively discussed in the literature, including International Standard ISO IEC 11172-2, entitled “Information technology — Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s -- Part 2: Video” dated August 1. 1993 and will not be further discussed here.
  • ISO IEC 11172-2 entitled "Information technology — Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s -- Part 2: Video” dated August 1. 1993 and will not be further discussed here.
  • a single video stream consists of a sequence of pictures. These pictures are also referred to as "frames.”
  • the MPEG video stream is normally created by subjecting video data representing a video picture or frame to several digital compression steps.
  • the MPEG-1 encoding scheme includes intra-frame compression which seeks to reduce redundancies within a frame and inter-frame compression which uses motion compensation to identify and eliminate redundancies between sequential frames. Motion compensation takes advantage of the movement of picture elements that remain approximately the same within a series of sequential frames but change position from frame to frame.
  • I-frames Intra
  • P-frames Predictive
  • B- frames Bidirectional
  • a fourth type of frame, the D-frame. is defined in the standard but is intended for use only as an indexing and overview feature and cannot be mixed with I.
  • B, and P frames I and P frames are collectively called reference frames since other frames can be based on them.
  • I frames contain all of the information needed to reconstruct one frame of video.
  • P- frames can use information from the previously displayed reference frame and can add new information.
  • B frames can use information from either the previously displayed reference frame, the next reference frame that will be displayed, or both, and can also add new information.
  • bitstream order is optimized to provide the necessary reference pictures at the appropriate time to allow efficient parsing and decoding when the stream is played forward. removing the need to back up or skip forward to display the stream. Because MPEG is optimized for forward play, backward play is especially challenging. Further, before the
  • SUBSTTTUTE SHEET (RULE 26) current invention efficient backward play, that is backward play of acceptable speed and quality, was not obtainable on a machine with only the memory resources required for acceptable forward play.
  • the MPEG standard also defines the concept of a Group of Pictures (GOP).
  • Each GOP contains at least one I-frame and may contain additional I. B, and P frames. There is no limit on the size of a GOP.
  • the GOP may begin in display order with one or more B-frames that refer to the last reference frame in the previous GOP. but no GOP may end with a frame that refers to the next GOP in the display order.
  • Each GOP begins with a header which contains parameters to assist in decoding the video stream. While such parameters can be different for every GOP in the stream, they typically are the same for all GOPs in a stream.
  • a sequence is a sequential group of GOPs in an MPEG stream. Each sequence begins with a sequence header which contains parameters which may be used to assist in decoding the sequence.
  • the video data is broken down into a luminance or Y component and two color difference components.
  • Cr red chrominance
  • Cb blue chrominance
  • the individual pictures can be represented as arrays of Y, Cr and Cb values.
  • the Cr and Cb values are subsampled with respect to the Y values by 2:1 in both the horizontal and vertical directions, therefore there is one Cr and one Cb value for each four Y values.
  • the Y component is represented by four 8x8 contiguous blocks for each 16x16 macroblock.
  • the Cr and Cb components are represented by a single 8x8 block for each component, but due to the subsampling discussed above the Cb 8x8 block and the Cr 8x8 block each cover the same area of the screen as the four 8x8 Y blocks.
  • the macroblock. therefore, consists of six 8x8 blocks, each limited to one component and all superimposed in the 16x16 pel area ofthe display covered by the four 8x8 Y blocks.
  • the six blocks comprising the macroblock are each subjected to a Discrete Cosine Transformation (DCT) algorithm that transforms them losslessly into 8x8 matrices that represent on each axis increasing horizontal and vertical frequency. Further compression steps take place to reduce the range of the values and encode them using a Huffman-type compression algorithm, but the specific algorithms used in the further compression steps are not relevant to the present invention. All macroblocks in an I picture are intra-coded. This means that all their DCT coefficients are encoded directly into the bitstream with no references to other pictures.
  • DCT Discrete Cosine Transformation
  • Each macroblock in a P picture may or may not have their DCT coefficients directly coded (herein referred to as "intra-coded” information), and each may or may not have a reference to a 16x16 pel area in the most recently displayed reference frame (such references are referred to as "motion vectors " in the ISO/IEC standard 1 1 172-2 or as "inter- " coded information). If both motion vector information and intra-coded information are present, the values are added. Either intra-coded information or an inter-coded reference to the next reference frame must be supplied for each macroblock.
  • Each macroblock in a B picture may or may not have intra-coded information, may or may not have a reference to a 16x16 pel area in the most recently displayed reference frame, and may or may not have a reference to a 16x16 pel area in the next reference frame that will be displayed. If references to both the previous and next reference frames arc present, the values in the two frames are averaged and added to the intra-coded information, if any. Either intra-coded information or an inter-coded reference, either to the next reference frame, the previous reference frame or both the next and previous reference frames, must be available for each macroblock. although the standard allows the information to be inherited from previous macroblocks in some cases.
  • the MPEG player constantly keeps the last two reference frames available for use in decoding B and P frames when they appear.
  • the first reference frame decoded is placed in the future buffer.
  • bitstream order When a new reference frame is encountered in the decoder ' s parsing of the bitstream (bitstream order), the previous "future " frame becomes the "past” frame and is normally displayed at that time.
  • the new reference frame is read into the future buffer and becomes the future frame.
  • These available reference frames are known as the "past” and “future” frames or pictures and are normally kept in portions of the computer or decoder memory known respectively as the '"past" and "future " buffers.
  • P frames may refer to past reference frames
  • B frames may refer to past and/or future reference frames.
  • the appropriate reference frames must be in the appropriate buffers of the MPEG player for the P and B frames to be properly decoded.
  • the MPEG bitstream is designed so that the proper frames will always be in the appropriate buffers when dependent frames are presented for decoding. If the past and future buffers contain the correct values and the MPEG decoder decodes the B picture, the correct picture will be displayed on the screen. However., the contents of these buffers change frequently during normal play. This makes it difficult to play a dependent frame except in the original linear video order.
  • the terms '"parsed" and "decoded" are virtually synonymous.
  • parsing or decoding may refer to only one or less than all steps necessary for complete decoding of the stream information.
  • a GOP. picture, or portion of a picture may be completely or incompletely parsed for reasons other than display.
  • the terms "parse " or "decode " may refer only to preliminary steps in the decoding process, such as those steps necessary to determine whether a certain picture is an I. P or B frame, or may refer to the entire process of decoding the picture and displaying the resultant bitmap.
  • frame and picture are also virtually synonymous. They both refer to a single video picture, whether or not it is coded.
  • Frame accurate access to the video stream is not necessary for broadcast, satellite or cable video programming applications. However, it is desirable for many other uses of MPEG. particularly in interactive, multimedia computer applications such as computer games. It would be desirable to use MPEG video "worlds" in interactive educational and game programs.
  • the MPEG standard has been designed primarily to support normal, forward linear playback of a digital video stream in display order. However, the standard also refers to possible additional operations including random access, fast search, reverse playback, error recovery, and editing.
  • the MPEG standard also mentions the possibility of reverse playback.
  • Reverse playback poses particular problems because of the directionality enforced by the MPEG standard in encoding groups of pictures. Only I frames can be individually accessed and decoded. Neither B nor P pictures contain sufficient information to generate a complete frame without reference to previous (bitstream order) pictures.
  • an MPEG stream has directionality and is incomprehensible if read backwards bit by bit. Further, the bitstream order of an MPEG stream has a definite directionality on the Group of Pictures level as well. Consequently, only reverse play of I frames can be achieved by simply reading the frames into the decoder in either reverse display order or in reverse bitstream order.
  • the MPEG standard suggests performing reverse playback by decoding GOP's in the ordinary fashion, storing the decoded bitmaps in a memory buffer and then displaying the bitmaps in reverse order. While this method results in a reverse playback with equal quality to the forward playback, by requiring storing of decoded pictures before playback, it places significant greater demands on computing resources, particularly memory resources than does forward MPEG.
  • Another method is to decode only the I frames in each group of pictures. While this method eliminates the bit map buffer requirement, it results in either loss of temporal resolution (where there is a significant number of B and P pictures skipped) or loss of compression (where the original video sequence is coded primarily in I frames to allow for smoother reverse playback).
  • Another method of creating a similar effect would be to avoid reverse play of an MPEG stream by storing "forward" and "backward " video contents in standard unidirectional MPEG streams to simulate reverse play by having the run-time system switch from the appropriate forward stream to the corresponding reverse stream when the "reverse " command is given by the user. While eliminating the need for memory resources which the bit map storage method requires and eliminating the loss of temporal resolution which the I frame only method may involve, such a system would double the storage requirements for the video information files which are to be made available for forward and reverse play. Further, such a solution would require limitation of the points along the video stream where a reverse command could be executed and or complete synchronization of the forward and reverse MPEG streams. Such a solution would also require a seek to the reverse stream every time a reverse command is given, slowing down navigation of the video world.
  • a method and system according to the present invention comprises a self-contained interactive multimedia computer system, such as a Microsoft Windows compatible personal computer with a 90 MHz Pentium processor.
  • SVGA video display and software or hardware MPEG decoder which is pre-programmed to allow the user to access MPEG video streams on a frame specific basis, play such video streams in the reverse direction, and create or play meaningful MPEG transitions which can be played while the system is seeking the "to" video stream of the transition, and to construct, edit and navigate multimedia video based environments and applications based thereon having such capabilities.
  • FIG. 1 is a block diagram of a video system for displaying video images in a PC environment, according to a preferred embodiment ofthe present invention:
  • FIG. 2 is a block diagram of a system, including an MPEG streamer, according to a preferred embodiment ofthe present invention:
  • FIG. 3 is an example of a video stream index according to a preferred embodiment of the present invention.
  • FIG. 4a is an example of a video stream shown in display order
  • FIG. 4b is an example ofthe video stream of FIG. 4a shown in bitstream order
  • FIG. 5 is a process flow diagram showing the process of a preferred embodiment ofthe present invention for frame specific access to a B frame in the video stream of FIG. 4.
  • FIG. 6 is a process flow diagram showing the process of a preferred embodiment ofthe present invention for frame specific access to a P frame in the video stream of FIG. 4.
  • FIG. 7 is a process flow diagram showing the process of a preferred embodiment ofthe present invention for frame specific access to an I frame in the video stream of FIG. 4.
  • FIG. 8 is a generalized flow diagram showing the process of frame specific access to a specific frame.
  • FIG. 9 is a process flow diagram showing the process of a preferred embodiment of the present invention for reverse play ofthe video stream of FIG. 4.
  • FIG. 10 is a process flow diagram showing the process of a preferred embodiment of the invention for transition from one video stream to another.
  • FIG. 11 and its parts 11a through Ilk is an illustration of a transition according to the invention.
  • FIG. 12 is an illustration of the appearance of successive displays during a push right transition.
  • FIG. 13 is a process flow diagram showing the process of a preferred embodiment of a turn transition, incorporating panning and composite pictures according to the invention.
  • FIG. 14 is an illustration of the relationship of panning the display over the FROM picture, composite picture and TO picture during a right turn transition according to the invention.
  • the present invention is directed to providing an apparatus and method for random playback and reverse playback of digitally compressed video bitstreams without loss of quality or temporal resolution while minimizing the impact of such playback on processing time and memory resources and to providing meaningful transitions between different video streams or video frames.
  • random playback we mean that any frame in a video stream may be the first in that stream to be displayed. Once such random access is achieved, the invention allows for normal forward play to continue from the randomly accessed frame or for reverse playback to begin from such frame.
  • reverse playback we mean the display of an MPEG video stream in reverse display order beginning at any selected frame within the video stream.
  • the GOP and Sequence header portions of the MPEG video stream are ignored and the MPEG player is set to use the default setting for all quantization matrices. It is a trivial matter to cause the MPEG player to load quantization matrices in the parsing process should this be desired in a particular application of the invention.
  • the sequence header portion ofthe MPEG video stream can be used to adjust the bit rate, in practice available decoders, ignore this parameter. The bit rate is not an issue as long as it is within the decoder ' s upper bound of bit rate handling capacity. Consequently, although it would be a trivial matter to have the MPEG streamer of the present invention adjust the bit rate parameter in the sequence header, the preferred embodiment, as do currently available MPEG players, ignores bit rate information in the sequence header.
  • the preferred embodiment of the present invention does not use the temporal reference field of the MPEG stream.
  • available decoders do not use temporal reference data.
  • SUBSTTTUTE SHEET (RULE 26) the temporal reference field, should it be desired to use the invention in combination with an MPEG player which uses this information.
  • the present invention may comprise either a hardware or software MPEG player.
  • Hardware players use special video processing chips, either on the computer's motherboard. or more commonly, on a special video card in the computer. Typically these chips speed up the process of decoding the MPEG video and/or the process of drawing the decoded video on the screen.
  • Software players do most of the work with program logic, and use only conventional computer hardware to display the decoded video.
  • the method and apparatus ofthe present invention preferably requires an MPEG player capable of parsing certain arbitrary sections of the bitstream without updating the display to display the resulting pictures on the screen, leaving the previously displayed picture on the screen and without playing any audio associated with the sections parsed.
  • an MPEG player where updating the display and audio is suppressed as playing the MPEG player in "suppression mode.”
  • the invention also preferably requires use of an MPEG player capable of accessing MPEG data stored in a memory buffer instead of merely the MPEG file. In the case of hardware or combination hardware/software decoders, this ability is usually a function of the player ' s software drivers, rather than a limitation of the hardware itself.
  • the computer system inco ⁇ orates a software MPEG decoder, such as SoftMotion from SAS Institute Inc. of Cary. North Carolina.
  • FIG. 1 there is shown a block diagram of a video system comprised of a multimedia computer system 10 which could be employed to implement the present invention.
  • a computer system 10 is shown having a Pentium® 90 MHz CPU based computer 1 with a 540MB internal hard drive, a CD ROM drive 8. a disk drive 9. a SVGA monitor 2. speakers 3. a keyboard 6. and a pointing device 7.
  • the computer could be connected to a local area network and/or modem for accessing resources not located within the computer ' s local drives.
  • the central component in such a system is the MPEG Streamer 23.
  • the MPEG Streamer can retrieve the video stream from an MPEG disk file 11. usually contained on a CD-ROM disk in a CD-ROM drive 8 as shown in FIG. 1.
  • As the data on the CD-ROM disk file is multiplexed with audio data and other data into a system stream 13.
  • the data must first be de-multiplexed by a de-multiplexer 15 in order for the video stream data 21 to be separated from the other types of data in the system stream.
  • the video stream data fed to the MPEG Streamer 23 also includes video indices which are placed by the streamer in the appropriate buffer 33, "TO" frames placed by the streamer in the "TO" frame buffer 27 and pre-constructed synthetic MPEG transition sequences which are placed in the transition buffer 29.
  • the system may include a transition generator 31 which generates the desired synthetic MPEG for transitions in response to the request for such from the MPEG streamer 23.
  • the MPEG streamer function is an important component of the preferred embodiment of the invention. It essentially constructs a continuous virtual MPEG stream out of the source MPEG to send to the decoder. As far as the MPEG player is concerned, the MPEG stream never ends and no seeking is ever requested. Consequently, the MPEG decoder never needs to initialize as it otherwise would have to do every time a seek is needed. Decoder initialization sequences may cause the skipping of some frames of a stream and will cause a delay as the player decodes initial frames before beginning to display video.
  • the streamer manufactures the continuous MPEG stream by assembling MPEG data from selected segments of the MPEG stream(s), and from transition data. In effect, the streamer does the seeking required outside of the MPEG player. When a seek is done, the streamer joins the new MPEG frames to the old ones so that the MPEG player sees what looks like a single MPEG stream with no interruptions. The streamer also injects the "synthetic " MPEG frames generated during transitions into its output stream.
  • the MPEG streamer accepts manual inputs as well as responding to positional data contained within the program to seek and access the appropriate video data to send to the MPEG player 37.
  • the video stream 39 sent to the MPEG player is referred to as the virtual video stream because the system may contain frames which are synthesized by the MPEG streamer and inserted into a "video stream" according to the invention, although in reality no preexisting stream with these characteristics exists.
  • the MPEG streamer 23 usually a software entity, is constructed so that, in conjunction with a de-multiplexer 15. it reads a base MPEG video stream, such as that on a video disc, and constructs a second valid MPEG stream 39 derived from the base stream and. possibly, other sources for the MPEG player to play.
  • Preferably such MPEG streamer creates an actual or virtual non-multiplexed MPEG video stream capable of being manipulated according to the method of the invention.
  • Such component or software entity may also include the ability to accept user input 25 and execute various seeking transactions as described herein to queue other data for "streaming " into the MPEG player.
  • the method and apparatus ofthe present invention uses the MPEG streamer ' s ability to construct a derived stream from various components and to manipulate the derived stream according to the invention to accomplish frame accurate access, reverse playback and other special effects.
  • the streamer has the ability to duplicate, omit or reorder pictures present in the base video stream and the ability to insert pictures into the video stream from other sources.
  • the MPEG player does not make use of the temporal reference field described in the MPEG standard and available in the MPEG stream. If the MPEG player on which the invention is to be practiced does use the temporal reference field, the MPEG streamer must contain means for adjusting the temporal reference data in the MPEG stream it creates to have the correct temporal reference data.
  • the functionality of the MPEG streamer used in the preferred embodiment ofthe invention is shown in the annotated source code for the interface definition of such a streamer attached hereto as Attachment 1 and incorporated herein by reference.
  • the method preferably creates an index of the MPEG stream to provide more rapid searching of said stream.
  • index preferably is an array of video stream offset numbers in bitstream order which indicates at what byte each picture starts and whether the picture is an I. P or B picture.
  • the index should also contain flags indicating whether the picture is the last picture in a GOP (in bitstream order) or the last picture in a sequence (in bitstream order). These last two flags allow the streamer to locate, read and parse the appropriate GOP and sequence headers where these are important in placing the player in the appropriate state.
  • index is preferably created for each MPEG stream which will be present in a product incorporating the invention during the creation of the product rather than being created during play of the product.
  • Such pre-constructed indices are loaded into buffer when appropriate during runtime.
  • the index may be stored in a separate file or in the file containing the video stream. If it is stored with the video stream, it may be stored as a composite stream, as user data, in the fields of the video stream (e.g., GOP timestamp. temporal reference, vbv delay, etc.). as a non-standard extension of the video data, or as a single chunk at a known location within the video stream.
  • FIG. 3 shows an example of an index according to the invention.
  • the headings are provided for illustration and convenience and are not necessarily present in the software index entity.
  • the first column in FIG. 3 indicates the type of frame of all of the frames in the MPEG stream indexed, in bitstream order.
  • the second column indicates whether the frame is the last frame in a GOP.
  • the third column indicates the offset byte number showing the location ofthe frame in the bitstream.
  • the index refers to the de-multiplexed MPEG stream offset number rather than the multiplexed system stream offset number.
  • the fourth column indicates whether the frame is the last frame in a sequence.
  • an MPEG player with the capability of suppressing display of a parsed MPEG stream while continuing to display the previously displayed picture and with the capability of generating a derived MPEG video stream can be made to exactly reproduce the state ofthe MPEG player when playing any given frame of video.
  • the decoder In normal forward play of an MPEG video stream, the decoder retains information from the most recently displayed reference frame and the next reference frame to be displayed in its past and future buffers respectively. It also retains information from the most recently
  • FIG. 4a and FIG. 4b illustrate typical frame sequences in an MPEG stream.
  • FIG. 4a shows the sequence in display order.
  • FIG. 4b shows the same sequence in bitstream order. Notice that the reference frames required to decode the intermediately displayed B frames appear before the intermediate B frames in bitstream order. In display order the intermediate B frames appear between the past and future reference frames on which they depend for complete decoding.
  • each P picture depends on the preceding I or P picture in bitstream order.
  • that reference picture in turn depends on an earlier reference frame (either P or I).
  • Each B frame depends on the previous two I or P pictures in bitstream order. If these necessary' reference pictures are not completely parsed and present in the buffers, the decoder is not in the proper state to completely decode the dependent frames and the P or B pictures which depend on the reference pictures cannot be displayed correctly.
  • the relevant GOP header and sequence header information must also be presented to the player before the frames depending on it are parsed. If all of this "state" information is not correct the player will not properly decode the B frames and P frames which occur later in bitstream order. No doubt due to this sequential dependence, the MPEG standard only suggests how to access I frames which can be read and displayed without reference to any other frames.
  • the MPEG player must be capable of accepting MPEG data from a memory buffer instead of merely from a file, such as found on a video disk or CD. Further, the MPEG player must be capable of executing instructions, according to the invention, requiring the player to parse certain arbitrary sections of the bitstream without updating the display to display the resulting pictures on the screen, leaving the previously displayed picture on the screen and without playing any audio associated with the sections
  • SUBSTTTUTE SHEET (RULE 26) parsed.
  • operation of an MPEG player where updating the display is suppressed is defined as operation of the player in "suppression mode.”
  • the MPEG player when operating in the suppression mode the MPEG player ignores the nominal picture rate ofthe stream and decodes the frames presented as quickly as possible.
  • Many existing hardware players are capable of executing instructions according to the invention and operating in suppression mode.
  • An example of a hardware player which is capable of executing the process of the invention with its current drivers is the Jakarta MPEG Video Graphics Accelerator sold by jazz Multimedia. Santa Clara. California. Applicants know of only one software player having the required capabilities — the "SoftMotion" MPEG player distributed by SAS Institute of Cary. North Carolina. While there has been little need for players able to perform this function prior to the current invention, the knowledge required to create such a player is well known in the art and will not be further discussed herein.
  • any frame in a bitstream will be properly parsed and displayed if the decoder is in the proper state to parse and display it.
  • the bitstream order of the frames, GOP headers and sequence headers automatically place the player in the proper state.
  • the method ofthe invention accomplishes the task of placing the MPEG player in the proper state to display an arbitrary "target" frame by using a previously constructed MPEG stream index as described above and illustrated in FIG. 3. While such data could easily be obtained from the MPEG video stream or from an array constructed during runtime, the use of a preconstructed array reduces processing time.
  • the index is analyzed to determine which 1 and P frames must be parsed prior to the parsing of the target frame in order to place the player in the proper state for parsing the target frame. Once this is determined, the player is directed to parse these I and P frames while in suppression mode. Then the picture display and audio are enabled. Finally, the MPEG player is directed to begin playing in the ordinary forward mode at the target location.
  • FIG. 4a The top row of FIG. 4a shows the picture type, either I. B or P.
  • the second row provides the picture display order.
  • FIG. 4b shows the same video stream in bitstream order.
  • each P picture depends on the preceding I or P picture in bitstream order or in display order. Therefore, in normal play I and P pictures are always parsed and displayed in the same order. Thus the position of I and P pictures relative to each other does not change from display order to bitstream order.
  • the parsing includes references to the preceding P frame or I frame which is present in the past buffer. Any B frames preceding the I or P frame and depending on it are then parsed and displayed. The I or P frame is then also displayed as it is read into the past buffer. The next I or P frame is then parsed and read into the forward buffer. The process then repeats itself through the entire video stream.
  • the necessary reference frames In order to provide frame specific access to an arbitrary target frame other than an I frame, the necessary reference frames must be properly parsed and residing in the appropriate buffer in order for the player to be in the proper state to parse the target frame. If the appropriate frames are not parsed and read into the appropriate player buffers, the P or B picture cannot be parsed and displayed correctly. Where the target frame is an I frame the display still must be disabled while seeking to the appropriate frame. Further, as I and P frames
  • SUBSTTTUTE SHEET (RULE 26) are not displayed until they are replaced in the future buffer by the next reference frame, the next reference frame must be placed in the future buffer in order to display the target frame.
  • the MPEG streamer After it decides the list of frames which must be parsed in order to parse the target frame, the MPEG streamer must construct a virtual MPEG stream out of the correct components of the original stream.
  • the virtual stream will be sent to the MPEG player to reproduce the correct system state.
  • MPEG player state information can be modified by MPEG sequence headers and by MPEG GOP headers.
  • the streamer performs the following steps:
  • sequence header and/or GOP header For each frame that is on the list to be parsed, determine which sequence header and which GOP header apply to that frame. If the sequence header and/or GOP header of the first picture are different from the last ones the player decoded, the different header must be copied in before the data for the first picture it applies to. For all pictures after the first, the sequence header and/or GOP header must be inserted if it is different from the one that applies to the preceding picture in the virtual stream.
  • Example 1 Frame specific access to a B frame
  • FIG. 5 shows the preferred process of the current invention used to access the video stream of FIG. 4 at picture 8.
  • the first step in the process illustrated in FIG. 5 is to analyze the bitstream.
  • the identity ofthe target frame is determined preferably by using the index.
  • picture 8 is a B frame.
  • the streamer determines that the target frame is a B frame.
  • SUBSTTTUTE SHEET (RULE 26) the index is further analyzed to determine the location in the video bitstream of the reference frames on which the B frame depends.
  • the target frame number 8 depends on the two preceding I or P pictures in bitstream order. In the example, these pictures are pictures 9 and 6. Thus, pictures 9 and 6 must be parsed correctly and resident in the appropriate buffers for frame 8 to be parsed. Also since picture 6 is a P frame, we must first parse the preceding I frame to correctly parse picture 6. Therefore, we must parse pictures 0. 3. 6, and 9 before attempting to parse and play picture 8.
  • the second step in the FIG. 5 process is to place the player in suppression mode.
  • the MPEG streamer creates a virtual video stream containing these reference frames pictures 0, 3, 6 and 9 in the proper order and sends them to the player for parsing.
  • GOP header and sequence header information was important for accurate parsing, these headers would also be sent to the decoder at the proper time.
  • the streamer then skips to the beginning of picture 8. Note that this sequence does not require skipping backward.
  • the fourth step in the process is to take the player out of suppression mode, re-enabling the picture display and audio.
  • the MPEG player begins normal play at the current file position.
  • FIG. 6 illustrates the preferred process for frame specific access to a reference frame (I or P frame). Notice that the process is slightly different than that for a B frame. One reason for the difference is that some of the B frames which would appear before the reference frame in display order occur after the reference picture in bitstream order. Thus "normal" play cannot be achieved immediately after parsing the desired reference frame.
  • the process of the invention uses the analysis obtained from the index to direct the player to skip these dependent B pictures which follow the target reference frame in bitstream order.
  • FIG. 6 illustrates the preferred process of the invention for frame specific access to picture 6:
  • the first step is to analyze the bitstream.
  • this step is accomplished at greater speed by referring to the preconstructed index.
  • picture 6. the target frame is a P frame, so it depends on the preceding I or P frame in bitstream order. Therefore picture 3 must be parsed correctly in order for picture 6 to be properly parsed.
  • picture 3 is also a P frame requiring picture 0
  • picture 0 must also be parsed. Since picture 6 is a reference frame, we will need to parse it before re-enabling the display.
  • the reference picture following the target frame is picture 9.
  • the second step is to place the player in suppression mode.
  • the third step is to parse pictures 0. 3 and 6 while the player is in suppression mode. then to seek to the beginning of picture 9.
  • the fourth step is to take the player out of suppression mode by re-enabling picture display and audio.
  • FIG. 7 illustrates the preferred embodiment of the present invention to access an I frame. Using the same video stream shown in FIG. 4 as do the previous examples to illustrate the steps required to access picture 0 (an I frame).
  • the first step is to use the pre-constructed index to analyze the bitstream.
  • Picture 0 is an I frame, so it does not depend on other frames.
  • the preferred embodiment uses an MPEG stream which does not convey state information to the player using sequence headers or GOP headers, the steps related to such headers, discussed above, do not need to be performed.
  • the second step is to place the player in suppression mode.
  • SUBSTTTUTE SHEET (RULE 26)
  • the third step is to parse picture 0. then seek to the beginning of picture 3 while the player remains in the suppression mode. In this particular stream, this action does nothing since pictures 0 and 3 are adjacent in bitstream order, but in other streams there may be B frames that are displayed before picture 0 that occur after picture 0 in the bitstream. In such a case the streamer would seek to the beginning of picture 3.
  • the fourth step is to take the player out of suppression mode by re-enabling the picture display and audio.
  • the last step is to begin normal play at the current file position. While the process actually carried out to perform frame specific access according to the invention varies depending on the type of picture sought, the logic is consistent. This logic is shown on the flow chart of FIG. 8.
  • the overall process is initiated in response to user input or program instruction.
  • user input or program instruction For example, in a computer game the user may "click" on a particular area of the screen within the current video frame being displayed. A click in this area may direct the program to begin playing a video stream at a particular location within the stream.
  • This instruction will then cause the system to select a particular target frame in the video stream to be accessed (either the same video stream or another video stream).
  • the system also puts the player in suppression mode and analyzes the target frame context.
  • the system accesses a preconstructed index such as that illustrated in FIG. 3. This index allows the system to determine the location ofthe target frame within the target video stream, whether the target frame is I. P or B and.
  • FIG. 8 shows, once the target frame type and location is identified, the process branches into three alternatives, which instruct the system how to get the MPEG player in the proper state for the three possible types of target frames.
  • the specific instructions are then carried out as described above and in FIG. 8, using, in the preferred embodiment, the MPEG streamer to locate and read the appropriate frames into the streamer buffer in the appropriate order to be fed by the streamer to the player while the player is in suppression mode.
  • the display and audio are enabled and the system begins normal play at its current position.
  • FIG. 8 illustrate.
  • SUBSTTTUTE SHEET (RULE 26) the bitstream position varies depending on the type of target frame.
  • the MPEG streamer is directed to skip or seek to the beginning of the target B frame once the preceding (in bitstream order) reference frames are parsed. Then the display is enabled and normal play is began. As B frames are displayed immediately as parsed, the B frame will be parsed and displayed immediately upon resumption of normal play. The appropriate I and P frames necessary to place the player in the appropriate state to accurately parse the B frame will have already been parsed into the appropriate buffers so that, once normal play is resumed, the B frame will be displayed as it is parsed.
  • reference frames either I or P. will not be displayed when initially parsed. Rather, reference frames are first parsed into the future buffer where they are used, along with the reference frame in the past buffer, to decode any intermediate B frames. The reference frame is not displayed until another reference frame is parsed into the future buffer, forcing the parsed reference frame into the past buffer and simultaneously displaying it.
  • the current invention takes advantage of this characteristic of MPEG players. Recall that P frames are dependent on the reference frame in the past buffer for accurate decoding. If the frame in the past buffer is a P frame, it in turn is dependent on any previous P frames between it and the closest previous I frame.
  • the MPEG streamer is directed to skip or seek to the most recent previous 1 frame which it sends to the player, then it sends in normal order all P frames between the I frame and the target P frame.
  • the streamer/player is not capable of skipping, all intermediate frames can be parsed while the player is in suppression mode as the streamer/player combination seeks toward the target frame.
  • the target P frame is then parsed into the future buffer and the player/streamer instructed to skip or seek to the next reference frame.
  • the display is enabled and normal play is resumed at this position.
  • the next reference frame is parsed into the future buffer, forcing the target frame into the past buffer and simultaneously displaying it.
  • the target frame is an I frame, which can be completely parsed without reference to other frames, we do not need to be concerned with the state of the MPEG player's past buffer.
  • the process of the invention merely assures that the I frame is parsed into the future buffer and the streamer/player is poised to parse the next reference frame upon enablement of the display and resumption of normal play.
  • the parsing of the next reference frame upon resumption of normal play will force the target I frame into the past buffer and simultaneously cause it to be displayed.
  • ISO 1 1 172-2 is the document defining the MPEG video standard. Appendix D. section 6.7 of this document contains a brief discussion of "Coding at lower picture rates.”
  • the standard recommends the use of a B or P picture inserted between other pictures in the stream to duplicate the previously displayed picture. Such B or P pictures would contain only zero motion vectors to the past reference frame, which would be in the player ' s past buffer. Such pictures essentially duplicate the picture in the past buffer in its entirety.
  • This previously disclosed technique can only be used to duplicate reference frames, not B pictures.
  • Appendix D also suggests encoding a stream at a lower temporal resolution (i.e. fewer pictures per second) and then "padding" the stream with inserted B or P pictures. However, such a stream must be encoded with all reference frames for such padding to work. This is the only use of such duplicator frames in the prior art.
  • the current invention uses duplicator frames in a novel and non-obvious way to facilitate reverse play and other special effects.
  • one technique used in the present invention is the creation of a B picture by the MPEG streamer that contains only zcro-motion- vector references to either the past or the future reference picture.
  • Such a past or future duplicator B picture effectively reproduces the picture in the past or future buffer in its entirety.
  • Such B pictures consist ofthe same bit sequence regardless of the contents of the picture being duplicated. Consequently the same past or future duplicator bit sequence can be used over and over again regardless ofthe content ofthe pictures being duplicated.
  • Duplicator B pictures which reproduce the picture in the past buffer are called past duplicators.
  • Those which reproduce the picture in the future buffer are called future duplicators.
  • the present invention makes use of the uniform property of such duplicators to create novel video effects, such as efficient reverse play.
  • SUBSTTTUTE SHEET (RULE 26) While this avoids the need for additional memory requirements which prior art reverse play methods require, it does require a great deal of extra processing overhead. If the frame to be displayed is near the end of a long GOP. the player may have to parse and suppress the display of many pictures to get the desired picture on the screen. Reverse MPEG can be accomplished with less processing overhead by the following methodology:
  • next frame to be displayed in reverse display order is another B frame
  • the past and future buffers are already set up correctly. Parse and display the B frame, then repeat this step for the next frame to be displayed.
  • step 3 If the frame to be displayed is a P frame or an I frame, skip to the beginning of the GOP and parse but do not display all P and I frames up to and including the frame to be displayed. Then generate a future duplicator B frame, parse it and display it. Go back to step 2.
  • FIG. 9 illustrates the process of a preferred embodiment of the invention for playing the FIG. 4 stream in reverse order.
  • the first step is to analyze the bitstream to be displayed in reverse order using the index.
  • the analysis reveals that the first frame to be played in reverse order is picture 9, a P frame.
  • the analysis also reveals that in order to place the player in the proper state to parse picture 9 the pictures on which it depends. 0. 3, and 6 must first be parsed.
  • the second step is to place the player in suppression mode and to parse pictures 0. 3. 6 and 9 while the player remains in suppression mode. This puts frame 6 in the past buffer and picture 9 in the future buffer.
  • the third step is to take the player out of suppression mode by re-enabling the display and audio.
  • the fourth step is to create, or retrieve from a buffer, a future duplicator B picture, parse it. and display it. This has the effect of displaying frame 9.
  • the fifth step is to parse and display pictures 8 and 7. which the analysis revealed to be B frames which depend on picture 9. .
  • the sixth step is to place the player back in suppression mode and to parse pictures 0. 3, and 6 while the player is in suppression mode. This places the player in the proper state to decode pictures 5 and 4 which are B frames but does not display picture 6. a P frame which remains in the future buffer.
  • the seventh step is performed.
  • the seventh step is to create a future duplicator B frame and display it. This has the effect of displaying an exact duplicate of picture 6.
  • the eighth step is to parse and display pictures 5 and 4. now that picture 6 has been displayed by duplication, although it remains in the future buffer where its presence is required to properly decode B frame pictures 5 and 4.
  • the ninth step is to place the player back in suppression mode and to parse pictures 0 and 3. This places the player in the proper state to decode pictures 1 and 2 which are B frames but does not display picture 3.
  • step 10 is performed.
  • Step 10 is the construction or retrieval from buffer, a future duplicator B frame which is then parsed and displayed.
  • the eleventh step is the parsing and displaying of pictures 2 and 1.
  • the twelfth step is the parsing of picture 0. However, as picture 0 is an I frame, it will not be displayed unless another reference frame is decoded to display it as it moves from the future buffer to the past buffer.
  • the thirteenth and final step is to display the contents of picture 0 by creating or retrieving a future duplicator B frame and displaying it while picture 0 is in the future buffer.
  • the current invention also uses duplicator frames in a further novel and non-obvious way to create a meaningful transition from one video stream to another.
  • the method requires two video frames, a "FROM” frame, which must be either an I or P frame of the pre-transition video stream and must be in the player ' s "future” buffer; and a "TO" frame which must be an I frame and is either the first frame to be decoded from the "TO" video stream or an I frame which duplicates the contents of this frame, such as one stored in a separate TO frame cache. Note that the requirement that the TO frame be an I frame does not require the target frame in the parent video stream to be an I frame.
  • the target frame in the TO video stream is not an I frame it can still function as the TO frame provided the I frame duplicate of the TO frame is in the TO buffer.
  • the recoding of the required TO frames into I frames is trivial.
  • the FROM frame be an I or P frame and that the TO frame be an I frame by using a decoder which is capable of treating the decoded data of a non-reference frame as if it had come from an I or P frame.
  • a decoder able to copy the decoded data from the non-reference frame being used as a FROM frame directly into the past or future buffer and able to copy the decoded data from the P frame or non-reference frame being used as a TO frame directly into the future buffer.
  • a transition according to the invention can be initiated from any FROM frame to any TO frame regardless of the frame type.
  • the first step of such a transition would be to copy the decoded information from the displayed FROM frame into the future buffer at the beginning ofthe transition.
  • the transition can proceed normally with the TO frame being decoded into the future buffer forcing the FROM frame into the past buffer.
  • the decoder could also be directed to copy the FROM frame directly into the past buffer rather than using the TO frame to force it into the past buffer as would happen in ordinary play.
  • Using a non-reference frame for the FROM frame with a decoder capable of this manipulation does not create any additional computational burdens on the system, however use of TO frames other than I frames does.
  • the TO frame must be decoded and the decoded frame information copied into the future buffer.
  • the use of a TO frame other than an I frame will require the system to first accurately decode the desired TO frame before it can be copied into the future buffer.
  • the system will be required at a minimum to parse all the reference frames upon which the TO frame depends in order to
  • the TO frames are I frames, either through the use of cached I frame duplicates of the TO frames or through restricting transitions to TO frames in the TO stream which are coded as I frames.
  • the transition is used not only to provide a meaningful transition from one video stream to the other or from one part of a stream to another part of the same stream by providing visual cues as to the direction of the transition thereby helping the viewer maintain orientation within the video environment, but also to provide such meaningful transition information during the time when the computer is seeking the TO video stream.
  • the transition technique of the present invention can be used for either one of these purposes or for both.
  • the example provided below employs the processes required to perform both functions, although it is understood that both functions are not necessary to practice the invention and the use of synthetic MPEG as disclosed herein is within the scope of the invention whether or not coupled with use of visually meaningful orientation cues and whether or not used to mask seeking time.
  • the MPEG streamer of the multimedia product has an additional buffer in which are maintained 1 frames
  • SUBSTTTUTE SHEET (RULE 26) corresponding to the TO frames which the run-time system would seek for each possible transition which the user might initiate from that point of the video environment.
  • the buffered frames might be left view, right view, and 180 degree turn midway on a department store aisle and might include left turn, right turn and 180 degree turn at the intersection of two aisles.
  • FIG. 2 illustrates an embodiment containing such a TO frame buffer.
  • the contents ofthe TO frame buffer may change from time to time depending on the location of the user within the video environment.
  • the transition can be performed after the TO frame is located in the target video stream sequence where the entire sequence is stored.
  • a system according to the invention is illustrated in FIG. 2.
  • FIG. 2 shows the preferred embodiment containing a dynamic TO frame buffer 27.
  • the embodiment illustrated in FIG. 2 also has a pre-constructed group of synthetic MPEG transition streams 29 (FIG. 2) which can be readily accessed to provide the desired push left, push right or other enabled transition effect.
  • FIG. 2 also illustrates use of a transition generator 31 which can be employed to create the desired synthetic MPEG stream on-the-fly.
  • the method of the invention can be practiced with either or both a pre-constructed synthetic MPEG transition buffer or a generator.
  • transition methods of the invention are useful whether or not they are used to mask seek time. Where seek time masking is not desired, the TO frame buffer can be eliminated and the TO frame sought directly from the target video stream. Whether the TO frame comes from the additional buffer or is copied from the actual TO frame, the streamer inserts the TO frame into the stream and causes the player to parse it. This forces the FROM frame onto the screen.
  • the player has now set up a situation in which the FROM frame is in the past buffer and the TO frame is in the future buffer.
  • the streamer can generate a series of B frames, all of which refer to either the FROM frame, the TO frame, or both, which appears to the user, when played, as if a transition is occurring.
  • the transition process is illustrated in the process flow diagram of FIG. 10.
  • the user of a multimedia software product is "navigating " along a path in a video environment which path is represented by a single MPEG stream and the user wishes to turn to the right at an intersection in the video environment being displayed
  • the user by means of any input device executes the appropriate navigational command which is recognized by the computer to require a push right transition to the target video stream at the TO frame.
  • a transition frame cache or buffer only selected, matched frames may be used for transitions. Therefore, upon receiving the push right instruction, in step two ofthe process the MPEG streamer continues to play the current FROM video stream until it encounters the next FROM frame in the stream for which a push right TO frame is correlated.
  • Such a frame may simply provide a view of a side wall or inaccessible portion ofthe environment, in which case no target video stream is sought, merely cached "side view" frames.
  • step two leads not only to step three where the FROM frame is sent to the MPEG player by the MPEG streamer, but also to steps A and B where, simultaneously, the streamer seeks the target video stream on the video stream storage media.
  • Steps A and B can be further facilitated by use of a pre-constructed video stream index 33 (FIG. 2). While the seeking steps are being performed, the streamer sends the appropriate TO frame from the TO frame buffer 27 (FIG. 2) to the player where it is parsed into the future buffer forcing the FROM frame into the past buffer as shown in step four of FIG. 10.
  • the MPEG streamer then either retrieves the appropriate pre-constructed synthetic MPEG transition stream from the transition buffer 29 (FIG. 2) or employs the transition generator 31 (FIG. 2) to create the appropriate synthetic MPEG stream on-the-fly as illustrated in step five of FIG. 10.
  • tlie seek function will also be completed and the player can resume normal play at the new position in the TO video stream. Note that if the TO frame is not an I frame it must be converted to an I frame using standard MPEG decoding/encoding steps before it can be injected into the TO
  • SUBSTTTUTE SHEET (RULE 26) frame cache.
  • the actual target frame will remain unaltered, but its representation in the cache will be as an I frame.
  • the synthetic MPEG which is sent to the player from the transition buffer or transition generator to create the push right transition desired by the user is performed by generating a series of B frames.
  • Each macroblock in the generated B frames refers to either the picture in the future buffer or the picture in the past buffer.
  • the first frame in the series contains mostly references to the past buffer, but all macroblocks referring to the past buffer use a motion vector that copies the macroblock in the FROM frame that is in the same row, but in the next column to the right.
  • the last column of macroblocks use a motion vector that copies the macroblock in the TO frame that is in the same row. but in the first column. This results in a picture that consists mostly of the FROM picture, but shifted to the left, and a small amount of the TO picture.
  • FIG. 11a labels each macroblock in the FROM picture with a code consisting ofthe letter of that picture, a digit indicating the row of the macroblock. and a digit indicating the column of the macroblock.
  • FIG. l ib does the same for the TO picture.
  • FIG. lie through FIG. Ilk show the contents of the intermediate B frames that would be generated to display a push right transition from frame A to frame B. Note that in the
  • FIG. 12 shows an illustration of a push right transition according to the invention as it would appear in the display.
  • the pictures illustrated are assumed to be only 6 macroblocks wide, although in actuality a 16 pel macroblock width increment would be substantially smaller than that illustrated.
  • the display is assumed to coincide with the bitmap of the picture. That is. it is assumed that there are no undisplayed portions of the picture.
  • the transition works equally well where the picture display is less that the entire bitmap. Further, transitions can be improved if the player used has the ability to "pan" the bitmap. This process is discussed in detail below in the Turn Transition section.
  • the FROM frame is shown.
  • the width of a column of macroblocks is illustrated by a dotted line 51 extending vertically through the FROM picture of FIG. 13. This width is denoted to be one macroblock column (16 pels) by the MB measurement shown 53.
  • the right margin of the FROM picture is depicted in pictures Tl through T5 of FIG. 12 by a vertical line 55.
  • Pictures Tl through T5 are sequential transition pictures generated according to the push right embodiment ofthe invention.
  • Tl contains all of the macroblock columns of the FROM picture except for the one to the farthest left.
  • the leftmost column of Tl is the macroblock column which was second from the leftmost in the FROM picture.
  • Tl also contains as its farthest right macroblock column the farthest left column of macroblocks from the TO Picture.
  • T2 contains all of the macroblocks of the FROM picture except for the two columns farthest to the left with the macroblock column which was third from the left in the FROM picture as the leftmost column of T2.
  • T2 contains the leftmost two macroblock columns of the TO picture as its rightmost two columns.
  • the successive transitional pictures. T3, T4, and T5. contain increasing numbers of columns of die TO picture and decreasingly fewer columns of macroblocks from the FROM picture until T5 which contains only the right most column of the FROM picture as its leftmost column and contains all columns of the TO picture except for the right most.
  • SUBSTTTUTE SHEET (RULE 26)
  • the speed of the transition can be varied by use of repeated transition frames. Further, the speed of the transition does not have to be constant. Use of varying numbers of repeated frames can create the illusion of acceleration or deceleration in turn speed. Use of this embodiment of the invention does have a minimum transition increment, however. The smallest increment of change from transition to transition is the macroblock. Thus in a push right transition, a row of macroblocks is the smallest incremental change from picture to picture.
  • Transitions are not limited to push rights and push lefts. Indeed, many transition effects can be generated since any of the macroblocks in the intermediate frames can display any 16x16 pel region in the FROM picture, any 16x16 pel region in the TO picture, or. by using both forward and backward motion vectors, the average of any 16x16 pel region in the FROM picture and any other 16x16 pel region in the TO picture.
  • transitions similar to these could be generated in software without the use of an MPEG player or an MPEG stream.
  • the decoder would need to make the bitmaps of the FROM and TO pictures available to the software, which would then scroll them across the screen in a meaningful way.
  • this does require the transition software to be in control with the resulting increase in computational demands.
  • the transition is done solely as an MPEG stream, as taught in the present invention, any MPEG hardware that is available will be able to run the transition with minimal CPU intervention.
  • the transition is done in software, rather than utilizing the present invention, the CPU will be quite busy displaying it. If the CPU is less busy performing the transition, it can spend more time anticipating the user ' s needs and reduce response time to the user's requests.
  • transitions described here are "invariant.”
  • the intermediate sequence of B frames for a given transition such as a push right, is identical. regardless ofthe content ofthe FROM and TO pictures.
  • This has significant advantages at run ⁇ time in that it is not necessary to perform laborious calculations or encoding to run the transition; a static block of data can be inserted into the bitstream by the streamer and then fed to the player to implement the transition.
  • the transitions can be performed during editing of a video work and the resulting linear video stream can be stored for future playback. Consequently, the part ofthe invention dealing with transitions also has application in more traditional linear-style editing of MPEG video.
  • a linear editor traditionally uses cut and paste techniques to create a new video stream out of one or more component streams, possibly with transitions between sections of the component streams.
  • the transition methods of the current invention could be employed to generate smooth transitions between the cut and pasted sequences at edit time and written to the disk as a part of the resulting edited video stream, thus creating a new linear video stream that has the effect of playing the transition.
  • the present invention can be employed in a runtime system to allow a large number of transitions to be accommodated, or can be used to splice video streams or sequences together at edit time.
  • transitions aspect of the present invention can also be used in conjunction with intra-bit map panning to generate a smooth transition.
  • This type of transition is a hybrid of the techniques previously described and existing techniques for panning bitmaps called a "turn” transition.
  • turn left” and turn right transitions analogous to "push left” and “push right”, but “turn up” and “turn down” would also be possible.
  • Turn transitions are much like push transitions, but can be made to occur much more smoothly than pushes since the from and to images do not have to move in 16-pel increments.
  • the player To execute a turn transition, the player must be placed in a mode which displays less than all ofthe video data in the pictures in the stream. At least a 16 pel column (where the turn transition is to the left or right) or row (where the turn transition is up or down) of video data must be offscreen at all times. This could be accomplished by any means known in the art. For example, pictures larger than the display can be accomplished by "stretching " the
  • SUBSTTTUTE SHEET (RULE 26) remaining pels to fill the onscreen area, or by reducing the size of the onscreen area so that only part ofthe source video is available.
  • the MPEG player in order to execute the hybrid "turn " transition, the MPEG player must have the capability of " panning " over the source video. Panning involves mapping a portion of the source video to the onscreen display area, leaving some portion of the source video offscreen. For example, if the pel with X coordinate of 16 and Y coordinate of 0 is onscreen at coordinate 0.0 (the upper left portion of the video display area) and coordinates are increasing down and to the right, the video picture is said to be panned 16 pels right. In this case, 16 pels of source video would be said to be offscreen to the left of the onscreen area. If no pels are offscreen to the left, the video is said to be panned fully left; if no pels are offscreen to the right, it is said to be panned fully right.
  • the player's past and future buffers are set up as for a push transition as illustrated in FIG. 10. steps 1 through 4.
  • the future buffer should contain a copy of the TO frame
  • the past buffer should contain a copy of the FROM frame.
  • the streamer or other runtime component decides on the number of pels the FROM and TO frames should appear to move between frames.
  • This number reflects the speed at which the "turn" will be perceived by the viewer of the display.
  • This decision can be made in a variety of ways.
  • the speed of the turn could vary according to the magnitude of the turn command from a user input device, such as a joystick or mouse.
  • the display will change at the constant rate of 4 pels per frame.
  • the speed of the transition it is well within the invention for the speed of the transition to be much less or much more depending on the effect desired. Further, the speed of the transition does not have to be constant. Note also that a variable number could be chosen for each frame to give the impression of accelerating or decelerating during the turn. The system could accelerate or decelerate or both during all or portions ofthe turn transition.
  • the third step illustrated in FIG. 13 requires the player to be instructed to pan the video display port to the right such that, at the end of the series of pans which make up this step, there are fewer than one panning increment's worth of pels of the FROM picture offscreen to the right of the display. This is done iteratively by sending past duplicator B frames to the screen and iteratively adjusting the panning amount by the panning increment. For example, if 80 pels are offscreen to the right when step 3 begins, and if the panning increment is 4, then 20 past duplicator frames will be sent as the picture is panned to the right 4 pels at a time. Notice that the duplicator frames in this panning step refer only to the FROM frame and contain no information from the TO frame.
  • the streamer determines which pel column of the FROM picture will be the leftmost pel column of the onscreen display area for the next frame by adding the panning increment to the leftmost currently visible pel column which has the effect of shifting the leftmost column to be displayed one panning increment to the right of the leftmost column displayed in the previous picture. Exactly where in the FROM picture this leftmost displayed column will be depends on the relationship between the width ofthe display port and the width ofthe picture being displayed.
  • the streamer constructs or retrieves a composite B which places the macroblock column of the FROM picture which contains the desired pel column in the leftmost macroblock column of the composite B frame. Succeeding columns contain the rest of the FROM picture and the first column(s) of the TO frame.
  • These composite B frames are similar to the ones used in the PUSH transition discussed above.
  • the fifth step illustrated in FIG. 13 is to send the composite picture to the player while also instructing the player to adjust its panning mode to the left by the amount that the FROM picture was shifted when creating the composite picture minus one panning increment.
  • the onscreen appearance will be that the FROM picture has shifted one panning increment to the left and the onscreen pel columns to the right of the FROM picture contain the first pel columns ofthe TO picture.
  • the streamer sends the player another copy ofthe composite video frame used in step 5. but with a panning instruction which serves to move the display one panning increment (eight pels) to the right.
  • This step is repeated with incremental panning adjustments until the number of pels remaining offscreen on the right side of the picture is less than one panning increment.
  • the seventh step illustrated in FIG. 13 is designed to determine whether further duplicator B frames must be constructed or retrieved.
  • the MPEG streamer determines which pel column should appear in the leftmost pel column of the visible display area by adding the panning increment to the leftmost currently visible pel coiumn. If this addition results in a leftmost visible pel column which is still in the range of the FROM picture ' s width, further intermediate B frames must be constructed or retrieved, and the streamer proceeds to step 8. If the resulting pel column would be outside the FROM picture ' s width, no further frames need be constructed; the TO picture can be used instead, and the streamer proceeds to step 9. which is the last step in the process.
  • step eight the streamer determines which macroblock column in the FROM picture contains the pel column that should be on the left-hand edge of the visible portion of the next frame and constructs or retrieves a duplicator B frame with the appropriate past and future buffer references to place this macroblock coiumn in the leftmost macroblock column of the B picture. Succeeding macroblock columns contain the portions of the FROM picture further to the right if any. followed by the first columns of the TO picture. After constructing the intermediate B frame, the streamer proceeds back to step five to display it appropriately.
  • steps A and B illustrated in FIG. 13 may be executed during the transition to mask seek time so that the TO video stream has been located and is ready to be sent to the streamer at the completion of the transition and the system is ready to accept any other user input available from this position on the TO frame. Note that in the best mode implementation of the invention the correlated FROM and
  • a camera lens or computerized rendering option is chosen so that a 90 degree field of view is obtained, thus giving the ability to make 90 degree left or right turns conveniently.
  • FIG. 14 illustrates a right turn according to the invention.
  • picture F. the FROM picture and picture T. the TO picture These pictures which were derived from actual "shot" scenes or rendered scenes are referred to as video resources to distinguish them from composite pictures created with the use of duplicator B frames according to the invention. These composite pictures are sometimes referred to herein as manufactured resources.
  • picture F and picture T each measure 704 pels by 240 pels and that the onscreen visible area is 352 pels by 240 pels.
  • the panning increment is a constant 8 pels, and that the topmost, leftmost pel in a picture is pel 0.0.
  • FIG. 14 the pictures are shown divided into two regions.
  • picture F has a left region F L 206 consisting of the first 352 pel columns of F (columns 0 through 351) and a right region F R 207 consisting of pel columns 352 through 703.
  • picture T has a left region T L 208 consisting of pel columns 0 through 351 and a right region T R 209 consisting of the remaining 352 pel columns of the picture.
  • FIGS. 14a, 14b, 14c and 14d illustrate a right turn transition from F to T.
  • the active video picture that is. the picture of which a portion is being displayed by the system, is shown by a solid-lined rectangle.
  • the active video picture is a video resource 201.
  • the lines of the rectangle are thin.
  • the active video picture is a manufactured resource 202.
  • the lines of the rectangle are thick.
  • the video picture is inactive, that is not currently being displayed, it is represented by a rectangle outlined by a single, dotted line 204.
  • a double, dotted line is also used to delineate between the left and right regions of a picture 205. whether or not the picture is an active picture.
  • the onscreen region of the video picture being displayed is shown as a shaded gray area 203.
  • the FROM frame could be either a video resource 201 or a manufactured resource 202.
  • the FROM frame is usually a video resource either taken directly from the video stream being played or taken from a cache of pictures of views which may or may not be not present on a video stream.
  • FIG. 14a shows the visible portion of the screen 203 before the transition is executed.
  • the middle portion of picture F is being displayed. That is the displayed portion of F consists of the right portion of region F L 206 and the left portion of region F R 207 which consists of pel columns 176 through pel column 527 of F.
  • the panning position is therefore 176 pels from the left edge of F.
  • the streamer must first add the FROM picture to the past buffer and the TO picture to the future buffer by passing them to the MPEG player to parse. Then the streamer generates a series of 22 past duplicator pictures while the panning position is adjusted to the following positions:
  • the picture is panned fully to the right, with no pels offscreen on the right. That is the display boundary coincides with the boundaries of F R and the displayed portion of the picture 203 covers all of F R . This position is shown in FIG. 14b
  • the streamer then obtains or creates a composite B picture consisting of the right 352 pel columns of picture F and the left 352 pel columns of picture T made in accordance with the
  • the visible appearance is that the portion of picture F R F, that was previously visible has shifted left 8 pels and 8 pels of picture T L T R are visible on the right edge of the picture.
  • the streamer continues to send copies of F R T L to the player while the panning position is adjusted to the following positions:
  • FIG. 14c shows the panning process at the intermediate point where the panning position reaches 176 pels from the right margin of picture F R T, .
  • the panning position reaches 344 we have the option of panning one further increment to 352 and direct the player to pan picture T L T R 8 pels to the right of the left margin, however we have chosen to cease the pan at the 344 pel position and. preferably using the frame specific access techniques described elsewhere in this disclosure, place the T t T R picture on the screen and simultaneously change the panning instruction to the player so that no pels are offscreen to the left. This position is shown in FIG. 14d.
  • This structure defines an MPEG transition tvpedef struct taeMPEGTRANS [ long mJLength, // Length of data in m_pData unsigned char *m_pData // Actual MPEG data for transition (all B frames),
  • This structure is used for seeking and to indicate the current frame in any ofthe ' frame sync functions Note that when you call MpegbeekSMPTE or MpegSeekFrame. only the SMPTE or FrameNum portions are looked at for input and the other portion is appropriately
  • MAX_PICS_PER_BUFFER 20 Max number of pictures per MPPBuffer tvpedef struct MPPPicInfoTag
  • MPPBuffer FREE 0 // Buffer is available
  • MPPBuffer PLAYER 1 // Buffer is in use by player
  • MPPBuffer PDATE 2 // Buffer is being updated do not use tvpedef struct MPPBufferTag struct MPPBufferTag *m_pNext, // Next buffer in chain (used by streamer only) void *m_pvData. // MPEG data long m JFiIeOffset. // File offset of MPEG data long mJLength, // Length of MPEG data long mJMax. // Maximum length allocated
  • MPSOpen initializes the streamer on the given file.
  • the file will usually be the clip hunk generated by the binder.
  • SVPRC MPSOpen (SVPSuperHandle &sup. const char *strFileName);
  • the IFrameNum parameter is the frame number of the cached copy of the TO frame that the
  • MPSDeCacheAll removes all previously-cached frames from the cache SVPRC MPSDeCacheAIKSVPSuperHandle sup);
  • / ⁇ ' MPSPlay Transition causes a transition to play using the MPEG frame '/ that is currently in the future buffer as the FROM frame and the / specified cached frame as the TO frame.
  • the transition to use is specified by the pMpegTrans structure passed in. SVPRC MPSPlay Transition(SVPSuperHandle sup,
  • MPSChopStream terminates the currently-playing MPEG stream after the specified frame number. Used for eliminating information that has already been read but has not yet been sent to the MPEG ' player in preparation for exiting the current clip.
  • SVPRC MPSChopStream (SVPSuperHandle sup, long IFrameNum);
  • MPSHoIdVideo is used to prevent the streamer from reading any more ' video from the clip hunk. Used before an MPSChopStream call to ' make sure the streamer doesn't read any more video before the ' transition takes place.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Television Signal Processing For Recording (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention concerne, pour un système vidéo au standard MPEG-1, un procédé et un dispositif permettant des effets de fondu enchaîné, d'accès à une trame spécifique et de restitution en défilement inverse d'images vidéo animées numériques comprimées. Pour réaliser un fondu enchaîné entre deux trames vidéo, le procédé consiste: à sélectionner une trame de départ (FROM) et une trame d'arrivée (TO); à générer une séquence de trames duplicatrices à dépendances bidirectionnelles, lesquelles trames font varier, en fonction d'un schéma prédéfini, les références vectorielles de mouvement par rapport à la trame FROM et à la trame TO; à mettre la trame FROM dans le tampon passé d'un décodeur et à placer la trame TO dans le tampon future d'un décodeur; à alimenter le décodeur en séquences de trames duplicatrices; à provoquer l'affichage des trames duplicatrices; et à commencer la restitution normale d'une séquence vidéo contenant la trame TO à la position de trame TO. Pour obtenir l'accès spécifique à une trame, le procédé consiste: à déterminer la position d'une trame cible; à déterminer le type de trame cible; à identifier les trames de référence auxquelles la trame cible fait directement ou indirectement référence; à analyser les trames de référence au moyen d'un décodeur pendant la suppression de l'affichage vidéo du décodeur; à rétablir l'affichage vidéo; et à commander la restitution normale par le décodeur à l'emplacement de la trame cible dans le flux binaire vidéo. Pour obtenir la restitution en défilement inverse, le procédé consiste à utiliser le procédé d'accès spécifique portant sur une succession de trames cibles dans laquelle chaque trame cible choisie précède immédiatement la trame cible antérieure dans l'ordre d'affichage.
EP97908727A 1996-02-20 1997-02-20 Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique Withdrawn EP0882358A2 (fr)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US1471696P 1996-02-20 1996-02-20
US1697596P 1996-05-06 1996-05-06
US16975P 1996-05-06
US80287097A 1997-02-19 1997-02-19
US802870 1997-02-19
US801254 1997-02-19
US08/801,254 US5959690A (en) 1996-02-20 1997-02-19 Method and apparatus for transitions and other special effects in digital motion video
PCT/US1997/002953 WO1997030544A2 (fr) 1996-02-20 1997-02-20 Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique
US14716P 2007-12-18

Publications (1)

Publication Number Publication Date
EP0882358A2 true EP0882358A2 (fr) 1998-12-09

Family

ID=27486409

Family Applications (1)

Application Number Title Priority Date Filing Date
EP97908727A Withdrawn EP0882358A2 (fr) 1996-02-20 1997-02-20 Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique

Country Status (2)

Country Link
EP (1) EP0882358A2 (fr)
WO (1) WO1997030544A2 (fr)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11215495A (ja) * 1998-01-27 1999-08-06 Canopus Co Ltd 画像データの圧縮伸張処理システムおよび画像データの圧縮伸張方法
CA2319519C (fr) * 1998-02-04 2013-04-30 Corporate Media Partners Doing Business As Americast Procede et systeme permettant de naviguer dans un contenu de maniere organisee et categorisee
US6564005B1 (en) 1999-01-28 2003-05-13 International Business Machines Corporation Multi-user video hard disk recorder
FI114433B (fi) 2002-01-23 2004-10-15 Nokia Corp Otossiirtymän koodaaminen videokoodauksessa
WO2004057609A1 (fr) * 2002-12-20 2004-07-08 Koninklijke Philips Electronics N.V. Creation d'effets de mise en forme sur des videos compressees mpeg-2
EP2348709B1 (fr) 2004-07-01 2014-01-01 Mitsubishi Electric Corporation Procédé de codage et appareil d'enregistrement de données video sur un support permettant un accès aléatoire
US20060239563A1 (en) * 2005-04-25 2006-10-26 Nokia Corporation Method and device for compressed domain video editing
EP2015305A1 (fr) * 2007-07-12 2009-01-14 Alcatel Lucent Dispositif et procédé de lecture inversée lisse d'un média
JP5326555B2 (ja) * 2008-12-25 2013-10-30 ソニー株式会社 情報処理装置、動画像切り出し方法および動画像切り出しプログラム
EP3249652B1 (fr) 2016-05-25 2020-07-15 Axis AB Procédé et appareil de lecture vidéo enregistrée
CN111031393A (zh) * 2019-12-26 2020-04-17 广州酷狗计算机科技有限公司 视频播放方法、装置、终端及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5185819A (en) * 1991-04-29 1993-02-09 General Electric Company Video signal compression apparatus for independently compressing odd and even fields
US5450599A (en) * 1992-06-04 1995-09-12 International Business Machines Corporation Sequential pipelined processing for the compression and decompression of image data
US5414455A (en) * 1993-07-07 1995-05-09 Digital Equipment Corporation Segmented video on demand system
US5521630A (en) * 1994-04-04 1996-05-28 International Business Machines Corporation Frame sampling scheme for video scanning in a video-on-demand system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO9730544A3 *

Also Published As

Publication number Publication date
WO1997030544A2 (fr) 1997-08-21
WO1997030544A3 (fr) 1997-10-02

Similar Documents

Publication Publication Date Title
US5959690A (en) Method and apparatus for transitions and other special effects in digital motion video
US5872575A (en) Method and system for the creation of and navigation through a multidimensional space using encoded digital video
US6285361B1 (en) Method and apparatus for clipping video segments from an audiovisual file
US6262777B1 (en) Method and apparatus for synchronizing edited audiovisual files
US6353700B1 (en) Method and apparatus for playing an MPEG data file backward
US6057832A (en) Method and apparatus for video-on-demand with fast play capability
US6871006B1 (en) Processing of MPEG encoded video for trick mode operation
US6771703B1 (en) Efficient scaling of nonscalable MPEG-2 Video
US6848117B1 (en) Video-on-demand system capable of performing a high-speed playback at a correct speed
US5557332A (en) Apparatus and method for reproducing a prediction-encoded video signal
DE69633552T2 (de) Verfahren und vorrichtung zum bildgenauen zugriff auf digitale audiovisuelle information
JP3662129B2 (ja) マルチメディア情報編集装置
US6157771A (en) Method and apparatus for seeking within audiovisual files
US6661430B1 (en) Method and apparatus for copying an audiovisual segment
US6400886B1 (en) Method and apparatus for stitching edited video segments
EP1239674A2 (fr) Procédé et appareil d'enregistrement de données radiodiffusées
WO1994029868A2 (fr) Systemes de montage video
US6201925B1 (en) Method and apparatus for editing video files
EP0882358A2 (fr) Procede et dispositif pour fondu enchaine, defilement inverse et autres effets speciaux appliques a l'image animee video numerique
US7194189B2 (en) Methods and apparatus for smooth reverse playback of MPEG encoded media
US10142605B1 (en) Systems and methods for playing a video clip of an encoded video file
US10283160B1 (en) Systems and methods for switching between multiple software video players linked to a single output
JPH10322680A (ja) 連鎖型多視点映像再生方法
US20060008248A1 (en) Optimal buffering and scheduling strategy for smooth reverse in a DVD player or the like
US5675383A (en) Encoding method, encoding unit, decoding method, decoding unit, recording medium, and transmission method

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 19980918

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): DE GB

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20000901