EP0622957A2 - Variable speed playback of digital video data stored in a non-tape media - Google Patents

Variable speed playback of digital video data stored in a non-tape media Download PDF

Info

Publication number
EP0622957A2
EP0622957A2 EP19940302715 EP94302715A EP0622957A2 EP 0622957 A2 EP0622957 A2 EP 0622957A2 EP 19940302715 EP19940302715 EP 19940302715 EP 94302715 A EP94302715 A EP 94302715A EP 0622957 A2 EP0622957 A2 EP 0622957A2
Authority
EP
Grant status
Application
Patent type
Prior art keywords
frames
averaging
components
mixed
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP19940302715
Other languages
German (de)
French (fr)
Other versions
EP0622957A3 (en )
Inventor
William S. Herz
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Electronics Inc
Original Assignee
Sony Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/782Television signal recording using magnetic recording on tape
    • H04N5/783Adaptations for reproducing at a rate different from the recording rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/91Television signal processing therefor
    • H04N5/92Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N5/926Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation
    • H04N5/9261Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation involving data reduction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/781Television signal recording using magnetic recording on disks or drums
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/84Television signal recording using optical recording
    • H04N5/85Television signal recording using optical recording on discs or drums
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/87Regeneration of colour television signals
    • H04N9/877Regeneration of colour television signals by assembling picture element blocks in an intermediate memory

Abstract

A method for high speed playback of digital video data (e.g. D1 or D2 data) in which, at lower speeds, the frames of data to be played back are made of a mixture of consecutive frames of data. The frames of data are mixed by averaging the pixel values over consecutive frames to make up the output frames. For high speed shuttle operation, non-consecutive frames are averaged on a pixel by pixel basis to create shuttle frames. In order to improve the appearance of the output, consecutive shuttle frames contain a common original frame so that a component of the original data spans two output frames.

Description

  • This invention relates generally to the field of playback of digital video as in a digital video tape recorder (VTR) or virtual digital VTR. More particularly, this invention relates to performing variable speed playback and shuttle of compressed digital video material in a manner which emulates the performance of a video tape recorder's variable speed playback and shuttle. The present invention is also useful in instances where temporal compression is needed (i.e. judicious merging elements of video frames to compensate for smaller display interval of a rapid video frame display).
  • In video tape recorders, variable speed playback of the video material is generally accomplished by increasing the tape speed during playback and providing dynamic tracking. In video tape editing application, such variable speed playback has a distinctive look which editors are accustomed to.
  • When a moving video image is stored digitally and retrieved at a higher than normal playback rate, this is generally accomplished by simply deleting (or not retrieving) frames of information. For example, with reference to FIGURE 1, if the ten sequential frames F₁ through F₁₀ which are shown in FIGURE 1 are to be played back at twice the normal playback rate, only alternate frames might be displayed. For example, frames F₁, F₃, F₅, F₇ and F₉ would be displayed at the normal frame rate while frames F₂, F₄, F₆, F₈ and F₁₀ would be omitted entirely. Consequently, only five frames of information would be displayed at a standard frame rate requiring only half the display time. Thus, the illusion that the image has been displayed at twice normal speed is created.
  • Unfortunately, when non-contiguous frames of digital video are consecutively displayed to produce a high speed image, a rough jumpy display results resembling a montage or a teleconferencing-type image in which updates are slower than real time resulting in discontinuous video. This type of implementation thus diverges substantially from the look of a video tape recorder's variable speed playback. In a video tape recorder, the motion is smooth and continuous but occurs at a higher rate. At high speed shuttle rates wherein discontinuous frames of video are consecutively displayed, the human brain has difficulty tracking the movement and little information is thus conveyed to the user.
  • It is desirable to create a simulation of this variable speed playback such as appears in analog VTR equipment for use in technology such as optical, magneto optical, magnetic and other storage media wherein, in general, the video frames are randomly accessible and not necessarily temporally ordered (e.g., frame 1 of a video sequence could be adjacent to frame 20 in a video sequence). Applications such as disk based non-linear editors for digital video are appropriate candidates for such technology.
  • It is an object of the present invention to provide an improved variable speed playback system for playing back compressed digital video.
  • It is another object of the invention to emulate the performance of a video tape recorder's variable speed playback and shuttle in a digital video environment, i.e. retaining the look of dynamic tracking.
  • These and other objects, advantages and features of the invention will become apparent to those skilled in the art upon consideration of the following description of the invention.
  • The present invention provides for mixing or proportional merging of two or more video frames to occupy proportionately less of a normal display time to yield a smooth update of the video image.
  • For variable speed play (approximately - 1X to +3X), a video mix is performed on adjacent video frames. Since no more than 3X speed is required, no more than three frames of video need to be mixed. The mix is accomplished by adding the respective pixel values of the respective frames and dividing by the number of frames. Thus, when two frames are mixed, each frame contributes a 50% weighting to the resulting frame. For fractional speeds (e.g. 2.5X) alternating "mixed frames" constructed of two frames and three frames respectively are generated to produce a long term average of 2.5X in speed.
  • For shuttle operation (e.g. ±3X to ±30X) a different technique is used. In this technique, non-contiguous frames are combined with consecutive resulting frames ("shuttle frames" or "bridge frames") using a common frame to produce the mix. This creates an image which is much smoother and easier for the user to understand. In instances where the system holds bursts of data from a storage media in a buffer until processing can occur or until new data is loaded (in this case compressed video frames), and where the buffers cannot hold contiguous frames, the shuttle frame is constructed in a slightly different manner. Assuming a maximum buffer fill rate of one non-contiguous frame per eight frame time intervals, only one new frame per eight frames of video can be output. This is handled by holding a given frame long enough to compensate for this temporal limiting factor.
  • According to the present invention, a method for high speed playback of digital video data (e.g. D1 or D2 data) in which, at lower speeds, the frames of data to be played back are made of a mixture of consecutive frames of data. The frames of data are mixed by averaging the pixel values over consecutive frames to make up the output frames. For high speed shuttle operation, non-consecutive frames are averaged on a pixel by pixel basis to create bridging frames (referred to herein as shuttle frames). In order to improve the appearance of the output, consecutive shuttle frames contain a common original frame so that a component of the original data spans two output frames.
  • One method of the present invention for playback of video data at a speed greater than normal playback speed, includes the steps of: generating a sequence of output frames including a mixed frame where the mixed frame has a component of two or more frames of the original video data; and displaying the sequence of output frames.
  • Another method for playback of video data at a speed greater than normal playback speed, includes the steps of: looking up a first number of frames in a look-up table; creating a first mixed frame by averaging the first number of consecutive frames of data on a pixel by pixel basis; looking up a second number of frames in a look-up table; creating a second mixed frame by averaging the second number of consecutive frames of data on a pixel by pixel basis; generating a sequence of output frames including a sequence of the mixed frames; displaying the sequence of output frames at a frame rate equal to a normal playback speed frame rate.
  • A method for high speed shuttle playback of video data according to the invention includes the steps of: generating a sequence of output frames including a plurality of mixed frames, the mixed frames having a component of two or more frames of said video data; each mixed frame including a component of a previous output frame; and displaying the sequence of output frames. The shuttle frames are generated at a rate of 1 shuttle frame for each M video frames, where M is generally an integer.
  • According to one embodiment of the invention, an apparatus for high speed playback of digital video data includes a mass storage device for storing frames of video data. An input device is used to select a playback speed. Frames of video data are retrieved from the storage device and a computer generates a sequence of output frames including a mixed frame, the mixed frame having a component of at least two frames of the video data. A display displays the sequence of output frames.
  • The features of the invention believed to be novel are set forth with particularity in the appended claims. The invention itself, however, both as to organization and method of operation, together with further objects and advantages thereof, may be best understood by reference to the following description taken in conjunction with the accompanying drawing.
  • Figure 1 is an illustration of ten sequential video frames as they would be played back at normal speed.
  • Figure 2 is an illustration of five mixed frames which would display the information from Figure 1 at two times the normal playback rate.
  • Figure 3 illustrates four mixed frames which would be played back to produce 2.5 times the normal playback rate of Figure 1.
  • Figure 4 is a flow chart describing the operation of the present invention for variable playback speeds.
  • Figure 5 is an illustration of a sequence of video frames as they would be played back at normal speed.
  • Figure 6 is an illustration of five consecutive shuttle frames representing a shuttle speed of 8X.
  • Figure 7 is an illustration of four consecutive shuttle frames representing a shuttle speed of 16X.
  • Figure 8 is a flow chart describing the operation of the present invention at shuttle playback speeds.
  • Figure 9 is an illustration of buffer operation in the present invention.
  • Figure 10 is an illustration of the hardware configuration of the present invention.
  • While this invention is susceptible to embodiment in many different forms, there is shown in the drawings that will herein be described in detail specific embodiments, with the understanding that the present disclosure is to be considered as an example of the principles of the invention and not intended to limit the invention to the specific embodiment shown and described.
  • Referring to Figure 1, a sequence of ten sequential frames of video is shown labeled F₁ through F₁₀. At standard video speeds, frames 1 through 10 would be sequentially displayed at a fixed frame rate (e.g. 1 frame every 33.33 ms for NTSC) to produce normal speed video. If it is desired to show this video at faster than normal rate, the present invention utilizes the following technique for generating the higher rate video. Consider first, the example shown in Figure 2 wherein the speed selected is twice the normal video rate. In this example, it is desirable to produce five sequential frames of video which represent the ten original frames of Figure 1 but which can be viewed in half the time. Thus, the current invention generates five "mixed frames" (MF) shown in Figure 2 as MF₁ through MF₅. In examples lower in speed than approximately three times the normal video rate (that is, at variable speed play rates) each mixed frame is generated as a combination of two or more of the original frames F₁ through F₁₀. In Figure 2 for example, MF₁ would be generated by summing the pixel values of frames F₁ and F₂ and dividing the sum by two. That is: MF₁ = (F₁ + F₂) ÷ 2
    Figure imgb0001
  • For D1 video, each pixel is represented as 2 eight bit bytes containing (YCR or YCB) where Y is luminance, CR is Red chrominance and CB is Blue chrominance (in PAL, Y, U and V). In this instance, the individual values of Y, CR, and CB are determined for each frame and then individually averaged and converted back to (YCR or YCB). While the present invention is illustrated in terms of D1 data, similar pixel by pixel combining of video frames can be accomplished using RGB images (as in 3 byte per pixel data representing high definition video) by separating each pixel into R, G and B components. The R, G and B component of each pixel is then averaged to produce the mixed frame. Similar techniques can be used for other formats.
  • In the case where a fractional speed increase is desired, a modification of the process is used. Consider for example the generation of mixed frames for a video rate of 2.5 times normal. In this instance, the mixed frame data is generated in a manner which produces a long term average of about 2.5 frames of original video data for each mixed frame of video data. In the example of Figure 3, MF₁ may be created by mixing together the information from F₁, F₂ and F₃ as follows: MF₁ = (F₁ + F₂ + F₃) ÷ 3
    Figure imgb0002

    MF₂ is then generated as MF₂ = (F₄ + F₅) ÷ 2.
    Figure imgb0003

    In this manner, alternating mixed frames are generated as combinations of two and three original data frames.
  • A similar scheme can be derived for virtually any fractional speed. For example, if a rate of 2.7 times normal is desired, seven out of ten frames of mixed frame data are generated as a combination of three original frames. The remaining frames are generated as a combination of two original frames. In order to retain as smooth an image as possible, the distribution of mixed frames created as a mixture of two and three original frames should be a relatively even distribution in time to the extent possible.
  • In the case of the preferred embodiment, as previously described, this mixing process is carried out on a pixel by pixel basis for D1 format video which is represented by two bytes per pixel. These two bytes represent Y, CR and CB data so, the Y data is added and divided by two, the CR data is added and divided by 2, and the CB data is added and divided by two to produce each pixel of the mixed frame MF₁. Thus, in general, a mixed frame of data can be represented by the equation:
    Figure imgb0004

    where i, j and k are integers with i representing a counter for an original data frame, j representing a counter for a mixed frame (MF) and k representing a number of original frames to be averaged. In one embodiment of the present invention, the original frames are processed in sequences of 10 frames so that the pattern repeats after ten mixed frames are generated. In this embodiment, the value of k can be read from a look-up table such as the following TABLE 1:
     N / M ... 2.3 2.4 2.5 2.6 ...
    1 ... 2 2 3 3 ...
    2 ... 2 3 2 2 ...
    3 ... 3 2 3 3 ...
    4 ... 2 3 2 2 ...
    5 ... 2 2 3 3 ...
    6 ... 3 2 2 3 ...
    7 ... 2 3 3 2 ...
    8 ... 2 2 2 3 ...
    9 ... 3 3 3 2 ...
    10 ... 2 2 2 3 ...
    TABLE 1
  • In TABLE 1, as shown in abbreviated form, all desired speeds in .1X increments between 1.1 and 3.0 are listed along with a number of original frames to be combined. When mixed frame number 10 is reached (N=10) the sequence starts over again. When coupled with a motion vector, a different distribution of frame weighting can be implemented in order to achieve an even more efficient use of frames with a more intelligent distribution.
  • The process for variable speed operation at forward speeds between 1.1 and 3 times normal is described by the flow chart of Figure 4. Those skilled in the art will appreciate that the analogous process for reverse speeds is also within the scope of the present invention. The process starts at step 20. When the user selects a playback speed, it is received at step 24 as a speed value M. Counters i, N and j are initialized to 1, 1 and 0 respectively at step 26. Counter j is a mixed frame counter, counter i is an original frame counter and N is a counter used to enter the look-up table. At step 30, j and N are incremented by 1. If N = 11 at step 32, the look-up table counter N is reset to 1 at step 36. In either event, at step 40, the look-up table is entered on the values of M and N to retrieve the value of k (the number of original frames to be mixed). Note that k may vary in value each time it is retrieved from the distribution table (i.e. each time step 40 is carried out).
  • When the value of k has been retrieved, the mixed frame is computed on a pixel by pixel basis as described by Equation 1 at step 44. The value of i is then incremented by the number of original frames processed (k) in step 48. The frame is then output at step 50 and the process loops back to step 30.
  • Those skilled in the art will recognize that the frame combinations described in TABLE 1 can be changed considerably without departing from the invention. For example, column 2.5 could equally be 2,3,2,3,2,3,2,3,2,3 (top to bottom). Also, the table can be restricted to the fractional part of the speed (0.0 to 0.9) with a similar algorithm wherein the integer part is added to the value looked up to arrive at k. Rather than use of a look-up table, various algorithms could be developed to provide the frame distribution and averaging. It is also noted, that for speeds between 1.0 and 2.0, the values of N will sometimes be equal to one, indicating that no mixing of frames occurs for that particular "mixed frame". Rather, that frame is simply equal to the original frame.
  • For reverse speeds, the present invention works in essentially the same manner. Since the video frames are randomly accessible, the only difference is that the frame numbers for the original data are renumbered in reverse order, or equivalently, the algorithm of Figure 4 can be modified suitably to accomplish the reverse process.
  • When high speed shuttle operation is desired in the context of the present invention, an additional problem surfaces. At higher speeds, the human brain finds it difficult to process sequential images which represent frames of information too far separated in time. For example, using conventional shuttling techniques at eight times normal speed, only every eighth frame would be displayed. Due to the generally significant differences in the image of frames separated this far in time, the resulting sequence of frames is difficult for an observer to follow and understand. In order to improve the ability to track such information, a variation of the present process is used. In Figure 5 an extended representation of the original data of Figure 1 is shown. The reader will understand that additional frames of original data follow frame 25 when viewing the data at normal speed.
  • In shuttle operation as illustrated in Figure 6 mixed frames of data, which will be referred to herein as shuttle frames, are generated as represented by SF₁ through SF₄ in Figure 6. Consider the example of playback at eight times normal speed for the example of Figure 6. In this example, shuttle frame SF₁ is made identical to frame F₁. It is generally better to begin (and end) with an unaltered frame to give the user a clear starting (and ending) image. Frame SF₂ is generated by mixing the pixel values of frame 1 and frame 9 so that SF₂ is given by the following equation: SF₂ = (F₁ + F₉) ÷ 2.
    Figure imgb0005

    In a similar manner, SF₃ and SF₄ are generated as: SF₃ = (F₉ + F₁₇) ÷ 2.
    Figure imgb0006
    SF₄ = (F₁₇ + F₂₅) ÷ 2.
    Figure imgb0007
  • Thus, to the extent possible, each shuttle frame contains a component from the previous frame (after the very first frame). That is, SF₂ and SF₁ both include frame F₁ as a component. In a like manner, SF₃ and SF₂ both contain the component F₉, and SF₄ and SF₃ both contain the component F₁₇. Due to the common subject matter contained in each adjacent shuttle frame, the human brain is more readily able to understand the high speed video action which is presented, and the video image more closely resembles that of shuttle on a VTR. Subsequent frames are generated in a similar manner, except that it is preferrable to end with an unaltered frame taken from the original collection of frames to provide a clear image at the end point.
  • In a similar manner, Figure 7 illustrates a shuttle speed of 16 times normal. Here once again SF₁ equals F₁. SF₂ is made up of pixel values created from F₁ and F₁₇ as: SF₂ = (F₁ + F₁₇) ÷ 2
    Figure imgb0008

    SF₃ is given by: SF₃ = (F₁₇ + F₃₃) ÷ 2.
    Figure imgb0009

    Each consecutive frame is thus constructed using a common component from the previous frame and a component which will become part of the following frame. While this example uses the 17th, 33rd, 49th, etc. frame, other suitable representative frames could be selected. At the end point, it is once again desirable to end with an unaltered frame to provide a clear end point image.
  • Turning now to Figure 8, a flow chart describing the process of shuttle speed playback as illustrated in FigureS 5 through 7 is shown starting at 80. Once again, the user enters the shuttle playback speed at 82 as a value M. In shuttle mode, the value of M is an integer between 3 and 30. The value of j (the shuttle frame counter) is initialized at step 86 to a value of 0 and incremented at step 90 by 1. For the first shuttle frame (SF₁, j=1), the value of the shuttle frame is simply that of original frame 1 (F₁). Thus, at step 92, the value of j is inspected. If j=1 at 92, SFj=F₁ at step 94 and the frame is output at step 96. If j>1 at step 92, the process goes to step 98 where SFj is computed as: SF j = [F M ( j -2)+1 + F M ( j -1)+1 ] 2
    Figure imgb0010

    Finally, at step 96, the computed frame SFj is output and the process returns to step 90 where j is incremented and the next frame is computed.
  • In the above shuttle play process, there is no provision for presenting an end point image having no mixed frame component. In general, as previously mentioned, it is desirable, but not required, to present an image which is unaltered as the last frame. Depending upon the number of frames to be presented, this generally leads to an additional frame presented at the beginning and end of the shuttle. Thus, to display e.g., 57 frames of video at 8X, one would expect a total of 57 ÷ 8 = 7.125 ≈ 7 shuttle frames to be produced. In fact, 7 shuttle frames are produced, but if it is desired to produce a clear image at the start and end, two additional frames, one at the beginning and one at the end, are presented for display (F₁ and F₅₇). For short sequences of shuttle frames, this obviously leads to an actual shuttle rate less than the selected value (in this example 8X is selected, but the actual effective rate is 57 frames ÷ 9 frames = 6.33X). As the length of the sequence increases, the number of additional frames remains two, so the effect is minimal. For example, if 1000 original frames are shuttled at 8X, 125 shuttle frames plus 2 frames (beginning and end) would be displayed resulting in an actual shuttle rate of 1000 frames ÷ 127 frames = 7.87X.
  • When the end point does not lie exactly at a multiple of the shuttle rate plus 1, (e.g. for 8X: 9, 17, 25, 33...), an adjustment of the algorithm is needed to complete the shuttle frame sequence. This can be accomplished in several ways, any of which are acceptable. For example, consider an 8X sequence of frames ending at frame F₃₀. Any of the following sequences produce an acceptable shuttle frame sequence:
    Figure imgb0011
    Figure imgb0012
  • In sequence 1, the last frame is not incorporated as part of the SF₄, but is simply presented as the last frame. In Sequence 2, the spacing between frames which are averaged is adjusted throughout the sequence so that the last frame is incorporated in SF₄. In sequence 3, the spacing between frames averaged to produce SF₄ is adjusted to be 13 frames (rather than 8) to compensate for the end point. In sequence 4, the spacing between the last two averaged frames is adjusted to be 5 and an additional frame is presented to compensate for the end point. Sequence 5 compensates for the end point by spacing the last two frames averaged by 5 and omits use of an un-mixed frame for SF₁. Sequence 6, also spaces the last two mixed frames by 5 to compensate for the end point, but omits use of un-mixed frames at both beginning and end. Each of these sequences are within the scope of the invention. Those skilled in the art will appreciate that many variations of this process are possible for compensating for the end point utilizing the present invention.
  • The present shuttle process can be used as long as the system's buffer can fill rapidly enough to hold all of the frames needed to construct the shuttle frames. However, when the buffer fill rate is too slow for high speed shuttle operation, a different process is used. Consider, for example, a system with a maximum buffer fill rate of one non-contiguous frame per eight frame time intervals as shown in Figure 9. Only one new frame per eight frames of video can be created. This problem is handled by holding a given frame long enough to compensate for this temporal limiting factor.
  • It is generally preferrable, as described above, for the starting and ending frames to not be mixed frames. For example, if frames F₁ through F₅₇ are to be displayed in 16X shuttle, the following sequence of frames are displayed: SF₁ = F1
    Figure imgb0013
    SF₂ = (F₁ + F₁₇)/2
    Figure imgb0014
    SF₃ = (F₁₇ + F₃₃)/2
    Figure imgb0015
    SF₄ = (F₃₃ + F₄₉)/2
    Figure imgb0016
    SF₅ = (F₄₉ + F₅₇)/2
    Figure imgb0017
    SF₆ = F₅₇
    Figure imgb0018
  • In this example, the frame spacing between frames which are averaged is adjusted at the last mixed frame SF₅ to 8 so that the sequence ends on a frame which is not a mixed frame. In this instance, the shuttle frames (after SF₁ and before SF₆ are generated as: SF j = F p n + F p n -1 2
    Figure imgb0019

    where pn = 1, 9, 17, 33, 49, 57. For different rates M and end points, the series Pn would, of course, differ with the ultimate goal being a long term average number of mixed frames of approximately 1/M of the total original frames being used and end points having no mixed frames. The methods of FigureS 4 and 8 account for not having a mixed frame as the first frame and can easily be modified to provide for having no mixed frame at the end point.
  • The shuttle play mode could also be implemented using a combination of more than two original frames to create the shuttle frames without departing from the invention.
  • Once again, for reverse shuttle speeds (e.g. -3 to -30), the present invention works in essentially the same manner. Since the video frames are randomly accessible, the only difference is that the frame numbers for the original data are renumbered in reverse order, or equivalently, the algorithm of Figure 8 can be suitably modified to accomplish the reverse process.
  • Referring now to Figure 10, a hardware embodiment of the present invention is shown. In the preferred embodiment, the present invention is implemented using a programmed digital computer such as CPU / controller 110 as both a central processor and as a control device. The computer includes memory such as random access memory (RAM) and read only memory (ROM) (not shown) in a conventional arrangement. An input device 114, which may be a general purpose input device such as a mouse or a computer keyboard, is used to select playback speed and other operational parameters. In other embodiments, dedicated input hardware can be devised to simulate the operation and feel of VTR controls, if desired.
  • The frames of original video data are stored on a mass storage device 120 such as, for example, a magneto-optical (MO) disk drive or an array of such drives operating in conjunction with conventional high speed hard disk drives. Access to the mass storage device 120 is through a buffer 126 which may include a high speed disk drive or RAM memory or a combination thereof.
  • In operation, compressed video frames are retrieved from mass storage device 120 in a burst transfer and stored in buffer 126 until needed. If a look-up table is used, it may also be stored in the mass storage device until needed. The compressed video frames are retrieved from the buffer 126 in accordance with the frame selection algorithm, as described previously, depending upon the operational mode (shuttle or high speed playback) and the speed selected via the input device 114. These frames are then decompressed by decompressor 130 and converted into their Y, CR and CB components (or other basic components such as RGB) and stored in a frame memory 136 with pixels at an address determined by address generator 140. The frames stored in frame memory 136 are then averaged on a pixel by pixel basis in a high speed arithmatic logic unit 145. The average values of each pixel is output to an output frame memory 150. The output of output frame memory 150 is provided, along with the frames stored in frame memory 136, to a multiplexer 158 for display on a display 162 at the predetermined frame rate of the system. The multiplexer 158 is controlled by CPU / controller 110 to select one of the four inputs for display.
  • The operation of the system may be enhanced by use of a motion vector detector 170. Motion vector detector 170 can be used to determine a more intelligent frame distribution for averaging to produce an enhanced image. Motion vector detector 170 receives frames from the CPU / controller 110 and determines which frames to combine and how to combine them to produce shuttle frames or mixed frames.
  • In the embodiment described above, the present invention is implemented using a dedicated ALU for performing the mixing function. However, those skilled in the art will recognize that a programmed general purpose processor operating at an adequately high speed could also be used. To compensate for the relatively low speed of MO disk drives, the data in the embodiment above is buffered in a high speed hard disk drive for display on a conventional video monitor. Those skilled in the art will appreciate that the present invention could also be implemented using dedicated hardware rather than the programmed control CPU without departing from the invention. Numerous other variations will occur to those skilled in the art upon consideration of the present invention.
  • Thus it is apparent that in accordance with the present invention, an apparatus that fully satisfies the objectives, aims and advantages is set forth above. While the invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications, permutations and variations will become apparent to those skilled in the art in light of the foregoing description. Accordingly, it is intended that the present invention embrace all such alternatives, modifications and variations as fall within the scope of the appended claims.

Claims (41)

  1. A method for playback of video data at a speed greater than normal playback speed, comprising the steps of:
       generating a sequence of output frames including a mixed frame, said mixed frame having a component of two or more frames of said video data; and
       displaying said sequence of output frames.
  2. The method of claim 1, wherein said mixed frame is created by the step of averaging two consecutive frames of data on a pixel by pixel basis.
  3. The method of claim 1, wherein said mixed frame is created by the step of averaging three consecutive frames of data on a pixel by pixel basis.
  4. The method of claim 1, wherein said mixed frame is created by the step of mixing two or more frames of data on a pixel by pixel basis.
  5. The method of claim 1, wherein said output frames are displayed at a frame rate equal to a normal playback speed frame rate.
  6. The method of claim 1, wherein said each of said output frames is a mixed frame created by the step of mixing two or more frames of data on a pixel by pixel basis.
  7. The method of claim 1, wherein a plurality of said output frames are mixed frames with certain of said mixed frames created by the step of mixing two frames of data on a pixel by pixel basis and certain others of said mixed frames are created by the step of mixing three frames of data on a pixel by pixel basis.
  8. The method of claim 1, further comprising the step of looking up a number of frames to be mixed in a look-up table prior to said generating step.
  9. The method of claim 8, wherein said output frames include a plurality of mixed frames and wherein said number of frames to be mixed can vary from mixed frame to mixed frame in accordance with said number of frames to be mixed called out by said look-up table.
  10. The method of claim 2, wherein said averaging step includes the steps of:
       separating each pixel of said frames into Y, CR, and CB components;
       averaging said Y components;
       averaging said CR components;
       averaging said CB components; and
       converting said averaged Y, CR, and CB components into a two byte D1 (YCR or YCB) word.
  11. The method of claim 2, wherein said averaging step includes the steps of:
       separating each pixel of said frames into R, G, and B components;
       averaging said R components;
       averaging said G components; and
       averaging said B components.
  12. A method for playback of video data at a speed greater than normal playback speed, comprising the steps of:
       looking up a first number of frames in a look-up table;
       creating a first mixed frame by averaging said first number of consecutive frames of data on a pixel by pixel basis, said averaging including:
       separating each pixel of said frames into Y, CR, and CB components;
       averaging said Y components;
       averaging said CR components;
       averaging said CB components;
       converting said averaged Y, CR, and CB components into a two byte D1 (YCR or YCB) word;
       looking up a second number of frames in a look-up table;
       creating a second mixed frame by averaging said second number of consecutive frames of data on a pixel by pixel basis, said averaging including:
       separating each pixel of said frames into Y, CR, and CB components;
       averaging said Y components;
       averaging said CR components;
       averaging said CB components;
       converting said averaged Y, CR, and CB components into a two byte D1 (YCR or YCB) word;
       generating a sequence of output frames including said first and second mixed frames; and
       displaying said sequence of output frames at a frame rate equal to a normal playback speed frame rate.
  13. A method for high speed shuttle playback of video data, comprising the steps of:
       generating a sequence of output frames including a plurality of mixed frames, said mixed frames having a component of two frames of said video data;
       each said mixed frame including a component of a previous output frame; and
       displaying said sequence of output frames.
  14. The method of claim 13, wherein a first said mixed frame is created by the step of averaging a first and a second non-consecutive frame of data on a pixel by pixel basis.
  15. The method of claim 14, wherein a second said mixed frame is created by the step of averaging a second and a third non-consecutive frame of data on a pixel by pixel basis.
  16. The method of claim 13, wherein said shuttle frames are generated at a rate of 1 shuttle frame for each M video frames, where M is an integer.
  17. The method of claim 16, wherein M is greater than or equal to three.
  18. The method of claim 17, wherein said mixed frames are shuttle frames SFj generated according to the equation: SF j = [ F M ( j -2)+1 + F M ( j -1)+1 ] 2
    Figure imgb0020
    where j = an integer shuttle frame counter, and the mathematical operations are carried out on a pixel by pixel basis.
  19. A method for high speed playback of video data, comprising the steps of:
       generating a sequence of output frames including a plurality of shuttle frames, said shuttle frames having a component of two or more frames of said video data;
       said shuttle frames are generated at a rate of 1 shuttle frame for each M video frames, where M is an integer greater than or equal to three;
       each said shuttle frame including a component of a previous output frame with the shuttle frames SFj generated according to the equation: SF j = [ F M ( j -2)+1 + F M ( j -1)+1 ] 2
    Figure imgb0021
    where j = an integer shuttle frame counter; and displaying said sequence of output frames.
  20. A method for playback of video data at a speed greater than normal playback speed, comprising the steps of:
       retrieving a plurality of frames of said video data from a storage device; and
       generating a sequence of output frames including a mixed frame, said mixed frame having a component of two or more frames of said video data.
  21. The method of claim 1, wherein said mixed frame is created by the step of averaging two or more frames of data on a pixel by pixel basis.
  22. The method of claim 21, wherein said averaging step includes the steps of:
       separating each pixel of said frames into Y, CR and CB components;
       averaging said Y components;
       averaging said CR components; and
       averaging said CB components.
  23. The method of claim 21, wherein said averaging step includes the steps of:
       separating each pixel of said frames into R, G, and B components;
       averaging said R components;
       averaging said G components; and
       averaging said B components.
  24. A method for high speed shuttle playback of video data, comprising the steps of:
       retrieving a plurality of frames of said video data from a storage device;
       generating a sequence of output frames including a plurality of mixed frames, said mixed frames having a component of two frames of said video data; and
       each said mixed frame including a component of a previous output frame.
  25. The method of claim 24, wherein said mixed frames are created by the step of averaging non-consecutive frames of data on a pixel by pixel basis.
  26. The method of claim 25, wherein said averaging step includes the steps of:
       separating each pixel of said frames into Y, CR and CB components;
       averaging said Y components;
       averaging said CR components; and
       averaging said CB components.
  27. The method of claim 25, wherein said averaging step includes the steps of:
       separating each pixel of said frames into R, G and B components;
       averaging said R components;
       averaging said G components; and
       averaging said B components.
  28. An apparatus for high speed playback of digital video data, comprising in combination:
       mass storage means for storing frames of said video data;
       input means for selecting a playback speed;
       means for retrieving said frames of video data;
       generating means for generating a sequence of output frames including a mixed frame, said mixed frame having a component of at least two frames of said video data; and
       display means for displaying said sequence of output frames.
  29. The apparatus of claim 28, wherein said generating means generates said mixed frame by averaging two consecutive frames of data on a pixel by pixel basis.
  30. The apparatus of claim 28, wherein said generating means generates said mixed frame by averaging two non-consecutive frames of data on a pixel by pixel basis.
  31. The apparatus of claim 28, further comprising a look-up table for looking up a number of frames to be mixed by said generating means.
  32. The apparatus of claim 31, wherein said output frames include a plurality of mixed frames and wherein said number of frames to be mixed can vary from mixed frame to mixed frame in accordance with said number of frames to be mixed called out by said look-up table.
  33. The apparatus of claim 29, wherein said generating means includes an arithmatic logic unit for averaging said frames.
  34. The apparatus of claim 30, wherein said generating means includes an arithmatic logic unit for averaging said frames.
  35. The apparatus of claim 28, wherein said generating means includes:
       means for separating each pixel of said frames into Y, CR, and CB components;
       averaging means for:
       averaging said Y components;
       averaging said CR components;
       averaging said CB components; and
       converting means for converting said averaged Y, CR, and CB components into a two byte D1 (YCR or YCB) word.
  36. The apparatus of claim 28, wherein said generating means includes:
       means for separating each pixel of said frames into R, G, and B components;
       averaging means for:
          averaging said R components;
          averaging said G components;
          averaging said B components; and
       converting means for converting said averaged R, G, and B components into a standard video format for display by said display means.
  37. The apparatus of claim 28, further comprising buffer means for buffering said frames stored in said mass storage means.
  38. The apparatus of claim 28, further comprising means for decompressing compressed frames stored in said mass storage means.
  39. An apparatus for high speed playback of digital video data, comprising in combination:
       mass storage means for storing compressed frames of said video data;
       buffer means for buffering said frames stored in said mass storage means;
       input means for selecting a playback speed;
       means for retrieving said frames of video data;
       decompressing means for decompressing compressed frames stored in said mass storage means;
       means for separating each pixel of said frames into Y, CR, and CB components;
       an arithmatic logic unit for:
          averaging said Y components;
          averaging said CR components;
          averaging said CB components;
       output memory means for storing said averaged Y, CR and CB components as output frames; and
       display means for displaying said sequence of output frames.
  40. An apparatus for high speed playback of digital video data, comprising in combination:
       mass storage means for storing compressed frames of said video data;
       buffer means for buffering said frames stored in said mass storage means;
       input means for selecting a playback speed;
       means for retrieving said frames of video data;
       decompressing means for decompressing compressed frames stored in said mass storage means;
       means for separating each pixel of said frames into R, G and B components;
       an arithmatic logic unit for:
          averaging said R components;
          averaging said G components;
          averaging said B components;
       output memory means for storing said averaged R G and B components as output frames; and
       display means for displaying said sequence of output frames.
  41. An apparatus for high speed shuttle playback of video data, comprising:
       generating means for generating a sequence of output frames including a plurality of mixed frames, said mixed frames having a component of two frames of said video data;
       each said mixed frame including a component of a previous output frame; and
       displaying means said sequence of output frames.
EP19940302715 1993-04-29 1994-04-18 Variable speed playback of digital video data stored in a non-tape media. Withdrawn EP0622957A3 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US08054579 US5543927A (en) 1993-04-29 1993-04-29 Variable speed playback of digital video stored in a non-tape media
US54579 1993-04-29

Publications (2)

Publication Number Publication Date
EP0622957A2 true true EP0622957A2 (en) 1994-11-02
EP0622957A3 true EP0622957A3 (en) 1995-06-07

Family

ID=21992080

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19940302715 Withdrawn EP0622957A3 (en) 1993-04-29 1994-04-18 Variable speed playback of digital video data stored in a non-tape media.

Country Status (3)

Country Link
US (3) US5543927A (en)
EP (1) EP0622957A3 (en)
JP (1) JP3557244B2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5592299A (en) * 1994-09-26 1997-01-07 Hitachi America, Ltd. Method and apparatus for reducing the amount of data required to represent a video frame
US5715176A (en) * 1996-01-23 1998-02-03 International Business Machines Corporation Method and system for locating a frame position in an MPEG data stream
WO2005111989A2 (en) * 2004-05-19 2005-11-24 Sony Computer Entertainment Inc. Image frame processing method and device for displaying moving images to a variety of displays

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1246084A3 (en) * 1995-03-15 2004-05-26 Kabushiki Kaisha Toshiba A communication method and the system thereof applied for the network having plural client systems and server systems
US6005600A (en) * 1996-10-18 1999-12-21 Silcon Graphics, Inc. High-performance player for distributed, time-based media
US7448062B1 (en) 1996-10-18 2008-11-04 Microsoft Corporation Seamless playback of multiple clips of media data across a data network
JPH10178619A (en) * 1996-12-19 1998-06-30 Nikon Corp Image reproducing device and electronic camera
US6560400B1 (en) 1997-07-10 2003-05-06 Hitachi Denshi Kabushiki Kaisha Video information editing method and system, and recording medium having the editing method stored
KR100269398B1 (en) * 1997-08-30 2000-10-16 구자홍 Mpeg video decoder for supporting replay and decoding method thereof
US6510554B1 (en) * 1998-04-27 2003-01-21 Diva Systems Corporation Method for generating information sub-streams for FF/REW applications
US6385771B1 (en) * 1998-04-27 2002-05-07 Diva Systems Corporation Generating constant timecast information sub-streams using variable timecast information streams
US6292454B1 (en) 1998-10-08 2001-09-18 Sony Corporation Apparatus and method for implementing a variable-speed audio data playback system
JP2003532345A (en) * 2000-04-25 2003-10-28 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ How to masking image display transition when switching video playback speed
US7089496B2 (en) * 2001-07-20 2006-08-08 Hewlett-Packard Development Company, L.P. System and method for scene detection information storage
US8392952B2 (en) * 2002-05-03 2013-03-05 Time Warner Cable Enterprises Llc Programming content processing and management system and method
US20050034171A1 (en) * 2002-05-03 2005-02-10 Robert Benya Technique for delivering programming content based on a modified network personal video recorder service
US7908626B2 (en) * 2002-05-03 2011-03-15 Time Warner Interactive Video Group, Inc. Network based digital information and entertainment storage and delivery system
US7073189B2 (en) * 2002-05-03 2006-07-04 Time Warner Interactive Video Group, Inc. Program guide and reservation system for network based digital information and entertainment storage and delivery system
US7111314B2 (en) * 2002-05-03 2006-09-19 Time Warner Entertainment Company, L.P. Technique for delivering entertainment programming content including interactive features in a communications network
US7610606B2 (en) * 2002-05-03 2009-10-27 Time Warner Cable, Inc. Technique for effectively providing various entertainment services through a communications network
US8312504B2 (en) 2002-05-03 2012-11-13 Time Warner Cable LLC Program storage, retrieval and management based on segmentation messages
US7114173B2 (en) * 2002-05-03 2006-09-26 Aol Time Warner Interactive Video Group, Inc. Technique for synchronizing deliveries of information and entertainment in a communications network
US8443383B2 (en) 2002-05-03 2013-05-14 Time Warner Cable Enterprises Llc Use of messages in program signal streams by set-top terminals
US7614066B2 (en) 2002-05-03 2009-11-03 Time Warner Interactive Video Group Inc. Use of multiple embedded messages in program signal streams
CN100593188C (en) * 2004-05-19 2010-03-03 索尼计算机娱乐公司 Image frame processing method and device for displaying moving images to a variety of displays
JP4283303B2 (en) * 2006-12-12 2009-06-24 ザイオソフト株式会社 Image display control apparatus, an image display control program and an image display control method
US20100251304A1 (en) 2009-03-30 2010-09-30 Donoghue Patrick J Personal media channel apparatus and methods
US9129655B2 (en) * 2009-06-25 2015-09-08 Visible World, Inc. Time compressing video content
KR101893075B1 (en) * 2012-02-28 2018-08-30 삼성디스플레이 주식회사 Organic Light Emitting Display Device and Driving Method Thereof
US9467723B2 (en) 2012-04-04 2016-10-11 Time Warner Cable Enterprises Llc Apparatus and methods for automated highlight reel creation in a content delivery network
JP2015122734A (en) * 2013-11-25 2015-07-02 パナソニックIpマネジメント株式会社 Imaging apparatus and imaging method
US9948962B2 (en) 2014-11-13 2018-04-17 Time Warner Cable Enterprises Llc Apparatus and methods for efficient delivery of electronic program guide data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BE1004491A3 (en) * 1990-07-20 1992-12-01 E V S Systems Procedure and system for processing video signals

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR1004491A (en) * 1947-04-30 1952-03-31 Coffin
JPH053793B2 (en) * 1984-07-31 1993-01-18 Matsushita Electric Ind Co Ltd
US5072296A (en) * 1990-02-23 1991-12-10 Massachusetts Institute Of Technology Video frame reduction/reconstruction method and apparatus
JP3161614B2 (en) * 1991-11-30 2001-04-25 ソニー株式会社 Moving picture decoding apparatus
JP2708312B2 (en) * 1992-03-05 1998-02-04 松下電器産業株式会社 Recording apparatus and reproducing apparatus
GB2268354B (en) * 1992-06-25 1995-10-25 Sony Broadcast & Communication Time base conversion

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BE1004491A3 (en) * 1990-07-20 1992-12-01 E V S Systems Procedure and system for processing video signals

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
RECORDS OF THE 13TH INTERNATIONAL TV SYMPOSIUM; 28 MAY-2 JUNE 1983, MONTREUX, CH, pages 409-422, WILKINSON ET AL. 'Replay of Digital Recordings at non-standard Tape Speeds' *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5592299A (en) * 1994-09-26 1997-01-07 Hitachi America, Ltd. Method and apparatus for reducing the amount of data required to represent a video frame
US5684539A (en) * 1994-09-26 1997-11-04 Hitachi America, Ltd. Method and apparatus for processing encoded video data to reduce the amount of data used to represent a video image
US5715176A (en) * 1996-01-23 1998-02-03 International Business Machines Corporation Method and system for locating a frame position in an MPEG data stream
WO2005111989A2 (en) * 2004-05-19 2005-11-24 Sony Computer Entertainment Inc. Image frame processing method and device for displaying moving images to a variety of displays
WO2005111989A3 (en) * 2004-05-19 2006-09-28 Sony Computer Entertainment Inc Image frame processing method and device for displaying moving images to a variety of displays
EP2309486A1 (en) * 2004-05-19 2011-04-13 Sony Computer Entertainment Inc. Image frame processing method and device for displaying moving images to a variety of displays
US8559798B2 (en) 2004-05-19 2013-10-15 Sony Corporation Image frame processing method and device for displaying moving images to a variety of displays

Also Published As

Publication number Publication date Type
US5543927A (en) 1996-08-06 grant
US5881202A (en) 1999-03-09 grant
JPH0799634A (en) 1995-04-11 application
EP0622957A3 (en) 1995-06-07 application
JP3557244B2 (en) 2004-08-25 grant
US5671318A (en) 1997-09-23 grant

Similar Documents

Publication Publication Date Title
US5523803A (en) DMD architecture and timing for use in a pulse-width modulated display system
US6424792B1 (en) Subtitle encoding/decoding method and apparatus
US4839743A (en) Interactive video and audio controller
US5021772A (en) Interactive real-time video processor with zoom pan and scroll capability
US4688106A (en) Video processing system using multi-head disc store
US4513374A (en) Memory system
US5309292A (en) Digital recording and/or reproduction apparatus of video signal rearranging main components within a fixed length block
US4992886A (en) Method and apparatus for encoding data within the subcode channel of a compact disc or laser disc
US5136391A (en) Digital video tape recorder capable of accurate image reproduction during high speed tape motion
US4361849A (en) Video disc vari-speed playback system
US5912710A (en) System and method for controlling a display of graphics data pixels on a video monitor having a different display aspect ratio than the pixel aspect ratio
US4233631A (en) Television system for displaying and recording paths of motion
US5657095A (en) System for Combining image signals
US4890833A (en) Apparatus for generating enhanced interactive video game playfield environments
US5493456A (en) Method and apparatus for increasing the recording time of a digital video tape recorder
US4752836A (en) Method and apparatus for reproducing video images to simulate movement within a multi-dimensional space
EP0469861A2 (en) Method of recording a digital video signal
US5815604A (en) Interactive image manipulation
US6771830B2 (en) Differential pulse code modulation image compression with varying levels of quantizers
US20060117357A1 (en) Methods and systems for controlling trick mode play speeds
US5384598A (en) System and method for frame differencing video compression and decompression with frame rate scalability
US4179709A (en) Video information bandwidth compression
US5450489A (en) System and method for authenticating software carriers
US4766541A (en) Apparatus for generating interactive video game playfield environments
US5146324A (en) Data compression using a feedforward quantization estimator

Legal Events

Date Code Title Description
AK Designated contracting states:

Kind code of ref document: A2

Designated state(s): DE FR GB

AK Designated contracting states:

Kind code of ref document: A3

Designated state(s): DE FR GB

17P Request for examination filed

Effective date: 19951130

17Q First examination report

Effective date: 19970113

ITF It: translation for a ep patent filed

Owner name: BIANCHETTI - BRACCO - MINOJA S.R.L.

18D Deemed to be withdrawn

Effective date: 20100730