Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/79—Processing of colour television signals in connection with recording
  • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
  • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
  • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/76—Television signal recording
  • H04N5/91—Television signal processing therefor
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/76—Television signal recording
  • H04N5/78—Television signal recording using magnetic recording
  • H04N5/781—Television signal recording using magnetic recording on disks or drums

Definitions

• the field of the present invention relates generally to digital data compression of video data, and more particularly relates to the coding of successive frames of video data in a selective manner to provide enhanced data compression.
• MPEG Motion Picture Expert Group
• the sequence of raw image data frames are divided into successive groups known as GOP's (group of pictures), respectively, and the coded GOP is comprised of independent frames I, predicted frames P and bidirectionally predicted frames B in such manner that GOP may be comprised, for example, as follows: I, B, B, P, B, B, P, B, B, P, B, B, P, B, B, P, B, B, B, B.
• the first P frame is derived from the previous I frame, while the remaining P frames are derived from the last previous P frame.
• the I and P frames are reference frames. Since each B frame is derived from the closest reference frames on either side of it, the pertinent P frames must be derived before the prior B frames can be derived.
• the high definition independent frames I at the beginning of each GOP are required because of the use of frame differential encoding to avoid accumulated error.
• the purpose of quantizing is to control the number of bits used in representing the changes between the frames.
• the corresponding portions of frames are conveyed by motion vectors indicating where blocks in the reference frames may be located in the frame being derived.
• PVR's Personal Video Recorders
• storage capacity in the order hundreds of Gigabytes (a Gigabyte represents 10 bytes), whereby a user will be able to store many hours of video content (a 120 Gigabyte disk presently stores up to 60 hours of broadcast quality video, and even more hours if a higher compression ratio is used). Users typically want to store increasingly greater video content or data, but typically users will not remove video content even if it isn't required.
• MPEG is (more or less) optimized for broadcast for decoders that only have a relatively small amount of memory.
• a video program can be started without history information from every I-frame.
• File sizes can be decreased by converting a stream of video data into files with a smaller size through use of higher compression ratios like MPEG-4 or DivX.
• these file formats don't use the common parts in the stored files, and quality will be reduced by transcoding. Since large quantities of video data or information can be stored on a hard disk, and it is possible to access such information therefrom almost instantaneously, it is no longer required to treat the video data as linear media, in view of the quantitative digitized data now available on a hard disk for permitting highly accurate reproduction of the original video data.
• Figure 1 is a block diagram showing three recorded MPEG files representing compressed video data divided into successive groups of pictures (GOP's);
• Figure 2a is a block schematic diagram for illustrating one embodiment of the invention;
• Figure 2b is a block schematic diagram illustrating another embodiment of the invention;
• Figure 2c is a block schematic diagram showing yet another embodiment of the invention;
• Figure 3 is a block schematic diagram showing an embodiment of the invention;
• Figure 4 is a block schematic diagram showing another embodiment of the invention that includes a virtual file system with an abstraction layer;
• Figure 5 is a sequence diagram of read requests relative to the embodiment of the invention of Figure 4;
• Figure 6 is a flowchart for an embodiment of the invention;
• Figure 7, is a block diagram showing show that a reference can point to another GOP and a differential code that is used to adapt the original GOP for another embodiment of the invention;
• Figure 8 is a block schematic diagram showing yet another embodiment of the invention, showing the transformation of a GOP into another representation
• the group of pictures (GOP) of recordings previously compressed are searched to identify redundant or identical frames/GOP's. For each GOP that is repeated one or more times, the GOP is replaced by a reference designation identifying that GOP or common block of frames. Not all GOP's need a reference, if a complete block is equal then only 1 reference with a length indicator is required. The first appearing common GOP is left in place, whereas the subsequent redundant or repeating corresponding frames or GOP's are identified by the reference designation for that frame or GOP.
• a GOP is defined as a group or number of pictures that can each be played independently.
• the method or idea is to first use a PVR to record video data online, followed by offline sequential searching through the stored video data or programs for frames that are identical. Next, pointer references (triggers) are used to identify repeating parts. On a typical PVR the total video file most recently recorded, and the video files from other recordings are accessible, thereby permitting reference to other video portions to be made.
• MPEG video contains I, P, and B frames, whereby video cannot be started from a random position, but only from an I-frame. Therefore, in the present invention a GOP is defined as a group of pictures that can be played as a separate unit. If a TV program contains triggers and the trigger part is already present on the hard disk, the video fragment doesn't even have to be recorded. Some examples: -When a sport event like soccer is recorded it is very likely that it contains replays. An offline optimization could be used to reduce the total file size.
• each frame is represented by an identifier or fingerprint.
• the easiest identifiers are an average colour index, a CRC code or audio fingerprinting, for example.
• the PVR may be possible to access an Internet server that has already calculated common blocks across popular programs. In this manner a reduction can be obtained in the number of CPU/disk cycles required to calculate identifiers for GOP's.
• the offline optimization schema could be implemented as a process that has a lower priority than the file recording processes of the PVR, and thus only be active if there's are enough CPU/disk cycles available for the optimization.
• a block schematic diagram is shown of three recorded MPEG files 2, 4, and 6, designated as a "First Recording, Second Recording, and Third Recording," respectively.
• PVR personal video recorder
• the invention is not limited to personal video recorders, and may be used in association with other video recorders.
• the recorded video data files can be reduced in size by eliminating redundant groups of pictures or GOP's.
• movies include picture data having adjacent audio track(s). It is assumed that the audio is either encoded in the GOP or present on the disk as a separate track of GOP's.
• the video data files can then be reduced offline through use of the present invention.
• a first embodiment of the invention as shown in Figure 2a, assume that it is determined that GOP5 of the first recording 2 is identical to GOP3.
• the video data representing GOP5 is then replaced by a simple reference designation, such as 19, for example, in association with a pointer, namely pointer reference 19, pointing to GOP3.
• a simple reference designation such as 19, for example, in association with a pointer, namely pointer reference 19, pointing to GOP3.
• a pointer reference 19 is detected, GOP3 with then be played back again, followed by the remaining video data for GOP6 and GOP7, in this example.
• the first recording 2 is reduced in size by removing the common blocks including GOP3, and GOP5, followed by assigning pointers 20 and 21 for pointing to the common block GOP3 for the appropriate beginning positions in the first recording 2 for GOP3 and GOP5, respectively, as shown.
• the pointer reference 22 shows that GOP5 is identical to GOP3, which is repeated to obtain GOP5, along with the further reference indication "two GOP's, play one time" meaning that after GOP3 is replayed for GOP5, GOP3 is then replayed as GOP6, followed by the playing of GOP7, for completing the playback of the first recording 2.
• Figure 3 Another embodiment of the invention is shown in Figure 3, showing an example of first, second, and third recordings 2, 4, 6, respectively, for which common blocks or common GOP's are detected between the programs of recordings 2, 4, and 6, respectively.
• Reference pointer 23 along with its associated coding, shows that for the first recording 2, after GOP3, GOP4 and GOP5 must be played, followed by GOP6 and GOP7.
• pointer reference 24 shows that for GOP1 1 followed by GOP12, the identical blocks GOP4 and GOP5, respectively are played to complete the playback of the second recording 4.
• a reference pointer 25 shows that after playing GOP15, GOP4 and GOP5 are next played in substitution of GOP 16 and GOP 17, respectively, followed by playing GOP 18, and GOP 19.
• the redundant blocks of video data or GOP's can be placed into a table to which associated pointer references point for retrieval.
• Figure 4 Another embodiment of the invention is shown in Figure 4, wherein a virtual file system is used with the PVR. Assume that a video recording 40 containing GOP1 through GOP6 represents the video data that is to be played back. Reference pointers 26 through 31 direct GOP's 1 through 6, respectively to an abstraction layer 42.
• the abstraction layer 42 is programmed to re-direct reads or GOP's to another position on the associated hard disk, and to adapt if necessary. During playback only GOP1 , GOP2, GOP5, and GOP6 are read from hard disk 44 via pointers 33 and 35, 32 and 36, 37, and 38, respectively, as shown. A translation table 46 is also stored on hard disk 44.
• the abstraction layer 42 includes a programmed area 48 for the "Normal Read Blocks 48.”
• the abstraction layer 42 also includes a program area 50 for reading blocks or GOP's from different positions on the hard disk 44, with possible adaptation. Also as shown, the abstraction layer 42 obtains from the translation table 46 information indicating that GOP3 shouldn't be read from disk but instead GOPl is read again.
• the length parameter indicates that also G0P4 will be replaced by GOP2.
• the length parameter is used make a sort reference parameter if a sequential group of GOP's can be replaced with another group of GOP's.
• a PVR is provided with a virtual file system in another embodiment of the invention.
• an abstraction layer 42 is programmed to be operative to re-direct reads from a recording 40 including groups of pictures GOPl through GOP6 connected via signal lines 26 through 31, respectively, to the abstraction layer 42.
• a video player in this example, reads GOP's 1 through 6, but only GOPl, GOP2, GOP5, and GOP6 are read from the hard disk 44. GOPl and GOP2 are either read twice, or they can be cached.
• the abstraction layer 42 includes a section 48 to normally read blocks GOPl, GOP2, GOP5, and GOP6 shown connected to the abstraction layer 42 area via signal lines 35 through 38, respectively.
• a section 50 of abstraction layer 42 includes an area for reading blocks from different positions on the associated hard disk, wherein the reads are made with possible adaptation. In this example, as indicated, blocks GOPl and GOP2 are read twice, the second time into abstraction layer area 50 via signal lines 32 and 33, respectively.
• the hard disk 44 includes a translation table 46, which is read into a translation table area 52 on the abstraction layer 42.
• GOP3 points to GOPl with a length equal to two.
• the reads are conducted between a video player 60, a file abstraction layer 42, and adapt block read 50, and a storage device 44, as previously mentioned.
• a video player 60 For blocks that are replaced some form of adaptation may be needed, i.e. for blocks that read via adaptation layer 42 signal lines 32, 33 it is possible that the Time Stamps, the duration of the block, audio and or the resolution must be adapted.
• GOPl and GOP2 are converted (signal lines 32 and 33) such that the video player receives data as if GOP3 and GOP4 are read from disk.
• Block 44 in Figure 4 represents a random storage device like a hard disk drive that is used to store data.
• the optimization process makes estimates of which blocks have a high chance of similarity with other blocks. Blocks so identified are searched first. Estimates are made by detecting which blocks are based on some similarity, such as in title, genres, and further based upon previous knowledge from optimizing similar files/titles.
• the first step 70 is to check which video titles are new since the last optimization was accomplished.
• the next step 71 is to check whether an Internet connection is available. If the connection is available, a request is then made in step 77 for a similarity table from an associated Internet server, followed by step 78 for adapting the table to the format used by the local device, a video player in this example.
• step 72 the table is then used to update a local reference table kept on hard disk, by adding new titles, and a GOP index map, as described below in the example of Table 1. If no Internet connection is available, updating step 72 is then directly entered from step 71.
• step 73 a first estimate is made as to which new titles have a high chance of having repeating GOP's, whereby these titles are placed in a list of titles to be scanned (scan list).
• step 74 a scan is performed for repeating GOP's per title, whereby after scanning a title and updating the GOP references, the title is removed from the aforesaid scan list.
• step 79 it is determined whether there are more titles to scan, and if so, the immediately previous scanning step 74 is repeated. If not, the next step 80 is to perform a scan for GOP's across MPEG titles.
• step 75 an estimate is made as to which titles have a high chance of having similar blocks, and the blocks identified are then placed into a scan list.
• step 81 a scan is made of the first two titles from the scan list for duplicate blocks or GOP's.
• step 82 it is determined whether there are more titles to scan, and if so the previous estimating step 75 is repeated for new scans that are made. A further check is then made in step 82 to determine if there are any new video titles on the hard disk which have not been processed for estimating whether they might be repetitive, and if more are detected then the process is repeated by returning to step 70. If no titles are left to scan, the preliminary optimization process is exited or terminated at step 76.
• the previously described estimating routine for the optimization process permits reference tables, such as Table 1 and Table 2, as shown below to be developed, for example.
• Table 1 shows three recorded programs along with their associated title, the genre to which they relate, the GOP identification reference referring to Table 2, the latter including a list of the GOP's used per title.
• the optimization process makes estimates as to which of the recordings have a high probability of including similarities, that is similar GOP's.
• the optimization process will then begin by processing the recordings likely have similarities.
• the process typically begins looking for similar blocks in the first recording.
• the probability values can be based on data gathered during a previous optimization, that is during a recording that was previously searched for similar blocks that typically has a low chance for detecting new similar blocks reprocessed. If so, recordings that are associated with a certain genre, or that are recorded from a common broadcast channel (i.e. with lots of commercials), could have a greater probability of duplicate GOP's.
• the reference relative to these duplications contains a counter of the number of common blocks to repeat, and a repetition factor for the number of times these blocks need to be played.
• GOP5 and GOP6 do not require a pointer to a physical sector on the hard disk.
• GOP5 has the same identifier as GOP3, and GOP6 has the same identifier as GOP4.
• the Translation Table 4 shows for example that GOP5 corresponds to GOP3, and GOP6 corresponds to GOP4. It is also indicated that these pairs of blocks must be played two times, that is for a length of two GOP's.
• the method of the present invention as described above for reducing the size of previously stored video data or information by using references presented on a basis of Group of Pictures (GOP's) or complete frames can be extended to replace portions of GOP's or frames with other portions that are present on the hard disk, and/or use other GOP's in trick play mode. It is also possible that GOP's that are substantially but not absolutely identical can be described through notation of the differences through use of a delta code. However, such an extension must be compromised in view of available processing power, complexity, and the overall efficiency of the optimization method. In the given examples a fingerprint or crc is used to identify similar blocks, however the given examples are simplified in the sense that the matches between GOP's are identified with an exact match (equal number for the GOP's).
• a GOP5 is recreated using both a reference to GOP3 via a pointer arrow 62, and a reference to Diff GOP3,5 via pointer arrow 63.
• Diff GOP3,5 contains a differential code that transforms GOP3 into a replacement for GOP5 that more closely resembles the original GOP5 than might otherwise be obtained.
• a dual reference is only viable if the code for Diff GOP3,5 is substantially smaller (less disk space or memory area) than the original code.
• An extension of this embodiment of the invention is shown in Figure 8, wherein a differential code is utilized, and GOP5 is reconstructed using GOP3 and Diff GOP3,5 as a starting point, in this embodiment. As shown by arrow 64, the first block 61 is copied directly to GOP5 in block 616.
• the remaining six blocks, namely 610 through 615, respectively, from GOP3, in this example, are transferred with the code in Diff GOP 3,5 into blocks 617 through 622, whereby blocks 623 and 624 are copied directly from Diff GOP3,5 in this example (see arrows 65).
• various embodiments of the present invention have been shown and described above, they are not meant to be limiting. Those of skill in the art may recognize certain modifications to these embodiments, which modifications are meant to be covered by the spirit and scope of the appended claims.
• the present method as described above shows how the file sizes for a PVR can be reduced, by eliminating redundancies in video data that are stored on an associated hard disk
• the present method is not so limited.
• the present method can be applied for use with other media that also are capable of being randomly accessed, such as a user having a collection of storage devices that are connected to each other, whereby the total file size therebetween can be decreased through use of the present invention.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Television Signal Processing For Recording (AREA)
• Signal Processing For Digital Recording And Reproducing (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=34102921

Family Applications (1)

Country Status (5)

Families Citing this family (11)

Family Cites Families (3)

• 2004
  • 2004-07-21 KR KR1020067001613A patent/KR20060038458A/ko not_active Application Discontinuation
  • 2004-07-21 CN CNA2004800210930A patent/CN1826805A/zh active Pending
  • 2004-07-21 JP JP2006520975A patent/JP2006528862A/ja active Pending
  • 2004-07-21 WO PCT/IB2004/051279 patent/WO2005011269A1/en not_active Application Discontinuation
  • 2004-07-21 EP EP04744634A patent/EP1652379A1/en not_active Withdrawn

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events