WO2002078357A1 - Storage of multi-media items - Google Patents

Storage of multi-media items Download PDF

Info

Publication number
WO2002078357A1
WO2002078357A1 PCT/IB2002/000761 IB0200761W WO02078357A1 WO 2002078357 A1 WO2002078357 A1 WO 2002078357A1 IB 0200761 W IB0200761 W IB 0200761W WO 02078357 A1 WO02078357 A1 WO 02078357A1
Authority
WO
WIPO (PCT)
Prior art keywords
quality
media item
stored
memory
item
Prior art date
Application number
PCT/IB2002/000761
Other languages
French (fr)
Inventor
Renatus J. Van Der Vleuten
Richard P. Kleihorst
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Priority to JP2002576449A priority Critical patent/JP2004524627A/en
Priority to EP02702668A priority patent/EP1374600A1/en
Publication of WO2002078357A1 publication Critical patent/WO2002078357A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/4147PVR [Personal Video Recorder]
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10Digital recording or reproducing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334Recording operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4335Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/7921Processing of colour television signals in connection with recording for more than one processing mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure

Definitions

  • the invention relates to storing of multi-media items in a memory.
  • An object of the invention is to provide advantageous use of the available storage capacity.
  • the invention provides a method of storing, a storage device and a recording apparatus as defined in the independent claims.
  • Advantageous embodiments are defined in the dependent claims.
  • a first embodiment of the invention provides storing at least one multi-media item in a memory, determining in an automated manner a quality level at which the multimedia item should be stored, which quality level at least partially depends on user preferences, and storing the multi-media item in the memory at the determined quality level. Automatically determining the quality of the multi-media item to be stored based on user preference has the advantage that a user need not indicate per item its desired quality.
  • more preferred multi-media items are stored at a higher quality than less preferred multi-media items. More preferred corresponds to a higher potential user interest and less preferred to a lower potential user interest.
  • the potential user interest is determined from the user preferences. Therefore the desired quality levels are determined automatically by monitoring user preferences.
  • the quality levels can cover the complete range from perfect lossless storage of an item down to completely deleting an item.
  • the quality levels correspond to the (estimated) user's preferences.
  • the storage system usage is also taken into account. In the case the storage capacity is not enough to store a new multi-media item, the quality levels of the previously stored multi-media items and of the multi-media item to be stored are reduced as much as is needed to store all items in the memory.
  • the respective reduced quality levels depend on the respective preference levels of the multi-media items. The more the item is preferred, the higher is its relative quality.
  • Multi-media items which have already been consumed or outputted once may be stored at a further reduced quality. This embodiment is especially useful for video items: if a video item has been viewed its potential interest for the user is lower. When the quality of a given multi-media item reduces below a given
  • the given multi-media item may be automatically deleted.
  • the threshold may be set as default by the set maker, but may also be set by the user.
  • WO 00/39707 discloses a video retrieval system that allows a user to quickly and easily select and receive stories of interest from a video stream.
  • the video retrieval system classifies stories and delivers samples of selected stories that match each user's current preference. This document further discloses the use of selective erasure.
  • the retrieval system identifies the segments that are on the recording multi-media that have the least correlation with the user's preferences. Instead of replacing the oldest segments with the newest segments, the segments of least potential interest to the user are replaced by the newest segments.
  • the retrieval system also terminates the recording of the newest segment when it determines, based on the classification of the newest segment by the classification system, that the newest segment is of no interest to the user, based on the user preferences.
  • characteristics of multi-media items consumed by the user are stored and a user profile is built based on these characteristics. Characteristics of multi-media items can be determined in a convenient way by monitoring meta data which is sent together with the multi-media items. The meta data indicates characteristics of the multi-media item, such as type of program, etc. See e.g WO 00/39707.
  • Fig. 1 shows a recording apparatus according to an embodiment of the invention
  • Fig. 2 shows an implementation of the storage memory according to an embodiment of the invention
  • Fig. 3 shows a schematic representation of a user interface for a recording apparatus
  • Fig. 4 shows an exemplary mapping of a Peak Signal to Noise Ration (PSNR) value to a color used as a quality indication
  • Fig. 5 shows a pseudo-code for converting a PSNR value into a color value (24-bit RGB);
  • Figs. 6 and 7 show a representation of a user interface for a recording apparatus after storing of 6 and 12 multi-media items respectively.
  • Fig. 1 shows a recording apparatus 2 according to an embodiment of the invention.
  • the recording apparatus comprises an internal data bus 20 to which several elements are coupled, e.g. an encoder, a storage medium, a decoder, an input I/O, a user interface and an output I/O.
  • the input I/O is coupled to a source device 1.
  • the source device 1 furnishes multi-media items to the recording apparatus 2.
  • the source device 2 may be present in the recording apparatus, e.g. as a camera in a digital camera application. Further, the source device may also be a transmitter from which the multi-media items are obtained.
  • the recording apparatus may further be coupled to an output device 3, such as a display and/or speaker.
  • the output device may also be present within the recording apparatus.
  • the multi-media items may be outputted. Further, the output device can be used as part of the user interface. Control means are further distributed in the recording apparatus over the several elements. It is also possible to use a Central Processing Unit (CPU) to control the several elements and the data transport on the internal data bus 20.
  • CPU Central Processing Unit
  • the invention may be incorporated in a storage device which may be implemented in an integrated circuit, in that case without a source device, output device and storage memory. In the following, two implementations of the recording apparatus of Fig. 1 are discussed.
  • the first implementation is based on the elastic memory concept.
  • the elastic memory (PHNL000110) is a concept for storing a variable number of multi-media items on a storage medium of fixed capacity. When only few items are stored, they are stored at high quality. If more items need to be stored, the quality (i.e., amount of storage space that is occupied) of the already stored items is automatically reduced to make room for the new items.
  • the elastic memory offers each individual user the option to select her/his personally preferred quality-capacity tradeoff. Some users may prefer to store a larger amount of data and are willing to accept a lower data quality. For example, because they own a small-screen television on which visual quality losses are less visible than on a large-screen television.
  • the elastic memory allows certain multi-media items to be stored at a higher quality than others according an embodiment of the present invention. This can be realized in the elastic memory by changing the measure (quality improvement or distortion reduction or perceptual significance, etc.) that is used to decide the relative importance of each data
  • the importance values of certain preferred multi-media items might be increased by 20%, when desired by the user.
  • an elastic memory application contains a large number of items, for example in case of "automatic" video program recording, such as in a "TiVo box", it is very inconvenient or even impossible for the user to indicate her/his preferred quality for each item, while the user might still prefer to have favorite programs stored at a higher than average quality.
  • the quality levels correspond to the (estimated) user's preferences. The quality levels can cover the complete range from perfect lossless storage of an item down to completely deleting an item.
  • the implementation can be done using the elastic memory concept, so only a solution for obtaining the desired quality levels is required.
  • these classifications can be used to set quality levels. This may be done by asking the user to enter preferences for one or more classes.
  • the quality levels may also be obtained by measuring which class of items is most frequently retrieved by the user and increasing the quality levels for more frequently retrieved classes and decreasing them for less frequently retrieved classes.
  • the classification information could, for example, be broadcast together with the program, or as side information in a program guide. Or it could be purchased from a service provider through the cable network or the internet. If no classification information is available, quality levels can be determined by monitoring both the time of storage and the frequency of retrieval.
  • a disadvantage of modifying the elastic memory block significance values is that the original values are not preserved.
  • significance modification parameters such as e.g. a scale factor.
  • Fig. 2 shows a main memory and an auxiliary memory to implement the elastic memory concept, with an exemplary content.
  • the main memory and the auxiliary memory may be present in the storage medium of Fig. 1. See for a detailed description of the use of the main memory and the auxiliary memory PHNL000110, which is incorporated by reference herein.
  • the main memory is divided into N memory parts for storing N data pieces.
  • N 11.
  • the auxiliary memory is typically smaller than the main memory and is used for administrative purposes.
  • the auxiliary memory comprises N records, each record comprising several fields.
  • a first field is a Pointer to Main Memory (P), which comprises a pointer to a location in the main memory 31 that holds the data associated with a given record.
  • a second field is an Object Identifier (I) that comprises information describing the data item to which the data piece stored in the main memory belongs.
  • the object identifier refers to, for example, an image out of a set of images stored by a digital camera. In a practical embodiment the object identifier is zero in case a memory part is not assigned to a data item, e.g.
  • a third field comprises a Significance (S).
  • S Significance
  • the S field gives a measure of the significance of the data piece stored in the main memory where the record refers to.
  • the entries in the significance fields are preferably non-negative numbers.
  • the auxiliary memory preferably has the property that the records are sorted on the significance. It is also possible to sort the records in a different manner, e.g. on the identifier to group all data pieces belonging to the same data item. Within the respective groups of data pieces, the data pieces may be sorted on significance.
  • the data item is coded in the encoder (see Fig. 1) to produce a scalable bit-stream, which is split in to data pieces. The pieces are subsequently processed.
  • the significance measure of each piece is first compared to that of the block with the lowest significance measure currently in the memory. If the significance of the new data piece is lower, it is not stored in the main memory, i.e. it is discarded. When a first piece is not stored, the processing can be stopped because the further data pieces have lower significance than the current data piece (which is a property of a scalable coding mechanism). Further, a scalable bit-stream wherein a data piece is missing in the sequence is not decodable. If the significance is higher, the new data piece is written to the main memory at the position of the current least significant data piece (which is obtained from the last position of the auxiliary memory in case the records are sorted on significance).
  • the last record of the auxiliary memory is replaced by the record data for the new data piece and the records in the auxiliary memory are re-ordered to restore the ordering on significance. It is advantageous to start processing with the most significant data piece of an item (and then with the subsequent data pieces having lower significance), because this is the order in which the data pieces are produced by the scalable coder, and further because these data pieces are not overwritten by blocks belonging to the same item since these are less significant.
  • the records in the auxiliary memory are subsequently processed and if the object identifier matches that of the data item to be extracted, the data piece in the main memory pointed to by the record is sent to the (scalable) decoder. Since the auxiliary memory is traversed starting at the highest significance, the data pieces will be extracted at the right order, allowing the decoder to form a progressively better reconstruction.
  • the records in the auxiliary memory are subsequently processed and if the object identifier matches that of the data item to be deleted, the significance of the record is set to a (predetermined) value that is lower than any value that can be produced by the encoder, e.g. zero.
  • the identifier is set to a predetermined value, e.g. zero to indicate that the memory part is not assigned to a data piece.
  • the records in the auxiliary memory are then re-ordered to restore the ordering on significance, i.e. such that the records of the lowest significance are placed at the end.
  • auxiliary memory is preferred. However, it is also possible to omit the auxiliary memory. In that case the significance of a data piece and an identifier to which data item it belongs should be stored in the main memory. Because pointers to data pieces in order of significance are in this case not available, searching the main memory takes more time. To reduce searching, the data pieces may be sorted in the main memory, at the cost of switching much larger amounts of data. Also a Content Addressable Memory can be used to implement the auxiliary memory, eliminating the need for sorting and searching in the auxiliary memory by a separate processor. Furthermore, more advanced data structures such as heaps or trees, which are generally known, may be used for performing the administrative functions, as an alternative to the preferred auxiliary memory data structure. These alternatives could be advantageous for example in a software implementation (or hardware when sufficient clock cycles are available), especially for a large number of memory parts.
  • the additional information is stored for each data item.
  • the additional information may include name, type of information, color, size, etc.
  • This additional information may be stored in the main memory, e.g. together with the first data piece.
  • the additional information is stored in the auxiliary memory, which makes it easier to retrieve this additional information.
  • a significance modification parameter is stored in the memory as additional information.
  • Fig. 2 an extra field 'Significance modification' has been added to the auxiliary memory.
  • the significance modification factor is a scale factor. As can be seen, for object (multi-media item) 1 the significance values stored in the significance field is 1.2.
  • any combination of two values out of the three values: original significance value, modified significance value and modification parameter can be stored.
  • the third value can always be calculated by combining the two stored values. For example, one may use the following relationship:
  • Modified significance value original significance value * modification parameter
  • the further implementation is based on MPEG storage such as currently used in the TiVo box.
  • the TiVo box allows a user to select a certain desired quality level for recording.
  • the quality levels are (basic, medium, high, best). This quality selection directly corresponds to a certain bit rate at which the video is encoded using MPEG.
  • the above-mentioned quality levels correspond to a storage capacity of (30, 18, 14, 9) hours, respectively.
  • the levels offered to the user are not true quality levels, since they specify a certain bit rate instead of a quality.
  • the resulting quality still depends on the content of the program that is recorded.
  • variable bit rate encoding mode e.g. constant quantizer step size
  • an average recording quality can be determined and the quality for recording certain programs can be automatically increased or decreased by e.g. 20%, depending on the automatically derived user preferences.
  • the TiVo box sets the bit rate of the program instead of the quality.
  • an average bit rate for recording is selected and this bit rate is automatically modified by a certain percentage depending on the automatically determined user preferences.
  • the total recording capacity, expressed in hours of video recording can at each time be exactly determined.
  • An important improvement on this "TiVo-like" method could be to also take into account the program content. If e.g.
  • the encoding bit rate may be adjusted to better approach the desired user preferences, by automatically increasing the bit rate for "difficult" content like sports or action. For example, a 20% higher than average desired quality could result in e.g. a 40% higher than average bit rate for a sports program, and only a 5% higher than average bit rate for a talkshow (since content with little movement will automatically get a higher quality because of the constant bit rate recording, even when a higher than average quality is preferred this content could even be recorded at a lower than average bit rate!). When no meta data for the content is available, this can possibly be obtained by analyzing previous recordings.
  • the complexity or "difficulty" i.e. the resulting program quality at a certain encoder setting
  • the complexity or "difficulty" can be analyzed from previous recordings and used to adjust the settings for future recordings.
  • the automatic adaptation of recording bit rate according to program content could also be implemented in the current TiVo system, in order to improve the correspondence between the quality setting selected by the user (basic, medium, high, best) and the actual resulting recording quality. This feature is therefore also applicable to embodiments outside the inventive concept of the current invention.
  • the invention can be applied in all multi-media storage systems that allow some stored data items to be stored at a higher quality than others, preferably with many possible different quality settings. Examples of applications are (automatic) video recording ("TiVO box") or a consumer home audio and/or video "jukebox".
  • Tele box a consumer home audio and/or video "jukebox”.
  • the invention can of course also be applied in case of multiple users (accessing for example a remote database or library).
  • One may store on the HD of the PC items at varying quality (bitrate) in an automated manner depending also on user preferences.
  • the invention is further applicable in transmission and downloading of e.g. MP3 files. Prior to transmission, the user preferences may be consulted by the transmitter in order to determine the bitrate of an item to be transmitted. This has the advantage that no more bits are sent than necessary.
  • the transmitter is able to allocate a limited bandwidth to respective multi-media items, which allocation is based on the user preferences.
  • items of more potential interest to the user can be allocated more bandwidth at the cost of the bandwidth of a less preferred item in order to provide the more preferred item at a higher quality than the less preferred item.
  • the receiver determines the quality the item should have and thereafter requests the transmitter to deliver the item at the determined quality (see Fig. 1)
  • the elastic memory allows a variable number of items to be stored on a storage device of fixed capacity.
  • the storage space occupied by each item is automatically adjusted in case subsequently more items have to be stored. This adjustment is accompanied by a corresponding quality reduction. Therefore, it is desirable to provide the user with current information about the media item quality.
  • the elastic memory also allows different items to be stored at a different quality, according to the user' s preferences. This feature must also be supported by the user interface. In the following user interface features related to the quality of the stored media items in a memory-based consumer device will be discussed within the scope of the current invention. These features are also applicable to other embodiments outside the scope of the current invention, in which other embodiments a quality of stored multi-media items may be modified.
  • a color can be used to indicate the quality of individual items (colored background, border, dot etc.) or the overall item quality (guaranteed minimum or average).
  • some representation of the stored item is used together with the quality indication.
  • the color could e.g. range from green for high quality through yellow for medium quality to red for low quality.
  • a linear scale could be used depending on the PSNR (see Fig. 5) or any other quality measure.
  • a bitrate of a stored media item may be used.
  • Each individual stored item can be assigned a "protect button". As long as this button is active, the associated media item is protected from any quality loss when subsequently more items are stored. This can be used, e.g. in a camera application to protect a picture that the user likes very much. This can be implemented by temporarily (while the button is active) replacing the elastic memory block significance values by a value that is larger than any value that can otherwise occur.
  • Each individual stored item can be assigned a "higher/lower than average quality” button or setting (i.e. allowing multiple values) which will influence the item quality when subsequently more items are stored.
  • This option can also be used as a "soft-delete” function: when the user does not like a certain item, it may be stored at a lower than average quality; under certain circumstances, it might even be automatically deleted.
  • This feature can be implemented, e.g., by scaling (multiplying by a constant) the elastic memory block sigmficance values for the items according to the quality setting. 4)
  • it is also desirable to be able to influence the quality at which the next item will be stored e.g.
  • a user may not be interested in constantly monitoring the item quality, but only be interested in maintaining a certain guaranteed item quality (which he/she can preset preselect). In that case, an "audio (e.g. a beep) and/or visual (e.g. a blinking light) quality warning signal" may be given when the item quality goes below the predetermined threshold, or, alternatively, is expected to go below this threshold after storing a certain additional number of items (for example two more pictures).
  • an "audio (e.g. a beep) and/or visual (e.g. a blinking light) quality warning signal” may be given when the item quality goes below the predetermined threshold, or, alternatively, is expected to go below this threshold after storing a certain additional number of items (for example two more pictures).
  • the stored items can be images, but e.g. also audio or video items that may also be represented by thumbnails (e.g. the CD/DVD cover print) and/or track names and/or individual scene thumbnails, in particular of key frames.
  • thumbnails e.g. the CD/DVD cover print
  • track names and/or individual scene thumbnails in particular of key frames.
  • Figure 4 shows a schematic representation of the user interface for two images.
  • the images that are shown are just the thumbnails; the full image is displayed if the user clicks on one of the thumbnails.
  • An image can be deleted by clicking on the Remove button below its thumbnail.
  • the numbers above the image thumbnails indicate the number of memory blocks (bl) allocated to the image, as well as its compression ratio (cr) (compared to the 24 bit/pixel original). This information would of course normally not be shown to the user.
  • the image quality is indicated by the color of the box around each thumbnail.
  • This color varies continuously from green, for very high image quality (60 dB PSNR), through yellow (40 dB PSNR) to red, for very poor image quality (20 dB PSNR); see Fig. 5.
  • the quality feedback may help the user to decide whether to take more pictures.
  • all images have about the same quality and more storage space is assigned to the complex images than to the less complex images.
  • more preferred images have a higher quality than less preferred images.
  • a multi-media item may be an image, an audio item, a video item or a combination of these, in particular as (part of) a program item, etc.
  • storing of at least one multi-media item in a memory wherein in an automated manner a quality level is determined at which the multi-media item should be stored, which quality level at least partially depends on user preferences.
  • the multi-media item is stored in the memory at the determined quality level. If a plurality of multi-media items is stored in the memory, a plurality of respective quality levels is determined based on the user preferences.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

Storing of at least one multi-media item in a memory is provided, wherein in an automated manner a quality level is determined at which the multi-media item should be stored, which quality level at least partially depends on user preferences. The multi-media item is stored in the memory at the determined quality level. If a plurality of multi-media items is stored in the memory, a plurality of respective quality levels is determined based on the user preferences.

Description

Storage of multi-media items
The invention relates to storing of multi-media items in a memory.
The Viewer's Guide for the Philips Personal Video Recorder, TiNo Software Version 1.3, page 34 discloses that a user can choose the video recording quality for the programs to be recorded. The higher quality settings take up more space, and will give fewer hours of recording time. A default recording quality can be set. It is recommended to record several shows at different recording qualities and to decide on a default setting based on the type of programs the user watches most often. When an individual recording is set up, the user can change its quality. Suggested settings are: basic (30 hours of recording time) for animation, medium (18 hours) for daytime talk shows, high (14 hours) for dramas and movies, best (9 hours) for sports and action.
An object of the invention is to provide advantageous use of the available storage capacity. To this end, the invention provides a method of storing, a storage device and a recording apparatus as defined in the independent claims. Advantageous embodiments are defined in the dependent claims.
A first embodiment of the invention provides storing at least one multi-media item in a memory, determining in an automated manner a quality level at which the multimedia item should be stored, which quality level at least partially depends on user preferences, and storing the multi-media item in the memory at the determined quality level. Automatically determining the quality of the multi-media item to be stored based on user preference has the advantage that a user need not indicate per item its desired quality.
In an embodiment of the invention, more preferred multi-media items are stored at a higher quality than less preferred multi-media items. More preferred corresponds to a higher potential user interest and less preferred to a lower potential user interest. The potential user interest is determined from the user preferences. Therefore the desired quality levels are determined automatically by monitoring user preferences. The quality levels can cover the complete range from perfect lossless storage of an item down to completely deleting an item. The quality levels correspond to the (estimated) user's preferences. In a further embodiment, the storage system usage is also taken into account. In the case the storage capacity is not enough to store a new multi-media item, the quality levels of the previously stored multi-media items and of the multi-media item to be stored are reduced as much as is needed to store all items in the memory. The respective reduced quality levels depend on the respective preference levels of the multi-media items. The more the item is preferred, the higher is its relative quality.
Multi-media items which have already been consumed or outputted once, may be stored at a further reduced quality. This embodiment is especially useful for video items: if a video item has been viewed its potential interest for the user is lower. When the quality of a given multi-media item reduces below a given
(predetermined) threshold, the given multi-media item may be automatically deleted. The threshold may be set as default by the set maker, but may also be set by the user.
Examples of applications are (automatic) video recording ("TiVo box") or a consumer home audio and/or video "jukebox". An implementation can be made using the elastic memory concept as proposed in non pre-published European patent application 00200890.2, filed 13.03.00 (our reference PHNL000110).
WO 00/39707 (our reference PHA 23.590) discloses a video retrieval system that allows a user to quickly and easily select and receive stories of interest from a video stream. The video retrieval system classifies stories and delivers samples of selected stories that match each user's current preference. This document further discloses the use of selective erasure. When a new segment requires an allocation of the recording multi-media, the retrieval system identifies the segments that are on the recording multi-media that have the least correlation with the user's preferences. Instead of replacing the oldest segments with the newest segments, the segments of least potential interest to the user are replaced by the newest segments. The retrieval system also terminates the recording of the newest segment when it determines, based on the classification of the newest segment by the classification system, that the newest segment is of no interest to the user, based on the user preferences.
The determination of user preferences is widely known in the art. Usually, characteristics of multi-media items consumed by the user are stored and a user profile is built based on these characteristics. Characteristics of multi-media items can be determined in a convenient way by monitoring meta data which is sent together with the multi-media items. The meta data indicates characteristics of the multi-media item, such as type of program, etc. See e.g WO 00/39707. Several aspects of the invention will be apparent from and elucidated with reference to the accompanying drawings.
In the drawings: Fig. 1 shows a recording apparatus according to an embodiment of the invention;
Fig. 2 shows an implementation of the storage memory according to an embodiment of the invention;
Fig. 3 shows a schematic representation of a user interface for a recording apparatus;
Fig. 4 shows an exemplary mapping of a Peak Signal to Noise Ration (PSNR) value to a color used as a quality indication;
Fig. 5 shows a pseudo-code for converting a PSNR value into a color value (24-bit RGB); Figs. 6 and 7 show a representation of a user interface for a recording apparatus after storing of 6 and 12 multi-media items respectively.
The drawings only show those elements that are necessary or useful to understand the invention.
Fig. 1 shows a recording apparatus 2 according to an embodiment of the invention. The recording apparatus comprises an internal data bus 20 to which several elements are coupled, e.g. an encoder, a storage medium, a decoder, an input I/O, a user interface and an output I/O. The input I/O is coupled to a source device 1. The source device 1 furnishes multi-media items to the recording apparatus 2. The source device 2 may be present in the recording apparatus, e.g. as a camera in a digital camera application. Further, the source device may also be a transmitter from which the multi-media items are obtained. The recording apparatus may further be coupled to an output device 3, such as a display and/or speaker. The output device may also be present within the recording apparatus. On the output device the multi-media items may be outputted. Further, the output device can be used as part of the user interface. Control means are further distributed in the recording apparatus over the several elements. It is also possible to use a Central Processing Unit (CPU) to control the several elements and the data transport on the internal data bus 20. The invention may be incorporated in a storage device which may be implemented in an integrated circuit, in that case without a source device, output device and storage memory. In the following, two implementations of the recording apparatus of Fig. 1 are discussed.
The first implementation is based on the elastic memory concept. The elastic memory (PHNL000110) is a concept for storing a variable number of multi-media items on a storage medium of fixed capacity. When only few items are stored, they are stored at high quality. If more items need to be stored, the quality (i.e., amount of storage space that is occupied) of the already stored items is automatically reduced to make room for the new items. The elastic memory offers each individual user the option to select her/his personally preferred quality-capacity tradeoff. Some users may prefer to store a larger amount of data and are willing to accept a lower data quality. For example, because they own a small-screen television on which visual quality losses are less visible than on a large-screen television.
The elastic memory allows certain multi-media items to be stored at a higher quality than others according an embodiment of the present invention. This can be realized in the elastic memory by changing the measure (quality improvement or distortion reduction or perceptual significance, etc.) that is used to decide the relative importance of each data
"block". For example, the importance values of certain preferred multi-media items might be increased by 20%, when desired by the user. Of course, it is also possible to assign "infinite importance" to a certain item, which prevents any of its data blocks to be overwritten and lets the item be stored at the highest possible quality. If an elastic memory application contains a large number of items, for example in case of "automatic" video program recording, such as in a "TiVo box", it is very inconvenient or even impossible for the user to indicate her/his preferred quality for each item, while the user might still prefer to have favorite programs stored at a higher than average quality. According to an embodiment of the present invention, the quality levels correspond to the (estimated) user's preferences. The quality levels can cover the complete range from perfect lossless storage of an item down to completely deleting an item.
The implementation can be done using the elastic memory concept, so only a solution for obtaining the desired quality levels is required. If the data to be stored has been classified, these classifications can be used to set quality levels. This may be done by asking the user to enter preferences for one or more classes. The quality levels may also be obtained by measuring which class of items is most frequently retrieved by the user and increasing the quality levels for more frequently retrieved classes and decreasing them for less frequently retrieved classes. The classification information could, for example, be broadcast together with the program, or as side information in a program guide. Or it could be purchased from a service provider through the cable network or the internet. If no classification information is available, quality levels can be determined by monitoring both the time of storage and the frequency of retrieval. For example, if a television program that is recorded from a certain channel every Wednesday from 20:00-21:00 hours is very frequently retrieved (i.e., watched) by the user, this program could be assigned a higher quality level, so it will be stored at a higher quality the next time it is recorded. In another example, if a user has a "jukebox" containing all her/his audio (from CDs, tapes) and/or video (from DVDs, tapes) content, items that are frequently retrieved may be preserved at a higher quality, in case new items have to be stored, than items that are infrequently retrieved. Furthermore, since newly stored items will probably be more frequently retrieved, they could initially be assigned a higher than average quality level.
A disadvantage of modifying the elastic memory block significance values is that the original values are not preserved. In order to preserve the original values, it is advantageous to add one or more additional fields to the auxiliary memory that store significance modification parameters (such as e.g. a scale factor). See Fig. 2. Such an enhanced elastic memory is applicable in any application in which an elastic memory is used and the significance values may be manipulated.
Fig. 2 shows a main memory and an auxiliary memory to implement the elastic memory concept, with an exemplary content. The main memory and the auxiliary memory may be present in the storage medium of Fig. 1. See for a detailed description of the use of the main memory and the auxiliary memory PHNL000110, which is incorporated by reference herein.
The main memory is divided into N memory parts for storing N data pieces. In this example N = 11. The auxiliary memory is typically smaller than the main memory and is used for administrative purposes. The auxiliary memory comprises N records, each record comprising several fields. A first field is a Pointer to Main Memory (P), which comprises a pointer to a location in the main memory 31 that holds the data associated with a given record. A second field is an Object Identifier (I) that comprises information describing the data item to which the data piece stored in the main memory belongs. The object identifier refers to, for example, an image out of a set of images stored by a digital camera. In a practical embodiment the object identifier is zero in case a memory part is not assigned to a data item, e.g. when the memory part is empty. A third field comprises a Significance (S). The S field gives a measure of the significance of the data piece stored in the main memory where the record refers to. The entries in the significance fields are preferably non-negative numbers. The auxiliary memory preferably has the property that the records are sorted on the significance. It is also possible to sort the records in a different manner, e.g. on the identifier to group all data pieces belonging to the same data item. Within the respective groups of data pieces, the data pieces may be sorted on significance. To add a data item, the data item is coded in the encoder (see Fig. 1) to produce a scalable bit-stream, which is split in to data pieces. The pieces are subsequently processed. The significance measure of each piece is first compared to that of the block with the lowest significance measure currently in the memory. If the significance of the new data piece is lower, it is not stored in the main memory, i.e. it is discarded. When a first piece is not stored, the processing can be stopped because the further data pieces have lower significance than the current data piece (which is a property of a scalable coding mechanism). Further, a scalable bit-stream wherein a data piece is missing in the sequence is not decodable. If the significance is higher, the new data piece is written to the main memory at the position of the current least significant data piece (which is obtained from the last position of the auxiliary memory in case the records are sorted on significance). Thereafter, the last record of the auxiliary memory is replaced by the record data for the new data piece and the records in the auxiliary memory are re-ordered to restore the ordering on significance. It is advantageous to start processing with the most significant data piece of an item (and then with the subsequent data pieces having lower significance), because this is the order in which the data pieces are produced by the scalable coder, and further because these data pieces are not overwritten by blocks belonging to the same item since these are less significant.
To extract a data item, the records in the auxiliary memory are subsequently processed and if the object identifier matches that of the data item to be extracted, the data piece in the main memory pointed to by the record is sent to the (scalable) decoder. Since the auxiliary memory is traversed starting at the highest significance, the data pieces will be extracted at the right order, allowing the decoder to form a progressively better reconstruction.
To delete a data item, the records in the auxiliary memory are subsequently processed and if the object identifier matches that of the data item to be deleted, the significance of the record is set to a (predetermined) value that is lower than any value that can be produced by the encoder, e.g. zero. Preferably, also the identifier is set to a predetermined value, e.g. zero to indicate that the memory part is not assigned to a data piece. The records in the auxiliary memory are then re-ordered to restore the ordering on significance, i.e. such that the records of the lowest significance are placed at the end.
The use of an auxiliary memory is preferred. However, it is also possible to omit the auxiliary memory. In that case the significance of a data piece and an identifier to which data item it belongs should be stored in the main memory. Because pointers to data pieces in order of significance are in this case not available, searching the main memory takes more time. To reduce searching, the data pieces may be sorted in the main memory, at the cost of switching much larger amounts of data. Also a Content Addressable Memory can be used to implement the auxiliary memory, eliminating the need for sorting and searching in the auxiliary memory by a separate processor. Furthermore, more advanced data structures such as heaps or trees, which are generally known, may be used for performing the administrative functions, as an alternative to the preferred auxiliary memory data structure. These alternatives could be advantageous for example in a software implementation (or hardware when sufficient clock cycles are available), especially for a large number of memory parts.
Preferably, some additional information is stored for each data item. The additional information may include name, type of information, color, size, etc. This additional information may be stored in the main memory, e.g. together with the first data piece. Preferably, the additional information is stored in the auxiliary memory, which makes it easier to retrieve this additional information. According to the embodiment of the current invention, a significance modification parameter is stored in the memory as additional information. In Fig. 2 an extra field 'Significance modification' has been added to the auxiliary memory. In this example, the significance modification factor is a scale factor. As can be seen, for object (multi-media item) 1 the significance values stored in the significance field is 1.2. This means that the (original) significance values should be multiplied with 1.2 to obtain the current significance values which are used in the elastic memory concept. The significance values of object 2 remain unmodified and the significance values of object 3 are multiplied with 0.9. This means that object 1 is given higher preference than objects 2 and 3. The block significance value used for the elastic memory algorithms is then determined by multiplying the modification factors with the original significance. This means in this example that object 1 will be retained at a higher level than object 2. For example, the second data block of object 2 with original and current significance at 1056 will sooner be overwritten than the second block of item 1, with current significance 1106 (1.2 * 922), although the original significance of the second block of object 1 is lower than the original significance of the second block of object 2. In this way, easy manipulation of the quality of objects (media items) is possible in the elastic memory concept.
It is further possible to directly store the current (modified) significance values in the significance fields and using those values in the elastic memory algorithms. In this case, the original significance values can be calculated by using the significance modification values.
In fact, any combination of two values out of the three values: original significance value, modified significance value and modification parameter can be stored. When needed, the third value can always be calculated by combining the two stored values. For example, one may use the following relationship:
Modified significance value = original significance value * modification parameter
The further implementation is based on MPEG storage such as currently used in the TiVo box. As already discussed above, the TiVo box allows a user to select a certain desired quality level for recording. Currently, the quality levels are (basic, medium, high, best). This quality selection directly corresponds to a certain bit rate at which the video is encoded using MPEG. Thus, the higher the quality setting, the lower the storage capacity or recording time of the TiVo box. The above-mentioned quality levels correspond to a storage capacity of (30, 18, 14, 9) hours, respectively.
Actually, the levels offered to the user are not true quality levels, since they specify a certain bit rate instead of a quality. The resulting quality still depends on the content of the program that is recorded. In the TiVo manual, it is therefore also suggested to select the "quality" setting based on the program content and to use e.g. the "best" quality setting for sports events or action movies.
To use automatic quality with MPEG storage, ideally a constant quality (e.g. constant quantizer step size) and thus variable bit rate encoding mode should be used. In that case, an average recording quality can be determined and the quality for recording certain programs can be automatically increased or decreased by e.g. 20%, depending on the automatically derived user preferences.
The practical problem with constant quality/variable bit rate recording is of course that it is difficult to use with a fixed storage amount, such as with a hard disk. Therefore, e.g. the TiVo box sets the bit rate of the program instead of the quality. We can automatically do the same: an average bit rate for recording is selected and this bit rate is automatically modified by a certain percentage depending on the automatically determined user preferences. Now the total recording capacity, expressed in hours of video recording, can at each time be exactly determined. An important improvement on this "TiVo-like" method could be to also take into account the program content. If e.g. meta-data is present about the content (like "soap", "talkshow", "football", "thriller", "action", etc.) the encoding bit rate may be adjusted to better approach the desired user preferences, by automatically increasing the bit rate for "difficult" content like sports or action. For example, a 20% higher than average desired quality could result in e.g. a 40% higher than average bit rate for a sports program, and only a 5% higher than average bit rate for a talkshow (since content with little movement will automatically get a higher quality because of the constant bit rate recording, even when a higher than average quality is preferred this content could even be recorded at a lower than average bit rate!). When no meta data for the content is available, this can possibly be obtained by analyzing previous recordings. For example, if a user watches a program that is recorded at the same time every day or week, the complexity or "difficulty" (i.e. the resulting program quality at a certain encoder setting) can be analyzed from previous recordings and used to adjust the settings for future recordings.
The automatic adaptation of recording bit rate according to program content (i.e. without the advanced automatic user-preferred quality determination according to the embodiment described above) could also be implemented in the current TiVo system, in order to improve the correspondence between the quality setting selected by the user (basic, medium, high, best) and the actual resulting recording quality. This feature is therefore also applicable to embodiments outside the inventive concept of the current invention.
The invention can be applied in all multi-media storage systems that allow some stored data items to be stored at a higher quality than others, preferably with many possible different quality settings. Examples of applications are (automatic) video recording ("TiVO box") or a consumer home audio and/or video "jukebox". The invention can of course also be applied in case of multiple users (accessing for example a remote database or library). One may store on the HD of the PC items at varying quality (bitrate) in an automated manner depending also on user preferences. The invention is further applicable in transmission and downloading of e.g. MP3 files. Prior to transmission, the user preferences may be consulted by the transmitter in order to determine the bitrate of an item to be transmitted. This has the advantage that no more bits are sent than necessary. Further, if the user preferences can be consulted by the transmitter, the transmitter is able to allocate a limited bandwidth to respective multi-media items, which allocation is based on the user preferences. In this embodiment, items of more potential interest to the user can be allocated more bandwidth at the cost of the bandwidth of a less preferred item in order to provide the more preferred item at a higher quality than the less preferred item. Another possible implementation is that the receiver determines the quality the item should have and thereafter requests the transmitter to deliver the item at the determined quality (see Fig. 1)
The elastic memory (PHNL000110EP.P) allows a variable number of items to be stored on a storage device of fixed capacity. The storage space occupied by each item is automatically adjusted in case subsequently more items have to be stored. This adjustment is accompanied by a corresponding quality reduction. Therefore, it is desirable to provide the user with current information about the media item quality. As an additional feature, the elastic memory also allows different items to be stored at a different quality, according to the user' s preferences. This feature must also be supported by the user interface. In the following user interface features related to the quality of the stored media items in a memory-based consumer device will be discussed within the scope of the current invention. These features are also applicable to other embodiments outside the scope of the current invention, in which other embodiments a quality of stored multi-media items may be modified.
There are various useful user-interface features related to the quality of the stored media items in a memory-based consumer device:
1) A color can be used to indicate the quality of individual items (colored background, border, dot etc.) or the overall item quality (guaranteed minimum or average). Advantageously, some representation of the stored item is used together with the quality indication. The color could e.g. range from green for high quality through yellow for medium quality to red for low quality. For images or video, e.g., a linear scale could be used depending on the PSNR (see Fig. 5) or any other quality measure. As a rough indication of the quality, a bitrate of a stored media item may be used. 2) Each individual stored item can be assigned a "protect button". As long as this button is active, the associated media item is protected from any quality loss when subsequently more items are stored. This can be used, e.g. in a camera application to protect a picture that the user likes very much. This can be implemented by temporarily (while the button is active) replacing the elastic memory block significance values by a value that is larger than any value that can otherwise occur.
3) Each individual stored item can be assigned a "higher/lower than average quality" button or setting (i.e. allowing multiple values) which will influence the item quality when subsequently more items are stored. This option can also be used as a "soft-delete" function: when the user does not like a certain item, it may be stored at a lower than average quality; under certain circumstances, it might even be automatically deleted. This feature can be implemented, e.g., by scaling (multiplying by a constant) the elastic memory block sigmficance values for the items according to the quality setting. 4) In addition to the previous option that influences the quality of already stored items, it is also desirable to be able to influence the quality at which the next item will be stored, e.g. using a "record/store next item at higher than average quality" button or setting. This can be implemented by scaling the item's block significance values before storing them. 5) A user may not be interested in constantly monitoring the item quality, but only be interested in maintaining a certain guaranteed item quality (which he/she can preset preselect). In that case, an "audio (e.g. a beep) and/or visual (e.g. a blinking light) quality warning signal" may be given when the item quality goes below the predetermined threshold, or, alternatively, is expected to go below this threshold after storing a certain additional number of items (for example two more pictures).
Some of the above features are implemented by modifying the elastic memory block significance values as discussed above.
The stored items can be images, but e.g. also audio or video items that may also be represented by thumbnails (e.g. the CD/DVD cover print) and/or track names and/or individual scene thumbnails, in particular of key frames.
An example of a possible user interface for a digital still image camera is shown in Figure 4, 6 and 7. Figure 4 shows a schematic representation of the user interface for two images. The images that are shown are just the thumbnails; the full image is displayed if the user clicks on one of the thumbnails. An image can be deleted by clicking on the Remove button below its thumbnail. The numbers above the image thumbnails indicate the number of memory blocks (bl) allocated to the image, as well as its compression ratio (cr) (compared to the 24 bit/pixel original). This information would of course normally not be shown to the user. The image quality is indicated by the color of the box around each thumbnail. This color varies continuously from green, for very high image quality (60 dB PSNR), through yellow (40 dB PSNR) to red, for very poor image quality (20 dB PSNR); see Fig. 5. The quality feedback may help the user to decide whether to take more pictures. In an elastic memory implementation all images have about the same quality and more storage space is assigned to the complex images than to the less complex images. In an embodiment according to the present invention, more preferred images have a higher quality than less preferred images.
A multi-media item may be an image, an audio item, a video item or a combination of these, in particular as (part of) a program item, etc.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of other elements or steps than those listed in a claim. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
In summary, storing of at least one multi-media item in a memory is provided, wherein in an automated manner a quality level is determined at which the multi-media item should be stored, which quality level at least partially depends on user preferences. The multi-media item is stored in the memory at the determined quality level. If a plurality of multi-media items is stored in the memory, a plurality of respective quality levels is determined based on the user preferences.

Claims

CLAIMS:
1. A method of storing at least one multi-media item in a memory, the method comprising: determining in an automated manner a quality level at which the multi-media item should be stored, which quality level at least partially depends on user preferences, and storing the multi-media item in the memory at the determined quality level.
2. A method as claimed in claim 1 , wherein the quality is progressively related to a potential user interest, which potential user interest is based on the user preferences.
3. A method as claimed in claim 1 , wherein the quality is proportional to a potential user interest, which potential user interest is based on the user preferences.
4. A method as claimed in claim 1 , wherein a plurality of multi-media items is stored in the memory and a plurality of respective quality levels is determined based on the user preferences.
5. A method as claimed in claim 1, the method further comprising: evaluating a quality level of at least one multi-media item which has previously been stored in the memory; and determining the quality at which the multi-media item should be stored based on the quality level of the at least one previously stored multi-media item and the user preferences.
6. A method as claimed in claim 5, wherein in the case that a capacity of the storage medium does not suffice to store the multi-media item at the determined quality, the quality of the previously stored multi-media item and the quality of the multi-media item to be stored are reduced taking into account the user preferences.
7. A method as claimed in claim 1, wherein a given quality corresponds to a given bit-rate and wherein the multi-media item is stored at a determined bitrate which corresponds with the determined quality.
8. A method as claimed in claim 7, wherein an average bit rate for recording is selected and this bit rate is automatically modified by a certain percentage depending on the user preferences to obtain the determined bitrate.
9. A method as claimed in claim 7 or 9, wherein the determined bitrate is adapted to take into account the content of the multi-media item, in particular a dependence of the bitrate on the content for a given quality.
10. A method of providing a user interface for a recording apparatus, which recording apparatus is capable of storing multi-media items at varying quality in a storage memory, the method comprising using a color to indicate the quality of at least one multimedia item available on the storage medium, which color corresponds to a quality of the multi-media item.
11. A method as claimed in claim 10, wherein the colored quality indication is displayed together with a representation of the stored item.
12. Device for storing at least one multi-media item in a memory, the device comprising: means for determining in an automated manner a quality level at which the multi-media item should be stored, which quality level at least partially depends on user preferences, and means for storing the multi-media item in the memory at the determined quality level.
13. Device for providing a user interface for a recording apparatus, which recording apparatus is capable of storing multi-media items at varying quality in a storage memory, the device comprising means for displaying a color to indicate the quality of at least one multi-media item available on the storage medium, which color corresponds to a quality of the multi-media item.
14. Recording apparatus comprising a storage memory on which multi-media items can be stored and a storage device according to claim 12 or 13.
15. A storage memory having stored thereon multi-media items, which multimedia items have been divided into successive data pieces of decreasing significance, which data pieces have been stored in respective memory parts of the storage medium, wherein on the storage medium at least two of the following parameters are available: an indication of a current significance of each data piece, an indication of an original significance and a significance modification parameter.
16. A method of storing at least one multi-media item in a memory, the method comprising: adapting in an automated manner a recording bit-rate at which the multi-media item is stored, in dependence on the item content and a quality setting selected by a user, and storing the multi-media item in the memory at the adapted recording bit-rate.
17. A device for storing at least one multi-media item in a memory, the device comprising: means for adapting in an automated manner a recording bit-rate at which the multi-media item is stored, in dependence on the item content and a quality setting selected by a user, and means for storing the multi-media item in the memory at the adapted recording bit-rate.
PCT/IB2002/000761 2001-03-26 2002-03-08 Storage of multi-media items WO2002078357A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2002576449A JP2004524627A (en) 2001-03-26 2002-03-08 Store multimedia items
EP02702668A EP1374600A1 (en) 2001-03-26 2002-03-08 Storage of multi-media items

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP01201159 2001-03-26
EP01201159.9 2001-03-26

Publications (1)

Publication Number Publication Date
WO2002078357A1 true WO2002078357A1 (en) 2002-10-03

Family

ID=8180079

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2002/000761 WO2002078357A1 (en) 2001-03-26 2002-03-08 Storage of multi-media items

Country Status (6)

Country Link
US (1) US20020149591A1 (en)
EP (1) EP1374600A1 (en)
JP (1) JP2004524627A (en)
KR (1) KR20030007711A (en)
CN (1) CN1460387A (en)
WO (1) WO2002078357A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2376831A (en) * 2001-04-20 2002-12-24 Paul Vincent Henderson Earpiece
JP2006196163A (en) * 2005-01-12 2006-07-27 Thomson Licensing Method for recording a/v signal on recording medium, and device for reading from and writing to recording medium
WO2007025852A2 (en) * 2005-08-29 2007-03-08 Siemens Aktiengesellschaft System and method for archiving surveillance videos

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004107756A1 (en) * 2003-05-30 2004-12-09 Koninklijke Philips Electronics N.V. Ascertaining show priority for recording of tv shows depending upon their viewed status
US20060018625A1 (en) * 2004-07-20 2006-01-26 Johnson Carolynn R User defined default recording mode rules
US20060204212A1 (en) * 2004-08-31 2006-09-14 Minoru Kinaka Information recording/reproducing device
US20060177122A1 (en) * 2005-02-07 2006-08-10 Sony Computer Entertainment Inc. Method and apparatus for particle manipulation using graphics processing
US7714838B2 (en) * 2006-04-27 2010-05-11 Research In Motion Limited Handheld electronic device having hidden sound openings offset from an audio source
US8155207B2 (en) 2008-01-09 2012-04-10 Cisco Technology, Inc. Processing and managing pictures at the concatenation of two video streams
US8875199B2 (en) * 2006-11-13 2014-10-28 Cisco Technology, Inc. Indicating picture usefulness for playback optimization
US8873932B2 (en) 2007-12-11 2014-10-28 Cisco Technology, Inc. Inferential processing to ascertain plural levels of picture interdependencies
US8416859B2 (en) 2006-11-13 2013-04-09 Cisco Technology, Inc. Signalling and extraction in compressed video of pictures belonging to interdependency tiers
US8958486B2 (en) 2007-07-31 2015-02-17 Cisco Technology, Inc. Simultaneous processing of media and redundancy streams for mitigating impairments
US8804845B2 (en) 2007-07-31 2014-08-12 Cisco Technology, Inc. Non-enhancing media redundancy coding for mitigating transmission impairments
CN101207799A (en) * 2007-11-22 2008-06-25 深圳市同洲电子股份有限公司 System and method for storing program and program ordering system
US8416858B2 (en) * 2008-02-29 2013-04-09 Cisco Technology, Inc. Signalling picture encoding schemes and associated picture properties
WO2009152450A1 (en) 2008-06-12 2009-12-17 Cisco Technology, Inc. Picture interdependencies signals in context of mmco to assist stream manipulation
US8705631B2 (en) 2008-06-17 2014-04-22 Cisco Technology, Inc. Time-shifted transport of multi-latticed video for resiliency from burst-error effects
US8699578B2 (en) 2008-06-17 2014-04-15 Cisco Technology, Inc. Methods and systems for processing multi-latticed video streams
US8971402B2 (en) 2008-06-17 2015-03-03 Cisco Technology, Inc. Processing of impaired and incomplete multi-latticed video streams
CN102210147B (en) 2008-11-12 2014-07-02 思科技术公司 Processing of a video [AAR] program having plural processed representations of a [AAR] single video signal for reconstruction and output
WO2010096767A1 (en) 2009-02-20 2010-08-26 Cisco Technology, Inc. Signalling of decodable sub-sequences
US8782261B1 (en) 2009-04-03 2014-07-15 Cisco Technology, Inc. System and method for authorization of segment boundary notifications
US8949883B2 (en) 2009-05-12 2015-02-03 Cisco Technology, Inc. Signalling buffer characteristics for splicing operations of video streams
US8279926B2 (en) 2009-06-18 2012-10-02 Cisco Technology, Inc. Dynamic streaming with latticed representations of video
JP2011014984A (en) * 2009-06-30 2011-01-20 Sony Corp Information processing apparatus and method and program
US10846743B2 (en) * 2015-08-14 2020-11-24 Google Llc Displaying content items based on user's level of interest in obtaining content
WO2018197144A1 (en) * 2017-04-28 2018-11-01 Asml Netherlands B.V. Optimizing a sequence of processes for manufacturing of product units

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0859523A2 (en) * 1997-02-17 1998-08-19 Sony Corporation Digital signal recording and/or reproduction methods and apparatus
EP0929072A2 (en) * 1997-12-12 1999-07-14 Kabushiki Kaisha Toshiba Digital recording system using variable recording rate
EP0999706A2 (en) * 1998-11-04 2000-05-10 Sharp Kabushiki Kaisha Dynamic management of embedded coded images in a digital storage device
US20020009289A1 (en) * 2000-07-21 2002-01-24 Taichiro Morishita Information compressing and recording device

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5764235A (en) * 1996-03-25 1998-06-09 Insight Development Corporation Computer implemented method and system for transmitting graphical images from server to client at user selectable resolution
EP0956701B1 (en) * 1997-02-03 2005-11-23 Sharp Kabushiki Kaisha An embedded image coder with rate-distortion optimization
US6594825B1 (en) * 1998-10-30 2003-07-15 Intel Corporation Method and apparatus for selecting a version of an entertainment program based on user preferences
US6446110B1 (en) * 1999-04-05 2002-09-03 International Business Machines Corporation Method and apparatus for representing host datastream screen image information using markup languages
US6522783B1 (en) * 1999-11-23 2003-02-18 Sharp Laboratories Of America, Inc. Re-indexing for efficient compression of palettized images
US6732175B1 (en) * 2000-04-13 2004-05-04 Intel Corporation Network apparatus for switching based on content of application data
GB0011426D0 (en) * 2000-05-11 2000-06-28 Charteris Limited A method for transforming documents written in different XML-based languages
WO2001098936A2 (en) * 2000-06-22 2001-12-27 Microsoft Corporation Distributed computing services platform
US6523036B1 (en) * 2000-08-01 2003-02-18 Dantz Development Corporation Internet database system
US6748583B2 (en) * 2000-12-27 2004-06-08 International Business Machines Corporation Monitoring execution of an hierarchical visual program such as for debugging a message flow
US20030101170A1 (en) * 2001-05-25 2003-05-29 Joseph Edelstein Data query and location through a central ontology model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0859523A2 (en) * 1997-02-17 1998-08-19 Sony Corporation Digital signal recording and/or reproduction methods and apparatus
EP0929072A2 (en) * 1997-12-12 1999-07-14 Kabushiki Kaisha Toshiba Digital recording system using variable recording rate
EP0999706A2 (en) * 1998-11-04 2000-05-10 Sharp Kabushiki Kaisha Dynamic management of embedded coded images in a digital storage device
US20020009289A1 (en) * 2000-07-21 2002-01-24 Taichiro Morishita Information compressing and recording device

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2376831A (en) * 2001-04-20 2002-12-24 Paul Vincent Henderson Earpiece
JP2006196163A (en) * 2005-01-12 2006-07-27 Thomson Licensing Method for recording a/v signal on recording medium, and device for reading from and writing to recording medium
WO2007025852A2 (en) * 2005-08-29 2007-03-08 Siemens Aktiengesellschaft System and method for archiving surveillance videos
WO2007025852A3 (en) * 2005-08-29 2007-08-02 Siemens Ag System and method for archiving surveillance videos

Also Published As

Publication number Publication date
US20020149591A1 (en) 2002-10-17
KR20030007711A (en) 2003-01-23
JP2004524627A (en) 2004-08-12
CN1460387A (en) 2003-12-03
EP1374600A1 (en) 2004-01-02

Similar Documents

Publication Publication Date Title
US20020149591A1 (en) Storage of multi-media items
US8798170B2 (en) Program recommendation apparatus
US7840112B2 (en) Gradually degrading multimedia recordings
EP1522934A2 (en) Image information describing method, video retrieval method, video reproducing method, and video reproducing apparatus
CN1505392A (en) Recording apparatus and recording method
EP1274238B1 (en) Method and apparatus for recording and searching audio/video signal
US6731808B2 (en) Compressed storage of data items
US8606078B2 (en) Data recording method
US20050232598A1 (en) Method, apparatus, and program for extracting thumbnail picture
US8437615B2 (en) Data recording method
US20070286571A1 (en) Information Recorder and Information Recording Program
US20080225940A1 (en) Digital video apparatus and method thereof for video playing and recording
EP1930903A1 (en) Recording method
US8170397B2 (en) Device and method for recording multimedia data
JP4257504B2 (en) 3D image data processing device, 3D image data transmission device, 3D image data recording device
EP1930904A1 (en) Recording method
KR100938057B1 (en) Recording method
EP1939883A1 (en) Data recording method
KR100606681B1 (en) Server data structure and method for service of multimedia data in order to providing VCR-like functionfast forward/fast rewind in Video On Demand system.
JPH11205753A (en) Animation perusing method and its device
JP4416586B2 (en) Video recording / playback device
JP2002150678A (en) Recording and reproducing device and information recording method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CN JP KR

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

WWE Wipo information: entry into national phase

Ref document number: 2002702668

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 028008219

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 1020027016029

Country of ref document: KR

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWP Wipo information: published in national office

Ref document number: 1020027016029

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 2002576449

Country of ref document: JP

WWP Wipo information: published in national office

Ref document number: 2002702668

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2002702668

Country of ref document: EP