US20100272187A1 - Efficient video skimmer - Google Patents

Efficient video skimmer Download PDF

Info

Publication number
US20100272187A1
US20100272187A1 US12/765,815 US76581510A US2010272187A1 US 20100272187 A1 US20100272187 A1 US 20100272187A1 US 76581510 A US76581510 A US 76581510A US 2010272187 A1 US2010272187 A1 US 2010272187A1
Authority
US
United States
Prior art keywords
video
comprises
full length
chapters
skimming
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/765,815
Inventor
Mehmet Reha Civanlar
Ofer Shapiro
Tal Shalom
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vidyo Inc
Original Assignee
Delta Vidyo Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US17235509P priority Critical
Application filed by Delta Vidyo Inc filed Critical Delta Vidyo Inc
Priority to US12/765,815 priority patent/US20100272187A1/en
Assigned to DELTA VIDYO, INC. reassignment DELTA VIDYO, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CIVANLAR, REHA, SHALOM, TAL, SHAPIRO, OFER
Publication of US20100272187A1 publication Critical patent/US20100272187A1/en
Assigned to VIDYO, INC. reassignment VIDYO, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DELTA VIDYO, INC.
Assigned to VENTURE LENDING & LEASING VI, INC. reassignment VENTURE LENDING & LEASING VI, INC. SECURITY AGREEMENT Assignors: VIDYO, INC.
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643Communication protocols
    • H04N21/64322IP
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/102Programmed access in sequence to addressed parts of tracks of operating record carriers
    • G11B27/105Programmed access in sequence to addressed parts of tracks of operating record carriers of operating discs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162User input
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234327Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4314Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/438Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
    • H04N21/4383Accessing a communication channel, e.g. channel tuning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/438Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
    • H04N21/4383Accessing a communication channel, e.g. channel tuning
    • H04N21/4384Accessing a communication channel, e.g. channel tuning involving operations to reduce the access time, e.g. fast-tuning for reducing channel switching latency
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440227Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8549Creating video summaries, e.g. movie trailer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/782Television signal recording using magnetic recording on tape
    • H04N5/783Adaptations for reproducing at a rate different from the recording rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/775Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television receiver
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/84Television signal recording using optical recording
    • H04N5/85Television signal recording using optical recording on discs or drums
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal

Abstract

Disclosed are a system, method, apparatus, and computer readable media containing instructions for displaying video files for rapid searching. In two different types of exemplary embodiments, a standalone video skimming system, and a video skimming system includes a server and a client system are disclosed, where the video file may be locally or remotely stored, or can be obtained from a live feed. The system displays many small windows simultaneously, in which different parts of the video chosen by the user are shown at the same time to shorten the skimming time. The video file is encoded using layered encoding to display smaller versions using lower layers, and without needing any processing to generate smaller versions of the video from the original full screen version. A video extractor is described for extracting the necessary bitstreams from a local video database containing layered encoded video files according to user specified window sizes, and distributing the signals over the electronic communications network channel. The system also includes a skimming control logic which can receive control commands from clients and invoke the video extractor to extract appropriate audio-visual signals there from for each command.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application claims the benefit of priority to U.S. Provisional Application Ser. No. 61,172,355, filed Apr. 24, 2009, which is hereby incorporated by reference herein in its entirety.
  • BACKGROUND
  • 1. Technical Field
  • The disclosed invention relates to techniques for searching for content in a compressed digital video file accessed from local storage or over a network such as the Internet. In particular, it relates to the use of layered video coding technology in connection with content searching for retrieving and displaying selected video segments.
  • 2. Background Art
  • Subject matter related to the present application can be found in co-pending U.S. patent application Ser. Nos. 12/015,956, filed Jan. 17, 2008 and entitled “System And Method For Scalable And Low-Delay Videoconferencing Using Scalable Video Coding,” 11/608,776, filed Dec. 8, 2006 and entitled “Systems And Methods For Error Resilience And Random Access In Video Communication Systems,” and 11/682,263, filed Mar. 5, 2007 and entitled “System And Method For Providing Error Resilience, Random Access And Rate Control In Scalable Video Communications,” and U.S. Pat. No. 7,593,032, filed Jan. 17, 2008 and entitled “System And Method For A Conference Server Architecture For Low Delay And Distributed Conferencing Applications,” each of which is hereby incorporated by reference herein in their entireties.
  • With increasing computing power and electronic storage capacity, and ubiquity of network bandwidth, the number of large digital video libraries accessible through the Internet is growing rapidly. On the low end of the performance spectrum, databases of popular websites such as YouTube (www.youtube.com) and Facebook (www.facebook.com) are becoming sources for millions of user videos. Currently, these videos are often of small size and low resolution. Considering the continuous increase in the resolutions of consumer video equipment, however, such databases are likely to contain higher resolution videos in the foreseeable future.
  • On the other end of the performance spectrum, high resolution video presentation technologies such as High-Definition TV (HDTV) are frequently used in entertainment and news video files. Even after applying digital video compression techniques, high resolution video results in large file sizes. Movies and broadcast TV content in standard TV (SDTV) resolution are still of considerable size. Two other important applications where higher resolution video may be used are surveillance video and video segments recorded for scientific experiments.
  • With the amount of digital video content growing, and that content being spread around the Internet, a technology is needed to search for content in these databases effectively and rapidly. Searching for content in video files has significant applications. Users of such video content searching technology can, for example, include: police officers looking for a specific scene in surveillance videos (content which can be distributed over many sites on the Internet, and can be many hours or days long); students and teachers looking for a specific presentation module; biologists looking for the instance when a particular biological event is triggered; or consumers of movies or news looking for a specific movie scene or news segment. Many users may be interested only in a small portion—perhaps only few seconds of content—of a full-length video. A noteworthy aspect of content searching in video files is that it normally requires a user's full attention and can't be performed as a background task while multi-tasking.
  • The following terms are used throughout the disclosure. A “full length video” can be any video footage that is meaningful as a unit, for example, a movie, a TV show, and similar video. A full length video may be divided into segments, which are called “video chapters”, or in short, “chapters”. Therefore, a full length video includes one or more concatenated chapters. Meta information related to a chapter is called “video index”, and the process by which video indices are generated is called “video indexing”. A video index may include information about the starting and ending times of the chapter, textual information about the chapter's content, and one or more images derived from the chapter that may represents its content. Video chapters may be indexed. One simple form of automatic indexing is to sub-divide a full length video into video chapters of a length that may be configured in the system or specified by the user.
  • A “raw video” represents uncompressed digitized video. After processing by a video encoder/compressor, raw video becomes “compressed video”. The term “transcoding” refers to the process of converting a compressed video into a different type of compressed video. Transcoding can involve, for example, the transforming of compressed video into raw video, and compressing this raw video into a different type of compressed video.
  • “Skimming” video, alternatively known as browsing, has been a technical challenge for a long time. There are some techniques commonly used to make skimming a full length video a more efficient process:
  • a. Fast forwarding: fast forwarding (also known as increasing the video playback speed) shortens the video viewing time. However, speeding up the video rate distorts the video information and may cause elimination of short events. This method has been the most popular browsing technique so far. Fast forwarding is discussed in more detail below.
  • b. Text Based Queries: This refers to a querying of metadata associated with the full length video or video chapters for specific textual information. For example, a text based query may be in the form of “scene with George falling off the bridge”. Text based queries today require the video to be annotated, mostly a manual process, before the video can be queried. Although text-based video query has been in existence for a long time, only few applications can afford the required intense human effort needed to intelligently categorize and annotate the videos. One example of video content that contains metadata which enables text based queries is medical records used in some systems.
  • c. Automatic Indexing: In the academic literature [for example, Cees G. M. Snoek and Marcel Worring, “Multimodal Video Indexing: A Review of the State-of-the-art,” Multimedia Tools and Applications, Volume 25, Number 1/January, 2005, Springer], techniques have been proposed to automatically index video for browsing representations based on information within the video. These indexing systems can use, for example, any of the following information aspects to generate video chapters:
      • Motion of the video;
      • Scene changes;
      • Image statistics—such as color and shape;
      • Audio information; and/or
      • Specific object types in the video.
  • The prior art, for example, Michael A. Smith, “Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques,” p.775, 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '97), 1997, discloses techniques based on one or more of the above aspects that generate a so-called “video skim”, a short synopsis version (or a small understandable portion) of the original indexed video. By viewing the video skim, a user may obtain an overview of the content of the full length video in a comparatively short period of time. A crude form of a video skim can, for example, consist of images or short scenes representing video chapters of a fixed duration. More complex skims may be generated using previously available text based annotation or automatic indexing techniques as mentioned above.
  • If a user becomes interested in one part of the skim, he/she can opt to view the associated video chapter in its relevant parts or in its entirety, and possibly at normal playback speed, without having to view the rest of the full length video. The critical aspect of creating a good video skim is context understanding, which is the key to choosing the significant images and words that should be included in the video skim. Today, when using any form of automated video skim generation, it is unfortunately quite frequent that a certain scene, in which a user may be interested, stays unidentified by the skimming process. Nevertheless, with advances in automated indexing techniques, video skims are becoming a useful reality today, and will likely be of even higher interest in the foreseeable future.
  • In summary, the automated context-sensitive generation of video skims, despite the significant research conducted over the past decade, has remained a task that is difficult, requiring high computational complexity and involving human interaction such as filtering and processing.
  • Once a video skim is generated, it often needs to be presented to the user.
  • In its crudest form, video skim generation and presentation can be performed simultaneously based on a full length video file that is available on the user premises. A software application running on the user's computer (or special purpose workstation), or dedicated hardware-based processing, can use any of the techniques mentioned below.
  • A) Video Skimming Using Raw Video: If the full length video on the local storage is in a raw format, then the skimming can be performed without the dedicated creation of a video skim, by simply “fast forwarding” until the desired segment is found. The fast forwarding process can be defined as “temporal sub-sampling”, or time domain sub-sampling of the frames of the video (e.g., skip every other frame for a playback speed-up of two (i.e., a 50% reduction in playback time), or show every fifth frame for a speed-up of five (i.e., a roughly 76% reduction in playback time), etc.). For example, referring to FIG. 1, a full-length video (101) contains even numbered (102, 104) and odd numbered (103, 105) frames. Using temporal sub-sampling a temporal sub-sampled sequence (106) is created, in which only the even-numbered frames of the full length video sequence are present (107, 108). This temporal sub-sampling results in a 50% reduction in the playback time. The speed-up can be varied by other sub-sampling intervals. When sub-sampling every fifth frame (110, 111) of the original full length video sequence is used in the temporal sub-sampled sequence (109), the reduction in playback time is rough 76%.
  • Other linear or non-linear sub-sampling factors can also be used.
  • There are at least three disadvantages of fast forwarding are as follows:
  • (1) The search may still take a long time depending on where the specific video segment of interest is located within the full length video sequence (particularly if it is located towards the end).
  • (2) The video segment of interest may be made unnoticeable or totally lost during sub-sampling as it may fall on the deleted frames (especially when large sub-sampling intervals are in use).
  • (3) The associated audio information, if any, often cannot be meaningfully presented.
  • Another method for skimming raw video is to subdivide the full length raw video into video chapters (which may be of preconfigured lengths or configured in real-time by the user), and allow the user to view more than one video chapter in parallel using separate small windows for each chapter. This process may require “spatial sub-sampling” to reduce the resolution of the original video to fit into smaller windows, because of display size limitations as illustrated in FIG. 2. The full length video (201) may include pictures of a certain spatial resolution, illustrated here by the spatial size of one of the pictures in this sequence (202). In one very simple form, spatial sub-sampling may be, for example, averaging the brightness and/or color component values of four spatially adjacent pixels (203) so as to create a single sub-sampled pixel (204). Doing so for all pixels of all pictures of the full-length video (201) yields a series of sub-sampled pictures (205) at the same frame rate as the full-length video, but at half of the spatial resolution in each dimension.
  • There are at least two specific disadvantages of this approach are as follows:
  • (1) Performing spatial sub-sampling in real-time to generate smaller versions of the full-length video is computationally intensive and time consuming. Depending on how many windows are generated and the size of each window, the sub-sampling may require significant computing resources.
  • (2) The information may be lost during sub-sampling due to side effects of spatial sub-sampling such as filtering or aliasing.
  • B) Video Skimming Using Compressed Video: If the full length video is in a compressed format (for example, the full length video is compliant with video compressions standards such as ITU-T Rec. H.264 or other video compression standards), then additional factors come into play. Using a compressed format eliminates the need for the very large uncompressed video files and is, therefore, in most cases, advantageous. However, the compressed video file can't be temporally sub-sampled randomly as the sequence of compressed frames may depend on other frames due to inter-picture prediction. Unless the entire video is decoded, only independently decodable reference (IDR) frames can be used in fast forwarding. If there are no IDR frames or if their frequency is low, then fast forwarding will not be feasible without decoding a large percentage of the coded pictures of the full length video sequence. One way of remedying this may be to transcode the compressed video with more frequent IDR frames. The disadvantages of this approach are as follows:
  • (1) It may be time consuming and/or computationally expensive.
  • (2) With an increase of the number of IDR frames, the compression ratio decreases. The transcoded full length sequence with a higher number of IDR frames may be significantly larger than the original compressed full length sequence.
  • (3) The disadvantages of fast forwarding with raw files still remain.
  • Another method for skimming compressed video is to subdivide the compressed video into video chapters and view each segment in parallel in separate small windows (as described for skimming raw video). The process of sub-dividing a compressed file suffers from similar disadvantages as temporal sub-sampling. Further using traditional video compression technologies, spatial sub-sampling is not possible in the compressed domain. In order to generate the required spatially smaller video sequences, a transcoding step may be necessary, with the spatial sub-sampling being performed after the decompression of the original video data and before the compression. Moreover, although use of compressed video file eliminates the disadvantage of storing a large file, the need to decode the file several times in real-time introduces significant additional cost and processing complexity to spatial re-sampling.
  • If the full length video file, be it in raw or compressed format, is not available locally, the problem of video skimming according to the described techniques is further exacerbated by the need to retrieve it in real-time to a local computer over a network like the public Internet. Particularly, if the file is in raw format, then the bandwidth requirements are impractically large (i.e., 45 Mbps for a reasonable speed download of an SDTV resolution sequence). Accordingly, given the issues of using raw and compressed video, and using temporal and spatial sub-sampling, there has not been an acceptable implementation of a practical real-time video skimmer in the market place.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 illustrates temporal sub-sampling existing in the prior art.
  • FIG. 2 illustrates spatial sub-sampling existing in prior art.
  • FIG. 3 is an exemplary video display screen of the video skimmer in accordance with the present invention.
  • FIG. 4 is a block diagram illustrating an exemplary, system of a standalone video skimmer in accordance with the present invention.
  • FIG. 5 is a block diagram illustrating an exemplary system with a client-server video skimmer server architecture in accordance with the present invention.
  • FIG. 6 is a block diagram illustrating an exemplary standalone video skimmer in accordance with the present invention.
  • FIG. 7 is a block diagram illustrating an exemplary client-server video skimmer in accordance with the present invention.
  • FIG. 8 is an exemplary method flow for controlling MBW configurations in accordance with the present invention.
  • FIG. 9 is an exemplary method flow chart for messaging between skimming control logic client and server in accordance with the present invention.
  • Throughout the drawings, the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the disclosed invention will now be described in detail with reference to the figures, it is done so in connection with the illustrative embodiments.
  • DESCRIPTION
  • A video skimmer, according to the present invention, is a system which implements an approach of displaying multiple chapters of the full length video file, that may appear to the user as if it were spatially and/or temporally sub-sampled, simultaneously, by using a video file that has been compressed using a layered (also known as scalable) encoder. The video file may be indexed or un-indexed. According to the invention, no transcoding, or sub-sampling in either temporal or spatial dimension may be required in order to enable the skimming process. The system works efficiently when the full length video file is available locally, or remotely and accessible only over a network, for example the Internet.
  • According to the invention, the video may be compressed using a layered codec, such as the one disclosed in ITU-T Recommendation H.264 Annex G (also known as SVC). In order to take full advantage of the invention, the scalable video bitstream that is stored, among other things, in the full length video file, should contain at least one low resolution version of the video content, advantageously as a base layer. The low resolution can be stored in the form of a base layer and one or more enhancement layers; however, the mentioned combination of base and/or enhancement layers, after decoding, still results in the low resolution. The resolution can be chosen such that it is suitable, after decoding, for displaying in a mini browsing window (MBW) of the video skimmer display. An MBW can be smaller in spatial size than a full window, which can be optimized to view the full resolution video. Full resolution video may be obtained by decoding a base layer and at least one enhancement layer more than required for the lower resolution. The sizes of the full window and any MBWs can be chosen by the user according to his/her user preferences. The system can include a user interface that can display many MBWs, and each MBW can display a specific video chapter of the full length video. The user interface can also allow the user to set his/her user preferences, for example, number of MBWs, size of each MBW, start time or duration of each video chapter, assignment of chapters to MBWs, and so forth.
  • The term “codec” is equally used herein to describe techniques for encoding and decoding, and for implementations of these techniques. An encoder converts input media data into a bitstream or a packet stream, and a decoder converts an input bitstream or packet stream into a media representation suitable for presentation to a user, for example digital or analog video ready for presentation through a monitor, or digital or analog audio ready for presentation through loudspeakers. A transcoder converts an input bitstream or packet stream compressed using a compression technique into its original media representation suitable for presentation to a user and then re-converts into an input bitstream or packet stream using another type of compression technique. Encoders and decoders can be dedicated hardware devices or building blocks of a software-based implementation running on a general purpose CPU.
  • Set-top-boxes and personal computers (PCs) can be built such that many encoders or decoders may run in parallel or quasi-parallel. For hardware encoders or decoders, one way to support multiple encoders/decoders is to integrate multiple of their instances in the set-top-box or PC. For software implementations, similar mechanisms can be employed.
  • Traditional video codecs used in video distribution systems provide only a single bit stream at a given bitrate, and no layers. As explained above, when a lower temporal or spatial resolution is required from a full length video file (such as for fast forwarding or for display at a smaller spatial size in a MBW), first, the full resolution file must be decoded to regenerate the raw (uncompressed) video, which then needs to be sub-sampled in temporal and/or spatial dimension, as the case may be, to produce a lower spatio-temporal resolution appropriate for the MBW. This process wastes significant bandwidth (if the full length video file is in a remote location and needs to be transported over a network), time, and computational resources. However, support for lower resolutions is beneficial in the video skimmer to enable display of many video chapters simultaneously, and without consuming processing time and power to generate them. The network bandwidth required to transport video for many MBWs may also be advantageously minimized.
  • In one embodiment, a skimmer may support “spatial skimming”. A full length video file available in a layered encoded format may readily carry a low resolution version of the actual video content, which may fit into MBWs of the video skimmer system without further spatial sub-sampling after decoding. The skimmer may simultaneously display more than one MBW showing more than one chapter. The user may enlarge the video of a chapter by clicking on the MBW once he/she identifies the scene of interest in an MBW. As a result, the skimmer can request and receive information that enables the skimmer to present to the user a high resolution version of the video content, as disclosed in the co-pending U.S. patent application entitled “Systems, Methods and Computer Readable Media for Instant Multi-Channel Video Content Browsing in digital Video Distribution Systems”, concurrently filed herewith.
  • In the same or another embodiment, a video skimmer can support temporal skimming. A full length video file available in a layered encoded format may readily carry a temporally sub-sampled lower layer. The skimmer may disregard the timing information in the lower layer and present the video as fast forward video. For example, if the full length video were originally available at 30 fps, and the temporally sub-sampled lower layer is available at 10 fps, the skimmer may display the 10 fps lower layer at 30 fps, thereby speeding up playback at a factor of 3. Once the user clicks on the MBW presenting the fast forward video, the skimmer may display the MBW's content in original speed (in the example, by slowing down playback speed to 10 fps). It may further request and receive temporal scalable enhancement layers that enable full temporal resolution of the MBW's content.
  • The video skimming advantageously uses a lower resolution (spatial and/or temporal) version of the video content from several video chapters to fit into more than one MBW. The user may view several MBWs simultaneously, and may assign specific video chapters to these MBWs. The video chapters may be generated by any of the options discussed before.
  • In the same or another embodiment, video chapters may be the result of subdividing the full length video into video chapters of a given length. For example, video chapters may be assigned 10 minutes intervals of the first 40 minutes of the full length video sequentially, and those video chapters may be displayed in 4 MBWs.
  • In the same or another embodiment, the user may decide to switch the assignment of video chapter to MBWs during the skimming process (e.g., switch to assign every 10 minutes of the next 40 minutes of the video sequentially to 4 MBWs). An exemplary user interface with 4 MBWs skimming a 40 minutes full-length video in only 10 minutes is shown in FIG. 3. The first MBW (301) on the screen (305) displays the first 10 minutes of the full length video. The second and third MBWs (302) (303) on the left side of the screen display minutes 10-20 and 20-30 of the full length video, respectively. As disclosed in the co-pending U.S. patent application, entitled “Systems, Methods and Computer Readable Media for Instant Multi-Channel Video Content Browsing in digital Video Distribution Systems”, concurrently filed herewith, MBWs can be of different shapes and sizes. Accordingly, the right MBW, which displays minutes 30-40 of the full length video, is twice the size of the other three MBWs. A person skilled in the art can easily construct other screen layouts with more or less MBWs in different sizes, covering different parts of the full length video.
  • DETAILED DESCRIPTION OF THE INVENTION
  • FIG. 4 shows a standalone video skimmer (401) with an attached display (402). The video skimmer can receive video content from a variety of sources: live feed video content, for example from a camera (403) connected to the video skimmer through interface (404); video content from a DVD (405) attached to the video skimmer through interface (406); or even in the form of a full length video file from an external and/or remote video database (407). The external remote video database can be located on the Internet (409) or other suitable networks, using network interfaces (410, 411). The MBWs are presented on display 402 (as depicted in FIG. 3). The video skimmer logic is part of the video skimmer (401). The video skimmer can be implemented based on a general purpose computer, e.g., a PC, a standalone computer or some other type of hardware, such as a set-top box in IPTV environment where the set-top box may be attached to a suitable network (409) such as the Internet.
  • In case a video file is retrieved from a remote database, one of the well known file transfer protocols such as FTP (RFC959 available at http://www.faqs.org/rfcs/rfc959.html) may be used to transmit video over the Internet (409) (or any other suitable network) over links (410, 411).
  • A set-top box can be hardware for the video skimmer (401). A TV connected to the set-top-box can be used as the display (402). The set-top-box translates the data received from the network (409) into a signal format the TV understands; traditionally, a combination of analog audio and video signals are used, but recently also all digital interfaces (such as HDMI) have become common. The set-top-box (on the TV side), therefore typically includes analog or digital audio/video outputs and interfaces.
  • Internally, a set-top-boxes can have a hardware architecture similar to a general purpose computer: a central processing unit (CPU) executes instructions stored in Random Access Memory (RAM) or read-only-memory (ROM), and utilizes interface hardware to connect to the network interface and to the audio/video output interface, as well as an interface to a form of user control (often in the form of a TV remote control, computer mouse, keyboard, or other input device), all under the control of the CPU. A set-top-box may also include one or more accelerator units (for example dedicated Digital Signal Processors, DSP) that may help the CPU with computationally complex tasks of video decoding and video processing, among others. Those units are typically present for reasons of cost efficiency, rather than for technical necessity.
  • General purpose computers can often be configured to act like a set-top-box. In some cases, additional hardware needs to be added to the general purpose computer to provide the interfaces a typical set-top-box contains, and/or additional accelerator hardware to augment the CPU for video decoding and processing.
  • The programmable parts of set-top-boxes, PCs, and other devices suitable as the basis of a video skimmer may require instructions, which may be supplied by a computer readable media (408).
  • The set-top-box or general purpose computer may run under an operating system such as Windows. The video skimmer is advantageously using an operating system that allows the simultaneous display of more than one motion video in on-screen windows.
  • Referring to FIG. 5, the internal architecture of a standalone video skimmer is now described. There are several options for video inputs to the video skimmer, as follows.
  • (a) Live video content may be fed from the camera (501), which attaches to a live video interface (502) through connection (503). The video interface may connect to frame capture (504). The frame capture may generate video frames that may feed into a layered encoder (505).
  • (b) Alternatively, the video may be obtained as a file download from a remote video database (506) attached to a network, such as the Internet (507), in which case the file download may be received through network connection and a file download interface (508). If the video database (506) delivers the video in an appropriate layered format then it can be processed directly (509). Otherwise the video may be transcoded into a layered encoded format by a transcoder (510).
  • (c) In the same or another embodiment, video content (in uncompressed format, or in a compressed format that may not be a layered format) may be received from a DVD or a similar storage medium such as a memory stick (511), in which case it is received on a digital video storage interface (512). If the received video is not in layered encoded format, then they may first be transcoded in transcoder (513) into layered encoded format; otherwise, it may be processed directly (514).
  • Regardless of the input mechanism as described above, the video may be sent as a full length video file in a layered encoding format in the local video database (515).
  • A video extractor (516) may be responsible to retrieve a video bit stream from video database (515) according to control commands it may receive from a skimming controller (517). The skimming control logic (517) may, for example, indicate the MBW size and the beginning and ending time markers of the video for that MBW to video extractor, which, in response to the indication, may find the corresponding layered encoded bitstream in the database.
  • The extracted video may be displayed in the MBW on display (519) after decoding by a layered decoder (518). A user interface (520) may send appropriate display commands to properly display the video. The user input commands may be received through a user interface device (521) (e.g., a keyboard, mouse or remote control device), which may be translated into proper display commands through user interface (520) and may be displayed on the display (519). The commands may be sent by the user interface (520) to the skimming control logic (517) for further processing. Such user commands can, for example, include: (a) selection of size and display location of a MBW, (b) a click or double click on a MBW that may result in a request to receive corresponding audio or full resolution video, (c) entering user desired skimming parameters such as index markers for video chapter, etc.
  • The skimmer control logic (517) may receive user commands and may translate them into appropriate actions for the video extractor (516), thereby enabling the video extractor (516) to extract only those video bits required for proper display.
  • An alternative implementation of a video skimmer may follow a client-server architecture as illustrated in FIG. 6. The video skimmer may be divided into two components: the video skimmer client (601) and the video skimming server (602). The video skimmer client (601) and display (603) are located at the user's premises. The video skimmer client (601) may be connected to the video skimming server (602) over a suitable network, such as the Internet (604). The video skimming server (602) may reside at any suitable location in the network and not necessarily in the user's premises. A local video database (605) may be advantageously placed co-located with the video skimming server (602).
  • The video skimming server may also serve non-co-located, but network attached databases, such as a remote video database (606), via a suitable network such as the Internet (604). However, if the video file is located in a remote video database, that video file may advantageously be downloaded to local video database (605) in its entirety before starting the skimming process, because the video extraction logic for skimming resides within the video skimming server (602).
  • A single video skimming server may serve many remote video databases and many video skimmer clients simultaneously. With the separation of the client and server, the video skimming service (server) may become a business for a service provider which can offer it to many subscribers, each subscriber deploying the client component for skimming.
  • Although, for simplicity, only a single client and a single server are shown in FIG. 6, the present invention also envisions distributed architectures of the skimming server. Similarly, one or more video skimming clients can run simultaneously on a single client computer.
  • The video skimmer server may be responsible for extracting the user-requested video file from a local video database, and may send layered encoded video chapters according to a user's requests across the Internet to the client. Note that the video chapters displayed in the MBWs may be sent using only those layers required for proper decoding and display in a MBW without spatial or temporal sub sampling, thereby significantly reducing the network bandwidth, compared to the transmission of all layers beneficial for decoding and display of the video in a main window (at full resolution).
  • In FIG. 7, the detailed architecture of the video skimming server (701) is shown. The video skimming server (701) can include, for example, a skimming control logic server (SCLS) (702) which can communicate with the corresponding skimming control logic client (SCLC) (704) in the video skimming client (703). The SCLC can, for example, specify the MBW layout (e.g., each MBW location and size, or number of MBWs) at the user's endpoint, and video chapter definitions when the video chapters are not indexed (e.g., video skimming start time, and/or video chapter lengths, or any other information that allows for suitable indexing). If the full length video is already indexed, then the server can have the capability to send indexed video chapters, and can also send the metadata associated with each indexed video chapter. If a skimmed version of the full length video is available at the server, then the server can have the capability to send the skims and the meta data associated with each skim.
  • The SCLS (702) can serve, and can be controlled by, many SCLCs (704) simultaneously.
  • One purpose of the SCLC (704) is to translate user input into a protocol format that the SCLS (702) understands. For example, when the user clicks on an MBW to view the content to a high-resolution video display in the main window, the SCLC (704) can send a request to the SCLS (702) to start sending all enhancement layers for decoding and display in the main window in high resolution, in addition to the layers for decoding and display in MBWs (MBW layers), for the video chapter associated with the MBW user clicks. Meanwhile, the server can continue to send the MBW layers of the video chapters being decoded and displayed in the other MBWs.
  • The skimming video may advantageously reside in a local video database (705). If the video file is in a remote database (706), then the file can be retrieved and placed in the local video database (705). The file in the remote database (706) can either be in suitable layered encoded format, in which case it can be placed directly deposited to the local video database (705). If the video file is encoded using another type of compression technology (including, for example, possibly loss-less compression or an uncompressed format), it can first be transcoded in transcoder (707), and then it can be deposited to the local video database (705).
  • The video skimmer server may reside in one or more computers. The video skimming client may reside in a PC, a standalone general purpose computer, or it may be an IPTV set-top box.
  • If the user endpoint is a set-top-box, it can use, for example, a TV as the display (708) to display the MBWs.
  • User commands for video skimming may be received through a user input device (709) (e.g., mouse, keyboard or remote control), which can be translated into information displayed on display (708) after the user interface (710). When a user selects a video file for skimming and the skimming configuration (e.g., number of MBWs, assignment of video chapters to MBWs), user interface (710) can send these requests to SCLC (704) which, in turn, may send appropriate skimming messages to the SCLS (702). The SCLS (702) can instruct video extractor (711) to extract appropriate layers of the encoded video that can be stored in the local video database (705). The video extractor (711) can extract the bitstreams and send them to the streaming server (712). The streaming server can send streamed bitstreams using protocols such as RTP (RFC 3550, available from http://www.faqs.org/rfcs/rfc3550.html). On the client side, a streaming client (713) can receive the RTP packets, extract the bitstreams, and send the bitstreams to layered decoder (714) which can decode the bitstreams into raw format ready for display. User Interface (710) can collaborate with the SCLC (704) to assign received bitstreams from the layered decoder (714) to appropriate MBW on the user's display (708).
  • The SCLC (704) can communicate with its server component SCLS (702) to specify:
  • a. set or change MBW configuration (e.g., number or alignment of MBWs, size of MBWs, location of MBW windows on the display)
  • b. select video chapters (e.g., un-indexed, indexed, or skimmed)
  • c. configure video chapters (e.g., video chapter start times, lengths, chapter location in video file, mapping to MBWs, etc.)
  • d. MBW video controls (e.g., dump content to main window in higher resolution; receive audio, pause/restart/stop video).
  • In summary, the video skimming client (703) can include, for example, the following functionalities, although others could be added:
  • a. Receive and process streamed media;
  • b. Decode streams of video chapters for display;
  • c. Display video chapters in MBWs;
  • d. Control skimming logic (e.g., user selection of MBWs, video chapters, start-stop type video controls, and other controls); and/or
  • e. Receive user commands from user interface devices.
  • In summary, video skimmer server (701) can include, for example, the following functionalities, although others could be added:
  • a. Encode and Transcode remote video for storage in local video database;
  • b. Extract appropriate bitstreams from the video database for display in an MBW
  • c. Receive, process and react to commands from skimming control logic client
  • d. Stream and send video towards client applications.
  • As shown in FIG. 8, in the same or another embodiment, certain user activities, recognized by the user interface, can lead to certain actions of the skimming control logic—either the local SCLC or the logic that is distributed between SCLS and SCLC. The user activities can be listed in the form of a menu structure. The top level menu can be invoked by pressing (821) a “Menu” button or by a similar user input activity.
  • On its top level menu, the user interface can offer different pull-down menus to input various user preferences, for example, to: save user settings (801), restore user settings (802), select all MBWs (803), select one MBW (804), select skimming mode (805), and/or cancel (806). By selecting to cancel (806), the user closes the top level menu and all sub-menus that may be open without further interference in the state of the system. If the user selects any of the other selections, he/she is presented with a sub-menu as follows:
  • If the user selects to save user default settings (801), he/she is presented with certain related sub-menu choices, for example, to: save MBW configuration (808), save default skimming configuration (807), and/or cancel (809). Electing to cancel (809) closes the sub-menu without any change in state, and returns to the main menu. Selecting to save default skimming configuration (807) saves, possibly after a confirmation, the current skimming configuration as a default. The skimming configuration can include aspects such as the length of each chapter (e.g., select a uniformly assigned length such as 10 minutes, select a length that is determined by hints or context—e.g., based on metadata that may be included in the full length video indicating different scenes, or any other suitable length determination method). Selecting to save default MBW configuration (808) saves the current MBW configuration as a default. The MBW configuration is being determined through the two main menu items discussed next.
  • When the user selects the main menu item to restore user settings (802), once selected, restores the previously saved user settings, as stored using the menu item for saving user settings (801).
  • If the user chooses to select all MBWs (803), he/she can set properties related to all MBWs currently being displayed. Sub-menu items can include, for example, to: close all MBWs (810), arrange all MBWs on screen (811) (which distributes the MBWs evenly over the available screen area), to resize all MBWs (812) (which allows the user to set the size of all MBWs), and to cancel (813).
  • If the user chooses to select one MBW (804), he/she first selects the MBW to which the subsequent changes apply. Alternatively, or in addition, this menu item can also advantageously be implemented as a context sensitive menu. For example, right-clicking on an MBW triggers this submenu without requiring the user to go through the main menu. The sub-menus can offer different related actions, for example, to:
  • Map content to the main screen (814) (which closes the skimming user interface and shows the chapter assigned to the MBW in full screen resolution),
  • Change MBW size (815) (which allows to set the size of the MBW without changing the size of other MBWs),
  • Assign chapter (816) (which allows the user to assign a chapter of the full length video to the selected MBW),
  • Move MBW (817) (which allows the user to select the MB W′s spatial position on the screen without affecting the positions of other MBWs,
  • Close MBW (818), to pause/start/stop video in MBW (819), and/or
  • Cancel (820).
  • The main menu item to select skimming mode (805) provides the following related options for skimming mode selection, which may include, for example, fixed interval (822), scene detection (823), hint track (824), and/or Cancel (825).
  • If the user selects the fixed interval sub-menu option (822), the user can select the length of the chapters by providing either an interval (in, for example, seconds, minutes, and/or hours), or by selecting the number of equally long chapters the full length video shall be divided into by the video skimmer.
  • If the user selects the scene detection sub-menu option (823) the video skimmer is instructed to assign each scene, as determined by a scene detection algorithm, to an MBW, up to a user-selectable maximum number of MBWs.
  • The hint track sub-menu option (824) determines the association of chapters to MBWs using a hint track that may be present in the full-length video; if the hint track is not present, this option may be grayed out.
  • The cancel sub-menu option (825) leaves the sub-menu without a state change.
  • FIG. 9 shows an exemplary message flow between the skimming control logic client (SCLC) (901) and the skimming control logic server (SCLS) (902), using a protocol such as HTTP, or employing other standard-based or proprietary protocol. The SCLC can request a specific video chapter mapping to an MBW for an already selected MBW configuration through a video chapter assignment request message (903). This message can contain information about the user (e.g., Client ID), the video file (e.g., File ID or file title), the MBW (e.g., dimensions), and/or the begin and end time markers of the requested video chapter.
  • If the request is valid (904), then the SCLS (902) can return a video chapter assignment response message (905) to the SCLC (901) indicating that the action is accepted, in which case the SCLS instructs (906) the video extractor to fetch defined bitstream for local database. If the request can't be implemented, then the SCLS returns a video chapter assignment response message (907) indicating that the action is not valid.

Claims (56)

1. A system for simultaneous display of a plurality chapters of a full length video file, wherein the full length video file comprises video in a layered coding format.
2. The system of claim 1, wherein the full length video file comprises:
a first layer representing a first spatial resolution; and
at least one enhancement layer based on the first layer representing at least one second spatial resolution.
3. The system of claim 2, wherein the first layer comprises a base layer.
4. The system of claim 1, wherein the full length video file comprises:
a first layer representing a first temporal resolution; and
at least one enhancement layer based on the first layer representing at least one second temporal resolution.
5. The system of claim 4, wherein the first layer comprises a base layer.
6. The system of claim 1, wherein the full length video file comprises one or more video files located in a local database.
7. The system of claim 1, wherein the full length video file comprises content received from a live feed.
8. The system of claim 1, wherein the full length video file comprises content received from a digital video storage interface.
9. The system of claim 1, wherein the full length video file comprises content received from a remote video database.
10. The system of claim 1, further comprising an input, coupled to the display system, adapted for permitting a user to control the simultaneous display of a plurality of chapters through at least one user preference.
11. The system of claim 10, wherein the at least one user preference comprises a preference to specify a number of windows.
12. The system of claim 10, wherein the at least one user preference comprises a preference to specify a duration for a video chapter.
13. The system of claim 10, wherein the at least one user preference comprises a preference to specify a start time for a video chapter.
14. The system of claim 10, wherein the at least one user preference comprises a preference to assign at least one chapter to at least one window.
15. The system of claim 1, further comprising one or more skimming control logic modules configured to control at least one aspect of a display system.
16. The system of claim 10, further comprising one or more skimming control logic modules configured to control at least one aspect of a display system.
17. The system of claim 16, wherein the skimming control logic further comprises logic adapted to translate one or more user preferences into video extraction commands.
18. A method of video skimming, comprising:
a. extracting a plurality of chapters of a full length video file, wherein the chapters are coded in a layered bitstream format,
b. decoding the plurality of chapters, and
c. simultaneously displaying each of the decoded plurality of chapters.
19. The method of claim 18 wherein the layered bitstream format comprises a first layer representing a first spatial resolution and at least one enhancement layer based on the first layer representing at least one second spatial resolution, the decoding further comprises:
a. decoding the first layer of the plurality of chapters; and
b. decoding at least one of the enhancement layers of the plurality of chapters.
20. The method of claim 18, further comprising extracting the full length video file from a local database.
21. The method of claim 18, further comprising receiving a full length video file content from a live feed.
22. The method of claim 18, further comprising receiving the full length video file content from a digital video storage interface.
23. The method of claim 18, further comprising receiving the full-length video file from a remote video database.
24. The method of claim 18, wherein the method further comprises receiving a user input to control the simultaneous display of a plurality of chapters through at least one user preference.
25. The method of claim 24, wherein the at least one user preference comprises a preference to specify a number of windows.
26. The method of claim 24, wherein the at least one user preference comprises a preference to specify a duration for a video chapter.
27. The method of claim 24, wherein the at least one user preference comprises a preference to specify a start time for a video chapter.
28. The method of claim 24, wherein the at least one user preference comprises a preference to assign at least one chapter to at least one window.
29. The method of claim 24, further comprising translating the at least one user preference into at least one video extraction command.
30. A computer readable media having computer executable instructions included thereon for performing a method of video skimming, comprising:
a. extracting a plurality of chapters of a full length video file, wherein the chapters are coded in a layered bitstream format, and
b. decoding the plurality of chapters, and
c. simultaneously displaying each of the decoded plurality of chapters.
31. The computer readable media of claim 30, wherein the layered bitstream format comprises a first layer representing a first spatial resolution and at least one enhancement layer based on the first layer representing at least one second spatial resolution, the decoding further comprises:
a. decoding the first layer of the plurality of chapters; and
b. decoding at least one of the enhancement layers of the plurality of chapters.
32. The computer readable media of claim 30, wherein the method further comprises extracting the full length video file from a local database.
33. The computer readable media of claim 30, wherein the method further comprises receiving a full length video file content from a live feed.
34. The computer readable media of claim 30, wherein the method further comprises receiving the full length video file content from a digital video storage interface.
35. The computer readable media of claim 30, wherein the method further comprises receiving the full-length video file from a remote video database.
36. The computer readable media of claim 30, wherein the method further comprises receiving a user input to control the simultaneous display of a plurality of chapters through at least one user preference.
37. The computer readable media of claim 36, wherein the at least one user preference comprises a preference to specify a number of windows.
38. The computer readable media of claim 36, wherein the at least one user preference comprises a preference to specify a duration for a video chapter.
39. The computer readable media of claim 36, wherein the at least one user preference comprises a preference to specify a start time for a video chapter.
40. The computer readable media of claim 36, wherein the at least one user preference comprises a preference to assign at least one chapter to at least one window.
41. The computer readable media of claim 36, wherein the method further comprises translating the at least one user preference into at least one video extraction command.
42. A video skimming server for preparing and distributing a plurality of chapters of a full length video file, wherein the full length video file comprises video coded in a layered bitstream format, comprising:
(a) a video extractor for extracting a plurality of encoded audio-visual signals from a full length video file;
(b) a streaming server for distributing the extracted audio-visual signals; and
(c) skimming control logic server, coupled to the video extractor, adapted for receiving at least one control message from a video skimmer client and instructing the video extractor to extract the audio-visual signals.
43. The video skimming server of claim 42, wherein the full length video file comprises one or more video files located in a local video database.
44. The video skimming server of claim 42, wherein the full length video file comprises content received from a live feed.
45. The video skimming server of claim 42, wherein the full length video file comprises content received from a digital storage interface.
46. The video skimming server of claim 42, wherein the full length video file comprises content received from a remote database.
47. The video skimming server of claim 46, further comprising a transcoder, coupled to the video extractor and to the remote database, for transcoding the content received from the remote database.
48. The video skimming server of claim 42, wherein the video extractor comprises at least a portion of a distributed server.
49. The video skimming server of claim 42, wherein the skimming control logic server comprises at least a portion of a distributed server.
50. A video skimmer client for preparing a customized display of a plurality of chapters of a full length video file, wherein the full length video file is coded in a layered bitstream format, comprising:
(a) at least one streaming client module configured to receive a plurality of chapters, wherein the chapters are in a layered bitstream format;
(b) one or more decoders configured to decode the chapters;
(c) a graphical user interface for receiving user input, wherein the graphical user interface is accessed through a video display; and
(d) a skimmer control logic client for sending at least one control message to a video skimmer server.
51. The video skimmer client of claim 50, wherein said video display comprises a television.
52. The video skimmer client of claim 50, wherein said video display comprises a computer monitor.
53. The video skimmer client of claim 50, wherein said video skimmer client comprises at least a portion of a television.
54. The video skimmer client of claim 50, wherein said receiver comprises at least a portion of a general purpose computer.
55. The video skimmer client of claim 50, wherein said video skimmer client comprises at least a portion of a set-top box.
56. The video skimmer client of claim 50, wherein said video skimmer client comprises at least a portion of a gaming console.
US12/765,815 2009-04-24 2010-04-22 Efficient video skimmer Abandoned US20100272187A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17235509P true 2009-04-24 2009-04-24
US12/765,815 US20100272187A1 (en) 2009-04-24 2010-04-22 Efficient video skimmer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/765,815 US20100272187A1 (en) 2009-04-24 2010-04-22 Efficient video skimmer

Publications (1)

Publication Number Publication Date
US20100272187A1 true US20100272187A1 (en) 2010-10-28

Family

ID=42992120

Family Applications (4)

Application Number Title Priority Date Filing Date
US12/765,793 Expired - Fee Related US8607283B2 (en) 2009-04-24 2010-04-22 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US12/765,815 Abandoned US20100272187A1 (en) 2009-04-24 2010-04-22 Efficient video skimmer
US12/765,767 Active 2031-04-07 US8341672B2 (en) 2009-04-24 2010-04-22 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US13/895,131 Active 2031-08-01 US9426536B2 (en) 2009-04-24 2013-05-15 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US12/765,793 Expired - Fee Related US8607283B2 (en) 2009-04-24 2010-04-22 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems

Family Applications After (2)

Application Number Title Priority Date Filing Date
US12/765,767 Active 2031-04-07 US8341672B2 (en) 2009-04-24 2010-04-22 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US13/895,131 Active 2031-08-01 US9426536B2 (en) 2009-04-24 2013-05-15 Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems

Country Status (7)

Country Link
US (4) US8607283B2 (en)
EP (1) EP2422469A4 (en)
JP (1) JP2012525076A (en)
CN (1) CN102422577A (en)
AU (1) AU2010238757A1 (en)
CA (1) CA2759729A1 (en)
WO (3) WO2010124133A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100293584A1 (en) * 2009-04-24 2010-11-18 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
EP2472519A1 (en) * 2011-01-04 2012-07-04 Samsung Electronics Co., Ltd. Multi-video rendering for enhancing user interface usability and user experience
WO2012121744A1 (en) * 2011-03-10 2012-09-13 Vidyo, Inc Adaptive picture rotation
US20120250755A1 (en) * 2011-03-29 2012-10-04 Lyrical Labs LLC Video encoding system and method
US20130088600A1 (en) * 2011-10-05 2013-04-11 Xerox Corporation Multi-resolution video analysis and key feature preserving video reduction strategy for (real-time) vehicle tracking and speed enforcement systems
EP2741518A1 (en) * 2012-12-04 2014-06-11 Alcatel Lucent Method for rendering a multimedia asset, a related system, media client and related media server
US8891935B2 (en) 2011-01-04 2014-11-18 Samsung Electronics Co., Ltd. Multi-video rendering for enhancing user interface usability and user experience
US20150213044A1 (en) * 2012-10-12 2015-07-30 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. File system, computer and method
US20160055886A1 (en) * 2014-08-20 2016-02-25 Carl Zeiss Meditec Ag Method for Generating Chapter Structures for Video Data Containing Images from a Surgical Microscope Object Area
US20160127728A1 (en) * 2014-10-30 2016-05-05 Kabushiki Kaisha Toshiba Video compression apparatus, video playback apparatus and video delivery system
US20160345009A1 (en) * 2015-05-19 2016-11-24 ScaleFlux Accelerating image analysis and machine learning through in-flash image preparation and pre-processing
RU170238U1 (en) * 2016-11-14 2017-04-18 Закрытое акционерное общество "Региональный научно-исследовательский экспертный центр" Complex for research of radio electronic devices
US9743099B2 (en) 2011-03-10 2017-08-22 Vidyo, Inc. Render-orientation information in video bitstream
US20170302977A1 (en) * 2014-09-22 2017-10-19 Airbus Ds Sas Method for transmitting video surveillance images
US20180052583A1 (en) * 2016-08-18 2018-02-22 Lg Electronics Inc. Terminal and controlling method thereof
US10284858B2 (en) * 2013-10-15 2019-05-07 Qualcomm Incorporated Support of multi-mode extraction for multi-layer video codecs

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102362494A (en) * 2009-06-19 2012-02-22 深圳Tcl新技术有限公司 Generation method of electronic program guide menu of television and TV set
KR20100137159A (en) * 2009-06-22 2010-12-30 삼성전자주식회사 Broadcast receiving apparatus and method for switching channel thereof
US8473998B1 (en) * 2009-07-29 2013-06-25 Massachusetts Institute Of Technology Network coding for multi-resolution multicast
TWI445329B (en) * 2009-08-05 2014-07-11
US9369508B2 (en) * 2010-10-06 2016-06-14 Humax Co., Ltd. Method for transmitting a scalable HTTP stream for natural reproduction upon the occurrence of expression-switching during HTTP streaming
CN102097081A (en) * 2011-01-27 2011-06-15 明基电通有限公司 Display device as well as driving method and displaying method thereof
US8689269B2 (en) * 2011-01-27 2014-04-01 Netflix, Inc. Insertion points for streaming video autoplay
EP2568471A1 (en) * 2011-09-12 2013-03-13 Alcatel Lucent A method for playing multimedia content, a related system and related playback module
GB2500245B (en) * 2012-03-15 2014-05-14 Toshiba Res Europ Ltd Rate optimisation for scalable video transmission
US9172737B2 (en) * 2012-07-30 2015-10-27 New York University Streamloading content, such as video content for example, by both downloading enhancement layers of the content and streaming a base layer of the content
KR101966921B1 (en) * 2012-09-12 2019-08-27 삼성전자주식회사 Method and Apparatus of managing muti-session
US9407961B2 (en) * 2012-09-14 2016-08-02 Intel Corporation Media stream selective decode based on window visibility state
EP2898506B1 (en) 2012-09-21 2018-01-17 Dolby Laboratories Licensing Corporation Layered approach to spatial audio coding
US20140096167A1 (en) * 2012-09-28 2014-04-03 Vringo Labs, Inc. Video reaction group messaging with group viewing
US8804042B2 (en) 2013-01-14 2014-08-12 International Business Machines Corporation Preemptive preloading of television program data
WO2014210284A1 (en) 2013-06-27 2014-12-31 Dolby Laboratories Licensing Corporation Bitstream syntax for spatial voice coding
US10257494B2 (en) * 2014-09-22 2019-04-09 Samsung Electronics Co., Ltd. Reconstruction of three-dimensional video
WO2016079276A1 (en) * 2014-11-21 2016-05-26 Takeda Gmbh Use of an anti-gm-csf antagonist and an anti-ccr2 antagonist in the treatment of an infectious disease
US9961004B2 (en) 2015-02-18 2018-05-01 Viasat, Inc. Popularity-aware bitrate adaptation of linear programming for mobile communications
US9716735B2 (en) 2015-02-18 2017-07-25 Viasat, Inc. In-transport multi-channel media delivery
CN106534879A (en) * 2016-11-08 2017-03-22 天脉聚源(北京)传媒科技有限公司 Live broadcasting switching method and system based on attention

Citations (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5555244A (en) * 1994-05-19 1996-09-10 Integrated Network Corporation Scalable multimedia network
US6055012A (en) * 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
US6167084A (en) * 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US6275531B1 (en) * 1998-07-23 2001-08-14 Optivision, Inc. Scalable video coding method and apparatus
US6292512B1 (en) * 1998-07-06 2001-09-18 U.S. Philips Corporation Scalable video coding system
US20020129374A1 (en) * 1991-11-25 2002-09-12 Michael J. Freeman Compressed digital-data seamless video switching system
US20020136162A1 (en) * 2001-03-21 2002-09-26 Ntt Docomo, Inc Communication quality control scheme using real time packet transmission state and transmission path congestion state
US20020163918A1 (en) * 2001-05-04 2002-11-07 Globespan Virata, Incorporated System and method for distributed processing of packet data containing audio information
US6496217B1 (en) * 2001-06-12 2002-12-17 Koninklijke Philips Electronics N.V. Video communication system using model-based coding and prioritzation techniques
US6498865B1 (en) * 1999-02-11 2002-12-24 Packetvideo Corp,. Method and device for control and compatible delivery of digitally compressed visual data in a heterogeneous communication network
US6510553B1 (en) * 1998-10-26 2003-01-21 Intel Corporation Method of streaming video from multiple sources over a network
US20030074674A1 (en) * 2001-10-17 2003-04-17 Magliaro Maximilian Matthew Method and system for dynamically adjusting video bit rates
US20030135631A1 (en) * 2001-12-28 2003-07-17 Microsoft Corporation System and method for delivery of dynamically scalable audio/video content over a network
US20030156824A1 (en) * 2002-02-21 2003-08-21 Koninklijke Philips Electronics N.V. Simultaneous viewing of time divided segments of a tv program
US6643496B1 (en) * 1998-03-31 2003-11-04 Canon Kabushiki Kaisha System, method, and apparatus for adjusting packet transmission rates based on dynamic evaluation of network characteristics
US20040001479A1 (en) * 2002-07-01 2004-01-01 Pounds Gregory E. Systems and methods for voice and data communications including a network drop and insert interface for an external data routing resource
US20040031058A1 (en) * 2002-05-10 2004-02-12 Richard Reisman Method and apparatus for browsing using alternative linkbases
US20040033061A1 (en) * 2000-05-05 2004-02-19 Hughes Robert K. Layered coding of image data using separate data storage tracks on a storage medium
US20040042549A1 (en) * 2002-08-27 2004-03-04 Hsiang-Chun Huang Architecture and method for fine granularity scalable video coding
US20040046801A1 (en) * 2001-10-16 2004-03-11 Liang-Jin Lin System and method for constructing an interactive video menu
US20040071354A1 (en) * 2002-10-11 2004-04-15 Ntt Docomo, Inc. Video encoding method, video decoding method, video encoding apparatus, video decoding apparatus, video encoding program, and video decoding program
US20040218816A1 (en) * 2003-04-30 2004-11-04 Nokia Corporation Picture coding method
US20050135477A1 (en) * 2000-07-11 2005-06-23 Microsoft Corporation Systems and methods with error resilience in enhancement layer bitstream of scalable video coding
US6912584B2 (en) * 1999-03-12 2005-06-28 Microsoft Corporation Media coding for loss recovery with remotely predicted data units
US20050147164A1 (en) * 2000-12-15 2005-07-07 Microsoft Corporation Drifting reduction and macroblock-based control in progressive fine granularity scalable video coding
US20050254575A1 (en) * 2004-05-12 2005-11-17 Nokia Corporation Multiple interoperability points for scalable media coding and transmission
US20050265450A1 (en) * 2004-05-04 2005-12-01 Raveendran Vijayalakshmi R Method and apparatus to construct bi-directional predicted frames for temporal scalability
US6973622B1 (en) * 2000-09-25 2005-12-06 Wireless Valley Communications, Inc. System and method for design, tracking, measurement, prediction and optimization of data communication networks
US20050275752A1 (en) * 2002-10-15 2005-12-15 Koninklijke Philips Electronics N.V. System and method for transmitting scalable coded video over an ip network
US20060015637A1 (en) * 2004-07-02 2006-01-19 Matrixstream Technologies Inc. System and method for transferring content via a network
US7012893B2 (en) * 2001-06-12 2006-03-14 Smartpackets, Inc. Adaptive control of data packet size in networks
US20060184966A1 (en) * 2005-02-14 2006-08-17 Hillcrest Laboratories, Inc. Methods and systems for enhancing television applications using 3D pointing
US20070003222A1 (en) * 2005-02-04 2007-01-04 Kosuke Shingai Printing based on motion picture
US20070073779A1 (en) * 2005-09-27 2007-03-29 Walker Gordon K Channel switch frame
US20070121584A1 (en) * 2005-11-25 2007-05-31 Chaoxin Qiu Caller ID information to internet protocol television displays
US20070195203A1 (en) * 2006-02-21 2007-08-23 Qualcomm Incorporated Multi-program viewing in a wireless apparatus
US20070203942A1 (en) * 2006-02-27 2007-08-30 Microsoft Corporation Video Search and Services
US20080148322A1 (en) * 2006-12-18 2008-06-19 At&T Knowledge Ventures, Lp System and method of providing video-on-demand content
US20080168523A1 (en) * 2006-12-29 2008-07-10 Prodea Systems, Inc. System And Method To Acquire, Aggregate, Manage, And Distribute Media
US20080273079A1 (en) * 2002-03-27 2008-11-06 Robert Craig Campbell Videophone and method for a video call
US20090021592A1 (en) * 2007-07-17 2009-01-22 Sony Corporation Display control apparatus, display control method, and program therefor
US20090025028A1 (en) * 2007-07-20 2009-01-22 At&T Intellectual Property, Inc. Systems, methods and computer products for internet protocol television voicemail monitoring
US20090187960A1 (en) * 2008-01-17 2009-07-23 Joon Hui Lee IPTV receiving system and data processing method
US20090252228A1 (en) * 2005-08-26 2009-10-08 Boyce Jill Macdonald Trick Play Using Temporal Layering
US20100161716A1 (en) * 2008-12-22 2010-06-24 General Instrument Corporation Method and apparatus for streaming multiple scalable coded video content to client devices at different encoding rates
US20100259595A1 (en) * 2009-04-10 2010-10-14 Nokia Corporation Methods and Apparatuses for Efficient Streaming of Free View Point Video
US20100275229A1 (en) * 2009-04-24 2010-10-28 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US7912098B2 (en) * 2007-03-29 2011-03-22 Alcatel Lucent System, method, and device using a singly encapsulated bundle and a tagger for re-encapsulation
US8094719B2 (en) * 2002-04-11 2012-01-10 Microsoft Corporation Streaming methods and systems

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6141053A (en) 1997-01-03 2000-10-31 Saukkonen; Jukka I. Method of optimizing bandwidth for transmitting compressed video data streams
US6836512B2 (en) * 2000-10-11 2004-12-28 Koninklijke Philips Electronics N.V. Spatial scalability for fine granular video encoding
US9161033B2 (en) * 2003-06-16 2015-10-13 Thomson Licensing Decoding method and apparatus enabling fast channel change of compressed video
US8327411B2 (en) 2003-12-01 2012-12-04 Sharp Laboratories Of America, Inc. Low-latency random access to compressed video
US20070195878A1 (en) 2004-04-06 2007-08-23 Koninklijke Philips Electronics, N.V. Device and method for receiving video data
KR20070090241A (en) * 2004-12-10 2007-09-05 코닌클리케 필립스 일렉트로닉스 엔.브이. System and method for layered digital video coding in a digital video recorder
KR100643291B1 (en) * 2005-04-14 2006-11-10 삼성전자주식회사 Apparatus and method of video encoding and decoding for minimizing random access delay
KR100667806B1 (en) * 2005-07-07 2007-01-12 삼성전자주식회사 Method and apparatus for video encoding and decoding
US8396134B2 (en) * 2006-07-21 2013-03-12 Vidyo, Inc. System and method for scalable video coding using telescopic mode flags
AU2006330457B2 (en) * 2005-12-22 2011-07-14 Vidyo, Inc. System and method for videoconferencing using scalable video coding and compositing scalable video conferencing servers
KR101396675B1 (en) * 2006-01-05 2014-05-16 텔레폰악티에볼라겟엘엠에릭슨(펍) Media content management
FR2898236A1 (en) * 2006-03-03 2007-09-07 Thomson Licensing Sas Method of transmitting audiovisual flows by anticipating controls of the user, receiver and transmitter for implementing the method
CN100588249C (en) * 2006-07-27 2010-02-03 腾讯科技(深圳)有限公司 Method, system and terminal for adjusting video quality
US8773494B2 (en) * 2006-08-29 2014-07-08 Microsoft Corporation Techniques for managing visual compositions for a multimedia conference call
CN101202906A (en) * 2006-12-11 2008-06-18 国际商业机器公司 Method and equipment for processing video stream in digital video broadcast system
US8230100B2 (en) * 2007-07-26 2012-07-24 Realnetworks, Inc. Variable fidelity media provision system and method
WO2009155963A1 (en) * 2008-06-23 2009-12-30 Ericsson Hungary Ltd Improving transmission of media streams of broadcast services in a multimedia broadcast transmission system
US8706863B2 (en) * 2008-07-18 2014-04-22 Apple Inc. Systems and methods for monitoring data and bandwidth usage

Patent Citations (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020129374A1 (en) * 1991-11-25 2002-09-12 Michael J. Freeman Compressed digital-data seamless video switching system
US5555244A (en) * 1994-05-19 1996-09-10 Integrated Network Corporation Scalable multimedia network
US6055012A (en) * 1995-12-29 2000-04-25 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
US6643496B1 (en) * 1998-03-31 2003-11-04 Canon Kabushiki Kaisha System, method, and apparatus for adjusting packet transmission rates based on dynamic evaluation of network characteristics
US6292512B1 (en) * 1998-07-06 2001-09-18 U.S. Philips Corporation Scalable video coding system
US6275531B1 (en) * 1998-07-23 2001-08-14 Optivision, Inc. Scalable video coding method and apparatus
US6167084A (en) * 1998-08-27 2000-12-26 Motorola, Inc. Dynamic bit allocation for statistical multiplexing of compressed and uncompressed digital video signals
US6510553B1 (en) * 1998-10-26 2003-01-21 Intel Corporation Method of streaming video from multiple sources over a network
US6498865B1 (en) * 1999-02-11 2002-12-24 Packetvideo Corp,. Method and device for control and compatible delivery of digitally compressed visual data in a heterogeneous communication network
US6912584B2 (en) * 1999-03-12 2005-06-28 Microsoft Corporation Media coding for loss recovery with remotely predicted data units
US20040033061A1 (en) * 2000-05-05 2004-02-19 Hughes Robert K. Layered coding of image data using separate data storage tracks on a storage medium
US20050135477A1 (en) * 2000-07-11 2005-06-23 Microsoft Corporation Systems and methods with error resilience in enhancement layer bitstream of scalable video coding
US6973622B1 (en) * 2000-09-25 2005-12-06 Wireless Valley Communications, Inc. System and method for design, tracking, measurement, prediction and optimization of data communication networks
US20050147164A1 (en) * 2000-12-15 2005-07-07 Microsoft Corporation Drifting reduction and macroblock-based control in progressive fine granularity scalable video coding
US20020136162A1 (en) * 2001-03-21 2002-09-26 Ntt Docomo, Inc Communication quality control scheme using real time packet transmission state and transmission path congestion state
US20020163918A1 (en) * 2001-05-04 2002-11-07 Globespan Virata, Incorporated System and method for distributed processing of packet data containing audio information
US7012893B2 (en) * 2001-06-12 2006-03-14 Smartpackets, Inc. Adaptive control of data packet size in networks
US6496217B1 (en) * 2001-06-12 2002-12-17 Koninklijke Philips Electronics N.V. Video communication system using model-based coding and prioritzation techniques
US20040046801A1 (en) * 2001-10-16 2004-03-11 Liang-Jin Lin System and method for constructing an interactive video menu
US20030074674A1 (en) * 2001-10-17 2003-04-17 Magliaro Maximilian Matthew Method and system for dynamically adjusting video bit rates
US20030135631A1 (en) * 2001-12-28 2003-07-17 Microsoft Corporation System and method for delivery of dynamically scalable audio/video content over a network
US20030156824A1 (en) * 2002-02-21 2003-08-21 Koninklijke Philips Electronics N.V. Simultaneous viewing of time divided segments of a tv program
US20080273079A1 (en) * 2002-03-27 2008-11-06 Robert Craig Campbell Videophone and method for a video call
US8094719B2 (en) * 2002-04-11 2012-01-10 Microsoft Corporation Streaming methods and systems
US20040031058A1 (en) * 2002-05-10 2004-02-12 Richard Reisman Method and apparatus for browsing using alternative linkbases
US20040001479A1 (en) * 2002-07-01 2004-01-01 Pounds Gregory E. Systems and methods for voice and data communications including a network drop and insert interface for an external data routing resource
US20040042549A1 (en) * 2002-08-27 2004-03-04 Hsiang-Chun Huang Architecture and method for fine granularity scalable video coding
US20040071354A1 (en) * 2002-10-11 2004-04-15 Ntt Docomo, Inc. Video encoding method, video decoding method, video encoding apparatus, video decoding apparatus, video encoding program, and video decoding program
US20050275752A1 (en) * 2002-10-15 2005-12-15 Koninklijke Philips Electronics N.V. System and method for transmitting scalable coded video over an ip network
US20040218816A1 (en) * 2003-04-30 2004-11-04 Nokia Corporation Picture coding method
US20050265450A1 (en) * 2004-05-04 2005-12-01 Raveendran Vijayalakshmi R Method and apparatus to construct bi-directional predicted frames for temporal scalability
US20050254575A1 (en) * 2004-05-12 2005-11-17 Nokia Corporation Multiple interoperability points for scalable media coding and transmission
US20060015637A1 (en) * 2004-07-02 2006-01-19 Matrixstream Technologies Inc. System and method for transferring content via a network
US20070003222A1 (en) * 2005-02-04 2007-01-04 Kosuke Shingai Printing based on motion picture
US20060184966A1 (en) * 2005-02-14 2006-08-17 Hillcrest Laboratories, Inc. Methods and systems for enhancing television applications using 3D pointing
US20090252228A1 (en) * 2005-08-26 2009-10-08 Boyce Jill Macdonald Trick Play Using Temporal Layering
US20070073779A1 (en) * 2005-09-27 2007-03-29 Walker Gordon K Channel switch frame
US20070121584A1 (en) * 2005-11-25 2007-05-31 Chaoxin Qiu Caller ID information to internet protocol television displays
US20070195203A1 (en) * 2006-02-21 2007-08-23 Qualcomm Incorporated Multi-program viewing in a wireless apparatus
US20070203942A1 (en) * 2006-02-27 2007-08-30 Microsoft Corporation Video Search and Services
US20080148322A1 (en) * 2006-12-18 2008-06-19 At&T Knowledge Ventures, Lp System and method of providing video-on-demand content
US20080168523A1 (en) * 2006-12-29 2008-07-10 Prodea Systems, Inc. System And Method To Acquire, Aggregate, Manage, And Distribute Media
US7912098B2 (en) * 2007-03-29 2011-03-22 Alcatel Lucent System, method, and device using a singly encapsulated bundle and a tagger for re-encapsulation
US20090021592A1 (en) * 2007-07-17 2009-01-22 Sony Corporation Display control apparatus, display control method, and program therefor
US20090025028A1 (en) * 2007-07-20 2009-01-22 At&T Intellectual Property, Inc. Systems, methods and computer products for internet protocol television voicemail monitoring
US20090187960A1 (en) * 2008-01-17 2009-07-23 Joon Hui Lee IPTV receiving system and data processing method
US20100161716A1 (en) * 2008-12-22 2010-06-24 General Instrument Corporation Method and apparatus for streaming multiple scalable coded video content to client devices at different encoding rates
US20100259595A1 (en) * 2009-04-10 2010-10-14 Nokia Corporation Methods and Apparatuses for Efficient Streaming of Free View Point Video
US20100275229A1 (en) * 2009-04-24 2010-10-28 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US20100293584A1 (en) * 2009-04-24 2010-11-18 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8341672B2 (en) 2009-04-24 2012-12-25 Delta Vidyo, Inc Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US8607283B2 (en) 2009-04-24 2013-12-10 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US9426536B2 (en) 2009-04-24 2016-08-23 Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US20100293584A1 (en) * 2009-04-24 2010-11-18 Delta Vidyo, Inc. Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US8891935B2 (en) 2011-01-04 2014-11-18 Samsung Electronics Co., Ltd. Multi-video rendering for enhancing user interface usability and user experience
EP2472519A1 (en) * 2011-01-04 2012-07-04 Samsung Electronics Co., Ltd. Multi-video rendering for enhancing user interface usability and user experience
WO2012121744A1 (en) * 2011-03-10 2012-09-13 Vidyo, Inc Adaptive picture rotation
US9743099B2 (en) 2011-03-10 2017-08-22 Vidyo, Inc. Render-orientation information in video bitstream
US10027970B2 (en) 2011-03-10 2018-07-17 Vidyo, Inc. Render-orientation information in video bitstream
US20120250755A1 (en) * 2011-03-29 2012-10-04 Lyrical Labs LLC Video encoding system and method
US9712835B2 (en) * 2011-03-29 2017-07-18 Lyrical Labs LLC Video encoding system and method
US8953044B2 (en) * 2011-10-05 2015-02-10 Xerox Corporation Multi-resolution video analysis and key feature preserving video reduction strategy for (real-time) vehicle tracking and speed enforcement systems
US20130088600A1 (en) * 2011-10-05 2013-04-11 Xerox Corporation Multi-resolution video analysis and key feature preserving video reduction strategy for (real-time) vehicle tracking and speed enforcement systems
US20150213044A1 (en) * 2012-10-12 2015-07-30 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. File system, computer and method
US10152485B2 (en) * 2012-10-12 2018-12-11 Fraunhofe-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. File system, computer and method
EP2741518A1 (en) * 2012-12-04 2014-06-11 Alcatel Lucent Method for rendering a multimedia asset, a related system, media client and related media server
US10284858B2 (en) * 2013-10-15 2019-05-07 Qualcomm Incorporated Support of multi-mode extraction for multi-layer video codecs
US20160055886A1 (en) * 2014-08-20 2016-02-25 Carl Zeiss Meditec Ag Method for Generating Chapter Structures for Video Data Containing Images from a Surgical Microscope Object Area
US10225586B2 (en) * 2014-09-22 2019-03-05 Airbus Ds Sas Method for transmitting video surveillance images
US20170302977A1 (en) * 2014-09-22 2017-10-19 Airbus Ds Sas Method for transmitting video surveillance images
US20160127728A1 (en) * 2014-10-30 2016-05-05 Kabushiki Kaisha Toshiba Video compression apparatus, video playback apparatus and video delivery system
US20160345009A1 (en) * 2015-05-19 2016-11-24 ScaleFlux Accelerating image analysis and machine learning through in-flash image preparation and pre-processing
US20180052583A1 (en) * 2016-08-18 2018-02-22 Lg Electronics Inc. Terminal and controlling method thereof
US10353536B2 (en) * 2016-08-18 2019-07-16 Lg Electronics Inc. Terminal and controlling method thereof
RU170238U1 (en) * 2016-11-14 2017-04-18 Закрытое акционерное общество "Региональный научно-исследовательский экспертный центр" Complex for research of radio electronic devices

Also Published As

Publication number Publication date
JP2012525076A (en) 2012-10-18
EP2422469A4 (en) 2012-10-31
US8607283B2 (en) 2013-12-10
US20100293584A1 (en) 2010-11-18
WO2010124136A1 (en) 2010-10-28
EP2422469A1 (en) 2012-02-29
WO2010124140A1 (en) 2010-10-28
CA2759729A1 (en) 2010-10-28
WO2010124133A1 (en) 2010-10-28
CN102422577A (en) 2012-04-18
AU2010238757A1 (en) 2011-11-03
US8341672B2 (en) 2012-12-25
US9426536B2 (en) 2016-08-23
US20130254817A1 (en) 2013-09-26
US20100275229A1 (en) 2010-10-28

Similar Documents

Publication Publication Date Title
US10264218B2 (en) Method, system, and apparatus for receiving, processing and rendering content responsive to user and device parameters
JP4955544B2 (en) Client / server architecture and method for zoomable user interface
US9185439B2 (en) Signaling data for multiplexing video components
US7739714B2 (en) System for transmitting digital data over a limited bandwidth link in plural blocks
CA2142801C (en) Frame sampling scheme for video in a video-on-demand system
US6445738B1 (en) System and method for creating trick play video streams from a compressed normal play video bitstream
ES2730894T3 (en) Manifest file updates for continuous streaming of encoded video data network
EP2919453B1 (en) Video stream switching
US5903264A (en) Video delivery system and method for displaying an indexing slider bar
EP1009170B1 (en) System, method and apparatus for a variable output video decoder
US20020154691A1 (en) System and process for compression, multiplexing, and real-time low-latency playback of networked audio/video bit streams
US20030159143A1 (en) Systems and methods for generating a real-time video program guide through video access of multiple channels
CN105187850B (en) The method and apparatus that the information of encoded video data is provided and receives encoded video data
US6414996B1 (en) System, method and apparatus for an instruction driven digital video processor
JP2014209751A (en) Video switching for streaming video data
US8677428B2 (en) System and method for rule based dynamic server side streaming manifest files
CA2573906C (en) System and method for playback of digital video pictures in compressed streams
US6989868B2 (en) Method of converting format of encoded video data and apparatus therefor
JP2008516477A (en) Video compression system
JP5089658B2 (en) Transmitting apparatus and transmitting method
CA2800614C (en) Viewing and recording streams
US8341672B2 (en) Systems, methods and computer readable media for instant multi-channel video content browsing in digital video distribution systems
US20060277316A1 (en) Internet protocol television
US20080101455A1 (en) Apparatus and method for multiple format encoding
KR100547317B1 (en) Simultaneous recording and playback apparatus with indexing/searching/browsing functionality

Legal Events

Date Code Title Description
AS Assignment

Owner name: DELTA VIDYO, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CIVANLAR, REHA;SHAPIRO, OFER;SHALOM, TAL;SIGNING DATES FROM 20100507 TO 20100511;REEL/FRAME:024451/0181

AS Assignment

Owner name: VIDYO, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DELTA VIDYO, INC.;REEL/FRAME:030985/0144

Effective date: 20130731

AS Assignment

Owner name: VENTURE LENDING & LEASING VI, INC., CALIFORNIA

Free format text: SECURITY AGREEMENT;ASSIGNOR:VIDYO, INC.;REEL/FRAME:031123/0712

Effective date: 20130813

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION