US20090276402A1 - Search system using media metadata tracks - Google Patents

Search system using media metadata tracks Download PDF

Info

Publication number
US20090276402A1
US20090276402A1 US12181186 US18118608A US2009276402A1 US 20090276402 A1 US20090276402 A1 US 20090276402A1 US 12181186 US12181186 US 12181186 US 18118608 A US18118608 A US 18118608A US 2009276402 A1 US2009276402 A1 US 2009276402A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
track
media stream
method
metadata
metadata track
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12181186
Inventor
Todd Stiers
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MobiTV Inc
Original Assignee
MobiTV Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • H04N9/8211Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal the additional signal being a sound signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • H04N9/8233Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal the additional signal being a character code signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction

Abstract

Real-time metadata tracks recorded to media streams allow search and analysis operations in a variety of contexts. Search queries can be performed using information in real-time metadata tracks such as closed captioning, sub-title, statistical tracks, miscellaneous data tracks. Media streams can also be augmented with additional tracks. The metadata tracks not only allow efficient searching and indexing, but also allow insertion of content specific advertising during appropriate portions of a media stream based on the content of the metadata tracks.

Description

    CROSS-REFERENCE To RELATED APPLICATIONS
  • The present invention claims priority to co-pending provisional U.S. patent application No. 61/049,739, filed May 1, 2008, which is incorporated herein by reference in its entirety.
  • TECHNICAL FIELD
  • The present disclosure relates to a search system using real-time media metadata tracks.
  • DESCRIPTION OF RELATED ART
  • A variety of conventional mechanisms allow search and analysis of data. For example, search algorithms can perform indexing of data included in disparate files to allow efficient search and retrieval. Media players can analyze data included in video clip headers and digital video disc (DVD) introductions to provide information to users. Furthermore, users can scour media catalogues and guides for text based information on media content.
  • However, conventional mechanisms for searching media content and other data have limitations. Consequently, it is desirable to provide improved techniques and mechanisms for performing media data searches.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The disclosure may best be understood by reference to the following description taken in conjunction with the accompanying drawings, which illustrate particular embodiments.
  • FIG. 1 illustrates an exemplary system for use with embodiments of the present invention.
  • FIG. 2 illustrates one example of a Real-Time Transport Protocol (RTP) packet.
  • FIG. 3 illustrates one example of an RTP stream.
  • FIG. 4 illustrates one example of modification of an RTP stream including removal and insertion of packets.
  • FIG. 5 illustrates one example of metadata content processing.
  • FIG. 6 illustrates an example of advertisement insertion.
  • DESCRIPTION OF EXAMPLE EMBODIMENTS
  • Reference will now be made in detail to some specific examples of the invention including the best modes contemplated by the inventors for carrying out the invention. Examples of these specific embodiments are illustrated in the accompanying drawings. While the invention is described in conjunction with these specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims.
  • For example, the techniques of the present invention will be described in the context of particular operations and pipelines. However, it should be noted that the techniques of the present invention apply to a variety of operations and pipelines. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. Particular example embodiments of the present invention may be implemented without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.
  • Various techniques and mechanisms of the present invention will sometimes be described in singular form for clarity. However, it should be noted that some embodiments include multiple iterations of a technique or multiple instantiations of a mechanism unless noted otherwise. For example, a system uses a processor in a variety of contexts. However, it will be appreciated that a system can use multiple processors can while remaining within the scope of the present invention unless otherwise noted. Furthermore, the techniques and mechanisms of the present invention will sometimes describe a connection between two entities. It should be noted that a connection between two entities does not necessarily mean a direct, unimpeded connection, as a variety of other entities may reside between the two entities. For example, a processor may be connected to memory, but it will be appreciated that a variety of bridges and controllers may reside between the processor and memory. Consequently, a connection does not necessarily mean a direct, unimpeded connection unless otherwise noted.
  • Overview
  • Real-time metadata tracks recorded to media streams allow search and analysis operations in a variety of contexts. Search queries can be performed using information in real-time metadata tracks such as closed captioning, sub-title, statistical tracks, miscellaneous data tracks. Media streams can also be augmented with additional tracks. The metadata tracks not only allow efficient searching and indexing, but also allow insertion of content specific advertising during appropriate portions of a media stream based on the content of the metadata tracks.
  • EXAMPLE EMBODIMENTS
  • The media streams that are delivered usually include at least an audio track and a video track, but the media streams may include more. For example, media streams may also include captions, subtitles, and real-time metadata real-time metadata describing activity relating directly to the content presented on the other tracks. Where this metadata track is represented as text, such as captions, it could be used to drive searches against ad inventories for timely, specific offers, interactions, time appropriate commercials, and banners for display along with the actual media playback.
  • According to various embodiments, a mobile player device itself has a real-time state that could be viewed as just another “track” of metadata. Location, temperature, battery condition, network performance, local-time, anything else happening on the device itself could be used to help filter the searches further to make them more relevant and useful to the viewer. These searches could be driven in real-time, with results optionally recorded as yet another time.
  • According to various embodiments, as media is played, available metadata (captions, tracks, online chat) is used to make queries a line at a time, or word or however fast the metadata is available. The query results are then used to display information back to the viewer, perhaps as a matching banner ad, or an interactive link, a customized commercial, or other asset as determined by the search engine rules. Recording of the results can be used for future playback if caching or archival of the current state is desired. Real-time query results can be used to leverage the latest information from the search engine being queried. Player disposition (location, date/time, temperature) could be used to refine search query and results at the user level.
  • In particular embodiments, with this technology, more frequent and highly contextual advertising search and results are possible. Rather than using the title or summary information for a search, the entire media stream can be used to drive search by including real time metadata, such as captions, chat responses, or editors notes. Multiple language tracks in existing DVDs can be used to query for secondary, related information and/or ads, supporting additional customization. If recorded in real time, the search results can be played on subsequent playback while bypassing the search functionality. The record feature acts as a cache and allows ads to be better produced/validated/verified. According to various embodiments, long tail content where not recorded can be intentionally searched on each playback to generate the latest query results available from the respective search engine.
  • According to various embodiments, leveraging the mobile player disposition information allows further refinement of search and results to include immediately relevant queries, or to be recorded as a media track itself. These impressions could represent the highest value to advertiser and user alike. In particular embodiments, this provides an ad driven media organization with the ability to sell more advertising in smaller, focused amounts using more frequent searches with more relevant information.
  • In still other examples, a user can search a metadata track for specific search terms. Based on the search terms, a specific portion of the media stream is returned. In some examples, the portion of the media stream returned is a portion where are character says particular words in the search terms.
  • FIG. 1 is a diagrammatic representation illustrating one example of a system that can use the techniques and mechanisms of the present invention. According to various embodiments, content servers 119, 121, 123, and 125 are configured to provide media content to a mobile device 101 using protocols such as RTP and RTCP. Although a mobile device 101 is shown, it should be recognized that other devices such as set top boxes and computer systems can also be used. In particular examples, the content servers 119, 121, 123, and 125 can themselves establish sessions with mobile devices and stream video and audio content to mobile devices. However, it is recognized that in many instances, a separate controller such as controller 105 or controller 107 can be used to perform session management using a protocol such as RTSP. It is recognized that content servers require the bulk of the processing power and resources used to provide media content mobile devices. Session management itself may include far fewer transactions. Consequently, a controller can handle a far larger number of mobile devices than a content server can. In some examples, a content server can operate simultaneously with thousands of mobile devices, while a controller performing session management can manage millions of mobile devices simultaneously.
  • By separating out content streaming and session management functions, a controller can select a content server geographically close to a mobile device 101. It is also easier to scale, as content servers and controllers can simply be added as needed without disrupting system operation. A load balancer 103 can provide further efficiency during session management using RTSP 133 by selecting a controller with low latency and high throughput.
  • According to various embodiments, the content servers 119, 121, 123, and 125 have access to a campaign server 143. The campaign server 143 provides profile information for various mobile devices 101. In some examples, the campaign server 143 is itself a content server or a controller. The campaign server 143 can receive information from external sources about devices such as mobile device 101. The information can be profile information associated with various users of the mobile device including interests and background. The campaign server 143 can also monitor the activity of various devices to gather information about the devices. The content servers 119, 121, 123, and 125 can obtain information about the various devices from the campaign server 143. In particular examples, a content server 125 uses the campaign server 143 to determine what type of media clips a user on a mobile device 101 would be interested in viewing.
  • According to various embodiments, the content servers 119, 121, 123, and 125 are also receiving media streams from content providers such as satellite providers or cable providers and sending the streams to devices using RTP 131. In particular examples, content servers 119, 121, 123, and 125 access database 141 to obtain desired content that can be used to supplement streams from satellite and cable providers. In one example, a mobile device 101 requests a particular stream. A controller 107 establishes a session with the mobile device 101 and the content server 125 begins streaming the content to the mobile device 101 using RTP 131. In particular examples, the content server 125 obtains profile information from campaign server 143.
  • In some examples, the content server 125 can also obtain profile information from other sources, such as from the mobile device 101 itself. Using the profile information, the content server can select a clip from a database 141 to provide to a user. In some instances, the clip is injected into a live stream without affecting mobile device application performance. In other instances, the live stream itself is replaced with another live stream. The content server handles processing to make the transition between streams and clips seamless from the point of view of a mobile device application. In still other examples, advertisements from a database 141 can be intelligently selected from a database 141 using profile information from a campaign server 143 and used to seamlessly replace default advertisements in a live stream. Content servers 119, 121, 123, and 125 have the capability to manipulate RTP packets to allow introduction and removal of media content.
  • FIG. 2 illustrates one example of an RTP packet. An RTP packet 201 includes a header 211. According to various embodiments, the header 211 includes information such as the version number, amount of padding, protocol extensions, application level, payload format, etc. The RTP packet 201 also includes a sequence number 213. Client applications receiving RTP packets expect that the sequence numbers for received packets be unique. If different packets have the same sequence number, erroneous operation can occur. RTP packets also have a timestamp 215 that allows jitter and synchronization calculations. Fields 217 and 219 identify the synchronization source and the contributing source. Extensions are provided in field 221.
  • According to various embodiments, data 231 holds actual media data such as MPEG frames. In some examples, a single RTP packet 201 holds a single MPEG frame. In many instances, many RTP packets are required to hold a single MPEG frame. In instances where multiple RTP packets are required for a single MPEG frame, the sequence numbers change across RTP packets while the timestamp 215 remains the same across the different RTP packets. Different MPEG frames include I-frames, P-frames, and B-frames. I-frames are intraframes coded completely by itself. P-frames are predicted frames which require information from a previous I-frame or P-frame. B-frames are bi-directionally predicted frames that require information from surrounding I-frames and P-frames.
  • Because different MPEG frames require different numbers of RTP packets for transmission, two different streams of the same time duration may require different numbers of RTP packets for transmission. Simply replacing a clip with another clip would not work, as the clips may have different numbers of RTP packets and having different impacts on the sequence numbers of subsequent packets.
  • According to various embodiments, each track of metadata can be represented in a stream of RTP packets for transport/recording and playback within a subsequent RTSP session. As background, the client player negotiates which RTP tracks to set up during negotiation of an RTSP session with a RTSP/RTP server. In particular embodiments, the client player has the ability to synchronize and use tracks it is requesting. It should be recognized that a variety of mechanisms can be used to packetize media in their native track formats into RTP, and many ways to record new metadata back into a file are contemplated. It should be noted that a new metadata track can be added to the disk content as new streams of RTP packets are synchronized to the audio and video RTP packet streams. Recording metadata tracks can occur on a client recording during playback or on the server during delivery, or in combination.
  • FIG. 3 illustrates one example of an RTP packet stream that may be used with the techniques of the present invention. An RTP packet stream 301 includes individual packets having a variety of fields and payload data. According to various embodiments, the fields include a timestamp 303, sequence 505, marker 307, etc. The packets also include payload data 309 holding MPEG frames such as I, P, and B-frames. Timestamps for different packets may be the same. In particular examples, several packets carrying portions of the same I-frame have the same time stamp. However, sequence numbers are different for each packet. Marker bits 307 can be used for different purposes, such as signaling the starting point of an advertisement.
  • According to various embodiments, packets with sequence numbers 4303, 4304, and 4305 carrying potions of the same I-frame and have the same timestamp of 6. Packets with sequence numbers 4306, 4307, 4308, and 4309 carry P, B, P, and P-frames and have timestamps of 7, 8, 9, and 10 respectively. Packets with sequence numbers 4310 and 4311 carry different portions of the same I-frame and both have the same timestamp of 11. Packets with sequence numbers 4312, 4313, 4314, 4315, and 4316 carry P, P, B, P, and B-frames respectively and have timestamps 12, 13, 14, 15, and 16. It should be noted that the timestamps shown in FIG. 3 are merely representational. Actual timestamps can be computed using a variety of mechanisms.
  • For many audio encodings, the timestamp is incremented by the packetization interval multiplied by the sampling rate. For example, for audio packets having 20 ms of audio sampled at 8,000 Hz, the timestamp for each block of audio increases by 160. The actual sampling rate may also differ slightly from this nominal rate. For many video encodings, the timestamps generated depend on whether the application can determine the frame number. If the application can determine the frame number, the timestamp is governed by the nominal frame rate. Thus, for a 30 f/s video, timestamps would increase by 3,000 for each frame. If a frame is transmitted as several RTP packets, these packets would all bear the same timestamp. If the frame number cannot be determined or if frames are sampled a periodically, as is typically the case for software codecs, the timestamp may be computed from the system clock
  • While the timestamp is used by a receiver to place the incoming media data in the correct timing order and provide playout delay compensation, the sequence numbers are used to detect loss. Sequence numbers increase by one for each RTP packet transmitted, timestamps increase by the time “covered” by a packet. For video formats where a video frame is split across several RTP packets, several packets may have the same timestamp. For example, packets with sequence numbers 4317 and 4318 have the same timestamp 17 and carry portions of the same I-frame.
  • FIG. 4 illustrates one example of RTP packet stream modification. An RTP packet stream 401 includes individual packets having a variety of fields and payload data. According to various embodiments, the fields include a timestamp 403, sequence 405, marker 407, etc. The packets also include payload data 409 holding MPEG frames such as I, P, and B-frames. Timestamps for different packets may be the same. In particular examples, several packets carrying portions of the same I-frame have the same time stamp. However, sequence numbers are different for each packet. Marker bits 407 can be used for different purposes, such as signaling the starting point of an advertisement. According to various embodiments, metadata searches allow the introduction of targeted advertising that can be inserted seamlessly into an RTP stream.
  • According to various embodiments, packets with sequence numbers 4303, 4304, and 4305 carrying potions of the same I-frame and have the same timestamp of 6. Packets with sequence numbers 4306, 4307, 4308, and 4309 carry P, B, P, and P-frames and have timestamps of 7, 8, 9, and 10 respectively. According to various embodiments, a content server removes multiple packets from an RTP packet stream 401, including packets with sequence numbers 4310 through 4316. The packets with sequence numbers 4310 and 4311 carry different portions of the same I-frame and both have the same timestamp of 11.
  • Packets with sequence numbers 4312, 4313, 4314, 4315, 4316 carry P, P, B, P, and B-frames respectively and have timestamps 12, 13, 14, 15, and 16. The spliced stream now ends at packet with sequence number 4309 carrying a P-frame. A B-frame is included in packet having sequence number 4307. It should be noted that B-frames sometimes may depend on information included in a subsequent I-frame which has been removed. Although having a few B-frames lacking reference frames is not extremely disruptive, it can sometimes be noticed. Therefore, the techniques of the present invention recognize that in some embodiments, the last packets left in a stream prior to splicing should be an I-frame or a P-frame.
  • According to various embodiments, now that a portion of the RTP stream has been removed, an RTP sequence 411 can be inserted. In particular examples, the RTP sequence inserted 411 begins with an I-frame for subsequent P and B-frames to reference. Without an I-frame for reference, an RTP sequence inserted may begin with a partial or incomplete picture. The packets for insertion are modified to have sequence numbers following the last sequence number of spliced packet stream 401. RTP insertion sequence 411 has sequence numbers 4310-4317 corresponding to packets carrying I, I, I, B, P, P, B, B, frames respectively, with the I-frame carried in three packets with the same time stamp of 11 and the B, P, P, B, an B-frames having timestamps of 12-16 respectively.
  • For example, packets with sequence numbers 4317 and 4318 have the same timestamp 17 and carry portions of the same I-frame. In some instances, the number of packets in the RTP sequence removed 421 will be exactly the same as the number of packets in the RTP sequence for insertion 411. However, in many instances, the number of packets removed and inserted will differ. For example, some frames may require more than one packet for transmission. Although timestamps can be configured to be the same, so that a 5 second clip can be replaced with another 5 second clip, the number of packets and consequently the sequence numbers can be thrown askew. According to various embodiments, packet with sequence number 4309 is referred to herein as a data stream end point packet. Packet with sequence number 4318 is referred to herein as a data stream restart point packet. Packets with sequence numbers 4310 and 4316 in removed sequence are referred to herein as the removed sequence start packet and the removed sequence end packet respectively. Packets with sequence numbers 4310 and 4316 in the insertion sequence are referred to herein as the insertion sequence start packet and the insertion sequence end packet respectively.
  • Consequently, the content server maintains a current sequence number per RTP data stream and modified subsequent packets after removing and inserting streams. For example, packets having timestamp 17 are modified to have sequence numbers 4318 and 4319 instead of 4317 and 4318. The content server then proceeds to update subsequent timestamps in the RTP data stream. According to various embodiments, this operation is uniquely performed at a content server because the content server has information about individual mobile devices and also is able to know information about the sequence numbers of an entire content stream. A content provider may not know information about individual mobile devices, whereas a network device or network switch may not receive all data packets in a sequence. Some packets may have been dropped while others may have been transmitted on different paths.
  • FIG. 5 is a process flow diagram showing one technique for performing metadata searches. At 501, metadata content for a media stream is extracted and analyzed. At 503, the metadata content is entered into a search engine to allow for indexing and/or later retrieval. In some examples, closed captioning information is indexed to allow searching of programming based on phrases stated during a program. At 505, metadata content is mapped to correspond to particular points in a media stream so that a search will not only retrieve a particular phrase but the video of the actors saying the particular phrase. In particular embodiments, the metadata is time sensitive metadata for a media stream. At 507, additional metadata such as viewer comments can also be added and mapped to particular points in a media stream. For example, a viewer may comment that a particular scene is extremely artistic and enter comments as metadata corresponding to a point in the media stream into a searchable database. At 509, search terms are received in a search query. At 511, metadata content and/or relevant portions of a media stream are returned based on the query.
  • FIG. 6 is a process flow diagram showing a technique for inserting targeted advertising. At 601, metadata content for a particular media stream is analyzed. According to various embodiments, analysis is done for only metadata content for an immediate time period such as 10 minutes of a most recently presented stream. At 603, targeted advertising is selected based on recent metadata content such as editorial content and closed captioning. In particular embodiments, advertisements corresponding to search patterns, tags, strings, and sequences may be selected and inserted into the media stream. At 605, selected advertising is ordered based on priority. At 607, additional criteria such as user profiles, device profiles, user preferences, etc. may be used to further filter or screen selected advertising. At 609, the media stream is modified to include targeted advertising. In some examples, the media stream is a live media stream that is dynamically modified to include relevant and targeted advertising. In other examples, the media stream may be a clip stored in a media library.
  • Although many of the components and processes are described above in the singular for convenience, it will be appreciated by one of skill in the art that multiple components and repeated processes can also be used to practice the techniques of the present invention.
  • Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Therefore, the present embodiments are to be considered as illustrative and not restrictive and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.

Claims (31)

  1. 1. A method, comprising:
    receiving a media stream, the media stream including a video track, an audio track, and a metadata track;
    analyzing the contents of the metadata track;
    providing advertising to a viewer using the contents of the metadata track.
  2. 2. The method of claim 1, wherein the advertising is provided by modifying the live media stream.
  3. 3. The method of claim 1, wherein the advertising is embedded in the video track.
  4. 4. The method of claim 1, wherein the advertising is provided by modifying the video track and the audio track.
  5. 5. The method of claim 2, wherein the media stream is a Real-Time Transport (RTP) stream.
  6. 6. The method of claim 1, wherein the metadata track comprises closed captioning information.
  7. 7. The method of claim 1, wherein the metadata track comprises editorial content.
  8. 8. The method of claim 1, wherein the metadata track comprises viewer comments.
  9. 9. A system, comprising:
    an interface operable to receive a media stream, the media stream including a video track, an audio track, and a metadata track;
    a processor operable to analyze the contents of the metadata track and provide advertising to a viewer using the contents of the metadata track.
  10. 10. The method of claim 9, wherein the advertising is provided by modifying the live media stream.
  11. 11. The method of claim 9, wherein the advertising is embedded in the video track.
  12. 12. The method of claim 9, wherein the advertising is provided by modifying the video track and the audio track.
  13. 13. The method of claim 10, wherein the media stream is a Real-Time Transport (RTP) stream.
  14. 14. The method of claim 9, wherein the metadata track comprises closed captioning information.
  15. 15. The method of claim 9, wherein the metadata track comprises editorial content.
  16. 16. The method of claim 9, wherein the metadata track comprises viewer comments.
  17. 17. An apparatus, comprising:
    means for receiving a media stream, the media stream including a video track, an audio track, and a metadata track;
    means for analyzing the contents of the metadata track;
    means for providing advertising to a user based on the contents of the metadata track.
  18. 18. A method, comprising:
    receiving a media stream, the media stream including a video track, an audio track, and a metadata track;
    indexing the contents of the metadata track;
    receiving a search query for contents in the metadata track;
    returning a portion of the media stream to the user corresponding to the search query.
  19. 19. The method of claim 18, wherein the metadata track comprises closed captioning information.
  20. 20. The method of claim 18, wherein the metadata track comprises editorial content.
  21. 21. The method of claim 18, wherein the metadata track comprises viewer comments.
  22. 22. The method of claim 18, wherein the media stream is a live media stream.
  23. 23. A system, comprising:
    an interface operable to receive a media stream, the media stream including a video track, an audio track, and a metadata track;
    indexing the contents of the metadata track;
    receiving a search query for contents in the metadata track;
    returning a portion of the media stream to the user corresponding to the search query.
  24. 24. The method of claim 23, wherein the metadata track comprises closed captioning information.
  25. 25. The method of claim 23, wherein the metadata track comprises editorial content.
  26. 26. The method of claim 23, wherein the metadata track comprises viewer comments.
  27. 27. The method of claim 23, wherein the media stream is a live media stream.
  28. 28. An apparatus, comprising:
    means for receiving a live media stream, the media stream including a video track, an audio track, and a metadata track;
    means for indexing the contents of the metadata track;
    means for receiving a search query for contents in the metadata track;
    means for returning a portion of the media stream to the user corresponding to the search query.
  29. 29. The apparatus of claim 28, wherein the metadata track comprises closed captioning information.
  30. 30. The apparatus of claim 28, wherein the metadata track comprises editorial content.
  31. 31. The apparatus of claim 28, wherein the metadata track comprises viewer comments.
US12181186 2008-05-01 2008-07-28 Search system using media metadata tracks Abandoned US20090276402A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US4973908 true 2008-05-01 2008-05-01
US12181186 US20090276402A1 (en) 2008-05-01 2008-07-28 Search system using media metadata tracks

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12181186 US20090276402A1 (en) 2008-05-01 2008-07-28 Search system using media metadata tracks
US13530310 US20120263438A1 (en) 2008-05-01 2012-06-22 Search system using media metadata tracks

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13530310 Continuation US20120263438A1 (en) 2008-05-01 2012-06-22 Search system using media metadata tracks

Publications (1)

Publication Number Publication Date
US20090276402A1 true true US20090276402A1 (en) 2009-11-05

Family

ID=41257147

Family Applications (3)

Application Number Title Priority Date Filing Date
US12181186 Abandoned US20090276402A1 (en) 2008-05-01 2008-07-28 Search system using media metadata tracks
US12245642 Abandoned US20090274437A1 (en) 2008-05-01 2008-10-03 Embedding data in media metadata tracks during playback
US13530310 Pending US20120263438A1 (en) 2008-05-01 2012-06-22 Search system using media metadata tracks

Family Applications After (2)

Application Number Title Priority Date Filing Date
US12245642 Abandoned US20090274437A1 (en) 2008-05-01 2008-10-03 Embedding data in media metadata tracks during playback
US13530310 Pending US20120263438A1 (en) 2008-05-01 2012-06-22 Search system using media metadata tracks

Country Status (1)

Country Link
US (3) US20090276402A1 (en)

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094963A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and systems for broadcast-like effect using fractional-storage servers
US20100095004A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Balancing a distributed system by replacing overloaded servers
US20100174603A1 (en) * 2008-10-14 2010-07-08 Robert Hughes System and Method for Advertising Placement and/or Web Site Optimization
US20100304860A1 (en) * 2009-06-01 2010-12-02 Andrew Buchanan Gault Game Execution Environments
US20110145370A1 (en) * 2009-08-31 2011-06-16 Bruno Nieuwenhuys Methods and systems to personalize content streams
US20120151079A1 (en) * 2010-12-13 2012-06-14 Jan Besehanic Methods and apparatus to measure media exposure
US8560331B1 (en) 2010-08-02 2013-10-15 Sony Computer Entertainment America Llc Audio acceleration
US20130290846A1 (en) * 2012-04-27 2013-10-31 Mobitv, Inc. Search-based navigation of media content
US8613673B2 (en) 2008-12-15 2013-12-24 Sony Computer Entertainment America Llc Intelligent game loading
US8840476B2 (en) 2008-12-15 2014-09-23 Sony Computer Entertainment America Llc Dual-mode program execution
US8863175B2 (en) * 2012-12-19 2014-10-14 Ebay Inc. Method and system for targeted commerce in network broadcasting
US8888592B1 (en) 2009-06-01 2014-11-18 Sony Computer Entertainment America Llc Voice overlay
US8968087B1 (en) 2009-06-01 2015-03-03 Sony Computer Entertainment America Llc Video game overlay
US20150264446A1 (en) * 2014-03-17 2015-09-17 Cbs Interactive Inc. Techniques to select advertisements using closed captioning data
US9349201B1 (en) 2006-08-03 2016-05-24 Sony Interactive Entertainment America Llc Command sentinel
US9426502B2 (en) 2011-11-11 2016-08-23 Sony Interactive Entertainment America Llc Real-time cloud-based video watermarking systems and methods
US20160269455A1 (en) * 2015-03-10 2016-09-15 Mobitv, Inc. Media seek mechanisms
US9498714B2 (en) 2007-12-15 2016-11-22 Sony Interactive Entertainment America Llc Program mode switching
US9878240B2 (en) 2010-09-13 2018-01-30 Sony Interactive Entertainment America Llc Add-on management methods

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191809A1 (en) 2008-01-30 2011-08-04 Cinsay, Llc Viral Syndicated Interactive Product System and Method Therefor
US8312486B1 (en) 2008-01-30 2012-11-13 Cinsay, Inc. Interactive product placement system and method therefor
EP2429238A4 (en) * 2009-08-24 2012-03-28 Huawei Tech Co Ltd Channel switching method, device and system
US9596522B2 (en) * 2010-06-04 2017-03-14 Mobitv, Inc. Fragmented file structure for live media stream delivery
JP5652642B2 (en) * 2010-08-02 2015-01-14 ソニー株式会社 Data generating apparatus and a data generation method, a data processing apparatus and data processing method
GB2497696A (en) * 2010-08-30 2013-06-19 Mobitv Inc Media rights management on multiple devices
US8713420B2 (en) * 2011-06-30 2014-04-29 Cable Television Laboratories, Inc. Synchronization of web applications and media
US20140351256A1 (en) * 2013-05-22 2014-11-27 Nice-Systems Ltd. System and method for generating metadata for a recorded session
US20160094501A1 (en) * 2014-09-26 2016-03-31 Line Corporation Method, system and recording medium for providing video contents in social platform and file distribution system
US9270563B1 (en) 2014-11-24 2016-02-23 Roku, Inc. Apparatus and method for content playback utilizing crowd sourced statistics

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040030682A1 (en) * 2000-11-21 2004-02-12 Porter Charles A. System and process for searching a network
US6721741B1 (en) * 2000-01-24 2004-04-13 Friskit, Inc. Streaming media search system
US6983287B1 (en) * 2002-07-01 2006-01-03 Microsoft Corporation Database build for web delivery
US20060047692A1 (en) * 2001-04-05 2006-03-02 Eric Rosenblum System and method for indexing, organizing, storing and retrieving environmental information
US20070055689A1 (en) * 1998-04-16 2007-03-08 Rhoads Geoffrey B Content Indexing and Searching using Content Identifiers and associated Metadata
US7206748B1 (en) * 1998-08-13 2007-04-17 International Business Machines Corporation Multimedia player toolkit for electronic content delivery
US7222163B1 (en) * 2000-04-07 2007-05-22 Virage, Inc. System and method for hosting of video content over a network
US7281034B1 (en) * 2000-01-24 2007-10-09 Friskit, Inc. System and method for media playback over a network using links that contain control signals and commands
US20070299870A1 (en) * 2006-06-21 2007-12-27 Microsoft Corporation Dynamic insertion of supplemental video based on metadata
US20080147608A1 (en) * 2006-12-14 2008-06-19 Yahoo! Inc. Video search and indexing systems and methods
US20090094113A1 (en) * 2007-09-07 2009-04-09 Digitalsmiths Corporation Systems and Methods For Using Video Metadata to Associate Advertisements Therewith
US7796779B1 (en) * 2006-08-24 2010-09-14 SugarSync, Inc. Efficient synchronization of changes to images

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030093790A1 (en) * 2000-03-28 2003-05-15 Logan James D. Audio and video program recording, editing and playback systems using metadata
US7295752B1 (en) * 1997-08-14 2007-11-13 Virage, Inc. Video cataloger system with audio track extraction
US6833865B1 (en) * 1998-09-01 2004-12-21 Virage, Inc. Embedded metadata engines in digital capture devices
US20020124262A1 (en) * 1999-12-01 2002-09-05 Andrea Basso Network based replay portal
US20010039571A1 (en) * 2000-01-06 2001-11-08 Atkinson Paul D. System and method for facilitating electronic commerce within public spaces
US20060090179A1 (en) * 2004-10-26 2006-04-27 Ya-Ling Hsu System and method for embedding supplemental information into a digital stream of a work of content
US20070136742A1 (en) * 2005-12-13 2007-06-14 General Instrument Corporation Method, apparatus and system for replacing advertisements in recorded video content
US7827191B2 (en) * 2005-12-14 2010-11-02 Microsoft Corporation Discovering web-based multimedia using search toolbar data
US20080284910A1 (en) * 2007-01-31 2008-11-20 John Erskine Text data for streaming video
US9955122B2 (en) * 2008-04-11 2018-04-24 Mobitv, Inc. Dynamic advertisement stream replacement

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080275906A1 (en) * 1998-04-16 2008-11-06 Rhoads Geoffrey B Content Indexing and Searching Using Content Identifiers and Associated Metadata
US20070055689A1 (en) * 1998-04-16 2007-03-08 Rhoads Geoffrey B Content Indexing and Searching using Content Identifiers and associated Metadata
US7206748B1 (en) * 1998-08-13 2007-04-17 International Business Machines Corporation Multimedia player toolkit for electronic content delivery
US6721741B1 (en) * 2000-01-24 2004-04-13 Friskit, Inc. Streaming media search system
US7281034B1 (en) * 2000-01-24 2007-10-09 Friskit, Inc. System and method for media playback over a network using links that contain control signals and commands
US7222163B1 (en) * 2000-04-07 2007-05-22 Virage, Inc. System and method for hosting of video content over a network
US20040030682A1 (en) * 2000-11-21 2004-02-12 Porter Charles A. System and process for searching a network
US20060047692A1 (en) * 2001-04-05 2006-03-02 Eric Rosenblum System and method for indexing, organizing, storing and retrieving environmental information
US6983287B1 (en) * 2002-07-01 2006-01-03 Microsoft Corporation Database build for web delivery
US20070299870A1 (en) * 2006-06-21 2007-12-27 Microsoft Corporation Dynamic insertion of supplemental video based on metadata
US7796779B1 (en) * 2006-08-24 2010-09-14 SugarSync, Inc. Efficient synchronization of changes to images
US20080147608A1 (en) * 2006-12-14 2008-06-19 Yahoo! Inc. Video search and indexing systems and methods
US20090094113A1 (en) * 2007-09-07 2009-04-09 Digitalsmiths Corporation Systems and Methods For Using Video Metadata to Associate Advertisements Therewith

Cited By (67)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9349201B1 (en) 2006-08-03 2016-05-24 Sony Interactive Entertainment America Llc Command sentinel
US9498714B2 (en) 2007-12-15 2016-11-22 Sony Interactive Entertainment America Llc Program mode switching
US20100174603A1 (en) * 2008-10-14 2010-07-08 Robert Hughes System and Method for Advertising Placement and/or Web Site Optimization
US20100094963A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and systems for broadcast-like effect using fractional-storage servers
US20100095012A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Fast retrieval and progressive retransmission of content
US20100094986A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Source-selection based Internet backbone traffic shaping
US20100094961A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and systems for requesting fragments without specifying the source address
US20100095184A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Obtaining Erasure-Coded Fragments Using Push and Pull Protocols
US20100094950A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and systems for controlling fragment load on shared links
US20100095013A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Fault Tolerance in a Distributed Streaming System
US20100094974A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Load-balancing an asymmetrical distributed erasure-coded system
US20100094969A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Reduction of Peak-to-Average Traffic Ratio in Distributed Streaming Systems
US8938549B2 (en) 2008-10-15 2015-01-20 Aster Risk Management Llc Reduction of peak-to-average traffic ratio in distributed streaming systems
US20100094973A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Random server selection for retrieving fragments under changing network conditions
US20100094959A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Hybrid open-loop and closed-loop erasure-coded fragment retrieval process
US20100094957A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and systems for fast segment reconstruction
US7818430B2 (en) * 2008-10-15 2010-10-19 Patentvc Ltd. Methods and systems for fast segment reconstruction
US7822856B2 (en) * 2008-10-15 2010-10-26 Patentvc Ltd. Obtaining erasure-coded fragments using push and pull protocols
US7822855B2 (en) * 2008-10-15 2010-10-26 Patentvc Ltd. Methods and systems combining push and pull protocols
US7840680B2 (en) * 2008-10-15 2010-11-23 Patentvc Ltd. Methods and systems for broadcast-like effect using fractional-storage servers
US7840679B2 (en) * 2008-10-15 2010-11-23 Patentvc Ltd. Methods and systems for requesting fragments without specifying the source address
US7844712B2 (en) * 2008-10-15 2010-11-30 Patentvc Ltd. Hybrid open-loop and closed-loop erasure-coded fragment retrieval process
US8874774B2 (en) 2008-10-15 2014-10-28 Aster Risk Management Llc Fault tolerance in a distributed streaming system
US20110055420A1 (en) * 2008-10-15 2011-03-03 Patentvc Ltd. Peer-assisted fractional-storage streaming servers
US8874775B2 (en) 2008-10-15 2014-10-28 Aster Risk Management Llc Balancing a distributed system by replacing overloaded servers
US8832292B2 (en) 2008-10-15 2014-09-09 Aster Risk Management Llc Source-selection based internet backbone traffic shaping
US20100094966A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Receiving Streaming Content from Servers Located Around the Globe
US8832295B2 (en) 2008-10-15 2014-09-09 Aster Risk Management Llc Peer-assisted fractional-storage streaming servers
US8825894B2 (en) 2008-10-15 2014-09-02 Aster Risk Management Llc Receiving streaming content from servers located around the globe
US8819260B2 (en) 2008-10-15 2014-08-26 Aster Risk Management Llc Random server selection for retrieving fragments under changing network conditions
US20100095004A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Balancing a distributed system by replacing overloaded servers
US8819259B2 (en) 2008-10-15 2014-08-26 Aster Risk Management Llc Fast retrieval and progressive retransmission of content
US8949449B2 (en) 2008-10-15 2015-02-03 Aster Risk Management Llc Methods and systems for controlling fragment load on shared links
US20100094968A1 (en) * 2008-10-15 2010-04-15 Patentvc Ltd. Methods and Systems Combining Push and Pull Protocols
US8819261B2 (en) 2008-10-15 2014-08-26 Aster Risk Management Llc Load-balancing an asymmetrical distributed erasure-coded system
US8840476B2 (en) 2008-12-15 2014-09-23 Sony Computer Entertainment America Llc Dual-mode program execution
US8613673B2 (en) 2008-12-15 2013-12-24 Sony Computer Entertainment America Llc Intelligent game loading
US20130203508A1 (en) * 2009-06-01 2013-08-08 Andrew Buchanan Gault Game Execution Environments
US9584575B2 (en) 2009-06-01 2017-02-28 Sony Interactive Entertainment America Llc Qualified video delivery
US20130217506A1 (en) * 2009-06-01 2013-08-22 Andrew Buchanan Gault Game Execution Environments
US8506402B2 (en) * 2009-06-01 2013-08-13 Sony Computer Entertainment America Llc Game execution environments
US9426198B2 (en) * 2009-06-01 2016-08-23 Sony Interactive Entertainment America Llc Game execution environments
US8784210B2 (en) * 2009-06-01 2014-07-22 Sony Computer Entertainment America Llc Game execution environments
US8944915B2 (en) * 2009-06-01 2015-02-03 Sony Computer Entertainment America Llc Game execution environments
US9723319B1 (en) 2009-06-01 2017-08-01 Sony Interactive Entertainment America Llc Differentiation for achieving buffered decoding and bufferless decoding
US20100304860A1 (en) * 2009-06-01 2010-12-02 Andrew Buchanan Gault Game Execution Environments
US8888592B1 (en) 2009-06-01 2014-11-18 Sony Computer Entertainment America Llc Voice overlay
US9203685B1 (en) 2009-06-01 2015-12-01 Sony Computer Entertainment America Llc Qualified video delivery methods
US8968087B1 (en) 2009-06-01 2015-03-03 Sony Computer Entertainment America Llc Video game overlay
US20130296051A1 (en) * 2009-06-01 2013-11-07 Sony Computer Entertainment America, LLC Game Execution Environments
US8641528B2 (en) * 2009-06-01 2014-02-04 Sony Computer Entertainment America Llc Game execution environments
US20110145370A1 (en) * 2009-08-31 2011-06-16 Bruno Nieuwenhuys Methods and systems to personalize content streams
US8676591B1 (en) 2010-08-02 2014-03-18 Sony Computer Entertainment America Llc Audio deceleration
US8560331B1 (en) 2010-08-02 2013-10-15 Sony Computer Entertainment America Llc Audio acceleration
US10039978B2 (en) 2010-09-13 2018-08-07 Sony Interactive Entertainment America Llc Add-on management systems
US9878240B2 (en) 2010-09-13 2018-01-30 Sony Interactive Entertainment America Llc Add-on management methods
US20120151079A1 (en) * 2010-12-13 2012-06-14 Jan Besehanic Methods and apparatus to measure media exposure
US9426502B2 (en) 2011-11-11 2016-08-23 Sony Interactive Entertainment America Llc Real-time cloud-based video watermarking systems and methods
US9785639B2 (en) * 2012-04-27 2017-10-10 Mobitv, Inc. Search-based navigation of media content
US20130290846A1 (en) * 2012-04-27 2013-10-31 Mobitv, Inc. Search-based navigation of media content
US8863175B2 (en) * 2012-12-19 2014-10-14 Ebay Inc. Method and system for targeted commerce in network broadcasting
US9648371B2 (en) 2012-12-19 2017-05-09 Paypal, Inc. Method and system for targeted commerce in network broadcasting
US20140359660A1 (en) * 2012-12-19 2014-12-04 Ebay Inc. Method and system for targeted commerce in network broadcasting
US9148685B2 (en) * 2012-12-19 2015-09-29 Paypal, Inc. Method and system for targeted commerce in network broadcasting
US9712885B2 (en) * 2014-03-17 2017-07-18 Cbs Interactive Inc. Techniques to select advertisements using closed captioning data
US20150264446A1 (en) * 2014-03-17 2015-09-17 Cbs Interactive Inc. Techniques to select advertisements using closed captioning data
US20160269455A1 (en) * 2015-03-10 2016-09-15 Mobitv, Inc. Media seek mechanisms

Also Published As

Publication number Publication date Type
US20120263438A1 (en) 2012-10-18 application
US20090274437A1 (en) 2009-11-05 application

Similar Documents

Publication Publication Date Title
Pantos et al. HTTP live streaming
US20100121936A1 (en) Apparatus and method for managing media content
US20080112690A1 (en) Personalized local recorded content
US20070078898A1 (en) Server-based system and method for retrieving tagged portions of media files
US20110231519A1 (en) Enhanced block-request streaming using url templates and construction rules
US20100169303A1 (en) Playlists for real-time or near real-time streaming
US20130332971A1 (en) Targeted high-value content in http streaming video on demand
US20110239078A1 (en) Enhanced block-request streaming using cooperative parallel http and forward error correction
US20110231569A1 (en) Enhanced block-request streaming using block partitioning or request controls for improved client-side handling
US20110119395A1 (en) Method and apparatus for adaptive streaming using segmentation
US20050198317A1 (en) Method and apparatus for sharing internet content
US20110119396A1 (en) Method and apparatus for transmitting and receiving data
US20130007223A1 (en) Enhanced block-request streaming system for handling low-latency streaming
US20080086754A1 (en) Peer to peer media distribution system and method
US20030030752A1 (en) Method and system for embedding information into streaming media
US20080255943A1 (en) Refreshing advertisements in offline or virally distributed content
US20120116883A1 (en) Methods and systems for use in incorporating targeted advertising into multimedia content streams
US20030088687A1 (en) Method and apparatus for automatically converting source video into electronic mail messages
US8752085B1 (en) Advertisement insertion into media content for streaming
US20100235472A1 (en) Smooth, stateless client media streaming
US20110080940A1 (en) Low latency cacheable media streaming
US20120124618A1 (en) Virtual insertion of advertisements
US20140150019A1 (en) Method and system for ad insertion in over-the-top live media delivery
US20070078883A1 (en) Using location tags to render tagged portions of media files
US20070136742A1 (en) Method, apparatus and system for replacing advertisements in recorded video content

Legal Events

Date Code Title Description
AS Assignment

Owner name: MOBITV, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STIERS, TODD;REEL/FRAME:021303/0057

Effective date: 20080725