WO2002021840A2 - Synchronizing a client with a media signal - Google Patents

Synchronizing a client with a media signal Download PDF

Info

Publication number
WO2002021840A2
WO2002021840A2 PCT/US2001/028246 US0128246W WO0221840A2 WO 2002021840 A2 WO2002021840 A2 WO 2002021840A2 US 0128246 W US0128246 W US 0128246W WO 0221840 A2 WO0221840 A2 WO 0221840A2
Authority
WO
WIPO (PCT)
Prior art keywords
action
client
actions
determining
media signal
Prior art date
Application number
PCT/US2001/028246
Other languages
French (fr)
Other versions
WO2002021840A3 (en
Inventor
Ivan Solvason
Original Assignee
Webchoice. Com
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Webchoice. Com filed Critical Webchoice. Com
Priority to AU2001290712A priority Critical patent/AU2001290712A1/en
Publication of WO2002021840A2 publication Critical patent/WO2002021840A2/en
Publication of WO2002021840A3 publication Critical patent/WO2002021840A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/08Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division
    • H04N7/087Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division with signal insertion during the vertical blanking interval only
    • H04N7/088Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division with signal insertion during the vertical blanking interval only the inserted signal being digital
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43076Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508Management of client data or end-user data
    • H04N21/4532Management of client data or end-user data involving end-user characteristics, e.g. viewer profile, preferences
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/454Content or additional data filtering, e.g. blocking advertisements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782Web browsing, e.g. WebTV
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/633Control signals issued by server directed to the network components or client
    • H04N21/6332Control signals issued by server directed to the network components or client directed to client
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/654Transmission by server directed to the client
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/654Transmission by server directed to the client
    • H04N21/6543Transmission by server directed to the client for forcing some client operations, e.g. recording
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/858Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17318Direct or substantially direct transmission and handling of requests

Definitions

  • the Internet and traditional radio and TV broadcasts both offer users/viewers access to a wide variety of information. Both mediums, generally, have different strengths. For example, the Internet enables users to view the content offered by a site at their own pace. Additionally, many sites provide a degree of inter-activity absent from traditional TV and radio broadcasts. For example, a web-page may solicit user input via forms or "applets" and respond accordingly. While Internet sites have greatly improved the quality and speed of network video transmission, traditional broadcasts generally retain the upper hand in image and sound quality.
  • the invention features a method of synchronizing a client with a media signal.
  • the method includes receiving one or more actions corresponding to the media signal, determining an application for handling the actions, and causing the determined application to handle at the actions .
  • Embodiments may include one or more of the following features.
  • the media signal may be an audio signal and/or a video signal.
  • Determining an application can include parsing action information.
  • parsing action information can include determining if the action information corresponds to a URI (Universal Resource Indicator) or a command.
  • the method can include navigating a browser to the URI.
  • the method may further include prompting a user before causing the determined application to handle the action.
  • the method may also include queuing the action.
  • the action may include an indication of when the determined application should handle the action (e.g., an SMPTE (Society of Motion Pictures and Television Engineers) time-code) .
  • the actions may be received from a server.
  • the method may also include receiving criteria associated with the action and evaluating the criteria.
  • the criteria may include one or more user characteristics (e.g., age, location, and gender) .
  • the invention features a computer program product, disposed on a computer readable medium, the computer program for synchronizing a client with a media signal.
  • the program includes instructions for causing a processor to receive one or more actions corresponding to the media signal, determine an application for handling at least one of the one or more received actions, and cause the determined application to handle at least one of the one or more received actions .
  • the invention features a method of synchronizing one or more clients with a media signal. The method includes receiving information identifying an action corresponding to the media signal, receiving information identifying criteria for the action, and transmitting the action and criteria toward the one or more clients.
  • Embodiments may include one or more of the following.
  • the action includes an identification of a time for processing the action and, for example, a URI or some application command or data.
  • Transmitting the action and criteria toward one or more clients may include transmitting the action and criteria to a server and determining at the server, based on the action criteria, whether to transmit the action to a client. Such a determination may be done based on user characteristics associated with the client.
  • a client receives the. action and determines, based on the action criteria, whether to execute the action.
  • the invention features a method of synchronizing a client with a media signal.
  • the method includes receiving one or more actions corresponding to the media signal, the actions including an indication of a time to execute each action.
  • the method also includes determining a delay time for the action and handling the action in accordance with the delay time .
  • Embodiments may include one or more of the following features. Determining a delay time may be based on the location of a client and/or on a broadcast medium used to transmit the media signal to a household of the client.
  • FIGs. 1A and IB are diagrams of systems for synchronizing a client with a media signal.
  • FIG. 2 is a flow-chart of a process for transmitting actions to a server.
  • FIG. 3 is a screenshot of a user interface for identifying actions corresponding to a media signal.
  • FIG. 4 is a flow-chart of a process for transmitting actions to clients.
  • FIG. 5 is a diagram of a client including instructions for processing actions.
  • FIG. 6 is a flow-chart of a process for receiving and processing actions. Detailed Description
  • FIG. 1A shows an example of a system 100 for synchronizing actions performed by computer 106 with a TV 104 or radio show.
  • a station 108 broadcasts a TV show (e.g., via airways, cable, or satellite)
  • a computer 106 web-browser may automatically navigate to Internet web-sites corresponding to the TV show content. That is, when the TV show mentions the U.S. Soccer Team, a user' s computer may automatically navigate a web-browser to a URI (Universal Resource Indicator) of
  • the system 100 can support a wide variety of applications. For example, inter-active web-pages may quiz and provide feedback to students viewing a broadcast school lesson.
  • the system 100 can provide Internet access that is synchronized with a media signal, the system 100 need not be limited to Internet navigation.
  • the system 100 can feature actions that transmit data and/or invoke different client 106 applications other than a browser.
  • actions may cause a client 106 to download and install protection software when a TV program reports the latest computer virus .
  • the system 100 can also tailor the actions performed by the clients 106, for example, based on user characteristics and/or preferences. That is, an action may cause a computer to present an adult with an option to charge an item shown on TV to their credit card, but not present this option to a child.
  • the server 112 and/or client 106 may store user characteristics such as an address, gender, age, and so forth.
  • the system 100 can be implemented in a variety of ways.
  • the system 100 of FIG. 1A includes a control computer 110 that enables an operator to define actions corresponding to a media signal 114 (e.g., TV or radio signal) .
  • the system 100 also includes a server 112 that receives actions from the control computer 110 and transmits (e.g., multicasts) the actions to clients 106 in households 102 viewing or listening to a particular broadcast.
  • control computer 110 can embed actions in the signal itself, for example, in a VBI (Vertical Blanking Interval) of a video signal.
  • VBI Very Blanking Interval
  • the server 112 can monitor the signal as broadcast, extract the actions, and, again, transmit them to the corresponding client 106.
  • clients 106 can monitor a broadcast or other media signal to monitor for actions without the server 112.
  • client software and/or hardware can reside in a wide variety of network devices such as network appliances, handheld PCs, PDAs (Personal Digital Assistants) , game consoles, Internet phones, WAP (Wireless Access Protocol) devices, set-top boxes, ReplayTV ® , TiVo ® , and so forth.
  • FIG. 1 shows a TV 104 as an example of an a device that presents a media signal
  • the media signal may also be presented by the same device running the client software.
  • the media signal may be streaming audio or video carried across the Internet to a client computer.
  • the media signal may come from a local source.
  • a VCR may output a media signal to for display and correlation with different actions downloaded from a server or from within the VCR signal.
  • an operator can use the control computer 110 to define 205 actions and specify how the actions should synchronize with the media signal (e.g., when the actions should occur relative to the start of the signal) .
  • Each action defines some operation the client should perform.
  • an action may include a command line identifying an application (e.g., "runvirus_install.exe”), a URI (e.g., www.tvshow.com) specifying a web-site a browser should visit, data representing the real or virtual coordinates of an object or person of a TV show for use by a game application, and so forth.
  • the control computer 110 can, in turn, transmit 204 the actions over a secure (e.g., encrypted) connection (e.g., a TCP/IP
  • This encryption may be based upon a security key.
  • the key can be changed at the administrative level. Once the key is decoded by the server, an action can be queued.
  • the system may also employee an encryption key at the end user level. As an additional measure of security, the system can also make use of firewall security at the server/control level as well as the client level .
  • the control computer 110 can offer a variety of different techniques for defining actions.
  • the control computer 110 can enable an operator to define actions in realtime. That is, as a signal broadcasts, an operator can define actions for the broadcast and immediately send them to the server 112 for transmission to clients 106. This can enable an operator to synchronize clients 106 during live broadcasts.
  • the same technique may also enable an operator to specify transmission delays. For example, an operator may define an action and specify that it not be transmitted or executed by the clients 106 for a period of ⁇ -seconds.
  • the user interface techniques described above feature an operator sitting and viewing the broadcast in real-time and preparing appropriate client computer actions.
  • control computer 110 may ease the demands on an operator, for example, by providing previously defined actions for selection, the process of defining actions in real-time may require some dexterity and quick-wittedness on the part of the operator to keep up with the on-going broadcast .
  • the control computer 110 may provide a user interface that enables the operator to define actions for pre-recorded material at his or her leisure.
  • the control computer 110 may provide a user interface that enables an operator to define an action and associate the action with a particular time-code.
  • video typically includes embedded SMPTE (Society of Motion Picture and Television Engineers) time-codes identifying each video frame.
  • SMPTE Society of Motion Picture and Television Engineers
  • an action of "Get Smart: 000100 : www.getsmart.com/episodel2" may define an action that causes a client computer's 106 web-browser to visit the "www.getsmart.com” web-site at the one-hundredth frame of Episode 12 of "Get Smart” .
  • the control computer 110 may use other time-codes other than SMPTE (e.g., NNTP time-codes). Additionally, the operator may simply enter a time offset for the action (e.g., one minute after broadcast begins).
  • the control computer 110 may embed the actions within a media signal such as the video blanking interval (VBI) of a video.
  • a server 112 can monitor the media signal, extract action definitions, and transmit the actions to clients 106.
  • a system 100 need not use a server 112 at all.
  • client computers 106 can monitor media signals received by the household 102 TVs 104 and process the embedded actions.
  • FIG. 3 illustrates an example of a user interface 300 presented to an operator of a control computer 110.
  • the interface 300 enables an operator to define actions 302 for a particular broadcast.
  • the action can include a URI, an application program command and parameters, and/or other instructions or data for transmission to the clients 106.
  • the action can also include synchronization information (e.g., a date, time, and/or frame 110 of an action) .
  • the interface 300 may also enable an operator to define whether the action is performed automatically or whether a user at the client computer 106 is prompted before execution.
  • the interface 300 may further enable an operator to define criteria that must be satisfied for performance of the action.
  • the operator may define multiple actions for the same signal time-code with different actions having different criteria. This enables the operator to tailor the behavior of client computers 110 for different classes of users.
  • the interface 300 also includes a list 304 of previously defined actions for selection and a list 306 of actions already assigned to the broadcast.
  • FIG. 4 shows a process 400 that a server 112 can use to transmit 408 actions received 402 from a control computer 110 or detected in a signal (e.g. , the VBI portion of a video).
  • the server 112 can log the actions.
  • the server 110 may dynamically add a description of broadcast actions to an HTML (HyperText Markup Language) or XML (extensible Markup Language) page listing actions for a particular show or broadcast. Users and contributors can access the logs to view actions for a particular broadcast long after the show has aired.
  • HTML HyperText Markup Language
  • XML extensible Markup Language
  • the server 110 may receive registration requests from different system 100 users.
  • Registration identifies a broadcast the user is viewing and, potentially, the IP (Internet Protocol) address of their client. Additionally, for new members, registration may involve providing demographic data such as an address, age, gender, income, and so forth. This data may be stored by the server 110 to tailor the actions delivered to a user's client 106. For existing members, registration may involve entering a user name so the server 110 can retrieve the previously stored demographic data for the user.
  • IP Internet Protocol
  • Registration and other interaction with the server 112 may occur via the Internet (e.g., a login web-page, a preferences web-page and so forth) or via a telephone voice response system that enables a user to customize and select different services transmitted to the client.
  • the Internet e.g., a login web-page, a preferences web-page and so forth
  • a telephone voice response system that enables a user to customize and select different services transmitted to the client.
  • the server 112 sends actions for the broadcast to clients 106 identified as viewing the show.
  • the server 112 can multicast the actions using UDP (User Datagram Protocol) or other protocols.
  • UDP User Datagram Protocol
  • the server 112 may select actions for transmission based on user demographic data.
  • the server 112 may handle timing issues by transmitting actions at the specified times.
  • the server 112 may include synchronization information in the action and leave it to the client 106 to process the action at the designated time.
  • the same show may be broadcast at slightly different times. For example, a satellite transmission generally lags cable transmission by tens of seconds. Additionally, different geographic locations broadcast at different times.
  • the server 112 can offset 408 transmission of actions based on the geographic location of a user (e.g., based on the user's zip code) and/or transmission sources (e.g., cable or satellite).
  • Communication between the clients 106 and server 110 need not be one way. That is, the clients 106 may send data to the server 110, for example, using TCP/IP. This enables the server 110 to monitor user behavior and can enable the server to select actions based on prior user behavior.
  • FIG. 5 shows an example of a client 106.
  • the client 106 includes an operating system 504 (e.g., Microsoft ® Windows 2000 ® ) and one or more applications 502 (e.g., Microsoft ® Explorer ® and Microsoft ® Word ® ) .
  • the client 106 also includes instructions 500 for processing actions. As shown, the processing instructions 500 can invoke other applications and can even make operating system 504 calls.
  • FIG. 6 illustrates a process 600 the instructions (500 of FIG. 5) may implement.
  • the process 600 may include registering the user with a server 602, for example, by navigating a browser to a log-in web-site associated with the server. Registering enables the client to identify a program being viewed or heard. Of course, a user need not actually be viewing or listening to a program.
  • the client may receive a stream of one or more actions 604 corresponding to the identified program as the program broadcast progresses.
  • the client determines an application for the action. For example, the client may parse information included in the action to identify whether an action includes a URI. If such parsing succeeds, the client can pass the URI to a browser. Alternatively, the client can look for names of applications and/or keyword such as "run" that indicate what operation the client should perform in response to the action (e.g., which application should handle the action).
  • the system(s) described above can use a variety of architectures. For example, many features provided by the server can instead be provided by the client and vice-versa.
  • the server 110 may transmit action criteria for evaluation 606 by the client.
  • either the server 110 or client 106 may specify that the action be queued 610 or executed 612 upon receipt or arrival of a specified time.
  • either the server 110 or client 106 may request that the client prompt a user before executing an action. This can enable, for example, a user to finish reading information at one web-site before moving on to another.
  • the techniques described herein are not limited to any particular hardware or software configuration; they may find applicability in any computing or processing environment .
  • the techniques may be implemented in hardware or software, or a combination of the two.
  • the techniques are implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements) , at least one input device, and one or more output devices.
  • Each program is preferably implemented in high level procedural or object oriented programming language to communicate with a computer system.
  • the programs can be implemented in assembly or machine language, if desired. In any case the language may be compiled or interpreted language.
  • Each such computer program is preferably stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic disk) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described herein.
  • the system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner.
  • Other embodiments are within the scope of the following claims.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)
  • Synchronisation In Digital Transmission Systems (AREA)

Abstract

A method of synchronizing a client with a media signal includes receiving one or more actions corresponding to the media singal, determining an application for handling the actions, and causing the determined application to handle the actions.

Description

SYNCHRONIZING A CLIENT WITH A MEDIA SIGNAL
Background
The Internet and traditional radio and TV broadcasts both offer users/viewers access to a wide variety of information. Both mediums, generally, have different strengths. For example, the Internet enables users to view the content offered by a site at their own pace. Additionally, many sites provide a degree of inter-activity absent from traditional TV and radio broadcasts. For example, a web-page may solicit user input via forms or "applets" and respond accordingly. While Internet sites have greatly improved the quality and speed of network video transmission, traditional broadcasts generally retain the upper hand in image and sound quality.
Summary In general, in one aspect, the invention features a method of synchronizing a client with a media signal. The method includes receiving one or more actions corresponding to the media signal, determining an application for handling the actions, and causing the determined application to handle at the actions . Embodiments may include one or more of the following features. The media signal may be an audio signal and/or a video signal. Determining an application can include parsing action information. For example, parsing action information can include determining if the action information corresponds to a URI (Universal Resource Indicator) or a command. For actions including a URI, the method can include navigating a browser to the URI.
The method may further include prompting a user before causing the determined application to handle the action. The method may also include queuing the action. The action may include an indication of when the determined application should handle the action (e.g., an SMPTE (Society of Motion Pictures and Television Engineers) time-code) . The actions may be received from a server.
The method may also include receiving criteria associated with the action and evaluating the criteria. The criteria may include one or more user characteristics (e.g., age, location, and gender) . In general, in another aspect, the invention features a computer program product, disposed on a computer readable medium, the computer program for synchronizing a client with a media signal. The program includes instructions for causing a processor to receive one or more actions corresponding to the media signal, determine an application for handling at least one of the one or more received actions, and cause the determined application to handle at least one of the one or more received actions . In general, in another aspect, the invention features a method of synchronizing one or more clients with a media signal. The method includes receiving information identifying an action corresponding to the media signal, receiving information identifying criteria for the action, and transmitting the action and criteria toward the one or more clients.
Embodiments may include one or more of the following. The action includes an identification of a time for processing the action and, for example, a URI or some application command or data. Transmitting the action and criteria toward one or more clients may include transmitting the action and criteria to a server and determining at the server, based on the action criteria, whether to transmit the action to a client. Such a determination may be done based on user characteristics associated with the client. In some embodiments, a client receives the. action and determines, based on the action criteria, whether to execute the action.
In general, in another aspect, the invention features a method of synchronizing a client with a media signal. The method includes receiving one or more actions corresponding to the media signal, the actions including an indication of a time to execute each action. The method also includes determining a delay time for the action and handling the action in accordance with the delay time . Embodiments may include one or more of the following features. Determining a delay time may be based on the location of a client and/or on a broadcast medium used to transmit the media signal to a household of the client.
Advantages will become apparent in view of the following description, including the figures and the claims.
Brief Description of the Drawings
FIGs. 1A and IB are diagrams of systems for synchronizing a client with a media signal. FIG. 2 is a flow-chart of a process for transmitting actions to a server.
FIG. 3 is a screenshot of a user interface for identifying actions corresponding to a media signal.
FIG. 4 is a flow-chart of a process for transmitting actions to clients.
FIG. 5 is a diagram of a client including instructions for processing actions.
FIG. 6 is a flow-chart of a process for receiving and processing actions. Detailed Description
FIG. 1A shows an example of a system 100 for synchronizing actions performed by computer 106 with a TV 104 or radio show. For example, as a station 108 broadcasts a TV show (e.g., via airways, cable, or satellite), a computer 106 web-browser may automatically navigate to Internet web-sites corresponding to the TV show content. That is, when the TV show mentions the U.S. Soccer Team, a user' s computer may automatically navigate a web-browser to a URI (Universal Resource Indicator) of
"www.soccer.com/usa/schedule". Similarly, when the TV show mentions a particular player, a user's web-browser may automatically navigate to a URI with a detailed biography. The system 100 can support a wide variety of applications. For example, inter-active web-pages may quiz and provide feedback to students viewing a broadcast school lesson.
Though the system 100 can provide Internet access that is synchronized with a media signal, the system 100 need not be limited to Internet navigation. For example, the system 100 can feature actions that transmit data and/or invoke different client 106 applications other than a browser. For example, actions may cause a client 106 to download and install protection software when a TV program reports the latest computer virus . In addition to synchronization of clients 106 with a signal, the system 100 can also tailor the actions performed by the clients 106, for example, based on user characteristics and/or preferences. That is, an action may cause a computer to present an adult with an option to charge an item shown on TV to their credit card, but not present this option to a child. To provide this feature, the server 112 and/or client 106 may store user characteristics such as an address, gender, age, and so forth. The system 100 can be implemented in a variety of ways. For example, the system 100 of FIG. 1A includes a control computer 110 that enables an operator to define actions corresponding to a media signal 114 (e.g., TV or radio signal) . The system 100 also includes a server 112 that receives actions from the control computer 110 and transmits (e.g., multicasts) the actions to clients 106 in households 102 viewing or listening to a particular broadcast.
In an alternate implementation shown in FIG. IB, the control computer 110 can embed actions in the signal itself, for example, in a VBI (Vertical Blanking Interval) of a video signal. Thus, instead of network transmission of the actions from the control computer 110 to the server 112, the server 112 can monitor the signal as broadcast, extract the actions, and, again, transmit them to the corresponding client 106. Additionally, clients 106 can monitor a broadcast or other media signal to monitor for actions without the server 112.
Though shown as personal computers 106 in FIG. 1A and IB, client software and/or hardware can reside in a wide variety of network devices such as network appliances, handheld PCs, PDAs (Personal Digital Assistants) , game consoles, Internet phones, WAP (Wireless Access Protocol) devices, set-top boxes, ReplayTV®, TiVo®, and so forth. Additionally, though FIG. 1 shows a TV 104 as an example of an a device that presents a media signal, the media signal may also be presented by the same device running the client software. For example, the media signal may be streaming audio or video carried across the Internet to a client computer. Further, while shown as being received from a remote location, the media signal may come from a local source. For example, a VCR may output a media signal to for display and correlation with different actions downloaded from a server or from within the VCR signal.
As shown in FIG. 2, an operator can use the control computer 110 to define 205 actions and specify how the actions should synchronize with the media signal (e.g., when the actions should occur relative to the start of the signal) . Each action defines some operation the client should perform. For example, an action may include a command line identifying an application (e.g., "runvirus_install.exe"), a URI (e.g., www.tvshow.com) specifying a web-site a browser should visit, data representing the real or virtual coordinates of an object or person of a TV show for use by a game application, and so forth. The control computer 110 can, in turn, transmit 204 the actions over a secure (e.g., encrypted) connection (e.g., a TCP/IP
(Transmission Control Protocol/Internet Protocol) connection) established 202 with the server 112. This encryption may be based upon a security key. The key can be changed at the administrative level. Once the key is decoded by the server, an action can be queued. The system may also employee an encryption key at the end user level. As an additional measure of security, the system can also make use of firewall security at the server/control level as well as the client level .
The control computer 110 can offer a variety of different techniques for defining actions. For example, the control computer 110 can enable an operator to define actions in realtime. That is, as a signal broadcasts, an operator can define actions for the broadcast and immediately send them to the server 112 for transmission to clients 106. This can enable an operator to synchronize clients 106 during live broadcasts. The same technique may also enable an operator to specify transmission delays. For example, an operator may define an action and specify that it not be transmitted or executed by the clients 106 for a period of Λ-seconds. The user interface techniques described above feature an operator sitting and viewing the broadcast in real-time and preparing appropriate client computer actions. While the control computer 110 may ease the demands on an operator, for example, by providing previously defined actions for selection, the process of defining actions in real-time may require some dexterity and quick-wittedness on the part of the operator to keep up with the on-going broadcast . Thus, the control computer 110 may provide a user interface that enables the operator to define actions for pre-recorded material at his or her leisure. Thus, the control computer 110 may provide a user interface that enables an operator to define an action and associate the action with a particular time-code. For example, video typically includes embedded SMPTE (Society of Motion Picture and Television Engineers) time-codes identifying each video frame. Thus, the user interface can enable an operator to associate an action with a particular time-code. For example, an action of "Get Smart: 000100 : www.getsmart.com/episodel2" may define an action that causes a client computer's 106 web-browser to visit the "www.getsmart.com" web-site at the one-hundredth frame of Episode 12 of "Get Smart" . The control computer 110 may use other time-codes other than SMPTE (e.g., NNTP time-codes). Additionally, the operator may simply enter a time offset for the action (e.g., one minute after broadcast begins). Again, instead of, or in addition to, transmitting actions to a server 112 via a network connection, the control computer 110 may embed the actions within a media signal such as the video blanking interval (VBI) of a video. In such embodiments, a server 112 can monitor the media signal, extract action definitions, and transmit the actions to clients 106. Additionally, in implementations that embed the actions in the video and/or audio signal, a system 100 need not use a server 112 at all. For example, client computers 106 can monitor media signals received by the household 102 TVs 104 and process the embedded actions.
FIG. 3 illustrates an example of a user interface 300 presented to an operator of a control computer 110. As shown, the interface 300 enables an operator to define actions 302 for a particular broadcast. The action can include a URI, an application program command and parameters, and/or other instructions or data for transmission to the clients 106. The action can also include synchronization information (e.g., a date, time, and/or frame 110 of an action) . The interface 300 may also enable an operator to define whether the action is performed automatically or whether a user at the client computer 106 is prompted before execution. The interface 300 may further enable an operator to define criteria that must be satisfied for performance of the action. For example, the operator may construct a Boolean expression of user characteristics (e.g., (gender = male) AND (age > 35) ) . The operator may define multiple actions for the same signal time-code with different actions having different criteria. This enables the operator to tailor the behavior of client computers 110 for different classes of users. As shown, the interface 300 also includes a list 304 of previously defined actions for selection and a list 306 of actions already assigned to the broadcast.
FIG. 4 shows a process 400 that a server 112 can use to transmit 408 actions received 402 from a control computer 110 or detected in a signal (e.g. , the VBI portion of a video). After receiving the actions, the server 112 can log the actions. For example, the server 110 may dynamically add a description of broadcast actions to an HTML (HyperText Markup Language) or XML (extensible Markup Language) page listing actions for a particular show or broadcast. Users and contributors can access the logs to view actions for a particular broadcast long after the show has aired.
Before and during a broadcast, the server 110 may receive registration requests from different system 100 users.
Registration identifies a broadcast the user is viewing and, potentially, the IP (Internet Protocol) address of their client. Additionally, for new members, registration may involve providing demographic data such as an address, age, gender, income, and so forth. This data may be stored by the server 110 to tailor the actions delivered to a user's client 106. For existing members, registration may involve entering a user name so the server 110 can retrieve the previously stored demographic data for the user.
Registration and other interaction with the server 112 may occur via the Internet (e.g., a login web-page, a preferences web-page and so forth) or via a telephone voice response system that enables a user to customize and select different services transmitted to the client.
During broadcast of a TV or radio program, the server 112 sends actions for the broadcast to clients 106 identified as viewing the show. For example, the server 112 can multicast the actions using UDP (User Datagram Protocol) or other protocols. Again, the server 112 may select actions for transmission based on user demographic data. The server 112 may handle timing issues by transmitting actions at the specified times. Alternatively, the server 112 may include synchronization information in the action and leave it to the client 106 to process the action at the designated time.
The same show may be broadcast at slightly different times. For example, a satellite transmission generally lags cable transmission by tens of seconds. Additionally, different geographic locations broadcast at different times. The server 112 can offset 408 transmission of actions based on the geographic location of a user (e.g., based on the user's zip code) and/or transmission sources (e.g., cable or satellite).
Communication between the clients 106 and server 110 need not be one way. That is, the clients 106 may send data to the server 110, for example, using TCP/IP. This enables the server 110 to monitor user behavior and can enable the server to select actions based on prior user behavior.
FIG. 5 shows an example of a client 106. As shown, the client 106 includes an operating system 504 (e.g., Microsoft® Windows 2000®) and one or more applications 502 (e.g., Microsoft® Explorer® and Microsoft® Word®) . The client 106 also includes instructions 500 for processing actions. As shown, the processing instructions 500 can invoke other applications and can even make operating system 504 calls.
FIG. 6 illustrates a process 600 the instructions (500 of FIG. 5) may implement. As shown, the process 600 may include registering the user with a server 602, for example, by navigating a browser to a log-in web-site associated with the server. Registering enables the client to identify a program being viewed or heard. Of course, a user need not actually be viewing or listening to a program.
Thereafter, the client may receive a stream of one or more actions 604 corresponding to the identified program as the program broadcast progresses. The client determines an application for the action. For example, the client may parse information included in the action to identify whether an action includes a URI. If such parsing succeeds, the client can pass the URI to a browser. Alternatively, the client can look for names of applications and/or keyword such as "run" that indicate what operation the client should perform in response to the action (e.g., which application should handle the action). The system(s) described above can use a variety of architectures. For example, many features provided by the server can instead be provided by the client and vice-versa. For example, instead of selection by the server 110, the server 110 may transmit action criteria for evaluation 606 by the client. Additionally, either the server 110 or client 106 may specify that the action be queued 610 or executed 612 upon receipt or arrival of a specified time. Further, either the server 110 or client 106 may request that the client prompt a user before executing an action. This can enable, for example, a user to finish reading information at one web-site before moving on to another.
Implementations
The techniques described herein are not limited to any particular hardware or software configuration; they may find applicability in any computing or processing environment . The techniques may be implemented in hardware or software, or a combination of the two. Preferably, the techniques are implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements) , at least one input device, and one or more output devices.
Each program is preferably implemented in high level procedural or object oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case the language may be compiled or interpreted language. Each such computer program is preferably stored on a storage medium or device (e.g., CD-ROM, hard disk, or magnetic disk) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner. Other embodiments are within the scope of the following claims.

Claims

What is claimed is :
1. A method of synchronizing a client with a media signal, the method comprising: receiving one or more actions corresponding to the media signal; determining an application for handling at least one of the one or more received actions; and causing the determined application to handle at least one of the one or more received actions.
2. The method of claim 1, wherein the media signal comprises at least one of the following: an audio signal and a video signal.
3. The method of claim 1, wherein determining an application comprises parsing action information.
4. The method of claim 3, wherein parsing action information comprises determining if the action information corresponds to a URI (Universal Resource Indicator) .
5. The method of claim 3, wherein parsing action information comprises determining if the action information corresponds to a command.
6. The method of claim 1, wherein causing the determined application to handle at least one of the actions comprises causing an Internet browser to navigate to a URI specified in the action.
7. The method of claim 1, further comprising prompting a user before causing the determined application to handle the action.
8. The method of claim 1, further comprising queuing the action.
9. The method of claim 1, wherein the action comprises at least one of the following: a URI and an application command.
10. The method of claim 9, wherein the action comprises an indication of when the determined application should handle the action.
11. The method of claim 10, wherein the indication comprises a SMPTE (Society of Motion Pictures and Television Engineers) time-code.
12. The method of claim 1, wherein receiving the at least one actions comprises receiving the at least one actions from a server.
13. The method of claim 1, further comprising receiving criteria associated with the action; and evaluating the criteria.
14. The method of claim 13, wherein the criteria comprises one or more user characteristics.
15. The method of claim 14, wherein the user characteristics comprises at least one of the following: age, location, and gender.
16. A computer program product, disposed on a computer readable medium, the computer program for synchronizing a client with a media signal, the program comprising instructions for causing a processor to: receive one or more actions corresponding to the media signal; determine an application for handling at least one of the one or more received actions; and cause the determined application to handle at least one of the one or more received actions.
18. A method of synchronizing one or more clients with a media signal, the method comprising: receiving information identifying an action corresponding to the media signal; receiving information identifying criteria for the action; transmitting the action and criteria toward the one or more clients.
19. The method of claim 18, wherein the action comprises identification of a time for processing the action and at least one of the following: a URI and an application command.
20. The method of claim 18, wherein transmitting the action and criteria toward one or more clients comprises transmitting the action and criteria to a server.
21. The method of claim 20, further comprising determining at the server, based on the action criteria, whether to transmit the action to a client.
22. The method of claim 21, wherein determining comprises determining based on user characteristics associated with the client.
23. The method of claim 22, wherein the user characteristics comprise one or more of the following: age, gender, and location.
24. The method of claim 18, further comprising receiving the action and criteria at a client; and determining at the client, based on the action criteria, whether to execute the action.
25. The method of claim 24, wherein determining comprises determining based on user characteristics.
26. The method of claim 25, wherein the user characteristics comprise one or more of the following: age, gender, and location.
27. A method of synchronizing a client with a media signal, the method comprising: receiving one or more actions corresponding to the media signal, the actions including an indication of a time to execute each action; determining a delay time for the action; and handling the action in accordance with the delay time.
28. The method of claim 27, wherein determining a delay time comprises determining a delay time based on the location of a client.
29. The method of claim 27, wherein determining a delay time comprises determining a delay time based on a broadcast medium used to transmit the media signal to a household of the client .
30. The method of claim 27, wherein handling the action comprises determining when to transmit the action from a server to the client .
31. The method of claim 27, wherein handling the action comprises determining when to execute the action at the client.
PCT/US2001/028246 2000-09-08 2001-09-10 Synchronizing a client with a media signal WO2002021840A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001290712A AU2001290712A1 (en) 2000-09-08 2001-09-10 Synchronizing a client with a media signal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US65749300A 2000-09-08 2000-09-08
US09/657,493 2000-09-08

Publications (2)

Publication Number Publication Date
WO2002021840A2 true WO2002021840A2 (en) 2002-03-14
WO2002021840A3 WO2002021840A3 (en) 2002-06-20

Family

ID=24637406

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/028246 WO2002021840A2 (en) 2000-09-08 2001-09-10 Synchronizing a client with a media signal

Country Status (2)

Country Link
AU (1) AU2001290712A1 (en)
WO (1) WO2002021840A2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2841724A1 (en) * 2002-06-28 2004-01-02 Thomson Licensing Sa SYNCHRONIZATION SYSTEM AND METHOD FOR AUDIOVISUAL PROGRAMS, DEVICES AND RELATED METHODS
GB2397991A (en) * 2002-12-31 2004-08-04 Orca Interactive Ltd Apparatus, method and a computer readable medium for providing interactive services
EP1579673A2 (en) * 2002-10-18 2005-09-28 Intellocity USA, Inc. Triggered video system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5973684A (en) * 1995-07-06 1999-10-26 Bell Atlantic Network Services, Inc. Digital entertainment terminal providing dynamic execution in video dial tone networks
US5990972A (en) * 1996-10-22 1999-11-23 Lucent Technologies, Inc. System and method for displaying a video menu
EP0982943A2 (en) * 1996-03-08 2000-03-01 Actv, Inc. An intergrated interactive video and internet system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5973684A (en) * 1995-07-06 1999-10-26 Bell Atlantic Network Services, Inc. Digital entertainment terminal providing dynamic execution in video dial tone networks
EP0982943A2 (en) * 1996-03-08 2000-03-01 Actv, Inc. An intergrated interactive video and internet system
US5990972A (en) * 1996-10-22 1999-11-23 Lucent Technologies, Inc. System and method for displaying a video menu

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2841724A1 (en) * 2002-06-28 2004-01-02 Thomson Licensing Sa SYNCHRONIZATION SYSTEM AND METHOD FOR AUDIOVISUAL PROGRAMS, DEVICES AND RELATED METHODS
WO2004004336A2 (en) 2002-06-28 2004-01-08 Thomson Licensing S.A. Audiovisual program synchronization system and method
WO2004004360A1 (en) * 2002-06-28 2004-01-08 Thomson Licensing S.A. Synchronization system and method for audiovisual programmes, associated devices and methods
US8612544B2 (en) * 2002-06-28 2013-12-17 Thomson Licensing Audiovisual program synchronization system and method
US8626872B2 (en) 2002-06-28 2014-01-07 Thomson Licensing Synchronization system and method for audiovisual programmes associated devices and methods
EP1579673A2 (en) * 2002-10-18 2005-09-28 Intellocity USA, Inc. Triggered video system
EP1579673A4 (en) * 2002-10-18 2007-05-23 Intellocity Usa Inc Triggered video system
GB2397991A (en) * 2002-12-31 2004-08-04 Orca Interactive Ltd Apparatus, method and a computer readable medium for providing interactive services
GB2397991B (en) * 2002-12-31 2006-07-05 Orca Interactive Ltd Apparatus,method and a computer readable medium for providing interactive services

Also Published As

Publication number Publication date
AU2001290712A1 (en) 2002-03-22
WO2002021840A3 (en) 2002-06-20

Similar Documents

Publication Publication Date Title
US10356477B2 (en) Interactive content system and method
KR102057107B1 (en) Receiving device, receiving method, program, and information processing system
CA2456984C (en) Interactive television tracking system
US7263712B2 (en) Enabling a PC-DTV receiver to share the resource cache with multiple clients
RU2556242C2 (en) Transmission device and method, receiving device and method and transmission and receiving system
WO2013118617A1 (en) Reception device, reception method, and program
US20020112002A1 (en) System and process for creating a virtual stage and presenting enhanced content via the virtual stage
US20030191805A1 (en) Methods, apparatus, and systems for on-line seminars
MX2015004659A (en) Apparatus and method for processing an interactive service.
US7725918B2 (en) Interactive television with embedded universal time codes
US8782733B2 (en) Method and system for updating an interaction document
JP2004500762A (en) Notify network users about television programming being watched by other network users
KR20030022882A (en) Method and apparatus for time shifting of broadcast content that has synchronized web content
JP3996770B2 (en) Television channel selection method and system
US20090064223A1 (en) Dtv viewing terminal, epg service system and method for exhibiting epg
EP1152619A2 (en) Method, device and system for digital communication
US20030037340A1 (en) Method and apparatus for managing TV broadcast content that has synchronized web applications
US7243365B1 (en) Apparatus and method for delivery of metadata on ATVEF transport B enabled platform
WO2002021840A2 (en) Synchronizing a client with a media signal
KR102611253B1 (en) Receiving devices, transmitting devices and data processing methods
JP2013118614A (en) Receiver, program, and broadcasting system
JP2003216531A (en) Content delivery method and system, and audience rating of content confirming device
KR20020096412A (en) Communication service device of real time internet broadcasting and control method thereof
EP1784982A2 (en) Program selection system for accessing streamed media content
KR100679399B1 (en) Method for surveying viewing rate using data broadcasting

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PH PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: A3

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PH PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

ENP Entry into the national phase

Ref document number: 2003108878

Country of ref document: RU

Kind code of ref document: A

Format of ref document f/p: F

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP