WO2003085991A2 - Mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control - Google Patents

Mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control Download PDF

Info

Publication number
WO2003085991A2
WO2003085991A2 PCT/US2003/006880 US0306880W WO03085991A2 WO 2003085991 A2 WO2003085991 A2 WO 2003085991A2 US 0306880 W US0306880 W US 0306880W WO 03085991 A2 WO03085991 A2 WO 03085991A2
Authority
WO
WIPO (PCT)
Prior art keywords
data streams
input data
control
control device
receiving device
Prior art date
Application number
PCT/US2003/006880
Other languages
French (fr)
Other versions
WO2003085991A3 (en
Inventor
Sho Kou
Original Assignee
Sony Electronics, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Electronics, Inc. filed Critical Sony Electronics, Inc.
Priority to EP03721338A priority Critical patent/EP1491051A4/en
Priority to JP2003583035A priority patent/JP4896374B2/en
Priority to KR10-2004-7014943A priority patent/KR20040111426A/en
Publication of WO2003085991A2 publication Critical patent/WO2003085991A2/en
Publication of WO2003085991A3 publication Critical patent/WO2003085991A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/775—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television receiver
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4135—Peripherals receiving signals from specially adapted client devices external recorder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4314—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43622—Interfacing an external recording device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43632—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wired protocol, e.g. IEEE 1394
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/454—Content or additional data filtering, e.g. blocking advertisements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information

Definitions

  • the present invention relates to the field of selecting and mixing data at a receiving device. More particularly, the present invention relates to the field of selecting and mixing streams of audio/video data at a receiving device to provide an output stream of data under control from an external device.
  • the IEEE standard "IEEE 1394-2000 Standard For A High Performance Serial Bus," Draft ratified in 2000, is an international standard for implementing an inexpensive liigh- speed serial bus architecture which supports both asynchronous and isochronous format data transfers.
  • Isochronous data transfers are real-time transfers which take place such that the time intervals between significant instances have the same duration at both the transmitting and receiving applications.
  • Each packet of data transferred isochronously is transferred in its own time period.
  • An example of an ideal application for the transfer of data isochronously would be from a video recorder to a television set. The video recorder records images and sounds and saves the data in discrete chunks or packets.
  • the video recorder then transfers each packet, representing the image and sound recorded over a limited time period, during that time period, for display by the television set.
  • the IEEE 1394-2000 serial bus architecture provides multiple channels for isochronous data transfer between applications. A six bit channel number is broadcast with the data to ensure reception by the appropriate application.
  • Asynchronous transfers are traditional data transfer operations which take place as soon as possible and transfer an amount of data from a source to a destination.
  • the IEEE 1394-2000 standard provides a high-speed serial bus for interconnecting digital devices thereby providing a universal I/O connection.
  • the IEEE 1394-2000 standard defines a digital interface for the applications thereby eliminating the need for an application to convert digital data to analog data before it is transmitted across the bus.
  • a receiving application will receive digital data from the bus, not analog data, and will therefore not be required to convert analog data to digital data.
  • the cable required by the IEEE 1394-2000 standard is very thin in size compared to other bulkier cables used to connect such devices. Devices can be added and removed from an IEEE 1394-2000 bus while the bus is active. If a device is so added or removed the bus will then automatically reconfigure itself for transmitting data between the then existing nodes.
  • a node is considered a logical entity with a unique address on the bus structure.
  • Each node provides a configuration ROM, a standardized set of control registers and its own address space. Because of these advantages the IEEE 1394-2000 standard provides for a unique networking structure that is capable of incorporating audio/video devices, media play/record devices, computing devices and display devices.
  • the IEEE 1394-2000 standard defines a protocol as illustrated in Figure 1.
  • This protocol includes a serial bus management block 10 coupled to a transaction layer 12, a link layer 14 and a physical layer 16.
  • the physical layer 16 provides the electrical and mechanical connection between a device or application and the IEEE 1394-2000 cable.
  • the physical layer 16 also provides arbitration to ensure that all devices coupled to the IEEE 1394-2000 bus have access to the bus as well as actual data transmission and reception.
  • the link layer 14 provides data packet delivery service for both asynchronous and isochronous data packet transport. This supports both asynchronous data transport, using an acknowledgement protocol, and isochronous data transport, providing real-time guaranteed bandwidth protocol for just-in-time data delivery.
  • the transaction layer 12 supports the commands necessary to complete asynchronous data transfers, including read, write and lock.
  • the transaction layer 12 also provides a path for isochronous management data to be transferred to the serial bus management block 10 via read operations with isochronous control compare-swap registers.
  • the serial bus management block 10 contains an isoclironous resource manager for managing isochronous data transfers.
  • the serial bus management block 10 also provides overall configuration control of the serial bus in the form of optimizing arbitration timing, guarantee of adequate electrical power for all devices on the bus, assignment of the cycle master, assignment of isoclironous channel and bandwidth resources and basic notification of errors.
  • a diverse range of products can be implemented with the ability to connect to an IEEE 1394-2000 serial bus network. These devices can have capabilities and functionality ranging from very simple to very complex. Specifically, a variety of audio/video devices, media play/record devices and computing/display devices are capable of being linked together over an IEEE 1394-2000 serial bus networking structure to support asynchronous and isochronous data transfers between the devices.
  • the IEEE 1394-2000 cable environment is a network of nodes connected by point-to- point links, including a port on each node's physical connection and the cable between them.
  • the physical topology for the cable environment of an IEEE 1394-2000 serial bus is a non- cyclic network of multiple ports, with finite branches.
  • the primary restriction on the cable environment is that nodes must be connected together without forming any closed loops.
  • FIG. 2 A block diagram of a conventional home audio/video network including a television and an analog video cassette recorder (VCR) is illustrated in Figure 2.
  • the television 20 is coupled to the analog VCR 40.
  • Video data and associated data are sent between the analog
  • VCR 40 VCR 40 and the television 20 in a known manner.
  • the television 20 includes an interface 32 which sends and receives audio and video signals to and from the analog VCR 40.
  • the interface 32 is coupled to an audio/video switch 26 for directing audio/video signals to and from the analog VCR 40.
  • a cable/antenna interface circuit 30 is coupled to receive input signals from a coaxial cable or an antenna and to pass those signals through a tuner 28 to the audio/video switch 26.
  • the audio/video switch 26 is coupled to a video random access memory (VRAM) circuit 24 for providing the video signals from the cable/antenna interface 30 or the analog VCR 40 to the display 22.
  • VRAM video random access memory
  • the analog VCR 40 includes a video source 46, such as a video tape which is being played by the analog VCR 40 or a television input.
  • the analog VCR 40 also includes a graphics source 48 which generates on-screen-display graphics to be displayed by the television 20 when the analog VCR 40 is sending data to be displayed by the television 20.
  • Such on-screen-display graphics include words or symbols representing commands being executed by the analog VCR 40, such as PLAY, STOP, REWIND, FAST-FORWARD, PAUSE and RECORD.
  • the on-screen-display graphics generated by the graphics source 48 typically also include a menu or menus displayed on the television 20 for the user which allow the user to program the analog VCR 40.
  • On-screen-display graphics generated by the graphics source 48 are provided to a mixer circuit 44.
  • the mixer circuit 44 also receives video data from the video source 46.
  • the mixer circuit 44 then combines the on-screen- display graphics from the graphics source 48 and the video data from the video source 46 into a video output which is transmitted through the interface circuit 42 to the television 20.
  • the video output from the analog VCR 40 is then processed by the television 20 and shown on the display 22.
  • the mixer circuit 44 will, as appropriate, either overlay the on-screen-display graphics onto the video data to form the video output or cause the video output to include only on-screen-display graphics from the graphics source 48 or video data from the video source 46.
  • the video data from the digital VCR is typically transmitted in a digital format such as MPEG.
  • the digital VCR is not typically responsible for encoding the video data in an MPEG format, but will record and transmit data previously encoded in a MPEG format by another source.
  • the digital VCR would have to include an MPEG encoder and have the ability to encode the on-screen-display graphics into an MPEG format and then combine the streams of data into a video output stream of data. Due to the cost of MPEG encoders, such a requirement is cost prohibitive in competitive consumer VCRs.
  • Some video source devices have the ability to generate a very high bandwidth graphics output which is sent to a display device, such as a television, for on-screen-display to the user.
  • This graphics output can be of very high resolution and color depth, including dynamic animation effects with multiple portions of the graphics data changing on a frequent basis.
  • This graphics data is typically mixed with, or overlaid on top of, a video signal.
  • it is relatively easy and inexpensive to provide capability within an analog device to combine on-screen-display graphics with a video stream of data and send the combined stream of data to a television or other display device for display to a user or users. It is also relatively easy and inexpensive for a device, such as a video game console to generate a pure graphics output stream and send this output stream over an analog video connection to the television.
  • the EIA-775, DTV 1394 Interface Specification, draft November 30, 1998, is a standard which defines a specification for a baseband digital interface to a digital television that provides a level of functionality that is similar to the analog system illustrated in Figure 2 and described above.
  • an audio/video source capable of producing analog audio and video and also an MPEG transport stream, is coupled to a digital television receiver. The analog signal is transmitted over a standard coaxial cable and the
  • MPEG data is transmitted isochronously over an IEEE 1394 serial bus. Bitmaps of the source on-screen-display data are sent by asynchronous connections separately over the IEEE 1394 serial bus and are mixed with the decoded MPEG video in the digital television prior to being presented on the display. This process removes the need for the audio/video source to perform the additional MPEG video decodes and re-encodes, described above.
  • the process of the EIA-775 specification requires the audio/video source and the digital television to be connected together by the IEEE 1394 serial bus. The process of the EIA-775 specification does not work with connections other than the IEEE 1394 serial bus.
  • a method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control includes a receiving device, a control device and one or more source devices.
  • the receiving device receives input data streams from the source devices.
  • the input data streams are provided as inputs to selectors within the receiving device.
  • the selectors output selective ones of the input data streams to a mixer, based on control information received from a control device.
  • the mixer then combines the selected input data streams into an output stream, based on control information received from the control device.
  • the interfaces and functions of the selectors and the mixer are provided to the control device in response to a request from the control device.
  • the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams.
  • a method of providing an output data stream comprises receiving one or more input data streams from one or more source devices at a receiving device, receiving control information from a control device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, selecting the selected input data streams based on the control information from the control device and mixing the selected input data streams into the output data stream based on the control information from the control device.
  • the method further comprises sending a request from the control device to the receiving device regarding selection and mixing functions and controls at the receiving device and providing the selection and mixing functions and controls from the receiving device to the control device.
  • the input data streams include one or more of audio/video data streams, on-screen- display data streams and graphics data streams.
  • the control device is preferably external of the receiving device. Alternatively, the control device is internal to the receiving device.
  • the method further comprises outputting the output data stream.
  • the method further comprises receiving control input at the control device and generating the control information based on the control input.
  • the one or more input streams are received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • the control information is received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • a method of controlling a receiving device providing a combination of one or more input data streams as an output data stream comprises sending a request from a control device to the receiving device regarding selection and mixing functions and controls at the receiving device, providing the selection and mixing functions and controls from the receiving device to the control device and providing control information from the control device to the receiving device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, wherein the selected input data streams are selected and mixed based on the control information.
  • the method further comprises receiving the one or more input data streams from one or more source devices at the receiving device, selecting the selected input data streams based on the control information from the control device and mixing the selected input data streams into the output data stream based on the control information from the control device.
  • the method further comprises outputting the output data stream.
  • the input data streams include one or more of audio/video data streams, on- screen-display data streams and graphics data streams.
  • the control device is external of the receiving device.
  • the method further comprises receiving control information at the control device and generating the control information based on the control input.
  • the one or more input streams are received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • the control information is received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • a receiving device for providing an output data stream comprises means for receiving one or more input data streams configured for coupling to one or more source devices, means for receiving control information configured for coupling to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, means for selecting coupled to the means for receiving one or more input data streams and the means for receiving control information for selecting the selected input data streams based on the control information from the control device and means for mixing coupled to the means for selecting and to the means for receiving control information for mixing the selected input data streams into the output data stream based on the control information from the control device.
  • the means for receiving control information provides selection and mixing functions and controls to the control device in response to a request from the control device.
  • the receiving device further comprises means for outputting coupled to the means for mixing for outputting the output data stream.
  • the input data streams include one or more of audio/video data streams, onscreen-display data streams and graphics data streams.
  • the control device is external of the receiving device.
  • the control device is internal of the receiving device.
  • the means for receiving one or more input data streams is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug- and-play standard.
  • the means for receiving control information is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a com ection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug- and-play standard.
  • a receiving device to provide an output data stream comprises a first interface to receive one or more input data streams configured to couple to one or more source devices, a second interface to receive control information configured to couple to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, one or more selectors coupled to the first interface and to the second interface to select the selected input data streams based on the control information from the control device and a mixer coupled to the selectors and the second interface to mix the selected input data streams into the output data stream based on the control information from the control device.
  • the second interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device.
  • the receiving device further comprises an output device coupled to the mixer to output the output data stream.
  • the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
  • the control device is external to the receiving device.
  • the control device is internal to the receiving device.
  • the first interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • the second interface is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • a control device to provide control information to a receiving device receiving one or more input data streams to control selection and mixing of the input data streams into an output data stream comprises an interface configured to couple to the receiving device to request selection and mixing function and control information from the receiving device and a control circuit coupled to the interface to receive the selection and mixing function and control information and to provide output control information to the receiving device, wherein the output control information includes information regarding selected ones of the one or more input data streams and a selected combination of the selected ones of the one or more input data streams.
  • the input data streams include one or more of audio/video data streams, on-screen- display data streams and graphics data streams.
  • the receiving device is external to the control device.
  • the receiving device is internal to the control device.
  • the interface is coupled to the receiving device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • a network of devices to provide an output data stream comprises one or more source devices configured to provide one or input data streams, a control device configured to provide control information regarding selection and mixing of the input data streams into the output data stream, the control device including a control interface configured to provide the control information and a control circuit coupled to the control interface to generate the control information and a receiving device comprising a first receiving interface coupled to the source devices to receive the one or more input data streams, a second receiving interface coupled to the control interface to receive the control information, one or more selectors coupled to the first receiving interface and the second receiving interface to select selected input data streams based on the control information and
  • the second receiving interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device.
  • the receiving device further comprises an output device coupled to the mixer to output the output data stream.
  • the input data streams include one or more of audio/video data streams, onscreen-display data streams and graphics data streams.
  • the control device is external to the receiving device.
  • the control device and the receiving device are integrated together.
  • the first receiving interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • the second receiving interface is coupled to control interface by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
  • Figure 1 illustrates a protocol defined by the IEEE 1394-2000 standard.
  • Figure 2 illustrates a block diagram of a conventional home audio/video network including a television and VCR.
  • Figure 3 illustrates a functional block diagram of an exemplary network of devices according to the present invention, including a first digital VCR, a second digital VCR, a computer system and a digital television.
  • Figure 4 illustrates a block diagram of the internal components of the computer system 120.
  • Figure 5 illustrates a flowchart showing the preferred steps followed by a control device when controlling the functions of selectors and a mixer within a receiving device.
  • Figure 6 illustrates a flowchart showing the preferred steps followed by a receiving device to provide an output data stream based on control instructions received from a control device.
  • the method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control of the preferred embodiment of the present invention includes a receiving device, a control device and one or more source devices.
  • the receiving device receives input data streams from the source devices.
  • the input data streams are streams of audio/video data and on-screen-display/graphics data.
  • the input data streams are streams of any appropriate data type to be selected and mixed into an output stream by the receiving device.
  • the input data streams are provided as inputs to selectors within the receiving device.
  • the selectors output selective ones of the input data streams to a mixer, based on control input received from a control device.
  • the selectors are turned off or on.
  • the selector When a selector is turned off, the selector will not output any input data stream.
  • the selector When a selector is turned on, the selector will output a selected one of the input data streams provided as inputs to the selector.
  • the outputs from the selectors are provided to a mixer.
  • the mixer then combines the selected input data streams into an output stream, based on control input received from the control device.
  • the mixer can select one of the input data streams or can output some combination of the input data streams.
  • the receiving device exposes the functionality and controls of the selectors and the mixer to the control device, allowing the control device to provide control information to control the selectors and the mixer within the receiving device.
  • control device As the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams.
  • the control device receives the control inputs from any appropriate input devices including a remote control, keyboard, mouse or other devices coupled to the control device either by a wired or wireless connection. This allows the control device to control which input data streams are output, how the input data streams are output and how the input data streams are combined for output, if appropriate. Accordingly, for video data streams, the display of multiple input data streams can be selected, displayed together in a tiled format or in separate windows, such as a picture-in-picture format, and on-screen- display/graphics data can be overlaid on video input data streams.
  • the output of one or more audio source streams can be selected or the audio source streams can be combined together for output.
  • a functional block diagram of an exemplary network of devices including a first digital VCR 80, a second digital VCR 100, a computer system 120 and a digital television 50 is illustrated in Figure 3. Relevant functional components of the first digital VCR 80, the second digital VCR 100 and the digital television 50 are also illustrated in Figure 3.
  • the digital television 50 includes an IEEE 1394-2000 interface circuit 76 through which data and commands are sent to and received from the digital VCR 80, the digital VCR 100 and the computer 120, over the IEEE 1394-2000 serial bus cables 126, 128 and 130, respectively.
  • the IEEE 1394-2000 interface circuit 76 includes a physical interface circuit 78.
  • the IEEE 1394-2000 serial bus cables 126, 128 and 130 are coupled to the plug A, the plug B and the plug C, respectively.
  • the IEEE 1394-2000 interface circuit 76 is coupled to a control circuit 74.
  • the control circuit 74 is coupled to provide audio/video data received from the digital VCRs 80 and 100 on the plugs A and B, to a decoder circuit 72.
  • the control circuit 74 is coupled to provide on-screen-display and graphics data to a decompressor circuit 66.
  • the control circuit 74 is also coupled to an on-screen-display selector 58, a secondary selector 60, a primary selector 62 and a mixer 56 to provide control information to the on-screen-display selector 58, the secondary selector 60, the primary selector 62 and the mixer 56.
  • the decoder circuit 72 includes an output A and an output B, which are both coupled to inputs of the primary selector 62.
  • the primary selector 62 includes a third input which is configured for coupling to another connection.
  • the decompressor circuit 66 is coupled to provide decompressed on-screen-display and graphics data to a buffer 64.
  • the buffer 64 is coupled to an input of the on-screen-display selector 58.
  • the digital television 50 includes a cable/antenna interface circuit 70 which is coupled to receive input signals from a coaxial cable or an antenna and to pass those signals through a tuner 68 to an input of the secondary selector 60.
  • the outputs of the on-screen-display selector 58, the secondary selector 60 and the primary selector 62 are coupled to inputs of the mixer 56.
  • An output of the mixer 56 is coupled to a VRAM circuit 54 for providing video and on-screen-display and graphics data to the display 52.
  • the digital VCR 80 includes an IEEE 1394-2000 interface circuit 88 through which data and commands are sent to and received from the digital television 50.
  • the IEEE 1394- 2000 interface circuit 88 includes a physical interface circuit 90.
  • the physical interface circuit 90 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 126.
  • the digital VCR 80 includes a video source 82, such as a video tape, which is being played by the VCR 80, or a television input.
  • the video source 82 is coupled to the IEEE 1394-2000 interface circuit 88 for transmitting video streams of data over the IEEE 1394- 2000 serial bus cable 126 to the digital television 50.
  • the digital VCR 80 also includes a graphics source 84 which generates on-screen-display graphics to be displayed by the television 50.
  • the graphics source 84 is preferably coupled to a compression circuit 86 which compresses the graphics data generated by the graphics source 84.
  • the compression circuit 86 is coupled to the IEEE 1394-2000 interface circuit 88 for transmitting the compressed graphics data over the IEEE 1394-2000 serial bus cable 126 to the digital television 50.
  • the digital VCR 100 includes an IEEE 1394-2000 interface circuit 108 through which data and commands are sent to and received from the digital television 50.
  • the IEEE 1394- 2000 interface circuit 108 includes a physical interface circuit 110.
  • the physical interface circuit 110 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 128.
  • the digital VCR 100 includes a video source 102, such as a video tape, which is being played by the digital VCR 100, or a television input.
  • the video source 102 is coupled to the IEEE 1394-2000 interface circuit 108 for transmitting video streams of data over the IEEE 1394-2000 serial bus cable 128 to the digital television 50.
  • the digital VCR 100 also includes a graphics source 104 which generates on-screen-display graphics to be displayed by the digital television 50.
  • the graphics source 104 is preferably coupled to a compression circuit 106 which compresses the graphics data generated by the graphics source 104.
  • the compression circuit 106 is coupled to the IEEE 1394-2000 interface circuit 108 for transmitting the compressed graphics data over the IEEE 1394-2000 serial bus cable 128 to the digital television 50.
  • the computer system 120 includes an IEEE 1394-2000 interface circuit 122 through which data and commands are sent to and received from the digital television 50.
  • the IEEE 1394-2000 interface circuit 122 includes a physical interface circuit 124.
  • the physical interface circuit 124 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 130.
  • the configuration illustrated in Figure 3 is exemplary only. It should be apparent that an audio/video network could include many different combinations of components. It should also be recognized that data, commands and parameters can be sent in either direction between the devices within the IEEE 1394-2000 network, as appropriate.
  • a block diagram of the internal components of the computer system 120 is illustrated in Figure 4.
  • the computer system 120 includes a central processor unit (CPU) 156, a main memory 150, a video memory 158, a mass storage device 152 and an IEEE 1394-2000 interface circuit 122, all coupled together by a conventional bidirectional system bus 154.
  • the interface circuit 122 includes the physical interface circuit 124 for sending and receiving communications over the IEEE 1394-2000 serial bus.
  • the physical interface circuit 124 is coupled to the digital television 50 over the IEEE 1394-2000 serial bus cable 130.
  • the interface circuit 122 is implemented on an IEEE 1394-2000 interface card within the computer system 120.
  • the mass storage device 152 may include both fixed and removable media using any one or more of magnetic, optical or magneto-optical storage technology or any other available mass storage technology.
  • the system bus 154 contains an address bus for addressing any portion of the memory 150 and 158.
  • the system bus 154 also includes a data bus for transferring data between and among the CPU 156, the main memory
  • the video memory 158 the video memory 158
  • the mass storage device 152 the interface circuit 122.
  • the computer system 120 is also coupled to a number of peripheral input and output devices including the keyboard 166, the mouse 168 and the associated display 164.
  • the keyboard 166 is coupled to the CPU 156 for allowing a user to input data and control commands into the computer system 120.
  • a conventional mouse 168 is coupled to the keyboard 166 for manipulating graphic images on the display 164 as a cursor control display. It should be apparent to those skilled in the art, that the mouse 168 could also be coupled to the CPU 156.
  • a port of the video memory 158 is coupled to a video multiplex and shifter circuit 160, which in turn is coupled to a video amplifier 162.
  • the video amplifier 162 drives the display 164.
  • the video multiplex and shifter circuitry 160 and the video amplifier 162 convert pixel data stored in the video memory 158 to raster signals suitable for use by the display 164.
  • the digital television 50 is a receiving device, which receives input streams of data from source devices coupled to the digital television 50.
  • the source devices 80 and 100 are coupled to the digital television 50 by the IEEE 1394- 2000 cables 126 and 128 and the cable input is coupled to the cable/antenna interface 70.
  • the source devices 80 and 100 could be coupled to the digital television 50 by any appropriate com ection means including wired and wireless connections and by any appropriate protocol, including other versions of the IEEE 1394 protocol, universal serial bus, and universal plug-and-play. It should also be apparent that input streams of data could originate from sources within the digital television 50.
  • the computer system 120 is a control device for controlling the operation of the selectors 58, 60 and 62 and the mixer 56 of the present invention, within the digital television 50.
  • the control device 120 is coupled to the digital television 50 by the IEEE 1394-2000 cable 130.
  • control device 120 could be coupled to the digital television 50 by any appropriate connection means including wired and wireless connections and by any appropriate protocol, including other versions of the IEEE 1394 protocol, universal serial bus, and universal plug-and-play.
  • control device 120 is an external device implemented by a computer system. It should be apparent to those skilled in the art that the control device can be any appropriate connected device with the capability to discover and control the selectors 58, 60 and 62 and the mixer 56, including an external control device, an internal control device, a source device and a remote control device.
  • the input data streams from the sources coupled to the digital television 50 are provided as inputs to the digital television 50.
  • the cable input is provided through the cable/antenna interface 70 to the tuner 68. After being tuned by the tuner 68, the cable input is then provided as an input to the secondary selector 60.
  • the input data streams from the source devices coupled to the digital television through the IEEE 1394-2000 serial bus network are provided through the IEEE 1394 interface circuit 76 to the control circuit 74.
  • the control circuit 74 is coupled to provide the on-screen-display/graphics input data streams to the decompressor 66.
  • the decompressor 66 decompresses the on-screen-display/graphics input data streams and provides the data to the buffer 64.
  • the decompressed on-screen-display/graphics input data is provided as an input to the OSD selector 58.
  • the OSD selector 58 selects input data to be output based on a URL address.
  • the control circuit 74 is coupled to provide encoded audio/video data from the IEEE 1394-2000 source devices to the decoder 72.
  • the decoder 72 then decodes the encoded audio/video data and provides the decoded data from the outputs A and B as inputs to the primary selector 62.
  • the outputs A and B from the decoder 72 correspond to the input plugs A and B of the IEEE 1394-2000 physical interface circuit 78.
  • the primary selector 62 has a third input for other connections.
  • the control input stream from the computer system 120 is provided through the IEEE 1394-2000 interface circuit 76 to the control circuit 74.
  • the control circuit 74 is coupled to provide control signals to the selectors 58, 60 and 62 and to the mixer 56 to control the operation of the selectors 58, 60 and 62 and the mixer 56. Under the control of the control device 120, the control circuit 74 controls the operation of the selectors 58, 60 and 62 to select one or none of the inputs coupled to each of the selectors 58, 60 and 62.
  • the OSD selector 58 can be controlled to output the input on-screen- display/graphics input data stream or to provide no output
  • the secondary selector 60 can be controlled to output the cable input data stream or to provide no output
  • the primary selector 62 can be controlled to output the input data streams corresponding to the IEEE
  • the selectors 58, 60 and 62 preferably will only output one of the input data streams at a time.
  • the outputs from the selectors 58, 60 and 62 are provided as inputs to the mixer circuit 56.
  • the control circuit 74 controls the operation of the mixer 56 to provide a mix of the input data streams from the selectors 58, 60 and 62.
  • the output of the mixer 56 is then provided to the VRAM 54 to be displayed on the display 52.
  • the control 74 controls the operation of the mixer 56 to combine the input data streams from the selectors 58, 60 and 62 into a combined output data stream to be displayed on the display 52.
  • the input data streams from the source devices coupled to the digital television 50 are selected, mixed and displayed, thereby providing opportunities for overlaying input data streams or displaying multiple input data streams at the same time in a tiled or picture-in-picture arrangement.
  • the output data stream from the mixer 56 is displayed on the display 52 within the receiving device 50. It should be understood by those skilled in the art, that alternatively, the output data stream from the mixer 56 can be stored for later display or transmitted from the receiving device 50 to another device for display or output at that device.
  • the control device 120 controls the operation and display of the digital television 50 to select input data streams at each of the selectors 58, 60 and 62 and to then mix the selected input data streams at the mixer 56 for display on the display 62.
  • the selectors 58, 60 and 62 and the mixer 56 are shown as separate blocks within the digital television 50. It should be understood that the functions of the selectors 58, 60 and 62 and the mixer 56 are preferably implemented within software within the digital television 50. Alternatively, the selectors 58, 60 and 62 and the mixer 56 are preferably implemented within firmware, hardware or some combination of software, firmware and hardware.
  • the control device 120 To provide control inputs to control the operation of the selectors 58, 60 and 62 and the mixer 62, the control device 120 first must discover each of the selectors 58, 60 and 62 and their respective inputs and the mixer 56. To discover the selectors 58, 60 and 62, the mixer 56 and their respective inputs, the control device 120 communicates with the digital television 50 over the IEEE 1394-2000 serial bus network to request this information from the digital television 50. In response to this request for information from the control device 120, the digital television 50 then exposes the functionality of the selectors 58, 60 and 62 and the mixer 56 to the control device 120.
  • the functionality and control of the selectors 58, 60 and 62 is preferably exposed to the control device 120, tlirough an XML document that describes the selectors 58, 60 and 62 and their inputs to the control device 120.
  • An example of such an XML document corresponding to the selectors 58, 60 and 62 of the example illustrated in Figure 3 is included below in Table I:
  • the primary selector 62 is used to select an input data stream from the source devices 80 and 100
  • the secondary selector 60 is used to select the input data stream from the cable/antenna interface 70
  • the on-screen-display selector 58 is designated to overlay on-screen-display/graphics data on the selected input data stream.
  • the "currentValue" tag can be written with any appropriate universal resource locator address (URL).
  • This OSD URL can correspond to a location within the buffer 64, representing on-screen-display/graphics data received from the source devices 80 and 100 or from the control device 120 or to any other appropriate URL, identifying information within the digital television 50 or external to the digital television 50.
  • the OSD URL can correspond to a location within the buffer 64, representing on-screen-display/graphics data received from the source devices 80 and 100 or from the control device 120 or to any other appropriate URL, identifying information within the digital television 50 or external to the digital television 50.
  • URL can also identify a web page to be overlaid on the selected input data stream.
  • the primary selector 62 outputs the input data stream corresponding to the IEEE 1394-2000 input plug A, the input data stream corresponding to the IEEE 1394-2000 input plug B, the input data stream corresponding to the other connections or is turned off and does not output any data stream.
  • the secondary selector 60 outputs the input data stream corresponding to the cable/antenna interface 70 or is turned off and does not output any data stream.
  • the OSD selector 58 outputs the input data stream corresponding to the specified URL or is turned off and does not output any data stream.
  • the mixer 56 is utilized to combine the inputs that have been selected by the selectors
  • the functionality and control of the mixer 56 is preferably exposed to the control device 120, through an XML document that describes the mixer 56 and its inputs to the control device 120.
  • An example of such an XML document corresponding to the mixer 56 of the example illustrated in Figure 3 is included below in Table II:
  • the mixer 56 combines the input data streams from the primary selector 62, the secondary selector 60 and the OSD selector 58 into an output data stream to be displayed on the display 52. Based on the values input to the mixer 56 from the control device 120, the mixer is either turned off or turned on. Preferably, when the mixer 56 is turned off, only the input data stream from the primary selector 62 is output from the mixer 56. When the mixer 56 is not turned off, the mixer 56 combines the input data streams provided from the selectors 58, 60 and 62, according to the values received from the control device 120.
  • the mixer 56 can combine the input data streams in any appropriate manner, under the control of the control device 120. As apparent from the XML document of Table II, the mixer 56 can combine the input data streams from the selectors 58, 60 and 62 in many different combinations, including with the input from the primary selector 62 on top, with the input from the secondary selector
  • the size and placement of the input data streams from the selectors 58, 60 and 62 within the display can also be controlled.
  • FIG. 5 A flowchart showing the preferred steps followed by the control device 120 when controlling the functions of the selectors 58, 60 and 62 and the mixer 56 within the receiving device 50 is illustrated in Figure 5.
  • the process begins at the step 200.
  • the control device sends a request to the receiving device 50 to request the functions and data from the receiving device 50, regarding the selectors and the mixer implemented within the receiving device 50, as well as the inputs to the selectors and the mixer.
  • the control device 120 receives the requested functions and data from the receiving device 50, thereby exposing the interface and functions of the selectors 58, 60 and 62 and the mixer 56 to the control device 120.
  • control device 120 it is determined if the control device 120 has received control input from a user or from a device coupled to the control device 120, regarding the display of the input data streams by the receiving device 50.
  • This control input can be input to the control device 120 by any appropriate means, including through a keyboard, mouse, a remote control or other appropriate device.
  • the control device 120 Once control input regarding the display of the input data streams by the receiving device 50 has been received by the control device 120, the control device 120 then generates control instructions and values based on this control input, at the step 208.
  • the control device 120 transmits these control instructions and values to the receiving device 50. The process then returns to the step 206 to wait until further control inputs are received by the control device.
  • the process begins at the step 250.
  • the functions of the selectors 58, 60 and 62 and the mixer 56 are exposed to the control device 120 at the step 254.
  • control instructions regarding the selection of and mixing of the input data streams has been received from the control device 120. Once it is determined that control instructions have been received from the control device 120 regarding the selection of and mixing of the input data streams at the step 256, then the receiving device 50 provides the control instructions to the selectors 58, 60 and 62 and the mixer 56 to appropriately control the selection and mixing of the input data streams based on the control instructions. The output data stream from the mixer 56 is then displayed by the receiving device 50 on the display 52. The process then returns to the step 256 to wait until further control instructions are received from the control device 120.
  • the digital television 50 selects and mixes input data streams from the source devices 80 and 100, from the cable/antenna interface 70 and on-screen- display/graphics data received from the address specified by a URL.
  • the computer system 120 first requests the functions and data for the selectors 58, 60 and 62 and the mixer 56 from the digital television 50.
  • the digital television 50 then exposes the functions and data representing the selectors 58, 60 and 62, the mixer 56 and their respective inputs to the computer system 120, preferably through the XML documents of Table I and Table ⁇ , as described above.
  • the functions and data representing the selectors, the mixer and their respective inputs are exposed to the computer system 120 in any appropriate manner.
  • the computer system 120 can then provide control information to the digital television 50 to control the output of the input data streams.
  • the computer system 120 provides control information to the digital television 50 to initially select the input data stream from the digital VCR 80 through the primary selector 62, to turn the secondary selector 60 off and to turn the OSD selector 58 off.
  • the mixer 56 is then only provided the input data stream from the primary selector 62 to output and display on the display 52.
  • the computer system 120 If the computer system 120 then receives a control input to display on-screen-display/graphics information, such as the current time, the computer system 120 generates the appropriate control information and provides this control information to the digital television 50 to turn on the OSD selector 58 and select a URL address corresponding to the current time.
  • This input data stream is then output from the OSD selector 58 and provided to the mixer 56.
  • the mixer 56 will then combine the input data stream from the primary selector 62 and from the OSD selector 58 to display the current time with the input data stream from the digital VCR 80 on the display 52.
  • the method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control of the preferred embodiment of the present invention includes a receiving device, a control device and one or more source devices.
  • the receiving device receives input data streams from the source devices.
  • the input data streams are streams of audio/video data and on-screen-display/graphics data.
  • the input data streams are streams of any appropriate data type to be selected and mixed into an output stream by the receiving device.
  • the input data streams are provided as inputs to selectors within the receiving device.
  • the selectors output selective ones of the input data streams to a mixer, based on control input received from a control device.
  • the mixer then combines the selected input data streams into an output stream, based on control input received from the control device.
  • the receiving device exposes the functionality and controls of the selectors and the mixer to the control device, allowing the control device to provide control information to control the selectors and the mixer within the receiving device.
  • the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams. This allows the control device to control which input data streams are output, how the input data streams are output and how the input data streams are combined for output.
  • the display of multiple input data streams can be selected, displayed together in a tiled format or in separate windows, such as a picture- in-picture format, and on-screen-display/graphics data can be overlaid on video input data streams.
  • the output of one or more audio source streams can be selected or the audio source streams can be combined together for output.
  • the combination of the selector and mixer functions of the present invention allow a control device to discover and control these functions within a receiving device using any appropriate protocol, not just the IEEE 1394-2000 serial bus protocol.
  • the interfaces and functions of the selectors and the mixer are exposed to external control devices or control points for external control of those functions within a receiving device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Systems (AREA)
  • Computer And Data Communications (AREA)
  • Studio Circuits (AREA)
  • Small-Scale Networks (AREA)

Abstract

A method and apparatus for enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control of a receiving device, control device and one or more source devices. The receiving device receives input from the source devices to selectors within the receiving device, which output to mixers, that combine the selected input streams to an output stream (Fig 2). As the control device receives control inputs, the control device provides control information to the selectors and mixer within the receiving device to control the output to the input data streams.

Description

A METHOD OF AND APPARATUS FOR SUPPORTING AND ENABLING
THE SELECTION AND MIXING OF MULTIPLE STREAMS OF
AUDIO/VIDEO DATA FROM MULTIPLE SOURCES WITHIN
A RECEIVING DEVICE ALLOWING EXTERNAL CONTROL
FIELD OF THE INVENTION:
The present invention relates to the field of selecting and mixing data at a receiving device. More particularly, the present invention relates to the field of selecting and mixing streams of audio/video data at a receiving device to provide an output stream of data under control from an external device.
BACKGROUND OF THE INVENTION:
The IEEE standard, "IEEE 1394-2000 Standard For A High Performance Serial Bus," Draft ratified in 2000, is an international standard for implementing an inexpensive liigh- speed serial bus architecture which supports both asynchronous and isochronous format data transfers. Isochronous data transfers are real-time transfers which take place such that the time intervals between significant instances have the same duration at both the transmitting and receiving applications. Each packet of data transferred isochronously is transferred in its own time period. An example of an ideal application for the transfer of data isochronously would be from a video recorder to a television set. The video recorder records images and sounds and saves the data in discrete chunks or packets. The video recorder then transfers each packet, representing the image and sound recorded over a limited time period, during that time period, for display by the television set. The IEEE 1394-2000 serial bus architecture provides multiple channels for isochronous data transfer between applications. A six bit channel number is broadcast with the data to ensure reception by the appropriate application.
This allows multiple applications to simultaneously transmit isochronous data across the bus structure. Asynchronous transfers are traditional data transfer operations which take place as soon as possible and transfer an amount of data from a source to a destination.
The IEEE 1394-2000 standard provides a high-speed serial bus for interconnecting digital devices thereby providing a universal I/O connection. The IEEE 1394-2000 standard defines a digital interface for the applications thereby eliminating the need for an application to convert digital data to analog data before it is transmitted across the bus. Correspondingly, a receiving application will receive digital data from the bus, not analog data, and will therefore not be required to convert analog data to digital data. The cable required by the IEEE 1394-2000 standard is very thin in size compared to other bulkier cables used to connect such devices. Devices can be added and removed from an IEEE 1394-2000 bus while the bus is active. If a device is so added or removed the bus will then automatically reconfigure itself for transmitting data between the then existing nodes. A node is considered a logical entity with a unique address on the bus structure. Each node provides a configuration ROM, a standardized set of control registers and its own address space. Because of these advantages the IEEE 1394-2000 standard provides for a unique networking structure that is capable of incorporating audio/video devices, media play/record devices, computing devices and display devices.
The IEEE 1394-2000 standard defines a protocol as illustrated in Figure 1. This protocol includes a serial bus management block 10 coupled to a transaction layer 12, a link layer 14 and a physical layer 16. The physical layer 16 provides the electrical and mechanical connection between a device or application and the IEEE 1394-2000 cable. The physical layer 16 also provides arbitration to ensure that all devices coupled to the IEEE 1394-2000 bus have access to the bus as well as actual data transmission and reception. The link layer 14 provides data packet delivery service for both asynchronous and isochronous data packet transport. This supports both asynchronous data transport, using an acknowledgement protocol, and isochronous data transport, providing real-time guaranteed bandwidth protocol for just-in-time data delivery. The transaction layer 12 supports the commands necessary to complete asynchronous data transfers, including read, write and lock. The transaction layer 12 also provides a path for isochronous management data to be transferred to the serial bus management block 10 via read operations with isochronous control compare-swap registers. The serial bus management block 10 contains an isoclironous resource manager for managing isochronous data transfers. The serial bus management block 10 also provides overall configuration control of the serial bus in the form of optimizing arbitration timing, guarantee of adequate electrical power for all devices on the bus, assignment of the cycle master, assignment of isoclironous channel and bandwidth resources and basic notification of errors.
A diverse range of products can be implemented with the ability to connect to an IEEE 1394-2000 serial bus network. These devices can have capabilities and functionality ranging from very simple to very complex. Specifically, a variety of audio/video devices, media play/record devices and computing/display devices are capable of being linked together over an IEEE 1394-2000 serial bus networking structure to support asynchronous and isochronous data transfers between the devices.
The IEEE 1394-2000 cable environment is a network of nodes connected by point-to- point links, including a port on each node's physical connection and the cable between them. The physical topology for the cable environment of an IEEE 1394-2000 serial bus is a non- cyclic network of multiple ports, with finite branches. The primary restriction on the cable environment is that nodes must be connected together without forming any closed loops.
A block diagram of a conventional home audio/video network including a television and an analog video cassette recorder (VCR) is illustrated in Figure 2. The television 20 is coupled to the analog VCR 40. Video data and associated data are sent between the analog
VCR 40 and the television 20 in a known manner.
Relevant internal components of the television 20 and the analog VCR 40 are also illustrated in Figure 2. The television 20 includes an interface 32 which sends and receives audio and video signals to and from the analog VCR 40. The interface 32 is coupled to an audio/video switch 26 for directing audio/video signals to and from the analog VCR 40. A cable/antenna interface circuit 30 is coupled to receive input signals from a coaxial cable or an antenna and to pass those signals through a tuner 28 to the audio/video switch 26. The audio/video switch 26 is coupled to a video random access memory (VRAM) circuit 24 for providing the video signals from the cable/antenna interface 30 or the analog VCR 40 to the display 22.
The analog VCR 40 includes a video source 46, such as a video tape which is being played by the analog VCR 40 or a television input. The analog VCR 40 also includes a graphics source 48 which generates on-screen-display graphics to be displayed by the television 20 when the analog VCR 40 is sending data to be displayed by the television 20. Such on-screen-display graphics include words or symbols representing commands being executed by the analog VCR 40, such as PLAY, STOP, REWIND, FAST-FORWARD, PAUSE and RECORD. The on-screen-display graphics generated by the graphics source 48 typically also include a menu or menus displayed on the television 20 for the user which allow the user to program the analog VCR 40. On-screen-display graphics generated by the graphics source 48 are provided to a mixer circuit 44. The mixer circuit 44 also receives video data from the video source 46. The mixer circuit 44 then combines the on-screen- display graphics from the graphics source 48 and the video data from the video source 46 into a video output which is transmitted through the interface circuit 42 to the television 20. The video output from the analog VCR 40 is then processed by the television 20 and shown on the display 22. The mixer circuit 44 will, as appropriate, either overlay the on-screen-display graphics onto the video data to form the video output or cause the video output to include only on-screen-display graphics from the graphics source 48 or video data from the video source 46. i an audio/video network including a digital television and a digital VCR coupled together by an IEEE 1394-1995 serial bus network, the video data from the digital VCR is typically transmitted in a digital format such as MPEG. The digital VCR is not typically responsible for encoding the video data in an MPEG format, but will record and transmit data previously encoded in a MPEG format by another source. In order for the digital VCR to have the ability to combine on-screen-display graphics with the video data, as described above, the digital VCR would have to include an MPEG encoder and have the ability to encode the on-screen-display graphics into an MPEG format and then combine the streams of data into a video output stream of data. Due to the cost of MPEG encoders, such a requirement is cost prohibitive in competitive consumer VCRs.
Some video source devices have the ability to generate a very high bandwidth graphics output which is sent to a display device, such as a television, for on-screen-display to the user. This graphics output can be of very high resolution and color depth, including dynamic animation effects with multiple portions of the graphics data changing on a frequent basis. This graphics data is typically mixed with, or overlaid on top of, a video signal. As described above, it is relatively easy and inexpensive to provide capability within an analog device to combine on-screen-display graphics with a video stream of data and send the combined stream of data to a television or other display device for display to a user or users. It is also relatively easy and inexpensive for a device, such as a video game console to generate a pure graphics output stream and send this output stream over an analog video connection to the television. However, for digital devices, it is quite expensive, to include within the device, the capability to generate graphics data or on-screen-display data and encode this data into a format such as MPEG video in which digital data is typically transmitted. It is cost prohibitive for most consumer devices such as VCRs and video game consoles to include the ability to generate on-screen-display graphics for combination with video and/or transmission to a digital display device as an MPEG stream of video. The EIA-775, DTV 1394 Interface Specification, draft November 30, 1998, is a standard which defines a specification for a baseband digital interface to a digital television that provides a level of functionality that is similar to the analog system illustrated in Figure 2 and described above. Within the EIA-775 specification, an audio/video source capable of producing analog audio and video and also an MPEG transport stream, is coupled to a digital television receiver. The analog signal is transmitted over a standard coaxial cable and the
MPEG data is transmitted isochronously over an IEEE 1394 serial bus. Bitmaps of the source on-screen-display data are sent by asynchronous connections separately over the IEEE 1394 serial bus and are mixed with the decoded MPEG video in the digital television prior to being presented on the display. This process removes the need for the audio/video source to perform the additional MPEG video decodes and re-encodes, described above. However, the process of the EIA-775 specification, requires the audio/video source and the digital television to be connected together by the IEEE 1394 serial bus. The process of the EIA-775 specification does not work with connections other than the IEEE 1394 serial bus.
SUMMARY OF THE INVENTION:
A method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control includes a receiving device, a control device and one or more source devices. The receiving device receives input data streams from the source devices. The input data streams are provided as inputs to selectors within the receiving device. The selectors output selective ones of the input data streams to a mixer, based on control information received from a control device. The mixer then combines the selected input data streams into an output stream, based on control information received from the control device. The interfaces and functions of the selectors and the mixer are provided to the control device in response to a request from the control device. As the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams.
In one aspect of the present invention, a method of providing an output data stream comprises receiving one or more input data streams from one or more source devices at a receiving device, receiving control information from a control device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, selecting the selected input data streams based on the control information from the control device and mixing the selected input data streams into the output data stream based on the control information from the control device. The method further comprises sending a request from the control device to the receiving device regarding selection and mixing functions and controls at the receiving device and providing the selection and mixing functions and controls from the receiving device to the control device. The input data streams include one or more of audio/video data streams, on-screen- display data streams and graphics data streams. The control device is preferably external of the receiving device. Alternatively, the control device is internal to the receiving device. The method further comprises outputting the output data stream. The method further comprises receiving control input at the control device and generating the control information based on the control input. The one or more input streams are received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard. The control information is received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
In a further aspect of the present invention, a method of controlling a receiving device providing a combination of one or more input data streams as an output data stream comprises sending a request from a control device to the receiving device regarding selection and mixing functions and controls at the receiving device, providing the selection and mixing functions and controls from the receiving device to the control device and providing control information from the control device to the receiving device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, wherein the selected input data streams are selected and mixed based on the control information. The method further comprises receiving the one or more input data streams from one or more source devices at the receiving device, selecting the selected input data streams based on the control information from the control device and mixing the selected input data streams into the output data stream based on the control information from the control device. The method further comprises outputting the output data stream. The input data streams include one or more of audio/video data streams, on- screen-display data streams and graphics data streams. Preferably, the control device is external of the receiving device. The method further comprises receiving control information at the control device and generating the control information based on the control input. The one or more input streams are received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard. The control information is received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
In still a further aspect of the present invention, a receiving device for providing an output data stream comprises means for receiving one or more input data streams configured for coupling to one or more source devices, means for receiving control information configured for coupling to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, means for selecting coupled to the means for receiving one or more input data streams and the means for receiving control information for selecting the selected input data streams based on the control information from the control device and means for mixing coupled to the means for selecting and to the means for receiving control information for mixing the selected input data streams into the output data stream based on the control information from the control device. The means for receiving control information provides selection and mixing functions and controls to the control device in response to a request from the control device. The receiving device further comprises means for outputting coupled to the means for mixing for outputting the output data stream. The input data streams include one or more of audio/video data streams, onscreen-display data streams and graphics data streams. Preferably, the control device is external of the receiving device. Alternatively, the control device is internal of the receiving device. The means for receiving one or more input data streams is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug- and-play standard. The means for receiving control information is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a com ection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug- and-play standard.
In a further aspect of the present invention, a receiving device to provide an output data stream comprises a first interface to receive one or more input data streams configured to couple to one or more source devices, a second interface to receive control information configured to couple to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, one or more selectors coupled to the first interface and to the second interface to select the selected input data streams based on the control information from the control device and a mixer coupled to the selectors and the second interface to mix the selected input data streams into the output data stream based on the control information from the control device. The second interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device. The receiving device further comprises an output device coupled to the mixer to output the output data stream. The input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams. Preferably, the control device is external to the receiving device. Alternatively, the control device is internal to the receiving device. The first interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
The second interface is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard. In still a further aspect of the present invention, a control device to provide control information to a receiving device receiving one or more input data streams to control selection and mixing of the input data streams into an output data stream comprises an interface configured to couple to the receiving device to request selection and mixing function and control information from the receiving device and a control circuit coupled to the interface to receive the selection and mixing function and control information and to provide output control information to the receiving device, wherein the output control information includes information regarding selected ones of the one or more input data streams and a selected combination of the selected ones of the one or more input data streams. The input data streams include one or more of audio/video data streams, on-screen- display data streams and graphics data streams. Preferably, the receiving device is external to the control device. Alternatively, the receiving device is internal to the control device. The interface is coupled to the receiving device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
In yet another aspect of the present invention a network of devices to provide an output data stream comprises one or more source devices configured to provide one or input data streams, a control device configured to provide control information regarding selection and mixing of the input data streams into the output data stream, the control device including a control interface configured to provide the control information and a control circuit coupled to the control interface to generate the control information and a receiving device comprising a first receiving interface coupled to the source devices to receive the one or more input data streams, a second receiving interface coupled to the control interface to receive the control information, one or more selectors coupled to the first receiving interface and the second receiving interface to select selected input data streams based on the control information and
- 1 - a mixer coupled to the selectors and the second receiving interface to mix the selected input data streams into the output data stream based on the control information from the control device. The second receiving interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device. The receiving device further comprises an output device coupled to the mixer to output the output data stream. The input data streams include one or more of audio/video data streams, onscreen-display data streams and graphics data streams. Preferably, the control device is external to the receiving device. Alternatively, the control device and the receiving device are integrated together. The first receiving interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard. The second receiving interface is coupled to control interface by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
BRIEF DESCRIPTION OF THE DRAWINGS:
Figure 1 illustrates a protocol defined by the IEEE 1394-2000 standard. Figure 2 illustrates a block diagram of a conventional home audio/video network including a television and VCR.
Figure 3 illustrates a functional block diagram of an exemplary network of devices according to the present invention, including a first digital VCR, a second digital VCR, a computer system and a digital television. Figure 4 illustrates a block diagram of the internal components of the computer system 120.
Figure 5 illustrates a flowchart showing the preferred steps followed by a control device when controlling the functions of selectors and a mixer within a receiving device.
Figure 6 illustrates a flowchart showing the preferred steps followed by a receiving device to provide an output data stream based on control instructions received from a control device.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT:
The method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control of the preferred embodiment of the present invention includes a receiving device, a control device and one or more source devices. The receiving device receives input data streams from the source devices. Preferably, the input data streams are streams of audio/video data and on-screen-display/graphics data. Alternatively, the input data streams are streams of any appropriate data type to be selected and mixed into an output stream by the receiving device. The input data streams are provided as inputs to selectors within the receiving device. The selectors output selective ones of the input data streams to a mixer, based on control input received from a control device. The selectors are turned off or on. When a selector is turned off, the selector will not output any input data stream. When a selector is turned on, the selector will output a selected one of the input data streams provided as inputs to the selector. The outputs from the selectors are provided to a mixer. The mixer then combines the selected input data streams into an output stream, based on control input received from the control device. The mixer can select one of the input data streams or can output some combination of the input data streams. Upon request from the control device, the receiving device exposes the functionality and controls of the selectors and the mixer to the control device, allowing the control device to provide control information to control the selectors and the mixer within the receiving device. As the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams. The control device receives the control inputs from any appropriate input devices including a remote control, keyboard, mouse or other devices coupled to the control device either by a wired or wireless connection. This allows the control device to control which input data streams are output, how the input data streams are output and how the input data streams are combined for output, if appropriate. Accordingly, for video data streams, the display of multiple input data streams can be selected, displayed together in a tiled format or in separate windows, such as a picture-in-picture format, and on-screen- display/graphics data can be overlaid on video input data streams. For audio data streams, the output of one or more audio source streams can be selected or the audio source streams can be combined together for output. A functional block diagram of an exemplary network of devices including a first digital VCR 80, a second digital VCR 100, a computer system 120 and a digital television 50 is illustrated in Figure 3. Relevant functional components of the first digital VCR 80, the second digital VCR 100 and the digital television 50 are also illustrated in Figure 3. The digital television 50 includes an IEEE 1394-2000 interface circuit 76 through which data and commands are sent to and received from the digital VCR 80, the digital VCR 100 and the computer 120, over the IEEE 1394-2000 serial bus cables 126, 128 and 130, respectively. The IEEE 1394-2000 interface circuit 76 includes a physical interface circuit 78. At the physical interface circuit 78, the IEEE 1394-2000 serial bus cables 126, 128 and 130 are coupled to the plug A, the plug B and the plug C, respectively. The IEEE 1394-2000 interface circuit 76 is coupled to a control circuit 74. The control circuit 74 is coupled to provide audio/video data received from the digital VCRs 80 and 100 on the plugs A and B, to a decoder circuit 72. The control circuit 74 is coupled to provide on-screen-display and graphics data to a decompressor circuit 66. The control circuit 74 is also coupled to an on-screen-display selector 58, a secondary selector 60, a primary selector 62 and a mixer 56 to provide control information to the on-screen-display selector 58, the secondary selector 60, the primary selector 62 and the mixer 56.
The decoder circuit 72 includes an output A and an output B, which are both coupled to inputs of the primary selector 62. The primary selector 62 includes a third input which is configured for coupling to another connection. The decompressor circuit 66 is coupled to provide decompressed on-screen-display and graphics data to a buffer 64. The buffer 64 is coupled to an input of the on-screen-display selector 58.
The digital television 50 includes a cable/antenna interface circuit 70 which is coupled to receive input signals from a coaxial cable or an antenna and to pass those signals through a tuner 68 to an input of the secondary selector 60. The outputs of the on-screen-display selector 58, the secondary selector 60 and the primary selector 62 are coupled to inputs of the mixer 56. An output of the mixer 56 is coupled to a VRAM circuit 54 for providing video and on-screen-display and graphics data to the display 52.
The digital VCR 80 includes an IEEE 1394-2000 interface circuit 88 through which data and commands are sent to and received from the digital television 50. The IEEE 1394- 2000 interface circuit 88 includes a physical interface circuit 90. The physical interface circuit 90 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 126. The digital VCR 80 includes a video source 82, such as a video tape, which is being played by the VCR 80, or a television input. The video source 82 is coupled to the IEEE 1394-2000 interface circuit 88 for transmitting video streams of data over the IEEE 1394- 2000 serial bus cable 126 to the digital television 50. The digital VCR 80 also includes a graphics source 84 which generates on-screen-display graphics to be displayed by the television 50. The graphics source 84 is preferably coupled to a compression circuit 86 which compresses the graphics data generated by the graphics source 84. The compression circuit 86 is coupled to the IEEE 1394-2000 interface circuit 88 for transmitting the compressed graphics data over the IEEE 1394-2000 serial bus cable 126 to the digital television 50.
The digital VCR 100 includes an IEEE 1394-2000 interface circuit 108 through which data and commands are sent to and received from the digital television 50. The IEEE 1394- 2000 interface circuit 108 includes a physical interface circuit 110. The physical interface circuit 110 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 128. The digital VCR 100 includes a video source 102, such as a video tape, which is being played by the digital VCR 100, or a television input. The video source 102 is coupled to the IEEE 1394-2000 interface circuit 108 for transmitting video streams of data over the IEEE 1394-2000 serial bus cable 128 to the digital television 50. The digital VCR 100 also includes a graphics source 104 which generates on-screen-display graphics to be displayed by the digital television 50. The graphics source 104 is preferably coupled to a compression circuit 106 which compresses the graphics data generated by the graphics source 104. The compression circuit 106 is coupled to the IEEE 1394-2000 interface circuit 108 for transmitting the compressed graphics data over the IEEE 1394-2000 serial bus cable 128 to the digital television 50. The computer system 120 includes an IEEE 1394-2000 interface circuit 122 through which data and commands are sent to and received from the digital television 50. The IEEE 1394-2000 interface circuit 122 includes a physical interface circuit 124. The physical interface circuit 124 is coupled to the digital television 50, over the IEEE 1394-2000 serial bus cable 130. The additional details and exemplary components of the computer system 120 will be described below in detail in reference to Figure 4.
The configuration illustrated in Figure 3 is exemplary only. It should be apparent that an audio/video network could include many different combinations of components. It should also be recognized that data, commands and parameters can be sent in either direction between the devices within the IEEE 1394-2000 network, as appropriate. A block diagram of the internal components of the computer system 120 is illustrated in Figure 4. The computer system 120 includes a central processor unit (CPU) 156, a main memory 150, a video memory 158, a mass storage device 152 and an IEEE 1394-2000 interface circuit 122, all coupled together by a conventional bidirectional system bus 154. The interface circuit 122 includes the physical interface circuit 124 for sending and receiving communications over the IEEE 1394-2000 serial bus. The physical interface circuit 124 is coupled to the digital television 50 over the IEEE 1394-2000 serial bus cable 130. In the preferred embodiment of the present invention, the interface circuit 122 is implemented on an IEEE 1394-2000 interface card within the computer system 120. However, it should be apparent to those skilled in the art that the interface circuit 122 can be implemented within the computer system 120 in any other appropriate manner, including building the interface circuit onto the motherboard itself. The mass storage device 152 may include both fixed and removable media using any one or more of magnetic, optical or magneto-optical storage technology or any other available mass storage technology. The system bus 154 contains an address bus for addressing any portion of the memory 150 and 158. The system bus 154 also includes a data bus for transferring data between and among the CPU 156, the main memory
150, the video memory 158, the mass storage device 152 and the interface circuit 122.
The computer system 120 is also coupled to a number of peripheral input and output devices including the keyboard 166, the mouse 168 and the associated display 164. The keyboard 166 is coupled to the CPU 156 for allowing a user to input data and control commands into the computer system 120. A conventional mouse 168 is coupled to the keyboard 166 for manipulating graphic images on the display 164 as a cursor control display. It should be apparent to those skilled in the art, that the mouse 168 could also be coupled to the CPU 156.
A port of the video memory 158 is coupled to a video multiplex and shifter circuit 160, which in turn is coupled to a video amplifier 162. The video amplifier 162 drives the display 164. The video multiplex and shifter circuitry 160 and the video amplifier 162 convert pixel data stored in the video memory 158 to raster signals suitable for use by the display 164. The digital television 50 is a receiving device, which receives input streams of data from source devices coupled to the digital television 50. In the example illustrated in Figure 3, the source devices 80 and 100 are coupled to the digital television 50 by the IEEE 1394- 2000 cables 126 and 128 and the cable input is coupled to the cable/antenna interface 70. It should be apparent however that the source devices 80 and 100 could be coupled to the digital television 50 by any appropriate com ection means including wired and wireless connections and by any appropriate protocol, including other versions of the IEEE 1394 protocol, universal serial bus, and universal plug-and-play. It should also be apparent that input streams of data could originate from sources within the digital television 50. hi the example illustrated in Figure 3, the computer system 120 is a control device for controlling the operation of the selectors 58, 60 and 62 and the mixer 56 of the present invention, within the digital television 50. h the example illustrated in Figure 3, the control device 120 is coupled to the digital television 50 by the IEEE 1394-2000 cable 130. It should also be apparent however that the control device 120 could be coupled to the digital television 50 by any appropriate connection means including wired and wireless connections and by any appropriate protocol, including other versions of the IEEE 1394 protocol, universal serial bus, and universal plug-and-play. Also, in the example illustrated in Figure 3, the control device 120 is an external device implemented by a computer system. It should be apparent to those skilled in the art that the control device can be any appropriate connected device with the capability to discover and control the selectors 58, 60 and 62 and the mixer 56, including an external control device, an internal control device, a source device and a remote control device.
The input data streams from the sources coupled to the digital television 50 are provided as inputs to the digital television 50. The cable input is provided through the cable/antenna interface 70 to the tuner 68. After being tuned by the tuner 68, the cable input is then provided as an input to the secondary selector 60. The input data streams from the source devices coupled to the digital television through the IEEE 1394-2000 serial bus network are provided through the IEEE 1394 interface circuit 76 to the control circuit 74. The control circuit 74 is coupled to provide the on-screen-display/graphics input data streams to the decompressor 66. The decompressor 66 decompresses the on-screen-display/graphics input data streams and provides the data to the buffer 64. From the buffer 64, the decompressed on-screen-display/graphics input data is provided as an input to the OSD selector 58. As will be discussed in detail below, the OSD selector 58 selects input data to be output based on a URL address. The control circuit 74 is coupled to provide encoded audio/video data from the IEEE 1394-2000 source devices to the decoder 72. The decoder 72 then decodes the encoded audio/video data and provides the decoded data from the outputs A and B as inputs to the primary selector 62. The outputs A and B from the decoder 72 correspond to the input plugs A and B of the IEEE 1394-2000 physical interface circuit 78. The primary selector 62 has a third input for other connections. The control input stream from the computer system 120 is provided through the IEEE 1394-2000 interface circuit 76 to the control circuit 74. The control circuit 74 is coupled to provide control signals to the selectors 58, 60 and 62 and to the mixer 56 to control the operation of the selectors 58, 60 and 62 and the mixer 56. Under the control of the control device 120, the control circuit 74 controls the operation of the selectors 58, 60 and 62 to select one or none of the inputs coupled to each of the selectors 58, 60 and 62. In the example of Figure 3, the OSD selector 58 can be controlled to output the input on-screen- display/graphics input data stream or to provide no output, the secondary selector 60 can be controlled to output the cable input data stream or to provide no output, and the primary selector 62 can be controlled to output the input data streams corresponding to the IEEE
1394-2000 input plugs A and B, the other connections or to provide no output. When not turned off, the selectors 58, 60 and 62 preferably will only output one of the input data streams at a time.
The outputs from the selectors 58, 60 and 62 are provided as inputs to the mixer circuit 56. Under the control of the control device 120, the control circuit 74 controls the operation of the mixer 56 to provide a mix of the input data streams from the selectors 58, 60 and 62. The output of the mixer 56 is then provided to the VRAM 54 to be displayed on the display 52. Under the control of the control device 120, the control 74 controls the operation of the mixer 56 to combine the input data streams from the selectors 58, 60 and 62 into a combined output data stream to be displayed on the display 52. In this manner, the input data streams from the source devices coupled to the digital television 50 are selected, mixed and displayed, thereby providing opportunities for overlaying input data streams or displaying multiple input data streams at the same time in a tiled or picture-in-picture arrangement. Preferably, the output data stream from the mixer 56 is displayed on the display 52 within the receiving device 50. It should be understood by those skilled in the art, that alternatively, the output data stream from the mixer 56 can be stored for later display or transmitted from the receiving device 50 to another device for display or output at that device.
Utilizing the selectors and mixer of the present invention, the control device 120 controls the operation and display of the digital television 50 to select input data streams at each of the selectors 58, 60 and 62 and to then mix the selected input data streams at the mixer 56 for display on the display 62. In the example of Figure 3, the selectors 58, 60 and 62 and the mixer 56 are shown as separate blocks within the digital television 50. It should be understood that the functions of the selectors 58, 60 and 62 and the mixer 56 are preferably implemented within software within the digital television 50. Alternatively, the selectors 58, 60 and 62 and the mixer 56 are preferably implemented within firmware, hardware or some combination of software, firmware and hardware.
To provide control inputs to control the operation of the selectors 58, 60 and 62 and the mixer 62, the control device 120 first must discover each of the selectors 58, 60 and 62 and their respective inputs and the mixer 56. To discover the selectors 58, 60 and 62, the mixer 56 and their respective inputs, the control device 120 communicates with the digital television 50 over the IEEE 1394-2000 serial bus network to request this information from the digital television 50. In response to this request for information from the control device 120, the digital television 50 then exposes the functionality of the selectors 58, 60 and 62 and the mixer 56 to the control device 120.
The functionality and control of the selectors 58, 60 and 62 is preferably exposed to the control device 120, tlirough an XML document that describes the selectors 58, 60 and 62 and their inputs to the control device 120. An example of such an XML document corresponding to the selectors 58, 60 and 62 of the example illustrated in Figure 3 is included below in Table I:
Selector:
<selectors>
<selector> <name>primaryS elector</name>
<dataType>string</dataType> <defaultValue>default value</defaultValue> <currentNalue>current value</currentNalue> <allowedNalueList> <allowedNalue>1394 Plug A</allowedNalue>
<allowedNalue>1394 Plug B</allowedNalue> <allowedNalue>Other Connections</allowedNalue> <allowedNalue>Off</allowedNalue> </allowedNalueList> </selector>
<selector>
<name>secondarySelector</name> <dataType>string</dataType> <defaultValue>default value</defaultValue> <currentNalue>current value</currentValue>
<allowedValueList>
<allowedValue>cable input</allowedValue> <allowedValue>Off</allowedValue>
</allowedValueList> </selector>
<selector>
<name>OSDSelector</name> <dataType>string</dataType> <defaultValue>default value</defaultValue> <currentValue>current value</currentValue>
<allowedValueList>
<allowedValue>URL</allowedValue> <allowedValue>Off</allowedValue>
</allowedValueList> </selector>
</selectors> TABLE I
hi the example of Figure 3 and of Table I, the primary selector 62 is used to select an input data stream from the source devices 80 and 100, the secondary selector 60 is used to select the input data stream from the cable/antenna interface 70 and the on-screen-display selector 58 is designated to overlay on-screen-display/graphics data on the selected input data stream. Within the OSD selector, the "currentValue" tag can be written with any appropriate universal resource locator address (URL). This OSD URL can correspond to a location within the buffer 64, representing on-screen-display/graphics data received from the source devices 80 and 100 or from the control device 120 or to any other appropriate URL, identifying information within the digital television 50 or external to the digital television 50. The OSD
URL can also identify a web page to be overlaid on the selected input data stream.
In the example of Figure 3 and of Table I, the primary selector 62 outputs the input data stream corresponding to the IEEE 1394-2000 input plug A, the input data stream corresponding to the IEEE 1394-2000 input plug B, the input data stream corresponding to the other connections or is turned off and does not output any data stream. The secondary selector 60 outputs the input data stream corresponding to the cable/antenna interface 70 or is turned off and does not output any data stream. The OSD selector 58 outputs the input data stream corresponding to the specified URL or is turned off and does not output any data stream. The mixer 56 is utilized to combine the inputs that have been selected by the selectors
58, 60 and 62. The functionality and control of the mixer 56 is preferably exposed to the control device 120, through an XML document that describes the mixer 56 and its inputs to the control device 120. An example of such an XML document corresponding to the mixer 56 of the example illustrated in Figure 3 is included below in Table II:
Video Mixer:
<process>
<name>Mixer</name> <dataType>string</dataType> <defaultValue>default</defaultValue>
<currentValue>current</currentValue> <allowedValueList>
<allowedValue>Off</allowedValue>
<allowedValue>Primary on top</allowedValue> <control>
<name>windowControl</name> <dataType>string</dataType> <defaultValue>default</defaultValue> <currentValue>current</currentValue> <allowedValueList>
<allowedValue>maximum</allowedValue> <allowedValue>minimum</allowedValue> <allowedValue>size</allowedValue> <allowedValue>restore</allowedValue> </allowedValueList>
</control> <allowedValue>Secondary on top</allowedValue> <control>
<name>windowControl</name> <dataType>string</dataType>
<defaultValue>default</defaultValue> <currentValue>current</currentValue> <allowedValueList>
<allowedValue>maxinιum</allowedValue> <allowedValue>minimum</allowedValue>
<allowedValue>size</allowedValue> <allowedValue>restore</allowedValue> </allowedValueList> </control> <allowedValue>OSD selector on top</allowedValue> <control> <name>windowControl</name>
<dataType>string</dataType> <defaultValue>default</defaultValue> <currentValue>current</currentValue> <allowedValueList> <allowedValue>maximum</allowedValue>
<allowedValue>minimum</allowedValue> <allowedValue>size</allowedValue> <allowedValue>restore</allowedValue> </allowedValueList> </control>
</allowedValueList> </process> TABLE π
The mixer 56 combines the input data streams from the primary selector 62, the secondary selector 60 and the OSD selector 58 into an output data stream to be displayed on the display 52. Based on the values input to the mixer 56 from the control device 120, the mixer is either turned off or turned on. Preferably, when the mixer 56 is turned off, only the input data stream from the primary selector 62 is output from the mixer 56. When the mixer 56 is not turned off, the mixer 56 combines the input data streams provided from the selectors 58, 60 and 62, according to the values received from the control device 120. The mixer 56 can combine the input data streams in any appropriate manner, under the control of the control device 120. As apparent from the XML document of Table II, the mixer 56 can combine the input data streams from the selectors 58, 60 and 62 in many different combinations, including with the input from the primary selector 62 on top, with the input from the secondary selector
60 on top and with the input from the OSD selector 58 on top. Also, in these combinations, the size and placement of the input data streams from the selectors 58, 60 and 62 within the display can also be controlled.
There is also preferably coordination between the OSD selector 58 and the mixer 56, such that when the OSD selector 58 is turned off, the mixer 56 will not attempt to perform an overlay function.
A flowchart showing the preferred steps followed by the control device 120 when controlling the functions of the selectors 58, 60 and 62 and the mixer 56 within the receiving device 50 is illustrated in Figure 5. The process begins at the step 200. At the step 202, the control device sends a request to the receiving device 50 to request the functions and data from the receiving device 50, regarding the selectors and the mixer implemented within the receiving device 50, as well as the inputs to the selectors and the mixer. At the step 204, the control device 120 receives the requested functions and data from the receiving device 50, thereby exposing the interface and functions of the selectors 58, 60 and 62 and the mixer 56 to the control device 120. At the step 206, it is determined if the control device 120 has received control input from a user or from a device coupled to the control device 120, regarding the display of the input data streams by the receiving device 50. This control input can be input to the control device 120 by any appropriate means, including through a keyboard, mouse, a remote control or other appropriate device. Once control input regarding the display of the input data streams by the receiving device 50 has been received by the control device 120, the control device 120 then generates control instructions and values based on this control input, at the step 208. At the step 210, the control device 120 then transmits these control instructions and values to the receiving device 50. The process then returns to the step 206 to wait until further control inputs are received by the control device.
A flowchart showing the preferred steps followed by the receiving device 50 to provide an output data stream based on control instructions received from the control device 120, is illustrated in Figure 6. The process begins at the step 250. At the step 252, it is determined if a request for functions and data has been received from the control device 120. Once it is determined at the step 252 that a request for functions and data has been received from the control device 120, then the functions of the selectors 58, 60 and 62 and the mixer 56 are exposed to the control device 120 at the step 254.
At the step 256, it is determined if control instructions regarding the selection of and mixing of the input data streams has been received from the control device 120. Once it is determined that control instructions have been received from the control device 120 regarding the selection of and mixing of the input data streams at the step 256, then the receiving device 50 provides the control instructions to the selectors 58, 60 and 62 and the mixer 56 to appropriately control the selection and mixing of the input data streams based on the control instructions. The output data stream from the mixer 56 is then displayed by the receiving device 50 on the display 52. The process then returns to the step 256 to wait until further control instructions are received from the control device 120.
As an example of the operation of the present invention within the audio/video network illustrated in Figure 3, the digital television 50 selects and mixes input data streams from the source devices 80 and 100, from the cable/antenna interface 70 and on-screen- display/graphics data received from the address specified by a URL. Regarding the example illustrated in Figure 3, the computer system 120 first requests the functions and data for the selectors 58, 60 and 62 and the mixer 56 from the digital television 50. The digital television 50 then exposes the functions and data representing the selectors 58, 60 and 62, the mixer 56 and their respective inputs to the computer system 120, preferably through the XML documents of Table I and Table π, as described above. Alternatively, the functions and data representing the selectors, the mixer and their respective inputs are exposed to the computer system 120 in any appropriate manner.
Once the computer system 120 has been exposed to the functions and data representing the selectors 58, 60 and 62 and the mixer 56, the computer system 120 can then provide control information to the digital television 50 to control the output of the input data streams. As an example, the computer system 120 provides control information to the digital television 50 to initially select the input data stream from the digital VCR 80 through the primary selector 62, to turn the secondary selector 60 off and to turn the OSD selector 58 off. The mixer 56 is then only provided the input data stream from the primary selector 62 to output and display on the display 52. If the computer system 120 then receives a control input to display on-screen-display/graphics information, such as the current time, the computer system 120 generates the appropriate control information and provides this control information to the digital television 50 to turn on the OSD selector 58 and select a URL address corresponding to the current time. This input data stream is then output from the OSD selector 58 and provided to the mixer 56. The mixer 56 will then combine the input data stream from the primary selector 62 and from the OSD selector 58 to display the current time with the input data stream from the digital VCR 80 on the display 52.
The method of and apparatus for supporting and enabling the selection and mixing of multiple input data streams from multiple sources within a receiving device allowing external control of the preferred embodiment of the present invention includes a receiving device, a control device and one or more source devices. The receiving device receives input data streams from the source devices. Preferably, the input data streams are streams of audio/video data and on-screen-display/graphics data. Alternatively, the input data streams are streams of any appropriate data type to be selected and mixed into an output stream by the receiving device. The input data streams are provided as inputs to selectors within the receiving device. The selectors output selective ones of the input data streams to a mixer, based on control input received from a control device. The mixer then combines the selected input data streams into an output stream, based on control input received from the control device. Upon request from the control device, the receiving device exposes the functionality and controls of the selectors and the mixer to the control device, allowing the control device to provide control information to control the selectors and the mixer within the receiving device. As the control device receives control inputs, the control device provides control information to the selectors and the mixer within the receiving device to control the output of the input data streams. This allows the control device to control which input data streams are output, how the input data streams are output and how the input data streams are combined for output. Accordingly, for video data streams, the display of multiple input data streams can be selected, displayed together in a tiled format or in separate windows, such as a picture- in-picture format, and on-screen-display/graphics data can be overlaid on video input data streams. For audio data streams, the output of one or more audio source streams can be selected or the audio source streams can be combined together for output. The combination of the selector and mixer functions of the present invention, allow a control device to discover and control these functions within a receiving device using any appropriate protocol, not just the IEEE 1394-2000 serial bus protocol. Using the present invention, the interfaces and functions of the selectors and the mixer are exposed to external control devices or control points for external control of those functions within a receiving device.
The present invention has been described in terms of specific embodiments incorporating details to facilitate the understanding of principles of construction and operation of the invention. Such reference herein to specific embodiments and details thereof is not intended to limit the scope of the claims appended hereto. It will be apparent to those skilled in the art that modifications may be made in the embodiment chosen for illustration without departing from the spirit and scope of the invention. Specifically, it will be apparent to those skilled in the art that while the illustrated embodiment utilizes an IEEE 1394-2000 serial bus structure, the present invention could also be implemented on any other appropriate digital interfaces or bus structures, or with any other appropriate protocols, including other or later versions of the IEEE 1394 serial bus, other local area network protocols or device connection protocols, including current or later versions of the USB protocol and universal plug-and-play protocol. It will also be apparent to those skilled in the art that while the illustrated embodiment utilizes video input data streams, the present invention could also be implemented with any appropriate input data streams, including audio data streams.

Claims

C L A I M SI Claim:
1. A method of providing an output data stream comprising: a. receiving one or more input data streams from one or more source devices at a receiving device; b. receiving control information from a control device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information; c. selecting the selected input data streams based on the control infoπnation from the control device; and d. mixing the selected input data streams into the output data stream based on the control information from the control device.
2. The method as claimed in claim 1 further comprising: a. sending a request from the control device to the receiving device regarding selection and mixing functions and controls at the receiving device; and b. providing the selection and mixing functions and controls from the receiving device to the control device.
3. The method as claimed in claim 1 wherein the input data streams include audio/video data streams.
4. The method as claimed in claim 1 wherein the input data streams include on- screen-display data streams.
5. The method as claimed in claim 1 wherein the input data streams include graphics data streams.
6. The method as claimed in claim 1 wherein the control device is external of the receiving device.
7. The method as claimed in claim 1 wherein the control device is internal to the receiving device.
8. The method as claimed in claim 1 further comprising outputting the output data stream.
9. The method as claimed in claim 1 further comprising receiving control input at the control device and generating the control information based on the control input.
10. The method as claimed in claim 1 wherein the one or more input streams are received over a network which substantially complies with a version of the IEEE 1394 standard.
11. The method as claimed in claim 1 wherein the one or more input streams are received over a connection which substantially complies with a version of universal serial bus standard.
12. The method as claimed in claim 1 wherein the one or more input streams are received over a connection which substantially complies with a version of universal plug-and- play standard.
13. The method as claimed in claim 1 wherein the control information is received over a network which substantially complies with a version of the IEEE 1394 standard.
14. The method as claimed in claim 1 wherein the control information is received over a connection which substantially complies with a version of universal serial bus standard.
15. The method as claimed in claim 1 wherein the control information is received over a connection which substantially complies with a version of universal plug-and-play standard.
16. A method of controlling a receiving device providing a combination of one or more input data streams as an output data stream comprising: a. sending a request from a control device to the receiving device regarding selection and mixing functions and controls at the receiving device; b. providing the selection and mixing functions and controls from the receiving device to the control device; and c. providing control information from the control device to the receiving device, including information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information, wherein the selected input data streams are selected and mixed based on the control information.
17. The method as claimed in claim 16 further comprising: a. receiving the one or more input data streams from one or more source devices at the receiving device; b. selecting the selected input data streams based on the control information from the control device; and c. mixing the selected input data streams into the output data stream based on the control information from the control device.
18. The method as claimed in claim 17 further comprising outputting the output data stream.
19. The method as claimed in claim 16 wherein the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
20. The method as claimed in claim 16 wherein the control device is external of the receiving device.
21. The method as claimed in claim 16 further comprising receiving control information at the control device and generating the control information based on the control input.
22. The method as claimed in claim 16 wherein the one or more input streams are received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
23. The method as claimed in claim 16 wherein the control information is received over one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and- play standard.
24. A receiving device for providing an output data stream comprising: a. means for receiving one or more input data streams configured for coupling to one or more source devices; b. means for receiving control information configured for coupling to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information; c. means for selecting coupled to the means for receiving one or more input data streams and the means for receiving control information for selecting the selected input data streams based on the control information from the control device; and d. means for mixing coupled to the means for selecting and to the means for receiving control information for mixing the selected input data streams into the output data stream based on the control information from the control device.
25. The receiving device as claimed in claim 24 wherein the means for receiving control information provides selection and mixing functions and controls to the control device in response to a request from the control device.
26. The receiving device as claimed in claim 24 further comprising means for outputting coupled to the means for mixing for outputting the output data stream.
27. The receiving device as claimed in claim 24 wherein the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
28. The receiving device as claimed in claim 24 wherein the control device is external of the receiving device.
/ 29. The receiving device as claimed in claim 24 wherein the control device is internal of the receiving device.
30. The receiving device as claimed in claim 24 wherein the means for receiving one or more input data streams is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
31. The receiving device as claimed in claim 24 wherein the means for receiving control information is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
32. A receiving device to provide an output data stream comprising: a. a first interface to receive one or more input data streams configured to couple to one or more source devices; b. a second interface to receive control information configured to couple to a control device, wherein the control information includes information regarding selected ones of the one or more input data streams, thereby providing selected input data streams, and mixing information; c. one or more selectors coupled to the first interface and to the second interface to select the selected input data streams based on the control information from the control device; and d. a mixer coupled to the selectors and the second interface to mix the selected input data streams into the output data stream based on the control information from the control device.
33. The receiving device as claimed in claim 32 wherein the second interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device.
34. The receiving device as claimed in claim 32 further comprising an output device coupled to the mixer to output the output data stream.
35. The receiving device as claimed in claim 32 wherein the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
36. The receiving device as claimed in claim 32 wherein the control device is external to the receiving device.
37. The receiving device as claimed in claim 32 wherein the control device is internal to the receiving device.
38. The receiving device as claimed in claim 32 wherein the first interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
39. The receiving device as claimed in claim 32 wherein the second interface is coupled to the control device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
40. A control device to provide control information to a receiving device receiving one or more input data streams to control selection and mixing of the input data streams into an output data stream comprising: a. an interface configured to couple to the receiving device to request selection and mixing function and control information from the receiving device; and b. a control circuit coupled to the interface to receive the selection and mixing function and control information and to provide output control information to the receiving device, wherein the output control information includes information regarding selected ones of the one or more input data streams and a selected combination of the selected ones of the one or more input data streams.
41. The control device as claimed in claim 40 wherein the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
42. The control device as claimed in claim 40 wherein the receiving device is external to the control device.
43. The control device as claimed in claim 40 wherein the receiving device is internal to the control device.
44. The control device as claimed in claim 40 wherein the interface is coupled to the receiving device by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
45. A network of devices to provide an output data stream comprising: a. one or more source devices configured to provide one or input data streams; b. a control device configured to provide control information regarding selection and mixing of the input data streams into the output data stream, the control device including: i. a control interface configured to provide the control information; and ii. a control circuit coupled to the control interface to generate the control information; and c. a receiving device comprising: i. a first receiving interface coupled to the source devices to receive the one or more input data streams; ii. a second receiving interface coupled to the control interface to receive the control information; iii. one or more selectors coupled to the first receiving interface and the second receiving interface to select selected input data streams based on the control information; and iv. a mixer coupled to the selectors and the second receiving interface to mix the selected input data streams into the output data stream based on the control information from the control device.
46. The network of devices as claimed in claim 45 wherein the second receiving interface provides functions and controls of the selectors and the mixer to the control device in response to a request from the control device.
47. The network of devices as claimed in claim 45 wherein the receiving device further comprises an output device coupled to the mixer to output the output data stream.
48. The network of devices as claimed in claim 45 wherein the input data streams include one or more of audio/video data streams, on-screen-display data streams and graphics data streams.
49. The network of devices as claimed in claim 45 wherein the control device is external to the receiving device.
50. The network of devices as claimed in claim 45 wherein the control device and the receiving device are integrated together.
51. The network of devices as claimed in claim 45 wherein the first receiving interface is coupled to the source devices by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a comiection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
52. The network of devices as claimed in claim 45 wherein the second receiving interface is coupled to confrol interface by one or more of a network which substantially complies with a version of the IEEE 1394 standard, a connection which substantially complies with a version of universal serial bus standard and a connection which substantially complies with a version of universal plug-and-play standard.
PCT/US2003/006880 2002-03-29 2003-03-05 Mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control WO2003085991A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP03721338A EP1491051A4 (en) 2002-03-29 2003-03-05 A method of and apparatus for supporting and enabling the selection and mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control
JP2003583035A JP4896374B2 (en) 2002-03-29 2003-03-05 Output data stream generating method and receiving apparatus
KR10-2004-7014943A KR20040111426A (en) 2002-03-29 2003-03-05 Mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/112,805 2002-03-29
US10/112,805 US7310808B2 (en) 2002-03-29 2002-03-29 Method of and apparatus for supporting and enabling the selection and mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control

Publications (2)

Publication Number Publication Date
WO2003085991A2 true WO2003085991A2 (en) 2003-10-16
WO2003085991A3 WO2003085991A3 (en) 2004-04-08

Family

ID=28673650

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/006880 WO2003085991A2 (en) 2002-03-29 2003-03-05 Mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control

Country Status (5)

Country Link
US (1) US7310808B2 (en)
EP (1) EP1491051A4 (en)
JP (1) JP4896374B2 (en)
KR (1) KR20040111426A (en)
WO (1) WO2003085991A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008056332A1 (en) 2008-11-07 2010-05-20 Giesecke & Devrient Gmbh Method for safe calculation of calculation result of cryptographic calculation, involves carrying out cryptographic calculation two times in data carrier

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040081434A1 (en) * 2002-10-15 2004-04-29 Samsung Electronics Co., Ltd. Information storage medium containing subtitle data for multiple languages using text data and downloadable fonts and apparatus therefor
US20060209210A1 (en) * 2005-03-18 2006-09-21 Ati Technologies Inc. Automatic audio and video synchronization
KR100708203B1 (en) 2006-02-24 2007-04-16 삼성전자주식회사 Method for granting control device and device for using thereof
US8607281B2 (en) 2006-09-07 2013-12-10 Porto Vinci Ltd. Limited Liability Company Control of data presentation in multiple zones using a wireless home entertainment hub
US9319741B2 (en) * 2006-09-07 2016-04-19 Rateze Remote Mgmt Llc Finding devices in an entertainment system
US8005236B2 (en) 2006-09-07 2011-08-23 Porto Vinci Ltd. Limited Liability Company Control of data presentation using a wireless home entertainment hub
US20080061578A1 (en) * 2006-09-07 2008-03-13 Technology, Patents & Licensing, Inc. Data presentation in multiple zones using a wireless home entertainment hub
US8935733B2 (en) * 2006-09-07 2015-01-13 Porto Vinci Ltd. Limited Liability Company Data presentation using a wireless home entertainment hub
US9386269B2 (en) 2006-09-07 2016-07-05 Rateze Remote Mgmt Llc Presentation of data on multiple display devices using a wireless hub
US9233301B2 (en) * 2006-09-07 2016-01-12 Rateze Remote Mgmt Llc Control of data presentation from multiple sources using a wireless home entertainment hub
US20080106640A1 (en) * 2006-11-06 2008-05-08 International Business Machines Corporation Method of multiple stream formatting in a multimedia system
KR101449025B1 (en) * 2008-03-19 2014-10-08 엘지전자 주식회사 Method and apparatus for managing and processing information of an object for multi-source-streaming
US9084021B2 (en) * 2012-12-12 2015-07-14 Jan-Christoph Bieselt Call-in participation for a live video broadcast
US10063853B2 (en) * 2013-12-16 2018-08-28 Barco N.V. Efficient error recovery

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH02100475A (en) * 1988-10-06 1990-04-12 Pioneer Electron Corp Video display device
JPH0419081A (en) * 1990-05-15 1992-01-23 Seiko Instr Inc In-vacuum conveyor robot
JP3269851B2 (en) * 1992-06-16 2002-04-02 株式会社日立製作所 Television signal display
JPH0730971A (en) * 1993-07-09 1995-01-31 Hitachi Ltd Av system controller
JP3270882B2 (en) * 1993-07-30 2002-04-02 ソニー株式会社 AV system control method
JP3228381B2 (en) * 1993-10-29 2001-11-12 ソニー株式会社 AV selector
JP3596568B2 (en) * 1995-10-31 2004-12-02 ソニー株式会社 Television receiver
JP3870436B2 (en) * 1995-11-02 2007-01-17 ソニー株式会社 Information signal recording device and communication method
GB2310114A (en) 1996-02-08 1997-08-13 Gec-Marconi Limited Data distribution systems in vehicles
MY116734A (en) * 1997-04-17 2004-03-31 Matsushita Electric Ind Co Ltd Image display device
CA2295785C (en) * 1997-06-25 2007-01-23 Samsung Electronics Co., Ltd. Programming tool for home networks
US6061746A (en) * 1998-04-30 2000-05-09 Compaq Computer Corporation Device bay system without 1394 PHY/Link interface to device bay controller having a software intercepting a GUID query and returning a stored unique identifier
US6219839B1 (en) * 1998-05-12 2001-04-17 Sharp Laboratories Of America, Inc. On-screen electronic resources guide
US6202210B1 (en) * 1998-08-21 2001-03-13 Sony Corporation Of Japan Method and system for collecting data over a 1394 network to support analysis of consumer behavior, marketing and customer support
JP4508330B2 (en) * 1999-01-25 2010-07-21 キヤノン株式会社 Display device
JP4193013B2 (en) * 1999-09-28 2008-12-10 ソニー株式会社 Information output device and connection relation management method
JP4135047B2 (en) * 1999-11-17 2008-08-20 ソニー株式会社 Information transmission system, information output device, information input device, and connection relation specifying method
JP3789716B2 (en) * 2000-03-17 2006-06-28 三洋電機株式会社 Television broadcast receiver
JP4859154B2 (en) * 2000-06-09 2012-01-25 キヤノン株式会社 Display control device, display control system, display control method, and storage medium
KR100424481B1 (en) * 2000-06-24 2004-03-22 엘지전자 주식회사 Apparatus and method for recording and reproducing a digital broadcasting service information on optical medium
US20020120944A1 (en) * 2001-02-23 2002-08-29 Wasilewski Louise Mary Remote control having a print button function

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of EP1491051A4
YANG ET AL.: "IP-Based UPNP Control of Network-Enabled IEEE1394 Devices", PROCEEDINGS OF INTERNATIONAL CONFERENCE ON CONTROL, AUTOMATION AND SYSTEMS, 2001, pages 464 - 467, XP008045713

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008056332A1 (en) 2008-11-07 2010-05-20 Giesecke & Devrient Gmbh Method for safe calculation of calculation result of cryptographic calculation, involves carrying out cryptographic calculation two times in data carrier

Also Published As

Publication number Publication date
US20030192062A1 (en) 2003-10-09
KR20040111426A (en) 2004-12-31
WO2003085991A3 (en) 2004-04-08
JP4896374B2 (en) 2012-03-14
EP1491051A2 (en) 2004-12-29
EP1491051A4 (en) 2010-04-21
JP2005522154A (en) 2005-07-21
US7310808B2 (en) 2007-12-18

Similar Documents

Publication Publication Date Title
US6593937B2 (en) Method of and apparatus for handling high bandwidth on-screen-display graphics data over a distributed IEEE 1394 network utilizing an isochronous data transmission format
EP0976242B1 (en) Display with one or more display windows and placement dependent cursor and function control
US8824500B2 (en) Method of controlling connection between nodes in digital interface
EP1016271B1 (en) Digital television apparatus for controlling a peripheral device via a digital bus
KR100230281B1 (en) Multi-media system for transmitting and receiving a program number and method for transmitting and receiving a program number
US8286204B2 (en) Data communication system, apparatus and control method therefor
US7310808B2 (en) Method of and apparatus for supporting and enabling the selection and mixing of multiple streams of audio/video data from multiple sources within a receiving device allowing external control
US6684110B1 (en) Control system, apparatus, and method in which display characteristics of an operation panel are changeable
JPH1188865A (en) Distributed type home network
JP2000253463A (en) Network control system, and target, controller and consumer used for the network control system
KR20000057102A (en) Method of transmitting and receiving data, electronic apparatus, and medium for supplying a data-transmitting/receiving program
JP2001145175A (en) Network control system and device and controller used therefor
JP2000278294A (en) Communication apparatus and communication method
KR100304898B1 (en) method for controlling connection between nodes in digital interface
KR20020024457A (en) Method for selecting icon in modular television appartus and remocon thereof
WO2000062176A1 (en) System for establishing and maintaining connections and confirming format compatibility between units, subunits and content

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): CN JP KR

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2003721338

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 1020047014943

Country of ref document: KR

WWE Wipo information: entry into national phase

Ref document number: 2003583035

Country of ref document: JP

WWP Wipo information: published in national office

Ref document number: 2003721338

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1020047014943

Country of ref document: KR