WO2002097584A2 - Adaptive video server - Google Patents

Adaptive video server Download PDF

Info

Publication number
WO2002097584A2
WO2002097584A2 PCT/US2002/017118 US0217118W WO02097584A2 WO 2002097584 A2 WO2002097584 A2 WO 2002097584A2 US 0217118 W US0217118 W US 0217118W WO 02097584 A2 WO02097584 A2 WO 02097584A2
Authority
WO
WIPO (PCT)
Prior art keywords
server
client
video
format
display
Prior art date
Application number
PCT/US2002/017118
Other languages
French (fr)
Inventor
Peter J. Cranstone
Kevin J. Kiley
Original Assignee
Hyperspace Communications, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hyperspace Communications, Inc. filed Critical Hyperspace Communications, Inc.
Publication of WO2002097584A2 publication Critical patent/WO2002097584A2/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234354Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering signal-to-noise ratio parameters, e.g. requantization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808Management of client data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808Management of client data
    • H04N21/25825Management of client data involving client display capabilities, e.g. screen resolution of a mobile phone
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808Management of client data
    • H04N21/25833Management of client data involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6582Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number

Definitions

  • Various embodiments relate generally to a multimedia communications system, and, more particularly, to a communications system having an adaptive video server.
  • RTSP Real Time Streaming Protocol
  • MULTI-USER, ON-DEMAND VIDEO SERVER SYSTEM INCLUDING INDEPENDENT, CONCURRENTLY OPERATING REMOTE DATA RETRIEVAL CONTROLLER discloses a video server having a mass storage unit for storing a plurality of movies in digital form, specifically in an MPEG format. Anderson further discloses a plurality of identical converters, one for each conventional television receiver, for decoding and decompressing the MPEG movies. The link between the video server and the converters is disclosed as being high bandwidth. There are several disadvantages with the video server of Anderson.
  • Exemplary of this approach includes a product available under the name FIREPRODUCER (http://www.firepad.com), which purports to deliver streams of live or stored video to multiple PalmOS handheld devices that run a client piece called FIREVIEWER.
  • FIREPRODUCER http://www.firepad.com
  • the output of this product is inflexible inasmuch as the proprietary video format is generally incompatible with other devices, or, at the very least, is not optimized or tailored to suit end user devices other than PalmOS.
  • Figure 1 is a simplified block diagram view of a system according to the invention.
  • Figure 2 is a simplified block diagram showing, in greater detail, an architectural organization of the video server of Figure 1.
  • Figure 3 is a simplified diagrammatic and block diagram view showing, in greater detail, the portions of the server software illustrated in Figure 2.
  • Figure 4 is a simplified block diagram of a client computing architecture according to the invention.
  • Figure 5 is a simplified block diagram view of an originating source of video data according to the invention.
  • Figure 6 is a simplified diagram of a display buffer of Figure 5.
  • Various embodiments of a video server adapt video data to a video format that is optimized for play back at a client device based on a number of factors. Another embodiment provides, with respect to an originating source of video data, for image acquisition and communication to a video server using computing devices having limited computing resources.
  • a client-server system may include a client configured to display video and which is coupled to a network.
  • the system also includes a server coupled to the network, and which is configured to determine computing resource characteristics of the client.
  • the server is further configured to select a transmit video format from a plurality of available video formats based on the determined computing resource characteristics of the client, and to communicate the video data to the client in the selected video format.
  • the selected transmit video format corresponds to a native display format associated with the client. This is useful in relieving the client with decoding, and decompression functions where the client has relatively limited central processing unit (CPU) computing ability, amounts of main memory, amounts of available memory, or the lack of a specialized graphics processor.
  • the server is also configured to translate, for example, color video data for display on a grey- scale display and/or to scale a horizontal and vertical display size so as to fit on the client display.
  • Figure 1 shows a client-server system 10 for multimedia communications in accordance with one embodiment of the present invention.
  • System 10 features an adaptive video server configured to select a transmit video format from a plurality of available video formats based on determined characteristics of the destination client.
  • an adaptive video server configured to select a transmit video format from a plurality of available video formats based on determined characteristics of the destination client.
  • Figure 1 further shows a video server 12 which may optionally have a stored multimedia database 14 associated therewith, and a plurality of client devices 16 ⁇ , 16 2 , 16 3 , . . ., 16 N connected to the video server 12 by way of a first network 18.
  • Network 18 may include a wireless communication portion 18w for communications with wireless clients such as client 16 ⁇ .
  • Figure 1 further shows a plurality of originating video sources 20 ⁇ , . . . , 20 N connected to video server 12 by way of a second network 22.
  • First network 18 and second network 22 may be the same network.
  • video means multimedia content including moving video, still video, audio, or a combination thereof.
  • Video server 12 is configured to communicate either (i) pre-stored multimedia content, such as contained in its local database 14, or (ii) streaming multimedia content received from originating sources 20, to client devices 16 that have connected therewith and have requested content.
  • Video server 12 may be implemented on conventional computing hardware.
  • server 12 may comprise a personal computer (PC) platform executing a windows operating system (e.g., Windows 98SE, Windows ME, Windows 2000, etc.), an Apple Macintosh computer system, a Unix-based computer, a Linux-based computer, and the like.
  • video server 12 includes conventional central processor(s), main memory, cache memory, hard disk drive, communication ports, keyboard, mouse, display and the like.
  • video server 12 may further include conventional server software, for example, conventional web server software such as the widely available Apache web server software for Linux-based systems (e.g., recently available Apache 1.3.19 version available at http://httpd.apache.org/). It should be understood, however, that server 12 need not comprise a web (or HTTP) server where, in particular, the client(s) 16 ⁇ , 16 2 , 16 3 , . . . , 16 N are not communicating with server 12 using the HTTP protocol.
  • server software for example, conventional web server software such as the widely available Apache web server software for Linux-based systems (e.g., recently available Apache 1.3.19 version available at http://httpd.apache.org/). It should be understood, however, that server 12 need not comprise a web (or HTTP) server where, in particular, the client(s) 16 ⁇ , 16 2 , 16 3 , . . . , 16 N are not communicating with server 12 using the HTTP protocol.
  • HTTP HyperText Transfer Protocol
  • the stored multimedia database 14 may comprise any type of content
  • Video formats including, without limitation, the Motion Picture Experts Group (MPEG) format (e.g., MPEG-1 , MPEG-2, MPEG-4, etc.), Microsoft video format-AVI format, Quick Time format, or any other type of format now known, or hereinafter developed.
  • MPEG Motion Picture Experts Group
  • Stored multimedia database 14 may reside, physically, on a hard disk drive, or other conventional apparatus (e.g., CD or DVD tower).
  • Client devices 16 ⁇ , 16 2 , 16 3 , . . . , 16N are each configured to display video and are, as shown, coupled to network 18.
  • Certain of the client devices 16 may include a client module to enhance the exchange of information between client 16 and server 12, the content of which will be described in greater detail in connection with Figure 3.
  • client devices 16 2 , 16 3 are shown having such a client module, while client devices 16 ⁇ and 16N do not have such a client module.
  • all clients have, at a minimum, capabilities to display video, as mentioned above.
  • the particular configuration of the plurality of client devices 16 varies greatly, and may be any one of a plurality of conventional computing devices commercially available.
  • client devices 16 may comprise a desktop or laptop personal computer (PC) executing a windows-based operating system (e.g., Windows 98SE, Windows ME, Windows 2000, etc.), an Apple Macintosh computer, a Unix work station, a Linux work station, a hand held computer such as a Microsoft Windows CE-based device ⁇ i.e., a Pocket PC), a Palm or Hand Spring connected organizer, any variety of laptop computers, wireless telephones having internet access (e.g., WAP enabled), such as, for example, a commercially available product from Kyocera (Webphone), video tablets, wireless wrist watches (e.g., commercially available from Matsucom), Internet appliances (e.g., 3COM's "Audrey”), as well as a variety of other computing devices, generally.
  • a windows-based operating system e.g., Windows 98SE, Windows ME, Windows 2000, etc.
  • Apple Macintosh computer e.g., a Unix work station, a Linux work station,
  • the client devices may be either connected to network 18 by way of a wire line, such as shown for client devices 16 2 , 16 3 and 16N or may be connected to network 18 by way of a wireless link, as for the case of client device 16 ⁇ connected to wireless portion 18 w of network 18 (connection shown in phantom line).
  • Video server 12 may select one of a plurality of video formats based on a profile of the requesting client device 16. The information contained in the profile will be described in connection with Figure 3.
  • video server 12 includes an operating system 24, server software 26, and, optionally, one or more virtual machines 28 ⁇ , 28 2 , . . ., 28N which may be established by server software 26 depending on the assessment of the profile of a requesting client device 16.
  • server software 26 it is quite often the case that content stored in multimedia database 14 is stored in a compressed form (e.g., MPEG-2) in order to reduce the amount of disk space taken up by the content, for example, multiple movies.
  • MPEG-2 a compressed form
  • server software 26 when it determines that a change in video format is desirable for a particular client device 16, establishes a respective virtual machine 28 in the server memory itself, where the requested multimedia content is played back/translated, or otherwise formatted so as to optimize the experience for a user of the requesting client device.
  • a similar construct is known in the art and is referred to as a "thin client” approach. The actual processing power, therefore, is drawn from the server, not the client.
  • FIG. 3 shows, in greater detail, the server software 26 according to one embodiment of the invention.
  • server software 26 includes input/output (I/O) 30, a client identity detect module 32, a client computer resource characteristic detect module 34, a client display characteristic detect module 36, a server-to-client bandwidth detect module 38, a client battery detect module 40, a video format select module 42, and a video data formatter module 44.
  • Video data formatted in a transmit video format is output as a stream of messages 45.
  • a respective client profile is developed using the detected client information.
  • Client identity detect module 32 is configured to detect an identity of a particular client device 16 and generate a signal SO indicative of such determined identity.
  • identification information may include the client device manufacturer, model number, operating system, and any special hardware or software.
  • signal SO is shown simply as a single signal line for clarity.
  • the identification may, in-fact, comprise, for example, text strings corresponding to the above described information, as well as perhaps numerical information.
  • Computing resource characteristic detect module 34 is configured to obtain information useful to assess, generally, the capability of a client device 16 to decode and decompress a multimedia content stream which, as described above, is commonly compressed.
  • computing resource characteristics may include the type and clock speed of a central processing unit (CPU), including any other characterizing computing ability (e.g., MMX capability), an amount of main memory, an amount of available memory, as well as an availability of any specialized graphics processor.
  • Module 34 determines one or more of the foregoing characteristics and generates a signal, designated S1 , indicative of such information. As with signal SO, it is shown as “single" line for clarity only.
  • Module 36 is configured to determine display characteristics of a client device 16. Display characteristics as used herein may include the availability of a color display and a color depth associated with each pixel, an availability of a grey-scale display and the number of levels of grey associated therewith, the availability of a black/white display, and a horizontal and vertical display size (e.g., in pixels). In some instances, determination of the identity of the client uniquely determines its display capabilities (e.g., a Palm Mix is a 160 x 160 pixel, 4-bit grey scale display). Display characteristic detect module 36 generates an output signal S2 indicative of the determined display characteristics of a particular client device 16. Again, signal S2, along with signals S3, S4 and S5, are shown as a single line for clarity purposes only.
  • Bandwidth detect module 38 is configured to detect an available bandwidth between video server 12 and a requesting client device 16 through network 18. Bandwidth detect module 38 generates a signal, designated S3 indicative of the determined available bandwidth. In addition, connectivity characteristics such as delay, turnaround time, whether the communication is "connectionless" or not (e.g., TCP/IP and opened ports, etc. vs. video data tunneled via HTTP), may also be included in the information in signal S3.
  • connectivity characteristics such as delay, turnaround time, whether the communication is "connectionless" or not (e.g., TCP/IP and opened ports, etc. vs. video data tunneled via HTTP), may also be included in the information in signal S3.
  • Battery detect module 40 is configured to determine whether a particular client device 16 is powered by way of batteries, and if so, the type and optionally the state of charge thereof. Battery detect module 40 generates an output signal, designated S4, indicative of the determined battery characteristics of the particular client device 16.
  • client devices 16 include a client module that is configured to communicate and cooperate with video server software 26 of video server 12 in obtaining the foregoing information.
  • client software 46 is configured to communicate with, for example, application programming interfaces (APIs) 47 of the operating system of the client device 16 in order to ascertain one or more of the items of information described above with respect to modules 32, 34, 36, 38, and 40.
  • APIs application programming interfaces
  • a set of publicly known APIs are available to determine a wide variety of client device information, such as the information set forth above.
  • Palm software development kit e.g., SDK 3.5
  • Similar software development kits are available for other devices (e.g., Microsoft Windows CE APIs for a pocket PC device, Windows 98 APIs for a desktop or laptop client device, etc.).
  • a particular client device 16 may comprise a Microsoft Windows-based operating system.
  • a windows video format e.g., a file having a * avi extension
  • the client identity may also uniquely determine some computing resource characteristics and/or screen display (as mentioned above).
  • the computing resource characteristics ⁇ i.e., signal S1 ) largely reflect the client device's capability to decode and decompress certain synchronized and compressed video formats such as the MPEG-2 format, and Real Time Video (RTV) format.
  • certain synchronized and compressed video formats such as the MPEG-2 format, and Real Time Video (RTV) format.
  • RTV Real Time Video
  • the video server 12 may determine that no compression should be used in order to provide the best experience to the user of that particular client device 16 in terms of frames per second, for example. This means that a bitmap video format would be sent.
  • the display characteristics may also be considered together with the other signals in determining what transmit video format to select.
  • a movie stored in multimedia storage database 14 in an MPEG-2 format that is in a color format, in a 1024 x 768 pixel size, for example, will not only have to be decoded and decompressed at the client device prior to display, but may also have to be converted from color to grey- scale, as well as to be scaled down to the display window of the client device (e.g., 160 x 160 pixels in a Palm OS connected organizer).
  • Sufficient computing resources may be unavailable to perform the decoding, decompression and the color-to-grey scale and scaling conversions in a suitably high frame rate.
  • the server software may perform the color-to-grey scale and size scaling transformations on the server side and send that in either compressed or uncompressed format to the client device 16.
  • the available bandwidth may impact selection of a video format as follows. For example, if a provisional selection by server 12 (perhaps based on other factors) of a video format would be to send uncompressed multimedia content to the client device 16, but the connection is bandwidth limited, for example over a wireless link, then a compromise must be made. The compromise reconciles (i) attempting to optimize frames per second at the client by reducing computationally intensive decoding and decompression (which may unfortunately increase the total payload), and (ii) total bandwidth from server to client (which may be limited and insufficient to carry uncompressed data). That is, some level of compression may be required in order to fit the video data within the maximum available bandwidth. A video format having some level of compression would be selected (e.g., MPEG-4), and the user of the particular client device 16 would be presented with a perhaps reduced frame rate than would otherwise be possible if an increased available bandwidth connection were available and no compression used.
  • MPEG-4 MPEG-4
  • the battery detect signal reflects the understanding that battery drain is a function of both computational activity and screen activity.
  • a client device dependent on batteries can ill afford a total drain.
  • Transferring an uncompressed bit map image to a display buffer at a client device 16, for example, is much less computationally intensive than decoding and decompressing an MPEG-2 multimedia stream.
  • Such video format bitmap; uncompressed
  • video server 12 may incorporate battery management into its decision structure.
  • video format select module 42 is responsive to one or more of the signals S0-S4 for selecting a transmit video format, which is represented by a video format selection signal S5.
  • video format select module 42 selects a transmit video format, which is then used by video data formatter module 44 in ensuring compliance of the requested video data with the selected video format.
  • Client device 16 includes the client software 46 referred to above, and is identified, for example, as a Windows 98-based machine including 256 MB of SDRAM, and 800 MHz Intel Pentium III central processor, 40 GB hard drive, with a 3D video graphics card having display settings set to 1024 x 768 pixels, 32 bit color depth (true color), connected to the internet via a cable modem having a bandwidth in excess of T1 speeds ⁇ i.e., greater than 1.5 Mbps), directly operating from 120 volt line voltage ⁇ i.e., no batteries).
  • the requested multimedia content is in an MPEG-2 stored video format.
  • Video format select module 42 makes a determination that the described client device 16 has sufficient computing resources to decode and decompress the requested MPEG-2 format movie in real time.
  • any color display characteristics ⁇ i.e., color depth) or scaling ⁇ i.e., display size) can also be accomplished at the client device 16 in real time.
  • the available bandwidth does not impose any limitations, nor is battery usage a factor.
  • server resource usage can be optimized ⁇ i.e., no need to make an MPEG-2-to-AVI translation), without any detriment on the perceived experience for the user at the client device end, by the server 12 passing through the retrieved MPEG movie to the requesting client device 16.
  • the transmitted video data is shown in Figure 3 as a message 45 containing video data in an MPEG video format ⁇ i.e., the selected transmit video format).
  • No virtual machine 28 is created by server software 26 in this Example 1. The foregoing constitutes one client profile.
  • the client device 16 comprises a so-called Pocket PC having 16 MB of
  • This client device 16 includes an Ethernet ⁇ i.e., a local area network) type connection to network 18, which is coupled to video server 12.
  • This client device 16 runs on batteries.
  • the requested multimedia content is again in an MPEG stored video format.
  • the transmit video format selected by module 42 in this second example, is an AVI video format, in view of the moderate level of computing resources available on such a client device, the display characteristics, and the operating system of the client device (e.g., Microsoft Windows-based operating system). The selection is communicated to video data formatter 44 via signal S5.
  • Video data formatter 44 establishes a virtual machine, for example virtual machine 28 ⁇ in the memory of server 12, for translating from MPEG to AVI.
  • MPEG and AVI codecs coders-decoders
  • the data stream being retrieved from stored multimedia database 14 is first converted from MPEG into raw video data, and which is then scaled, and whose color depth is further corrected to match that of the client device, and then encoded using an AVI codec.
  • the outgoing stream from video data formatter 44 is sent out via I/O 30 as a stream of messages 45 containing video data in the transmit video format (here, the AVI format). Effectively, much of the processing power required to obtain playback in the client's native format, working from the original compressed MPEG file, is actually borne by the server, not the client. This enhances the client's experience.
  • the foregoing constitutes a second client profile.
  • the client device 16 is a Palm OS connected organizer with a 160 x
  • the client device 16 in this example includes client software 46 for enhanced information gathering and communication upstream to video server 12 through network 18W, 18.
  • Video format select module 42 selects an uncompressed, 16-level grey-scale video format with a scaled size of approximately 160 x 160 pixels for communication to this client 16.
  • the video format selection is provided via signal S5 to video data formatter 44.
  • Video data formatter 44 initiates another virtual machine 282. As the requested content, which is in an MPEG format, is retrieved from stored multimedia 14, it is, in effect, running (play back) within virtual machine 282.
  • Video data formatter 44 in this example, is configured to convert from color to the 16- level grey scale, as well as to perform a scaling operation to reduce the size so as to be immediately usable for display on the client device 16.
  • the selected transmit video format is a native video format associated with the described client device 16. All that the client device needs to do is to transfer the incoming, uncompressed (native video format) data from the receive buffer to the display buffer.
  • the foregoing example further illustrates how computing resources are drawn from the device in the chain best adapted to handle (in this example, the server).
  • the foregoing describes a third client profile.
  • a whole spectrum of pass through, format conversion, and other variations will be possible depending on all of the factors described above, and yet remain with the spirit and scope of the present invention.
  • a plurality of client devices 16 of the above-described variety of configurations are serviced concurrently by video server 12.
  • video data formatter 44 can be configured to select streaming video feeds that are provided on an alternate input thereto in addition to, or in lieu of, retrieving stored multimedia from stored multimedia database 14.
  • the video server 12 may be adaptive depending on the profile of the client device being serviced in order to provide a better experience to the user. This arrangement overcomes many of the problems in the art, with single purpose or inflexible servers.
  • streaming video is provided from a plurality of physically separate locations comprising respective originating video source(s) designated 20 ⁇ , . . ., 20 N .
  • Each source 20 ⁇ , . . ., 20 N include an imaging device 48, a computing device 50 cooperating together to image and stream video data corresponding to a respective view 52.
  • Figure 5 shows a single originating source 20 ⁇ in greater detail.
  • Imaging device 48 may be any one of a number of conventional imaging devices, such as still digital cameras, video cameras, and the like. In a constructed embodiment, however, a low-cost web cam is employed, which is a commercially available component, for example, available from D-Link, capable of capturing 30 frames per second, 64,000,000 colors, having a universal serial bus (USB), interface, model no. DSB-C300. Of course, any number of alternative imaging devices, comparable in function, may be substituted therefore.
  • D-Link universal serial bus
  • Computing device 50 ⁇ may comprise conventional computing apparatus, having a central processing unit (CPU) 54 coupled, as shown in diagrammatic fashion, to a common bus 55, to which communication is enabled with a main memory 56, a display buffer 58, and an input/output (I/O) 60.
  • CPU central processing unit
  • I/O input/output
  • image data 62 generated by imaging device 48 ⁇ , which corresponds to view 52, is provided to device 50 ⁇ in a compressed format, such as an *.avi video format.
  • software provided with imaging devices of the type described ⁇ i.e., web cams
  • allow a user to, among other things, save the incoming video stream 62 to a local hard drive, or to simply allow the image data 62 to be continuously displayed on a locally connected display in a "preview" mode ⁇ i.e., the software converts the image data 62 into a bit map and moves the bit map into the display buffer 58).
  • the incoming image data 62 in the * avi format, for example, is not stored to a local hard drive, but is rather allowed to free run in the "preview" mode.
  • FIG. 6 shows display buffer 58 in greater detail, having a first portion
  • a module executing from main memory 56 is configured to retrieve the bit map 68 from display buffer 58 and forward the same via I/O 60 as a message 64 containing the bit map data, destined through network 22 to video server 12.
  • Figure 6 is simplified, and that the bit map portion 68 of display buffer 58 need not be linear and contiguous, as shown.
  • a plurality of such configurations may be deployed at a plurality of distinct physical locations, for example purposes only, for distributed security monitoring.
  • the stream of messages 64 comprising frames in the bit map video format, when routed through video server 12, and when destined for, for example, a relatively low powered client device (e.g., a Palm OS connected organizer, as described above in Example 3), may simply "pass through" the bit map to the client device 16.
  • a relatively low powered client device e.g., a Palm OS connected organizer, as described above in Example 3
  • the embodiment of Figure 5 can be used in conjunction with the video server to form an end-to-end adaptive video capture and client data server system for improved video communication.
  • a user with a handheld device is then enabled to monitor a plurality of different views at a remote location.
  • a flexible approach to the distribution of multimedia content wherein resource intensive tasks are allocated to a computing device best suited/best capable of performing such resource intensive tasks.
  • the video server is the node in the system having the greatest computing power, most of the conditioning work on the video data is performed there.
  • the client has extensive capabilities, then the video server just serves up the content, which is usually compressed (MPEG, AVI), and allows the client to decode and decompress it.
  • the originating source i.e., image or video capture source
  • a bitmap only is extracted and sent upstream to the video server.

Abstract

A client-server system includes a client configured to display video wherein the client is coupled to a network, and a server that is also coupled to the network. The server is configured to determine computing resource characteristics of the client. The server is also configured to select a transmit video format from a plurality of available video formats based on the determined computing resource characteristics of the client. The server is then adapted to communicate video data to the client device in the selected transmit video format.

Description

ADAPTIVE VIDEO SERVER
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit of U.S. Provisional Patent
Application "ADAPTIVE VIDEO SERVER," No. 60/294,835, filed May 31 , 2001 (attorney docket number 36120-8001 US00).
BACKGROUND
[0002] Various embodiments relate generally to a multimedia communications system, and, more particularly, to a communications system having an adaptive video server.
[0003] There has been much investigation into improvements in the distribution of multimedia content over networks for varied purposes such as entertainment, education and the like. While initially multimedia was distributed mainly over terrestrial broadcasting channels (e.g., network television), the distribution channels have evolved to include community access television (CATV, a.k.a. "cable" TV), digital satellite distribution, and the Internet.
[0004] Sophisticated digital transport networks have been developed, generally, for each of the above channels. Particularly, protocols have been developed for the transport of multimedia content over the Internet. For example, documents entitled Experimental Internet Stream Protocol, Version 2, RFC 1190 (October 1990; available at http://www.rfc-editor.org/go.html) and Real Time Streaming Protocol (RTSP), RFC 2326 (April 1998; available at http://www.rfc-editor.org/go.html) describe such standards, both herein incorporated by reference. The Real Time Streaming Protocol (RTSP) document discloses an application-level protocol for the control over the delivery of data with real-time properties, such data including audio and video sourced from either live feeds and stored clips. The Real Time Streaming Protocol (RTSP) document further contemplates choice from among a variety of Internet delivery channels (such as UDP, multicast UDP and TCP). These protocols, however, only govern how the data is transferred, not what the data should be {i.e., in what format the multimedia should be packaged).
[0005] One prevailing multimedia format, namely the Motion Picture Experts
Group (MPEG) format, is disclosed in exemplary fashion in U.S. Patent No. 6,148,142 issued to Anderson entitled "MULTI-USER, ON-DEMAND VIDEO SERVER SYSTEM INCLUDING INDEPENDENT, CONCURRENTLY OPERATING REMOTE DATA RETRIEVAL CONTROLLER". Anderson discloses a video server having a mass storage unit for storing a plurality of movies in digital form, specifically in an MPEG format. Anderson further discloses a plurality of identical converters, one for each conventional television receiver, for decoding and decompressing the MPEG movies. The link between the video server and the converters is disclosed as being high bandwidth. There are several disadvantages with the video server of Anderson.
[0006] One limitation is that is requires identical MPEG converter devices.
This requirement limits the utility of the video server to use only over closed networks where end-user devices can be controlled to be the same. Another limitation is that is requires a high bandwidth connection between the video server and the converter. With the large number of dial-up analog modem users (e.g., 56 kbps), and particularly in view of the gaining acceptance of wireless connections {e.g., 9.6 kbps), this requirement also limits the potential audience the video server may service. Yet another disadvantage is that is requires sufficient computing power in the converter to decode and decompress MPEG video in real time. Many handheld computing devices scarcely have the memory to store an MPEG codec, much less the computing power to decode and decompress in real time. In addition, even were the user willing to accept substantially less than real time playback {e.g., a frame every 20 seconds), such use would quickly run down the handheld's battery. Again, the video server of Anderson has utility only for a limited audience of devices. [0007] One approach taken in the art ostensibly to address the problems that arise when delivering multimedia to low memory/computing resource devices (e.g., Palm connected organizers, Pocket PCs, and the like) involves packaging the content in a proprietary format for playback using a proprietary viewer so as to allegedly improve on the frames/per/second playback speed. Exemplary of this approach includes a product available under the name FIREPRODUCER (http://www.firepad.com), which purports to deliver streams of live or stored video to multiple PalmOS handheld devices that run a client piece called FIREVIEWER. However, the output of this product is inflexible inasmuch as the proprietary video format is generally incompatible with other devices, or, at the very least, is not optimized or tailored to suit end user devices other than PalmOS.
[0008] Another approach, for example Microsoft's Netmeeting product, requires several "hard" connections be opened {i.e., ports) on the destination client device, which also must run a Microsoft Windows-based operating system. This makes such a product difficult to use through a firewall or where hard connections as described above are not available or permitted.
[0009] Another approach taken in the art to address the problem of variable end-user devices is seen in U.S. Patent 5,905,524 to Sauer entitled "DIGITAL ISDN VIDEO SERVER". Sauer discloses a video server that provides video and audio signals with different video formats and different transmission rates for different video telephones. Sauer further discloses that the different formats allow the selection of a signal quality which is suitable for a subscriber, and which can operate in accordance with the known H.261 ITU standard. The H.261 ITU standard provides for two resolution choices, with a spectrum of bit rate choices (all of which are multiples of 64kbits, particularly suited for ISDN). The system of Sauer, however, is compression-based and thus presumes significant resources in the end-user device as well as an ISDN connection.
[ooio] There remains, therefore, a need for an video server that minimizes, overcomes and/or eliminates one or more of the problems set forth above. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a simplified block diagram view of a system according to the invention. [0012] Figure 2 is a simplified block diagram showing, in greater detail, an architectural organization of the video server of Figure 1. [0013] Figure 3 is a simplified diagrammatic and block diagram view showing, in greater detail, the portions of the server software illustrated in Figure 2. [0014] Figure 4 is a simplified block diagram of a client computing architecture according to the invention. [0015] Figure 5 is a simplified block diagram view of an originating source of video data according to the invention. [0016] Figure 6 is a simplified diagram of a display buffer of Figure 5.
DETAILED DESCRIPTION
[0017] Various embodiments of a video server adapt video data to a video format that is optimized for play back at a client device based on a number of factors. Another embodiment provides, with respect to an originating source of video data, for image acquisition and communication to a video server using computing devices having limited computing resources.
[0018] A client-server system may include a client configured to display video and which is coupled to a network. The system also includes a server coupled to the network, and which is configured to determine computing resource characteristics of the client. The server is further configured to select a transmit video format from a plurality of available video formats based on the determined computing resource characteristics of the client, and to communicate the video data to the client in the selected video format.
[0019] In one embodiment, the selected transmit video format corresponds to a native display format associated with the client. This is useful in relieving the client with decoding, and decompression functions where the client has relatively limited central processing unit (CPU) computing ability, amounts of main memory, amounts of available memory, or the lack of a specialized graphics processor. In an alternative embodiment, the server is also configured to translate, for example, color video data for display on a grey- scale display and/or to scale a horizontal and vertical display size so as to fit on the client display.
[0020] Other objects, features, and advantages will become apparent to one of ordinary skill in the art from the following detailed description and drawings which illustrate the invention by way of example, but not by way of limitation.
[0021] Referring now to the drawings wherein like reference numerals are used to identify identical components in the various views, Figure 1 shows a client-server system 10 for multimedia communications in accordance with one embodiment of the present invention.
[0022] System 10 features an adaptive video server configured to select a transmit video format from a plurality of available video formats based on determined characteristics of the destination client. As described in the Background, a variety of end-user devices are available in commerce, each having unique operating and display characteristics, and for which certain video formats are more desirable and suitable than others.
[0023] Figure 1 further shows a video server 12 which may optionally have a stored multimedia database 14 associated therewith, and a plurality of client devices 16ι, 162, 163, . . ., 16N connected to the video server 12 by way of a first network 18. Network 18 may include a wireless communication portion 18w for communications with wireless clients such as client 16ι. Figure 1 further shows a plurality of originating video sources 20ι, . . . , 20N connected to video server 12 by way of a second network 22. First network 18 and second network 22 may be the same network. As used herein, video means multimedia content including moving video, still video, audio, or a combination thereof.
[0024] Video server 12 is configured to communicate either (i) pre-stored multimedia content, such as contained in its local database 14, or (ii) streaming multimedia content received from originating sources 20, to client devices 16 that have connected therewith and have requested content. Video server 12 may be implemented on conventional computing hardware. For purposes of example only, server 12 may comprise a personal computer (PC) platform executing a windows operating system (e.g., Windows 98SE, Windows ME, Windows 2000, etc.), an Apple Macintosh computer system, a Unix-based computer, a Linux-based computer, and the like. As such, video server 12 includes conventional central processor(s), main memory, cache memory, hard disk drive, communication ports, keyboard, mouse, display and the like. In addition to the functionality described herein, video server 12 may further include conventional server software, for example, conventional web server software such as the widely available Apache web server software for Linux-based systems (e.g., recently available Apache 1.3.19 version available at http://httpd.apache.org/). It should be understood, however, that server 12 need not comprise a web (or HTTP) server where, in particular, the client(s) 16ι, 162, 163, . . . , 16N are not communicating with server 12 using the HTTP protocol.
[0025] The stored multimedia database 14 may comprise any type of content
(e.g., entertainment, educational, etc.) in any one of a number of known video formats including, without limitation, the Motion Picture Experts Group (MPEG) format (e.g., MPEG-1 , MPEG-2, MPEG-4, etc.), Microsoft video format-AVI format, Quick Time format, or any other type of format now known, or hereinafter developed. Stored multimedia database 14 may reside, physically, on a hard disk drive, or other conventional apparatus (e.g., CD or DVD tower).
[0026] Client devices 16ι, 162, 163, . . . , 16N are each configured to display video and are, as shown, coupled to network 18. Certain of the client devices 16 may include a client module to enhance the exchange of information between client 16 and server 12, the content of which will be described in greater detail in connection with Figure 3. For example, client devices 162, 163 are shown having such a client module, while client devices 16ι and 16N do not have such a client module. However, all clients have, at a minimum, capabilities to display video, as mentioned above. The particular configuration of the plurality of client devices 16 varies greatly, and may be any one of a plurality of conventional computing devices commercially available. For purposes of example only, client devices 16 may comprise a desktop or laptop personal computer (PC) executing a windows-based operating system (e.g., Windows 98SE, Windows ME, Windows 2000, etc.), an Apple Macintosh computer, a Unix work station, a Linux work station, a hand held computer such as a Microsoft Windows CE-based device {i.e., a Pocket PC), a Palm or Hand Spring connected organizer, any variety of laptop computers, wireless telephones having internet access (e.g., WAP enabled), such as, for example, a commercially available product from Kyocera (Webphone), video tablets, wireless wrist watches (e.g., commercially available from Matsucom), Internet appliances (e.g., 3COM's "Audrey"), as well as a variety of other computing devices, generally. The client devices may be either connected to network 18 by way of a wire line, such as shown for client devices 162, 163 and 16N or may be connected to network 18 by way of a wireless link, as for the case of client device 16ι connected to wireless portion 18w of network 18 (connection shown in phantom line).
[0027] As should be understood, especially in view of the previous description of the range of devices for client device 16, not all video formats may be practically played back on any particular client device 16. Video server 12 may select one of a plurality of video formats based on a profile of the requesting client device 16. The information contained in the profile will be described in connection with Figure 3.
[0028] Referring now to Figure 2, a simplified block diagram view of video server 12 is shown. As illustrated, video server 12 includes an operating system 24, server software 26, and, optionally, one or more virtual machines 28ι, 282, . . ., 28N which may be established by server software 26 depending on the assessment of the profile of a requesting client device 16. In particular, it is quite often the case that content stored in multimedia database 14 is stored in a compressed form (e.g., MPEG-2) in order to reduce the amount of disk space taken up by the content, for example, multiple movies. However, many end-user client devices 16 have neither the computing resources to decode and decompress an MPEG-2 stream in real time, nor have the appropriate display to show the decoded and decompressed MPEG-2 video even after such processing has been completed. For example, the original movie may be "full screen" and in color, while the client display may be a much smaller grey scale screen. Accordingly, server software 26, when it determines that a change in video format is desirable for a particular client device 16, establishes a respective virtual machine 28 in the server memory itself, where the requested multimedia content is played back/translated, or otherwise formatted so as to optimize the experience for a user of the requesting client device. A similar construct is known in the art and is referred to as a "thin client" approach. The actual processing power, therefore, is drawn from the server, not the client.
[0029] Figure 3 shows, in greater detail, the server software 26 according to one embodiment of the invention. As shown in block, functional form, server software 26 includes input/output (I/O) 30, a client identity detect module 32, a client computer resource characteristic detect module 34, a client display characteristic detect module 36, a server-to-client bandwidth detect module 38, a client battery detect module 40, a video format select module 42, and a video data formatter module 44. Video data formatted in a transmit video format is output as a stream of messages 45. A respective client profile is developed using the detected client information.
[0030] Client identity detect module 32 is configured to detect an identity of a particular client device 16 and generate a signal SO indicative of such determined identity. For example, such identification information may include the client device manufacturer, model number, operating system, and any special hardware or software. It should be appreciated that signal SO is shown simply as a single signal line for clarity. Of course, the identification may, in-fact, comprise, for example, text strings corresponding to the above described information, as well as perhaps numerical information. [0031] Computing resource characteristic detect module 34 is configured to obtain information useful to assess, generally, the capability of a client device 16 to decode and decompress a multimedia content stream which, as described above, is commonly compressed. As used herein, computing resource characteristics may include the type and clock speed of a central processing unit (CPU), including any other characterizing computing ability (e.g., MMX capability), an amount of main memory, an amount of available memory, as well as an availability of any specialized graphics processor. Module 34 determines one or more of the foregoing characteristics and generates a signal, designated S1 , indicative of such information. As with signal SO, it is shown as "single" line for clarity only.
[0032] Module 36 is configured to determine display characteristics of a client device 16. Display characteristics as used herein may include the availability of a color display and a color depth associated with each pixel, an availability of a grey-scale display and the number of levels of grey associated therewith, the availability of a black/white display, and a horizontal and vertical display size (e.g., in pixels). In some instances, determination of the identity of the client uniquely determines its display capabilities (e.g., a Palm Mix is a 160 x 160 pixel, 4-bit grey scale display). Display characteristic detect module 36 generates an output signal S2 indicative of the determined display characteristics of a particular client device 16. Again, signal S2, along with signals S3, S4 and S5, are shown as a single line for clarity purposes only.
[0033] Bandwidth detect module 38 is configured to detect an available bandwidth between video server 12 and a requesting client device 16 through network 18. Bandwidth detect module 38 generates a signal, designated S3 indicative of the determined available bandwidth. In addition, connectivity characteristics such as delay, turnaround time, whether the communication is "connectionless" or not (e.g., TCP/IP and opened ports, etc. vs. video data tunneled via HTTP), may also be included in the information in signal S3.
[0034] Battery detect module 40 is configured to determine whether a particular client device 16 is powered by way of batteries, and if so, the type and optionally the state of charge thereof. Battery detect module 40 generates an output signal, designated S4, indicative of the determined battery characteristics of the particular client device 16.
[0035] As described above, in one embodiment, client devices 16 include a client module that is configured to communicate and cooperate with video server software 26 of video server 12 in obtaining the foregoing information. In such an embodiment, as shown in Figure 4, such client software 46 is configured to communicate with, for example, application programming interfaces (APIs) 47 of the operating system of the client device 16 in order to ascertain one or more of the items of information described above with respect to modules 32, 34, 36, 38, and 40. For example, in the case of a Palm OS based handheld computer, a set of publicly known APIs are available to determine a wide variety of client device information, such as the information set forth above. In this regard, calling the publicly documented APIs to obtain the aforementioned information requires no more than routine application of programming capabilities of one of ordinary skill in the art and therefore need not be described in detail. For further illustration purposes only, a Palm software development kit (e.g., SDK 3.5) is freely available and widely distributed, for example, at http://www.palmos.com. Similar software development kits are available for other devices (e.g., Microsoft Windows CE APIs for a pocket PC device, Windows 98 APIs for a desktop or laptop client device, etc.).
[0036] As a matter of general principle, the aforementioned information associated with a particular client device 16 may affect the selection of a video format. A general overview of the selection criteria will now be set forth, with concrete examples to follow.
[0037] As to the identity signal SO, such information may be dispositive in selecting a transmit video format. For example, a particular client device 16 may comprise a Microsoft Windows-based operating system. In such case, a windows video format (e.g., a file having a * avi extension) may be used inasmuch as it has been determined that the windows operating system video drivers prefer incoming video data via an * avi format {i.e., as opposed to an MPEG format or a Quick Time format for example). The client identity may also uniquely determine some computing resource characteristics and/or screen display (as mentioned above).
[0038] The computing resource characteristics {i.e., signal S1 ) largely reflect the client device's capability to decode and decompress certain synchronized and compressed video formats such as the MPEG-2 format, and Real Time Video (RTV) format. For example, it is contended in the literature that 17 frames per second are needed by the human visual cortex to assimilate real time video. Depending on the client device's display characteristics, and the available bandwidth through the network to the client device, the video server 12 may determine that no compression should be used in order to provide the best experience to the user of that particular client device 16 in terms of frames per second, for example. This means that a bitmap video format would be sent.
[0039] The display characteristics (signal S2) may also be considered together with the other signals in determining what transmit video format to select. For example, a movie stored in multimedia storage database 14 in an MPEG-2 format, that is in a color format, in a 1024 x 768 pixel size, for example, will not only have to be decoded and decompressed at the client device prior to display, but may also have to be converted from color to grey- scale, as well as to be scaled down to the display window of the client device (e.g., 160 x 160 pixels in a Palm OS connected organizer). Sufficient computing resources may be unavailable to perform the decoding, decompression and the color-to-grey scale and scaling conversions in a suitably high frame rate. In such case, the server software may perform the color-to-grey scale and size scaling transformations on the server side and send that in either compressed or uncompressed format to the client device 16.
[0040] The available bandwidth, designated S3, may impact selection of a video format as follows. For example, if a provisional selection by server 12 (perhaps based on other factors) of a video format would be to send uncompressed multimedia content to the client device 16, but the connection is bandwidth limited, for example over a wireless link, then a compromise must be made. The compromise reconciles (i) attempting to optimize frames per second at the client by reducing computationally intensive decoding and decompression (which may unfortunately increase the total payload), and (ii) total bandwidth from server to client (which may be limited and insufficient to carry uncompressed data). That is, some level of compression may be required in order to fit the video data within the maximum available bandwidth. A video format having some level of compression would be selected (e.g., MPEG-4), and the user of the particular client device 16 would be presented with a perhaps reduced frame rate than would otherwise be possible if an increased available bandwidth connection were available and no compression used.
[0041] The battery detect signal, designated S4, reflects the understanding that battery drain is a function of both computational activity and screen activity. A client device dependent on batteries can ill afford a total drain. Transferring an uncompressed bit map image to a display buffer at a client device 16, for example, is much less computationally intensive than decoding and decompressing an MPEG-2 multimedia stream. Such video format (bitmap; uncompressed) would be selected to reduce battery drain. Therefore, video server 12 may incorporate battery management into its decision structure.
[0042] With continued reference to Figure 4, the video format select module
42 is responsive to one or more of the signals S0-S4 for selecting a transmit video format, which is represented by a video format selection signal S5. Applying the foregoing principles, video format select module 42 selects a transmit video format, which is then used by video data formatter module 44 in ensuring compliance of the requested video data with the selected video format. Several examples will now be set forth. EXAMPLE 1
[0043] Client device 16 includes the client software 46 referred to above, and is identified, for example, as a Windows 98-based machine including 256 MB of SDRAM, and 800 MHz Intel Pentium III central processor, 40 GB hard drive, with a 3D video graphics card having display settings set to 1024 x 768 pixels, 32 bit color depth (true color), connected to the internet via a cable modem having a bandwidth in excess of T1 speeds {i.e., greater than 1.5 Mbps), directly operating from 120 volt line voltage {i.e., no batteries). The requested multimedia content is in an MPEG-2 stored video format. Video format select module 42 makes a determination that the described client device 16 has sufficient computing resources to decode and decompress the requested MPEG-2 format movie in real time. Moreover, any color display characteristics {i.e., color depth) or scaling {i.e., display size) can also be accomplished at the client device 16 in real time. In the described example, the available bandwidth does not impose any limitations, nor is battery usage a factor. While the identification of the client device 16 as a Microsoft Windows-based machine may in some situations counsel in favor of an * avi video format, server resource usage can be optimized {i.e., no need to make an MPEG-2-to-AVI translation), without any detriment on the perceived experience for the user at the client device end, by the server 12 passing through the retrieved MPEG movie to the requesting client device 16. The transmitted video data is shown in Figure 3 as a message 45 containing video data in an MPEG video format {i.e., the selected transmit video format). No virtual machine 28 is created by server software 26 in this Example 1. The foregoing constitutes one client profile.
EXAMPLE 2
[0044] The client device 16 comprises a so-called Pocket PC having 16 MB of
ROM, 32 MB of RAM, and a 240 x 320 TFT LCD screen display with more than 4,000 colors. This client device 16 includes an Ethernet {i.e., a local area network) type connection to network 18, which is coupled to video server 12. This client device 16 runs on batteries. The requested multimedia content is again in an MPEG stored video format. The transmit video format selected by module 42, in this second example, is an AVI video format, in view of the moderate level of computing resources available on such a client device, the display characteristics, and the operating system of the client device (e.g., Microsoft Windows-based operating system). The selection is communicated to video data formatter 44 via signal S5. Video data formatter 44 establishes a virtual machine, for example virtual machine 28τ in the memory of server 12, for translating from MPEG to AVI. Both MPEG and AVI codecs (coders-decoders) are publicly known and commercially available, as known to those of ordinary skill in the art. Thus, the data stream being retrieved from stored multimedia database 14 is first converted from MPEG into raw video data, and which is then scaled, and whose color depth is further corrected to match that of the client device, and then encoded using an AVI codec. The outgoing stream from video data formatter 44 is sent out via I/O 30 as a stream of messages 45 containing video data in the transmit video format (here, the AVI format). Effectively, much of the processing power required to obtain playback in the client's native format, working from the original compressed MPEG file, is actually borne by the server, not the client. This enhances the client's experience. The foregoing constitutes a second client profile.
EXAMPLE 3
[0045] The client device 16 is a Palm OS connected organizer with a 160 x
160, 16-grey level display, connected by way of a wireless connection (e.g., GSM/PCN, CDMA 2G, 2.5G, etc.). The bandwidth may be 9.6 kbps-14.4 kbps. The client device 16 in this example includes client software 46 for enhanced information gathering and communication upstream to video server 12 through network 18W, 18. Video format select module 42 selects an uncompressed, 16-level grey-scale video format with a scaled size of approximately 160 x 160 pixels for communication to this client 16. The video format selection is provided via signal S5 to video data formatter 44. Video data formatter 44 initiates another virtual machine 282. As the requested content, which is in an MPEG format, is retrieved from stored multimedia 14, it is, in effect, running (play back) within virtual machine 282. Video data formatter 44, in this example, is configured to convert from color to the 16- level grey scale, as well as to perform a scaling operation to reduce the size so as to be immediately usable for display on the client device 16. In this example, the selected transmit video format is a native video format associated with the described client device 16. All that the client device needs to do is to transfer the incoming, uncompressed (native video format) data from the receive buffer to the display buffer. The foregoing example further illustrates how computing resources are drawn from the device in the chain best adapted to handle (in this example, the server). The foregoing describes a third client profile.
[0046] It should be understood that a whole spectrum of pass through, format conversion, and other variations will be possible depending on all of the factors described above, and yet remain with the spirit and scope of the present invention. In one embodiment, a plurality of client devices 16 of the above-described variety of configurations, are serviced concurrently by video server 12. It should be further understood that video data formatter 44 can be configured to select streaming video feeds that are provided on an alternate input thereto in addition to, or in lieu of, retrieving stored multimedia from stored multimedia database 14. The video server 12 may be adaptive depending on the profile of the client device being serviced in order to provide a better experience to the user. This arrangement overcomes many of the problems in the art, with single purpose or inflexible servers.
[0047] With continued reference to Figure 1 , streaming video is provided from a plurality of physically separate locations comprising respective originating video source(s) designated 20ι, . . ., 20N. Each source 20ι, . . ., 20N include an imaging device 48, a computing device 50 cooperating together to image and stream video data corresponding to a respective view 52. [0048] Figure 5 shows a single originating source 20ι in greater detail.
Imaging device 48 may be any one of a number of conventional imaging devices, such as still digital cameras, video cameras, and the like. In a constructed embodiment, however, a low-cost web cam is employed, which is a commercially available component, for example, available from D-Link, capable of capturing 30 frames per second, 64,000,000 colors, having a universal serial bus (USB), interface, model no. DSB-C300. Of course, any number of alternative imaging devices, comparable in function, may be substituted therefore.
[0049] Computing device 50ι may comprise conventional computing apparatus, having a central processing unit (CPU) 54 coupled, as shown in diagrammatic fashion, to a common bus 55, to which communication is enabled with a main memory 56, a display buffer 58, and an input/output (I/O) 60. In a common configuration, image data 62 generated by imaging device 48ι, which corresponds to view 52, is provided to device 50ι in a compressed format, such as an *.avi video format. Commonly, software provided with imaging devices of the type described {i.e., web cams), allow a user to, among other things, save the incoming video stream 62 to a local hard drive, or to simply allow the image data 62 to be continuously displayed on a locally connected display in a "preview" mode {i.e., the software converts the image data 62 into a bit map and moves the bit map into the display buffer 58). The incoming image data 62, in the * avi format, for example, is not stored to a local hard drive, but is rather allowed to free run in the "preview" mode. In many computationally, and computing resource weak computing devices 50 it may be difficult to rescale, change from color-to-black and white, or color-to- grey scale, or compress in another format, or do any other conditioning of the image data 62 in real time in addition to the original decompression from AVI format {i.e., a significant portion of the resources have already been consumed converting the incoming image data 62 to a bit map video format for transfer to the display buffer 58). [0050] Figure 6 shows display buffer 58 in greater detail, having a first portion
66, a portion 68 containing the bit map of the captured view 52, and a remainder portion of the display buffer 70. Accordingly, a module executing from main memory 56 is configured to retrieve the bit map 68 from display buffer 58 and forward the same via I/O 60 as a message 64 containing the bit map data, destined through network 22 to video server 12. It should be understood that Figure 6 is simplified, and that the bit map portion 68 of display buffer 58 need not be linear and contiguous, as shown. Through the foregoing, with a low cost imaging device 48, and a low cost computing device 50, which need not even have a display attached or a keyboard attached thereto, can together cooperate to form a digital image acquisition package. A plurality of such configurations may be deployed at a plurality of distinct physical locations, for example purposes only, for distributed security monitoring. The stream of messages 64 comprising frames in the bit map video format, when routed through video server 12, and when destined for, for example, a relatively low powered client device (e.g., a Palm OS connected organizer, as described above in Example 3), may simply "pass through" the bit map to the client device 16. To that end, the embodiment of Figure 5 can be used in conjunction with the video server to form an end-to-end adaptive video capture and client data server system for improved video communication. A user with a handheld device is then enabled to monitor a plurality of different views at a remote location.
[0051] In this embodiment, a flexible approach to the distribution of multimedia content is provided, wherein resource intensive tasks are allocated to a computing device best suited/best capable of performing such resource intensive tasks. If the video server is the node in the system having the greatest computing power, most of the conditioning work on the video data is performed there. If the client has extensive capabilities, then the video server just serves up the content, which is usually compressed (MPEG, AVI), and allows the client to decode and decompress it. Likewise, if the originating source {i.e., image or video capture source) is resource limited, a bitmap only is extracted and sent upstream to the video server. The system described herein provides a comprehensive approach applicable across many of the common client devices today, and is particularly suited for wireless, and/or handheld computing devices.

Claims

1. A client-server system comprising: a client configured to display video coupled to a network; a server coupled to said network configured to determine computing resource characteristics of said client, said server being further configured to select a transmit video format from a plurality of video formats based on said determined characteristics and to communicate video data to said client in said transmit video format.
2. The system of claim 1 wherein said video data is stored on said server in a stored video format.
3. The system of claim 2 wherein said stored video format is the same as said transmit video format.
4. The system of claim 3 wherein said stored video format is a Motion Picture Experts Group (MPEG) compliant video format.
5. The system of claim 4 wherein said stored video format is an MPEG-2 compliant format.
6. The system of claim 2 wherein said stored video format is different than said transmit video format.
7. The system of claim 6 wherein said stored video format is a Motion Picture Experts Group (MPEG) compliant video format and said transmit format is a native video format associated with said client.
8. The system of claim 1 wherein said computing resource characteristics is selected from the group comprising central processing unit (CPU) computing ability, an amount of main memory, an amount of available memory, and an availability of a specialized graphics processor.
9. The system of claim 1 wherein said server is configured to determine display characteristics of said client, said server being configured to select said transmit video format further as a function of said display characteristic.
10. The system of claim 9 wherein said display characteristic is selected from the group comprising an availability of color display and color depth associated therewith, an availability of grey-scale display and a number of levels of gray associated therewith, an availability of black/white display, and a horizontal and vertical display size.
11. The system of claim 1 wherein said server is configured to determine a bandwidth through said network between said server and said client, said server being configured to select said transmit video format further as a function of said determined bandwidth.
12. The system of claim 1 wherein said video data is fed to said server from an originating source.
13. The system of claim 12 wherein said originating source comprises: a computing device having a main memory and a video buffer; and an imaging device coupled to said computing device; said computing device being configured to receive image data from said imaging device of a first view and store a bitmap of said first view in a portion of said video buffer, said computing device being further configured to communicate said portion of said video buffer to said server.
14. The system of claim 12 wherein said video data is fed to said server from a plurality of originating sources.
15. The system of claim 12 wherein said server is configured to transmit said video data to a plurality of clients.
16. The system of claim 12 wherein said plurality of sources are disposed in at least two different physical locations.
17. The system of claim 1 wherein said server is configured to determine an identity of said client, said server being configured to select said transmit video format further as a function of said determined identity.
18. The system of claim 1 wherein said server is configured to transmit video data to a plurality of clients wherein a first client differs from a second client in at least one of computing resource characteristic, display characteristic, bandwidth through said network to said server, or identity.
19. The system of claim 18 wherein said server is further configured to transmit said video data to said plurality of client concurrently.
20. The system of claim 1 wherein said server is configured to determine (i) display characteristics of said client, (ii) a bandwidth through said network between said server and said client, and (iii) an identity of said client, said server being configured to select said transmit video format further as a function of said determined display characteristic, said determined bandwidth, and said determined identity.
21. The system of claim 1 wherein said server is configured to determine a battery characteristic associated with said client, said server being further configured to select said transmit format further as a function of said determined battery characteristic.
22. A client-server system comprising: a client configured to display video coupled to a network; a server coupled to said network configured to determine computing resource characteristics of said client available for processing compressed video, said server being configured to select a transmit video format from a plurality of video formats based on said determined characteristics and to communicate video data to said client in said transmit video format.
23. The system of claim 22 wherein said server is configured to determine at least one of (i) a display characteristic of said client, (ii) a bandwidth through said network between said server and said client, (iii) an identity of said client, and (iv) a battery characteristic associated with said client, said server being further configured to select said transmit format further as a function of at least one of said determined display characteristic, said determined bandwidth, said determined identity or said determined battery characteristic.
24. The system of claim 23 wherein said video data is fed to said server from an originating source, said originating source comprising: a computing device having a main memory and a video buffer; and an imaging device coupled to said computing device; said computing device being configured to receive image data from said imaging device of a first view and store a bitmap of said first view in a portion of said video buffer, said computing device being further configured to communicate said portion of said video buffer to said server in said bitmap format.
25. A method of communicating video data over a network to a client comprising: determining a computing resource characteristic of said client; and selecting a transmit video format based on the determined computing resource characteristic of said client.
26. The method of claim 25 wherein the transmit video format is selected based on at least one of a display characteristic of said client, an identity of said client or an available bandwidth to said client.
PCT/US2002/017118 2001-05-31 2002-05-31 Adaptive video server WO2002097584A2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US29483501P 2001-05-31 2001-05-31
US60/294,835 2001-05-31
USNONE 2002-10-22

Publications (1)

Publication Number Publication Date
WO2002097584A2 true WO2002097584A2 (en) 2002-12-05

Family

ID=23135141

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/017118 WO2002097584A2 (en) 2001-05-31 2002-05-31 Adaptive video server

Country Status (1)

Country Link
WO (1) WO2002097584A2 (en)

Cited By (130)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004100500A2 (en) * 2003-05-05 2004-11-18 Thomson Licensing S.A. System and method for communicating with a display device via a network
EP1684515A1 (en) * 2005-01-24 2006-07-26 Microsoft Corporation System and method for gathering and reporting screen resolutions of attendees of a collaboration session
WO2006113776A2 (en) * 2005-04-20 2006-10-26 Jupiter Systems Audiovisual signal routing and distribution system
WO2007017407A1 (en) * 2005-08-05 2007-02-15 France Telecom Method for controlling a multimedia device from a mobile terminal, corresponding computer programs, multimedia device and server
WO2007030370A1 (en) * 2005-09-07 2007-03-15 Scientific-Atlanta, Inc. Optimizing data rate for video services
WO2008094595A3 (en) * 2007-01-29 2008-10-02 Ntt Docomo Inc Optimized content-based and royalty-based encoding and distribution of media data
WO2009002324A1 (en) * 2007-06-28 2008-12-31 Thomson Licensing Method, apparatus and system for providing display device specific content over a network architecture
US20090070473A1 (en) * 2005-03-16 2009-03-12 Marc Baum Method for Data Routing in Networks
WO2010047706A1 (en) * 2008-10-23 2010-04-29 Hewlett-Packard Development Company, L.P. Decompressing a video stream on a first computer system, and scaling and displaying the video stream on a second computer system
US7934232B1 (en) 2000-05-04 2011-04-26 Jerding Dean F Navigation paradigm for access to television services
US7962370B2 (en) 2000-06-29 2011-06-14 Rodriguez Arturo A Methods in a media service system for transaction processing
US7975277B1 (en) 2000-04-03 2011-07-05 Jerding Dean F System for providing alternative services
US7992163B1 (en) 1999-06-11 2011-08-02 Jerding Dean F Video-on-demand navigational system
US8006273B2 (en) 2001-06-29 2011-08-23 Rodriguez Arturo A Updating download options for unavailable media content
US8006262B2 (en) 2001-06-29 2011-08-23 Rodriguez Arturo A Graphic user interfaces for purchasable and recordable media (PRM) downloads
US8020184B2 (en) 1999-06-11 2011-09-13 Jerding Dean F Channel control system for exiting from an interactive program guide
US8032914B2 (en) 2000-11-10 2011-10-04 Rodriguez Arturo A Systems and methods for dynamically allocating bandwidth in a digital broadband delivery system
US8037504B2 (en) 1999-06-11 2011-10-11 Jerding Dean F Video on demand system with selectable options of configurable random-access control
US8069259B2 (en) 2000-06-09 2011-11-29 Rodriguez Arturo A Managing removal of media titles from a list
GB2484406A (en) * 2010-10-07 2012-04-11 Ocean Blue Software Filtering available content for consumer device based on device identifier requested by server
US8161388B2 (en) 2004-01-21 2012-04-17 Rodriguez Arturo A Interactive discovery of display device characteristics
CN102474652A (en) * 2009-07-16 2012-05-23 纳瑞塔有限责任公司 Systems and methods for managing content in real-time
US8191093B2 (en) 2001-06-29 2012-05-29 Rodriguez Arturo A Providing information pertaining to audio-visual and personal bi-directional services
EP2437509A3 (en) * 2010-09-30 2013-03-06 Comcast Cable Communications, LLC Delivering content in multiple formats
US8442377B2 (en) 2011-08-18 2013-05-14 International Business Machines Corporation Intelligent recording
US8516525B1 (en) 2000-06-09 2013-08-20 Dean F. Jerding Integrated searching system for interactive media guide
US8547997B2 (en) 2005-04-20 2013-10-01 Jupiter Systems Capture node for use in an audiovisual signal routing and distribution system
US8553716B2 (en) 2005-04-20 2013-10-08 Jupiter Systems Audiovisual signal routing and distribution system
CN103430508A (en) * 2011-04-01 2013-12-04 华为技术有限公司 Streaming media service processing system, method and network device therefor
US8606949B2 (en) 2005-04-20 2013-12-10 Jupiter Systems Interconnection mechanism for multiple data streams
US8640172B2 (en) 2001-06-29 2014-01-28 Cisco Technology, Inc. System and method for characterization of purchasable and recordable media (PRM)
JP2014017003A (en) * 2013-08-27 2014-01-30 Thomson Licensing Method, apparatus and system for providing display device specific content over network architecture
EP2696552A1 (en) * 2012-08-07 2014-02-12 NTT DoCoMo, Inc. Method, system and network for transmitting multimedia data to a plurality of clients
US8707153B2 (en) 2000-06-09 2014-04-22 Cisco Technology, Inc. Displaying comment data corresponding to a video presentation
US8745656B2 (en) 2002-02-11 2014-06-03 Cisco Technology, Inc. Tracking of presented television advertisements
US20140379852A1 (en) * 2013-06-20 2014-12-25 Wipro Limited System and method for subscribing to a content stream
GB2483045B (en) * 2009-06-01 2015-03-11 Sony Comp Entertainment Us Qualified video delivery
US9287727B1 (en) 2013-03-15 2016-03-15 Icontrol Networks, Inc. Temporal voltage adaptive lithium battery charger
US9306809B2 (en) 2007-06-12 2016-04-05 Icontrol Networks, Inc. Security system with networked touchscreen
US9349276B2 (en) 2010-09-28 2016-05-24 Icontrol Networks, Inc. Automated reporting of account and sensor information
US9380327B2 (en) 2011-12-15 2016-06-28 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
US9412248B1 (en) 2007-02-28 2016-08-09 Icontrol Networks, Inc. Security, monitoring and automation controller access and use of legacy security control panel information
US9450776B2 (en) 2005-03-16 2016-09-20 Icontrol Networks, Inc. Forming a security network including integrated security system components
US20160274759A1 (en) 2008-08-25 2016-09-22 Paul J. Dawes Security system with networked touchscreen and gateway
US9509935B2 (en) 2010-07-22 2016-11-29 Dolby Laboratories Licensing Corporation Display management server
US9510065B2 (en) 2007-04-23 2016-11-29 Icontrol Networks, Inc. Method and system for automatically providing alternate network access for telecommunications
US9531593B2 (en) 2007-06-12 2016-12-27 Icontrol Networks, Inc. Takeover processes in security network integrated with premise security system
US9609003B1 (en) 2007-06-12 2017-03-28 Icontrol Networks, Inc. Generating risk profile using data of home monitoring and security system
US9621408B2 (en) 2006-06-12 2017-04-11 Icontrol Networks, Inc. Gateway registry methods and systems
US9628440B2 (en) 2008-11-12 2017-04-18 Icontrol Networks, Inc. Takeover processes in security network integrated with premise security system
US9654751B2 (en) 2006-12-21 2017-05-16 Thomson Licensing Method, apparatus and system for providing color grading for displays
US9729342B2 (en) 2010-12-20 2017-08-08 Icontrol Networks, Inc. Defining and implementing sensor triggered response rules
US9867143B1 (en) 2013-03-15 2018-01-09 Icontrol Networks, Inc. Adaptive Power Modulation
US9928975B1 (en) 2013-03-14 2018-03-27 Icontrol Networks, Inc. Three-way switch
CN107851425A (en) * 2015-08-05 2018-03-27 索尼公司 Message processing device, information processing method and program
US10051078B2 (en) 2007-06-12 2018-08-14 Icontrol Networks, Inc. WiFi-to-serial encapsulation in systems
US10062245B2 (en) 2005-03-16 2018-08-28 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US10062273B2 (en) 2010-09-28 2018-08-28 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US10079839B1 (en) 2007-06-12 2018-09-18 Icontrol Networks, Inc. Activation of gateway device
US10078958B2 (en) 2010-12-17 2018-09-18 Icontrol Networks, Inc. Method and system for logging security event data
US10091014B2 (en) 2005-03-16 2018-10-02 Icontrol Networks, Inc. Integrated security network with security alarm signaling system
US10127801B2 (en) 2005-03-16 2018-11-13 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US10142392B2 (en) 2007-01-24 2018-11-27 Icontrol Networks, Inc. Methods and systems for improved system performance
EP3413572A1 (en) * 2005-12-29 2018-12-12 Rovi Guides, Inc. An interactive media guidance system having multiple devices
US10156959B2 (en) 2005-03-16 2018-12-18 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US10156831B2 (en) 2004-03-16 2018-12-18 Icontrol Networks, Inc. Automation system with mobile interface
US10178203B1 (en) 2014-09-23 2019-01-08 Vecima Networks Inc. Methods and systems for adaptively directing client requests to device specific resource locators
US10200504B2 (en) 2007-06-12 2019-02-05 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US10237237B2 (en) 2007-06-12 2019-03-19 Icontrol Networks, Inc. Communication protocols in integrated systems
US10313303B2 (en) 2007-06-12 2019-06-04 Icontrol Networks, Inc. Forming a security network including integrated security system components and network devices
US10339791B2 (en) 2007-06-12 2019-07-02 Icontrol Networks, Inc. Security network integrated with premise security system
US10348575B2 (en) 2013-06-27 2019-07-09 Icontrol Networks, Inc. Control system user interface
US10365810B2 (en) 2007-06-12 2019-07-30 Icontrol Networks, Inc. Control system user interface
US10380871B2 (en) 2005-03-16 2019-08-13 Icontrol Networks, Inc. Control system user interface
US10382452B1 (en) 2007-06-12 2019-08-13 Icontrol Networks, Inc. Communication protocols in integrated systems
US10389736B2 (en) 2007-06-12 2019-08-20 Icontrol Networks, Inc. Communication protocols in integrated systems
US10423309B2 (en) 2007-06-12 2019-09-24 Icontrol Networks, Inc. Device integration framework
US10498830B2 (en) 2007-06-12 2019-12-03 Icontrol Networks, Inc. Wi-Fi-to-serial encapsulation in systems
US10522026B2 (en) 2008-08-11 2019-12-31 Icontrol Networks, Inc. Automation system user interface with three-dimensional display
US10523689B2 (en) 2007-06-12 2019-12-31 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US10530839B2 (en) 2008-08-11 2020-01-07 Icontrol Networks, Inc. Integrated cloud system with lightweight gateway for premises automation
US10559193B2 (en) 2002-02-01 2020-02-11 Comcast Cable Communications, Llc Premises management systems
US10616075B2 (en) 2007-06-12 2020-04-07 Icontrol Networks, Inc. Communication protocols in integrated systems
US10645347B2 (en) 2013-08-09 2020-05-05 Icn Acquisition, Llc System, method and apparatus for remote monitoring
US10666523B2 (en) 2007-06-12 2020-05-26 Icontrol Networks, Inc. Communication protocols in integrated systems
US10721087B2 (en) 2005-03-16 2020-07-21 Icontrol Networks, Inc. Method for networked touchscreen with integrated interfaces
US10747216B2 (en) 2007-02-28 2020-08-18 Icontrol Networks, Inc. Method and system for communicating with and controlling an alarm system from a remote server
US10785319B2 (en) 2006-06-12 2020-09-22 Icontrol Networks, Inc. IP device discovery systems and methods
US10979389B2 (en) 2004-03-16 2021-04-13 Icontrol Networks, Inc. Premises management configuration and control
US10999254B2 (en) 2005-03-16 2021-05-04 Icontrol Networks, Inc. System for data routing in networks
US11089122B2 (en) 2007-06-12 2021-08-10 Icontrol Networks, Inc. Controlling data routing among networks
US11113950B2 (en) 2005-03-16 2021-09-07 Icontrol Networks, Inc. Gateway integrated with premises security system
US11146637B2 (en) 2014-03-03 2021-10-12 Icontrol Networks, Inc. Media content management
US11182060B2 (en) 2004-03-16 2021-11-23 Icontrol Networks, Inc. Networked touchscreen with integrated interfaces
US11201755B2 (en) 2004-03-16 2021-12-14 Icontrol Networks, Inc. Premises system management using status signal
US11212192B2 (en) 2007-06-12 2021-12-28 Icontrol Networks, Inc. Communication protocols in integrated systems
US11218878B2 (en) 2007-06-12 2022-01-04 Icontrol Networks, Inc. Communication protocols in integrated systems
US11237714B2 (en) 2007-06-12 2022-02-01 Control Networks, Inc. Control system user interface
US11244545B2 (en) 2004-03-16 2022-02-08 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US11258625B2 (en) 2008-08-11 2022-02-22 Icontrol Networks, Inc. Mobile premises automation platform
US11277465B2 (en) 2004-03-16 2022-03-15 Icontrol Networks, Inc. Generating risk profile using data of home monitoring and security system
US11310199B2 (en) 2004-03-16 2022-04-19 Icontrol Networks, Inc. Premises management configuration and control
US11316958B2 (en) 2008-08-11 2022-04-26 Icontrol Networks, Inc. Virtual device systems and methods
US11316753B2 (en) 2007-06-12 2022-04-26 Icontrol Networks, Inc. Communication protocols in integrated systems
US11343380B2 (en) 2004-03-16 2022-05-24 Icontrol Networks, Inc. Premises system automation
US11368327B2 (en) 2008-08-11 2022-06-21 Icontrol Networks, Inc. Integrated cloud system for premises automation
US11405463B2 (en) 2014-03-03 2022-08-02 Icontrol Networks, Inc. Media content management
US11424980B2 (en) 2005-03-16 2022-08-23 Icontrol Networks, Inc. Forming a security network including integrated security system components
US11423756B2 (en) 2007-06-12 2022-08-23 Icontrol Networks, Inc. Communication protocols in integrated systems
US11451409B2 (en) 2005-03-16 2022-09-20 Icontrol Networks, Inc. Security network integrating security system and network devices
US11489812B2 (en) 2004-03-16 2022-11-01 Icontrol Networks, Inc. Forming a security network including integrated security system components and network devices
US11496568B2 (en) 2005-03-16 2022-11-08 Icontrol Networks, Inc. Security system with networked touchscreen
US11582065B2 (en) 2007-06-12 2023-02-14 Icontrol Networks, Inc. Systems and methods for device communication
US11601810B2 (en) 2007-06-12 2023-03-07 Icontrol Networks, Inc. Communication protocols in integrated systems
US11615697B2 (en) 2005-03-16 2023-03-28 Icontrol Networks, Inc. Premise management systems and methods
US11646907B2 (en) 2007-06-12 2023-05-09 Icontrol Networks, Inc. Communication protocols in integrated systems
US11677577B2 (en) 2004-03-16 2023-06-13 Icontrol Networks, Inc. Premises system management using status signal
US11700142B2 (en) 2005-03-16 2023-07-11 Icontrol Networks, Inc. Security network integrating security system and network devices
US11706279B2 (en) 2007-01-24 2023-07-18 Icontrol Networks, Inc. Methods and systems for data communication
US11706045B2 (en) 2005-03-16 2023-07-18 Icontrol Networks, Inc. Modular electronic display platform
US11729255B2 (en) 2008-08-11 2023-08-15 Icontrol Networks, Inc. Integrated cloud system with lightweight gateway for premises automation
US11750414B2 (en) 2010-12-16 2023-09-05 Icontrol Networks, Inc. Bidirectional security sensor communication for a premises security system
US11758026B2 (en) 2008-08-11 2023-09-12 Icontrol Networks, Inc. Virtual device systems and methods
US11792036B2 (en) 2008-08-11 2023-10-17 Icontrol Networks, Inc. Mobile premises automation platform
US11792330B2 (en) 2005-03-16 2023-10-17 Icontrol Networks, Inc. Communication and automation in a premises management system
US11811845B2 (en) 2004-03-16 2023-11-07 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US11816323B2 (en) 2008-06-25 2023-11-14 Icontrol Networks, Inc. Automation system user interface
US11831462B2 (en) 2007-08-24 2023-11-28 Icontrol Networks, Inc. Controlling data routing in premises management systems
US11916928B2 (en) 2008-01-24 2024-02-27 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US11916870B2 (en) 2004-03-16 2024-02-27 Icontrol Networks, Inc. Gateway registry methods and systems

Cited By (255)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8037504B2 (en) 1999-06-11 2011-10-11 Jerding Dean F Video on demand system with selectable options of configurable random-access control
US8056106B2 (en) 1999-06-11 2011-11-08 Rodriguez Arturo A Video on demand system with dynamic enablement of random-access functionality
US7992163B1 (en) 1999-06-11 2011-08-02 Jerding Dean F Video-on-demand navigational system
US8020184B2 (en) 1999-06-11 2011-09-13 Jerding Dean F Channel control system for exiting from an interactive program guide
US7992166B2 (en) 2000-04-03 2011-08-02 Jerding Dean F Providing alternative services based on receiver configuration and type of display device
US7975277B1 (en) 2000-04-03 2011-07-05 Jerding Dean F System for providing alternative services
US9380347B2 (en) 2000-05-04 2016-06-28 Cisco Technology, Inc. Hypertext service guide menu display
US8739212B2 (en) 2000-05-04 2014-05-27 Cisco Technology, Inc. Configuration of presentations of selectable TV services according to usage
US7934232B1 (en) 2000-05-04 2011-04-26 Jerding Dean F Navigation paradigm for access to television services
US8516525B1 (en) 2000-06-09 2013-08-20 Dean F. Jerding Integrated searching system for interactive media guide
US8707153B2 (en) 2000-06-09 2014-04-22 Cisco Technology, Inc. Displaying comment data corresponding to a video presentation
US8069259B2 (en) 2000-06-09 2011-11-29 Rodriguez Arturo A Managing removal of media titles from a list
US7962370B2 (en) 2000-06-29 2011-06-14 Rodriguez Arturo A Methods in a media service system for transaction processing
US8032914B2 (en) 2000-11-10 2011-10-04 Rodriguez Arturo A Systems and methods for dynamically allocating bandwidth in a digital broadband delivery system
US8006262B2 (en) 2001-06-29 2011-08-23 Rodriguez Arturo A Graphic user interfaces for purchasable and recordable media (PRM) downloads
US8006273B2 (en) 2001-06-29 2011-08-23 Rodriguez Arturo A Updating download options for unavailable media content
US8191093B2 (en) 2001-06-29 2012-05-29 Rodriguez Arturo A Providing information pertaining to audio-visual and personal bi-directional services
US8640172B2 (en) 2001-06-29 2014-01-28 Cisco Technology, Inc. System and method for characterization of purchasable and recordable media (PRM)
US10559193B2 (en) 2002-02-01 2020-02-11 Comcast Cable Communications, Llc Premises management systems
US8745656B2 (en) 2002-02-11 2014-06-03 Cisco Technology, Inc. Tracking of presented television advertisements
WO2004100500A2 (en) * 2003-05-05 2004-11-18 Thomson Licensing S.A. System and method for communicating with a display device via a network
WO2004100500A3 (en) * 2003-05-05 2005-04-28 Thomson Licensing Sa System and method for communicating with a display device via a network
US8161388B2 (en) 2004-01-21 2012-04-17 Rodriguez Arturo A Interactive discovery of display device characteristics
US9615139B2 (en) 2004-01-21 2017-04-04 Tech 5 Determining device that performs processing of output pictures
US11449012B2 (en) 2004-03-16 2022-09-20 Icontrol Networks, Inc. Premises management networking
US11656667B2 (en) 2004-03-16 2023-05-23 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US11378922B2 (en) 2004-03-16 2022-07-05 Icontrol Networks, Inc. Automation system with mobile interface
US10692356B2 (en) 2004-03-16 2020-06-23 Icontrol Networks, Inc. Control system user interface
US10691295B2 (en) 2004-03-16 2020-06-23 Icontrol Networks, Inc. User interface in a premises network
US10735249B2 (en) 2004-03-16 2020-08-04 Icontrol Networks, Inc. Management of a security system at a premises
US11811845B2 (en) 2004-03-16 2023-11-07 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US11810445B2 (en) 2004-03-16 2023-11-07 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US11893874B2 (en) 2004-03-16 2024-02-06 Icontrol Networks, Inc. Networked touchscreen with integrated interfaces
US10754304B2 (en) 2004-03-16 2020-08-25 Icontrol Networks, Inc. Automation system with mobile interface
US11782394B2 (en) 2004-03-16 2023-10-10 Icontrol Networks, Inc. Automation system with mobile interface
US11757834B2 (en) 2004-03-16 2023-09-12 Icontrol Networks, Inc. Communication protocols in integrated systems
US10796557B2 (en) 2004-03-16 2020-10-06 Icontrol Networks, Inc. Automation system user interface with three-dimensional display
US11368429B2 (en) 2004-03-16 2022-06-21 Icontrol Networks, Inc. Premises management configuration and control
US10890881B2 (en) 2004-03-16 2021-01-12 Icontrol Networks, Inc. Premises management networking
US10979389B2 (en) 2004-03-16 2021-04-13 Icontrol Networks, Inc. Premises management configuration and control
US10992784B2 (en) 2004-03-16 2021-04-27 Control Networks, Inc. Communication protocols over internet protocol (IP) networks
US11043112B2 (en) 2004-03-16 2021-06-22 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US10156831B2 (en) 2004-03-16 2018-12-18 Icontrol Networks, Inc. Automation system with mobile interface
US11082395B2 (en) 2004-03-16 2021-08-03 Icontrol Networks, Inc. Premises management configuration and control
US11677577B2 (en) 2004-03-16 2023-06-13 Icontrol Networks, Inc. Premises system management using status signal
US11410531B2 (en) 2004-03-16 2022-08-09 Icontrol Networks, Inc. Automation system user interface with three-dimensional display
US11626006B2 (en) 2004-03-16 2023-04-11 Icontrol Networks, Inc. Management of a security system at a premises
US11625008B2 (en) 2004-03-16 2023-04-11 Icontrol Networks, Inc. Premises management networking
US11916870B2 (en) 2004-03-16 2024-02-27 Icontrol Networks, Inc. Gateway registry methods and systems
US10142166B2 (en) 2004-03-16 2018-11-27 Icontrol Networks, Inc. Takeover of security network
US11153266B2 (en) 2004-03-16 2021-10-19 Icontrol Networks, Inc. Gateway registry methods and systems
US11159484B2 (en) 2004-03-16 2021-10-26 Icontrol Networks, Inc. Forming a security network including integrated security system components and network devices
US11601397B2 (en) 2004-03-16 2023-03-07 Icontrol Networks, Inc. Premises management configuration and control
US11175793B2 (en) 2004-03-16 2021-11-16 Icontrol Networks, Inc. User interface in a premises network
US11182060B2 (en) 2004-03-16 2021-11-23 Icontrol Networks, Inc. Networked touchscreen with integrated interfaces
US11588787B2 (en) 2004-03-16 2023-02-21 Icontrol Networks, Inc. Premises management configuration and control
US11537186B2 (en) 2004-03-16 2022-12-27 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US11184322B2 (en) 2004-03-16 2021-11-23 Icontrol Networks, Inc. Communication protocols in integrated systems
US11489812B2 (en) 2004-03-16 2022-11-01 Icontrol Networks, Inc. Forming a security network including integrated security system components and network devices
US10447491B2 (en) 2004-03-16 2019-10-15 Icontrol Networks, Inc. Premises system management using status signal
US11201755B2 (en) 2004-03-16 2021-12-14 Icontrol Networks, Inc. Premises system management using status signal
US11244545B2 (en) 2004-03-16 2022-02-08 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US11277465B2 (en) 2004-03-16 2022-03-15 Icontrol Networks, Inc. Generating risk profile using data of home monitoring and security system
US11310199B2 (en) 2004-03-16 2022-04-19 Icontrol Networks, Inc. Premises management configuration and control
US11343380B2 (en) 2004-03-16 2022-05-24 Icontrol Networks, Inc. Premises system automation
US7599989B2 (en) 2005-01-24 2009-10-06 Microsoft Corporation System and method for gathering and reporting screen resolutions of attendees of a collaboration session
EP1684515A1 (en) * 2005-01-24 2006-07-26 Microsoft Corporation System and method for gathering and reporting screen resolutions of attendees of a collaboration session
US11113950B2 (en) 2005-03-16 2021-09-07 Icontrol Networks, Inc. Gateway integrated with premises security system
US10841381B2 (en) 2005-03-16 2020-11-17 Icontrol Networks, Inc. Security system with networked touchscreen
US10380871B2 (en) 2005-03-16 2019-08-13 Icontrol Networks, Inc. Control system user interface
US11824675B2 (en) 2005-03-16 2023-11-21 Icontrol Networks, Inc. Networked touchscreen with integrated interfaces
US10721087B2 (en) 2005-03-16 2020-07-21 Icontrol Networks, Inc. Method for networked touchscreen with integrated interfaces
US9450776B2 (en) 2005-03-16 2016-09-20 Icontrol Networks, Inc. Forming a security network including integrated security system components
US11792330B2 (en) 2005-03-16 2023-10-17 Icontrol Networks, Inc. Communication and automation in a premises management system
US11706045B2 (en) 2005-03-16 2023-07-18 Icontrol Networks, Inc. Modular electronic display platform
US11367340B2 (en) 2005-03-16 2022-06-21 Icontrol Networks, Inc. Premise management systems and methods
US10930136B2 (en) 2005-03-16 2021-02-23 Icontrol Networks, Inc. Premise management systems and methods
US11424980B2 (en) 2005-03-16 2022-08-23 Icontrol Networks, Inc. Forming a security network including integrated security system components
US10999254B2 (en) 2005-03-16 2021-05-04 Icontrol Networks, Inc. System for data routing in networks
US11700142B2 (en) 2005-03-16 2023-07-11 Icontrol Networks, Inc. Security network integrating security system and network devices
US11451409B2 (en) 2005-03-16 2022-09-20 Icontrol Networks, Inc. Security network integrating security system and network devices
US10156959B2 (en) 2005-03-16 2018-12-18 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US10062245B2 (en) 2005-03-16 2018-08-28 Icontrol Networks, Inc. Cross-client sensor user interface in an integrated security network
US11496568B2 (en) 2005-03-16 2022-11-08 Icontrol Networks, Inc. Security system with networked touchscreen
US9059863B2 (en) * 2005-03-16 2015-06-16 Icontrol Networks, Inc. Method for data routing in networks
US11595364B2 (en) 2005-03-16 2023-02-28 Icontrol Networks, Inc. System for data routing in networks
US10091014B2 (en) 2005-03-16 2018-10-02 Icontrol Networks, Inc. Integrated security network with security alarm signaling system
US20090070473A1 (en) * 2005-03-16 2009-03-12 Marc Baum Method for Data Routing in Networks
US10127801B2 (en) 2005-03-16 2018-11-13 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US11615697B2 (en) 2005-03-16 2023-03-28 Icontrol Networks, Inc. Premise management systems and methods
US8606949B2 (en) 2005-04-20 2013-12-10 Jupiter Systems Interconnection mechanism for multiple data streams
US10469553B2 (en) 2005-04-20 2019-11-05 Jupiter Systems, Llc Interconnection mechanism for multiple data streams
US8547997B2 (en) 2005-04-20 2013-10-01 Jupiter Systems Capture node for use in an audiovisual signal routing and distribution system
WO2006113776A3 (en) * 2005-04-20 2007-09-20 Jupiter Systems Audiovisual signal routing and distribution system
WO2006113776A2 (en) * 2005-04-20 2006-10-26 Jupiter Systems Audiovisual signal routing and distribution system
US8553716B2 (en) 2005-04-20 2013-10-08 Jupiter Systems Audiovisual signal routing and distribution system
US9549011B2 (en) 2005-04-20 2017-01-17 Infocus Corporation Interconnection mechanism for multiple data streams
EP3054629A1 (en) * 2005-08-05 2016-08-10 Orange Method for controlling a multimedia device from a mobile terminal, corresponding computer program, multimedia device and server
US8122103B2 (en) 2005-08-05 2012-02-21 France Telecom Method for controlling a multimedia device from a mobile terminal, corresponding computer programs, multimedia device and server
WO2007017407A1 (en) * 2005-08-05 2007-02-15 France Telecom Method for controlling a multimedia device from a mobile terminal, corresponding computer programs, multimedia device and server
WO2007030370A1 (en) * 2005-09-07 2007-03-15 Scientific-Atlanta, Inc. Optimizing data rate for video services
EP3413572A1 (en) * 2005-12-29 2018-12-12 Rovi Guides, Inc. An interactive media guidance system having multiple devices
US10785319B2 (en) 2006-06-12 2020-09-22 Icontrol Networks, Inc. IP device discovery systems and methods
US10616244B2 (en) 2006-06-12 2020-04-07 Icontrol Networks, Inc. Activation of gateway device
US11418518B2 (en) 2006-06-12 2022-08-16 Icontrol Networks, Inc. Activation of gateway device
US9621408B2 (en) 2006-06-12 2017-04-11 Icontrol Networks, Inc. Gateway registry methods and systems
US9654751B2 (en) 2006-12-21 2017-05-16 Thomson Licensing Method, apparatus and system for providing color grading for displays
US11418572B2 (en) 2007-01-24 2022-08-16 Icontrol Networks, Inc. Methods and systems for improved system performance
US10225314B2 (en) 2007-01-24 2019-03-05 Icontrol Networks, Inc. Methods and systems for improved system performance
US11706279B2 (en) 2007-01-24 2023-07-18 Icontrol Networks, Inc. Methods and systems for data communication
US10142392B2 (en) 2007-01-24 2018-11-27 Icontrol Networks, Inc. Methods and systems for improved system performance
US11412027B2 (en) 2007-01-24 2022-08-09 Icontrol Networks, Inc. Methods and systems for data communication
JP2011030222A (en) * 2007-01-29 2011-02-10 Ntt Docomo Inc Optimized content-based and royalty-based encoding and distribution of media data
JP2010517476A (en) * 2007-01-29 2010-05-20 株式会社エヌ・ティ・ティ・ドコモ Optimized content-based and royalty-based encoding and distribution of media data
KR101007173B1 (en) * 2007-01-29 2011-01-12 가부시키가이샤 엔티티 도코모 Optimized content-based and royalty-based encoding and distribution of media data
WO2008094595A3 (en) * 2007-01-29 2008-10-02 Ntt Docomo Inc Optimized content-based and royalty-based encoding and distribution of media data
US9412248B1 (en) 2007-02-28 2016-08-09 Icontrol Networks, Inc. Security, monitoring and automation controller access and use of legacy security control panel information
US10657794B1 (en) 2007-02-28 2020-05-19 Icontrol Networks, Inc. Security, monitoring and automation controller access and use of legacy security control panel information
US11194320B2 (en) 2007-02-28 2021-12-07 Icontrol Networks, Inc. Method and system for managing communication connectivity
US11809174B2 (en) 2007-02-28 2023-11-07 Icontrol Networks, Inc. Method and system for managing communication connectivity
US10747216B2 (en) 2007-02-28 2020-08-18 Icontrol Networks, Inc. Method and system for communicating with and controlling an alarm system from a remote server
US9510065B2 (en) 2007-04-23 2016-11-29 Icontrol Networks, Inc. Method and system for automatically providing alternate network access for telecommunications
US11663902B2 (en) 2007-04-23 2023-05-30 Icontrol Networks, Inc. Method and system for providing alternate network access
US11132888B2 (en) 2007-04-23 2021-09-28 Icontrol Networks, Inc. Method and system for providing alternate network access
US10140840B2 (en) 2007-04-23 2018-11-27 Icontrol Networks, Inc. Method and system for providing alternate network access
US10672254B2 (en) 2007-04-23 2020-06-02 Icontrol Networks, Inc. Method and system for providing alternate network access
US10142394B2 (en) 2007-06-12 2018-11-27 Icontrol Networks, Inc. Generating risk profile using data of home monitoring and security system
US11316753B2 (en) 2007-06-12 2022-04-26 Icontrol Networks, Inc. Communication protocols in integrated systems
US11582065B2 (en) 2007-06-12 2023-02-14 Icontrol Networks, Inc. Systems and methods for device communication
US11894986B2 (en) 2007-06-12 2024-02-06 Icontrol Networks, Inc. Communication protocols in integrated systems
US10444964B2 (en) 2007-06-12 2019-10-15 Icontrol Networks, Inc. Control system user interface
US10382452B1 (en) 2007-06-12 2019-08-13 Icontrol Networks, Inc. Communication protocols in integrated systems
US9531593B2 (en) 2007-06-12 2016-12-27 Icontrol Networks, Inc. Takeover processes in security network integrated with premise security system
US10666523B2 (en) 2007-06-12 2020-05-26 Icontrol Networks, Inc. Communication protocols in integrated systems
US11601810B2 (en) 2007-06-12 2023-03-07 Icontrol Networks, Inc. Communication protocols in integrated systems
US11611568B2 (en) 2007-06-12 2023-03-21 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US9609003B1 (en) 2007-06-12 2017-03-28 Icontrol Networks, Inc. Generating risk profile using data of home monitoring and security system
US10365810B2 (en) 2007-06-12 2019-07-30 Icontrol Networks, Inc. Control system user interface
US11625161B2 (en) 2007-06-12 2023-04-11 Icontrol Networks, Inc. Control system user interface
US10339791B2 (en) 2007-06-12 2019-07-02 Icontrol Networks, Inc. Security network integrated with premise security system
US10616075B2 (en) 2007-06-12 2020-04-07 Icontrol Networks, Inc. Communication protocols in integrated systems
US11632308B2 (en) 2007-06-12 2023-04-18 Icontrol Networks, Inc. Communication protocols in integrated systems
US11646907B2 (en) 2007-06-12 2023-05-09 Icontrol Networks, Inc. Communication protocols in integrated systems
US11722896B2 (en) 2007-06-12 2023-08-08 Icontrol Networks, Inc. Communication protocols in integrated systems
US10313303B2 (en) 2007-06-12 2019-06-04 Icontrol Networks, Inc. Forming a security network including integrated security system components and network devices
US11237714B2 (en) 2007-06-12 2022-02-01 Control Networks, Inc. Control system user interface
US10389736B2 (en) 2007-06-12 2019-08-20 Icontrol Networks, Inc. Communication protocols in integrated systems
US11218878B2 (en) 2007-06-12 2022-01-04 Icontrol Networks, Inc. Communication protocols in integrated systems
US10237237B2 (en) 2007-06-12 2019-03-19 Icontrol Networks, Inc. Communication protocols in integrated systems
US11212192B2 (en) 2007-06-12 2021-12-28 Icontrol Networks, Inc. Communication protocols in integrated systems
US10051078B2 (en) 2007-06-12 2018-08-14 Icontrol Networks, Inc. WiFi-to-serial encapsulation in systems
US10423309B2 (en) 2007-06-12 2019-09-24 Icontrol Networks, Inc. Device integration framework
US10200504B2 (en) 2007-06-12 2019-02-05 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US10498830B2 (en) 2007-06-12 2019-12-03 Icontrol Networks, Inc. Wi-Fi-to-serial encapsulation in systems
US10079839B1 (en) 2007-06-12 2018-09-18 Icontrol Networks, Inc. Activation of gateway device
US11423756B2 (en) 2007-06-12 2022-08-23 Icontrol Networks, Inc. Communication protocols in integrated systems
US10523689B2 (en) 2007-06-12 2019-12-31 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US11089122B2 (en) 2007-06-12 2021-08-10 Icontrol Networks, Inc. Controlling data routing among networks
US9306809B2 (en) 2007-06-12 2016-04-05 Icontrol Networks, Inc. Security system with networked touchscreen
WO2009002324A1 (en) * 2007-06-28 2008-12-31 Thomson Licensing Method, apparatus and system for providing display device specific content over a network architecture
US11815969B2 (en) 2007-08-10 2023-11-14 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US11831462B2 (en) 2007-08-24 2023-11-28 Icontrol Networks, Inc. Controlling data routing in premises management systems
US11916928B2 (en) 2008-01-24 2024-02-27 Icontrol Networks, Inc. Communication protocols over internet protocol (IP) networks
US11816323B2 (en) 2008-06-25 2023-11-14 Icontrol Networks, Inc. Automation system user interface
US10530839B2 (en) 2008-08-11 2020-01-07 Icontrol Networks, Inc. Integrated cloud system with lightweight gateway for premises automation
US11641391B2 (en) 2008-08-11 2023-05-02 Icontrol Networks Inc. Integrated cloud system with lightweight gateway for premises automation
US11711234B2 (en) 2008-08-11 2023-07-25 Icontrol Networks, Inc. Integrated cloud system for premises automation
US11190578B2 (en) 2008-08-11 2021-11-30 Icontrol Networks, Inc. Integrated cloud system with lightweight gateway for premises automation
US11729255B2 (en) 2008-08-11 2023-08-15 Icontrol Networks, Inc. Integrated cloud system with lightweight gateway for premises automation
US10522026B2 (en) 2008-08-11 2019-12-31 Icontrol Networks, Inc. Automation system user interface with three-dimensional display
US11616659B2 (en) 2008-08-11 2023-03-28 Icontrol Networks, Inc. Integrated cloud system for premises automation
US11316958B2 (en) 2008-08-11 2022-04-26 Icontrol Networks, Inc. Virtual device systems and methods
US11368327B2 (en) 2008-08-11 2022-06-21 Icontrol Networks, Inc. Integrated cloud system for premises automation
US11758026B2 (en) 2008-08-11 2023-09-12 Icontrol Networks, Inc. Virtual device systems and methods
US11792036B2 (en) 2008-08-11 2023-10-17 Icontrol Networks, Inc. Mobile premises automation platform
US11258625B2 (en) 2008-08-11 2022-02-22 Icontrol Networks, Inc. Mobile premises automation platform
US10375253B2 (en) 2008-08-25 2019-08-06 Icontrol Networks, Inc. Security system with networked touchscreen and gateway
US20160274759A1 (en) 2008-08-25 2016-09-22 Paul J. Dawes Security system with networked touchscreen and gateway
WO2010047706A1 (en) * 2008-10-23 2010-04-29 Hewlett-Packard Development Company, L.P. Decompressing a video stream on a first computer system, and scaling and displaying the video stream on a second computer system
US9628440B2 (en) 2008-11-12 2017-04-18 Icontrol Networks, Inc. Takeover processes in security network integrated with premise security system
US10813034B2 (en) 2009-04-30 2020-10-20 Icontrol Networks, Inc. Method, system and apparatus for management of applications for an SMA controller
US11601865B2 (en) 2009-04-30 2023-03-07 Icontrol Networks, Inc. Server-based notification of alarm event subsequent to communication failure with armed security system
US11665617B2 (en) 2009-04-30 2023-05-30 Icontrol Networks, Inc. Server-based notification of alarm event subsequent to communication failure with armed security system
US11553399B2 (en) 2009-04-30 2023-01-10 Icontrol Networks, Inc. Custom content for premises management
US11778534B2 (en) 2009-04-30 2023-10-03 Icontrol Networks, Inc. Hardware configurable security, monitoring and automation controller having modular communication protocol interfaces
US11356926B2 (en) 2009-04-30 2022-06-07 Icontrol Networks, Inc. Hardware configurable security, monitoring and automation controller having modular communication protocol interfaces
US10332363B2 (en) 2009-04-30 2019-06-25 Icontrol Networks, Inc. Controller and interface for home security, monitoring and automation having customizable audio alerts for SMA events
US11284331B2 (en) 2009-04-30 2022-03-22 Icontrol Networks, Inc. Server-based notification of alarm event subsequent to communication failure with armed security system
US11223998B2 (en) 2009-04-30 2022-01-11 Icontrol Networks, Inc. Security, monitoring and automation controller access and use of legacy security control panel information
US11129084B2 (en) 2009-04-30 2021-09-21 Icontrol Networks, Inc. Notification of event subsequent to communication failure with security system
US10674428B2 (en) 2009-04-30 2020-06-02 Icontrol Networks, Inc. Hardware configurable security, monitoring and automation controller having modular communication protocol interfaces
US10275999B2 (en) 2009-04-30 2019-04-30 Icontrol Networks, Inc. Server-based notification of alarm event subsequent to communication failure with armed security system
US9426720B2 (en) 2009-04-30 2016-08-23 Icontrol Networks, Inc. Controller and interface for home security, monitoring and automation having customizable audio alerts for SMA events
US11856502B2 (en) 2009-04-30 2023-12-26 Icontrol Networks, Inc. Method, system and apparatus for automated inventory reporting of security, monitoring and automation hardware and software at customer premises
US10237806B2 (en) 2009-04-30 2019-03-19 Icontrol Networks, Inc. Activation of a home automation controller
GB2483045B (en) * 2009-06-01 2015-03-11 Sony Comp Entertainment Us Qualified video delivery
CN102474652A (en) * 2009-07-16 2012-05-23 纳瑞塔有限责任公司 Systems and methods for managing content in real-time
EP2454717A2 (en) * 2009-07-16 2012-05-23 Nagrastar L.L.C. Systems and methods for managing content in real-time
EP2454717A4 (en) * 2009-07-16 2013-04-03 Nagrastar L L C Systems and methods for managing content in real-time
US9509935B2 (en) 2010-07-22 2016-11-29 Dolby Laboratories Licensing Corporation Display management server
US10327021B2 (en) 2010-07-22 2019-06-18 Dolby Laboratories Licensing Corporation Display management server
US10223903B2 (en) 2010-09-28 2019-03-05 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US10127802B2 (en) 2010-09-28 2018-11-13 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US10062273B2 (en) 2010-09-28 2018-08-28 Icontrol Networks, Inc. Integrated security system with parallel processing architecture
US11900790B2 (en) 2010-09-28 2024-02-13 Icontrol Networks, Inc. Method, system and apparatus for automated reporting of account and sensor zone information to a central station
US9349276B2 (en) 2010-09-28 2016-05-24 Icontrol Networks, Inc. Automated reporting of account and sensor information
US11398147B2 (en) 2010-09-28 2022-07-26 Icontrol Networks, Inc. Method, system and apparatus for automated reporting of account and sensor zone information to a central station
US11444995B2 (en) 2010-09-30 2022-09-13 Tivo Corporation Delivering content in multiple formats
US10965726B2 (en) 2010-09-30 2021-03-30 Tivo Corporation Delivering content in multiple formats
EP2437509A3 (en) * 2010-09-30 2013-03-06 Comcast Cable Communications, LLC Delivering content in multiple formats
US8458362B2 (en) 2010-09-30 2013-06-04 Comcast Cable Communications, Llc Delivering content in multiple formats
US9596283B2 (en) 2010-09-30 2017-03-14 Comcast Cable Communications, Llc Delivering content in multiple formats
US10506010B2 (en) 2010-09-30 2019-12-10 Comcast Cable Communications, Llc Delivering content in multiple formats
GB2484406A (en) * 2010-10-07 2012-04-11 Ocean Blue Software Filtering available content for consumer device based on device identifier requested by server
US11750414B2 (en) 2010-12-16 2023-09-05 Icontrol Networks, Inc. Bidirectional security sensor communication for a premises security system
US10741057B2 (en) 2010-12-17 2020-08-11 Icontrol Networks, Inc. Method and system for processing security event data
US10078958B2 (en) 2010-12-17 2018-09-18 Icontrol Networks, Inc. Method and system for logging security event data
US11341840B2 (en) 2010-12-17 2022-05-24 Icontrol Networks, Inc. Method and system for processing security event data
US9729342B2 (en) 2010-12-20 2017-08-08 Icontrol Networks, Inc. Defining and implementing sensor triggered response rules
US11240059B2 (en) 2010-12-20 2022-02-01 Icontrol Networks, Inc. Defining and implementing sensor triggered response rules
EP2675129A1 (en) * 2011-04-01 2013-12-18 Huawei Technologies Co., Ltd. Streaming media service processing system, method and network device therefor
US10374865B2 (en) 2011-04-01 2019-08-06 Huawei Technologies Co., Ltd. System for processing streaming media service and method and network device thereof
EP2919436A1 (en) * 2011-04-01 2015-09-16 Huawei Technologies Co., Ltd. Apparatus for processing streaming media service
EP2675129A4 (en) * 2011-04-01 2014-05-28 Huawei Tech Co Ltd Streaming media service processing system, method and network device therefor
CN103430508A (en) * 2011-04-01 2013-12-04 华为技术有限公司 Streaming media service processing system, method and network device therefor
US9071812B2 (en) 2011-08-18 2015-06-30 International Business Machines Corporation Intelligent recording
US8442377B2 (en) 2011-08-18 2013-05-14 International Business Machines Corporation Intelligent recording
US11818374B2 (en) 2011-12-15 2023-11-14 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
US9380327B2 (en) 2011-12-15 2016-06-28 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
US10652562B2 (en) 2011-12-15 2020-05-12 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
US11057633B2 (en) 2011-12-15 2021-07-06 Comcast Cable Communications, Llc System and method for synchronizing timing across multiple streams
EP2696552A1 (en) * 2012-08-07 2014-02-12 NTT DoCoMo, Inc. Method, system and network for transmitting multimedia data to a plurality of clients
US20140047071A1 (en) * 2012-08-07 2014-02-13 Ntt Docomo, Inc. Method, system and network for transmitting multimedia data to a plurality of clients
CN103581701A (en) * 2012-08-07 2014-02-12 株式会社Ntt都科摩 Method, system and network for transmitting multimedia data to a plurality of clients
US9928975B1 (en) 2013-03-14 2018-03-27 Icontrol Networks, Inc. Three-way switch
US11553579B2 (en) 2013-03-14 2023-01-10 Icontrol Networks, Inc. Three-way switch
US9287727B1 (en) 2013-03-15 2016-03-15 Icontrol Networks, Inc. Temporal voltage adaptive lithium battery charger
US9867143B1 (en) 2013-03-15 2018-01-09 Icontrol Networks, Inc. Adaptive Power Modulation
US10117191B2 (en) 2013-03-15 2018-10-30 Icontrol Networks, Inc. Adaptive power modulation
US10659179B2 (en) 2013-03-15 2020-05-19 Icontrol Networks, Inc. Adaptive power modulation
US20140379852A1 (en) * 2013-06-20 2014-12-25 Wipro Limited System and method for subscribing to a content stream
US11296950B2 (en) 2013-06-27 2022-04-05 Icontrol Networks, Inc. Control system user interface
US10348575B2 (en) 2013-06-27 2019-07-09 Icontrol Networks, Inc. Control system user interface
US10645347B2 (en) 2013-08-09 2020-05-05 Icn Acquisition, Llc System, method and apparatus for remote monitoring
US10841668B2 (en) 2013-08-09 2020-11-17 Icn Acquisition, Llc System, method and apparatus for remote monitoring
US11438553B1 (en) 2013-08-09 2022-09-06 Icn Acquisition, Llc System, method and apparatus for remote monitoring
US11722806B2 (en) 2013-08-09 2023-08-08 Icn Acquisition, Llc System, method and apparatus for remote monitoring
US11432055B2 (en) 2013-08-09 2022-08-30 Icn Acquisition, Llc System, method and apparatus for remote monitoring
JP2014017003A (en) * 2013-08-27 2014-01-30 Thomson Licensing Method, apparatus and system for providing display device specific content over network architecture
US11146637B2 (en) 2014-03-03 2021-10-12 Icontrol Networks, Inc. Media content management
US11405463B2 (en) 2014-03-03 2022-08-02 Icontrol Networks, Inc. Media content management
US10178203B1 (en) 2014-09-23 2019-01-08 Vecima Networks Inc. Methods and systems for adaptively directing client requests to device specific resource locators
EP3316247A4 (en) * 2015-08-05 2018-12-26 Sony Corporation Information processing device, information processing method, and program
US10511767B2 (en) 2015-08-05 2019-12-17 Sony Corporation Information processing device, information processing method, and program
CN107851425A (en) * 2015-08-05 2018-03-27 索尼公司 Message processing device, information processing method and program

Similar Documents

Publication Publication Date Title
WO2002097584A2 (en) Adaptive video server
US6981045B1 (en) System for redirecting requests for data to servers having sufficient processing power to transcast streams of data in a desired format
US6961754B2 (en) Interactive access, manipulation, sharing and exchange of multimedia data
US7339993B1 (en) Methods for transforming streaming video data
US7143432B1 (en) System for transforming streaming video data
US7069573B1 (en) Personal broadcasting and viewing method of audio and video data using a wide area network
US7114174B1 (en) Computer program product for transforming streaming video data
US7047305B1 (en) Personal broadcasting system for audio and video data using a wide area network
US5748234A (en) Video processing system and method
KR100679343B1 (en) Method and apparatus for a client-sever system with heterogeneous clients
JP5120943B2 (en) Content providing apparatus, content providing method, client, information processing method of client, and program
US20090300701A1 (en) Area of interest processing of video delivered to handheld device
US20070024705A1 (en) Systems and methods for video stream selection
CN102457544B (en) Method and system for acquiring screen image in screen sharing system based on Internet
US8843983B2 (en) Video decomposition and recomposition
US5784099A (en) Video camera and method for generating time varying video images in response to a capture signal
US20060192848A1 (en) Video conferencing system
WO2008021126A2 (en) Method and apparatus for encoding and distributing media data
KR19990087916A (en) Internet convolution audio/video server
US20040001091A1 (en) Method and apparatus for video conferencing system with 360 degree view
US20040073953A1 (en) Audio/video apparatus for use with a cable television network
CN108810468B (en) Video transmission device and method for optimizing display effect
CN109963110A (en) Processing method, device, storage medium and the calculating equipment of multipart video-meeting
US20100049832A1 (en) Computer program product, a system and a method for providing video content to a target system
CN103581695A (en) System and method for achieving access of mobile terminal to global eye

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SD SE SG SI SK SL TJ TM TN TR TT TZ UA UG UZ VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WA Withdrawal of international application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP