CN102158474B - Virtual User Interface - Google Patents

Virtual User Interface Download PDF

Info

Publication number
CN102158474B
CN102158474B CN201110041831.XA CN201110041831A CN102158474B CN 102158474 B CN102158474 B CN 102158474B CN 201110041831 A CN201110041831 A CN 201110041831A CN 102158474 B CN102158474 B CN 102158474B
Authority
CN
China
Prior art keywords
user interface
client
connection
virtual
server
Prior art date
Application number
CN201110041831.XA
Other languages
Chinese (zh)
Other versions
CN102158474A (en
Inventor
D·赫尔斯
J·托马斯
Original Assignee
盖斯特泰克交互娱乐有限公司
Filing date
Publication date
Priority to US12/688,343 priority Critical patent/US9229734B2/en
Priority to US12/688343 priority
Application filed by 盖斯特泰克交互娱乐有限公司 filed Critical 盖斯特泰克交互娱乐有限公司
Publication of CN102158474A publication Critical patent/CN102158474A/en
Application granted granted Critical
Publication of CN102158474B publication Critical patent/CN102158474B/en

Links

Abstract

本发明涉及虚拟用户接口。 The present invention relates to a virtual user interface. 描述了能够实现用于各种各样的应用和计算环境中的任何一种的用户接口虚拟化的方法和设备。 It describes a user interface apparatus and method of virtualizing any of a wide variety of applications can be realized and used in a computing environment.

Description

虚拟用户接口 Virtual User Interface

技术领域 FIELD

[0001]本发明涉及提供一种用于机顶盒或类似客户端设备的虚拟用户接口。 [0001] The present invention relates to providing a virtual user interface for a set-top box or similar client device. 更特别地,本发明使用户接口虚拟化能够用于各种各样应用和计算环境中的任一种。 More particularly, the present invention enables user interface virtualization can be used in any of a wide variety of applications and computing environments.

背景技术 Background technique

[0002]用于使用信息或采用机顶盒的娱乐系统促进用户的交互的一种传统方法包括提供一种在机顶盒上本地运行的用户接口应用。 [0002] The set-top box using the information or entertainment system using a conventional method to promote user interaction includes providing a user interface application running locally on the set-top box. 这种应用可以是例如展现菜单等级(menuhierarchy)的Flash播放器。 Such an application may be for example to show the menu level (menuhierarchy) Flash Player. 不幸地是,由于机顶盒的典型性能,与这种接口相关联的动画(animat1n)通常以不满意的方式得以渲染,即播放器将花将近数秒来渲染它接收到的任何新信息(例如,从前一菜单屏幕渲染新的子菜单屏幕)。 Unfortunately, due to the typical performance set-top boxes, and animations associated with this kind of interface (animat1n) generally unsatisfactory way to render that player will take a few seconds to render almost any new information it received (for example, the former a menu screen rendering of the new sub-menu screen). 考虑到用户已经习惯于与桌上计算机系统上的用户接口交互,这种运行级别和用户体验是不可接受的。 Taking into account users are used to interact with the user interface on a desktop computer system, this operating level and the user experience is unacceptable.

[0003]由于与传统机顶盒相关联的性能限制和部署成本,希望提供在其中传送媒体的软件部件尽可能地是设备独立的解决方案。 [0003] Since the set-top box associated with the traditional performance limitations and deployment costs, desirable to provide a transmission medium in which software components as much as possible independent equipment solutions.

发明内容 SUMMARY

[0004]根据本发明,提供了用于能够实现用户接口虚拟化的各种方法、设备和计算机程序产品。 [0004] According to the present invention, various methods can be realized for the virtual user interface, apparatus and computer program product. 根据一个具体实施例,提供了一种在网络上提供虚拟用户接口的系统。 According to a particular embodiment, a system for providing a virtual user interface over a network. 该系统包括部署在网络上的一个或多个服务器,该一个或多个服务器被配置为管理多个虚拟帧缓冲器和多个用户接口应用的操作。 The system includes one or more servers deployed on the network, the one or more servers is configured to manage operation of a plurality of virtual frame buffers and a plurality of user interface applications. 该一个或多个服务器进一步被配置为将每个用户接口应用与对应的其中一个虚拟帧缓冲器相关联。 The one or more servers are further configured to each of the user interface applications with a corresponding one of the virtual frame buffer associated. 每个用户接口应用被配置为将对应的图形用户接口渲染到(render to)对应的虚拟帧缓冲器。 Each user interface application is configured to render a corresponding graphical user interface to (render to) the corresponding virtual frame buffer. 该一个或多个服务器进一步被配置为通过网络与多个客户端的每一个建立第一专用、持久、双向的连接以实施虚拟用户接口会话,将虚拟帧缓冲器之一分配给客户端的每一个,并通过第一连接将图形用户接口的更新从每个虚拟帧缓冲器转发给对应的客户端。 The one or more servers are further configured to establish a first private network through each of the plurality of clients, persistent, bi-directional connection to conduct a virtual user interface session, to allocate one of the virtual frame buffers to each client, and updating the graphical user interface will be forwarded through the first connection from each virtual frame buffer to the corresponding client. 这些更新中的至少一些包括用于在对应的客户端将图形用户接口与附加视觉信息相混合的透明信息。 These updates include at least some of the customer for a corresponding graphical user interface with additional visual information terminal phase mixture transparency information. 该一个或多个服务器进一步被配置为通过第一连接从每个客户端接收用户输入,并将该用户输入提供给对应的用户接口应用。 The one or more servers are further configured to receive user input from each client via the first connection, and provides the user input to the user interface corresponding to the application. 该用户输入代表和每个客户端相关联的用户与对应图形用户接口的表示(representat1n)的交互。 The user input corresponding to a user and a representative of each of the graphical user interface associated client representation (representat1n) interaction. 该系统还包括多个具有关联显示器的客户端。 The system further includes a plurality of clients with associated displays. 每个客户端被配置为通过第一连接从对应的虚拟帧缓冲器接收图形用户接口更新,并使用这些更新和透明信息在关联显示器上渲染具有附加视觉信息的对应图形用户接口的表示。 Each client is configured to receive from a graphical user interface to the corresponding virtual frame buffer via the first connection update, and use these to update and transparency information corresponding to rendering a graphical user having the additional visual information on the associated display represented interface. 每个客户端进一步被配置为通过第一连接将用户输入发送给该一个或多个服务器。 Each client is further configured to send the one or more servers via the first connection to the user input.

[0005]根据其他具体实施例,提供了用于在网络上提供虚拟用户接口的方法、设备和计算机程序产品。 [0005] According to another embodiment, a method for providing a virtual user interface over a network, devices and computer program products. 根据这些实施例,多个虚拟帧缓冲器的操作在部署在网络上的一个或多个服务器上得以管理。 According to these embodiments, a plurality of virtual frame buffers is managed on the operation of one or more servers deployed on the network. 响应于来自远程客户端的连接请求,在该一个或多个服务器的第一个上运行的用户接口应用与分配的其中一个虚拟帧缓冲器相关联。 In response to a connection request from a remote client, a user interface application and distribution of which in a first run of the one or more servers on a virtual frame buffer associated. 该用户接口应用被配置为将图形用户接口渲染到分配的虚拟帧缓冲器。 The user interface application is configured to render a graphical user interface to the allocated virtual frame buffer. 通过网络第一专用、持久、双向的连接在第一服务器和远程客户端之间得以建立以实施虚拟用户接口会话。 The first private network, persistent, bi-directional connection between the first server and the remote client to establish a virtual user interface session embodiment. 通过第一连接将该图形用户接口的更新从分配的虚拟帧缓冲器转发到远程客户端。 Forwarded to the remote client via the first connection updating the graphical user interface from the allocated virtual frame buffer. 这些更新的至少一些包括用于在远程客户端将图形用户接口与附加视觉信息混合的透明信息。 These updated for at least some of the remote client comprises information Graphical User Interface mixed with additional visual transparency information. 通过第一连接从远程客户端接收用户输入,并将用户输入提供给用户接口应用。 Receiving through the first connection from a remote client user input, and user input to a user interface application. 该用户输入代表和远程客户端相关联的用户与图形用户接口的表示的交互。 The user input and interaction on behalf of a remote client user associated with the representation of the graphical user interface.

[0006]仍然根据其他具体实施例,提供了用于通过网络与虚拟用户接口交互的方法、设备和计算机程序产品。 [0006] According to still another embodiment, there is provided a method for interacting with a virtual user interface of a network, apparatus and computer program product. 通过网络将连接请求发送给服务器。 Via the network connection request to the server. 该服务器被配置为管理多个虚拟帧缓冲器的操作,并且响应于连接请求,将运行在该服务器上的用户接口应用与分配的其中一个虚拟帧缓冲器相关联。 The server is configured to manage operation of a plurality of virtual frame buffers and, in response to the connection request, the user interface application running on the assigned server which is associated a virtual frame buffer. 用户接口应用被配置为将图形用户接口渲染到分配的虚拟帧缓冲器。 User interface application is configured to render a graphical user interface to the allocated virtual frame buffer. 通过网络建立与服务器的第一专用、持久、双向的连接以实施虚拟用户接口会话。 The server through a network to establish a first dedicated, persistent, bi-directional connection to conduct a virtual user interface session. 通过第一连接从分配的虚拟帧缓冲器接收图形用户接口的更新。 Receiving updates the graphical user interface from the allocated virtual frame buffer via the first connection. 这些更新的至少一些包括用于将图形用户接口与附加视觉信息混合的透明信息。 These updates at least some of the graphical user interface comprises a visual information and the additional information transparent mixing. 使用这些更新和透明信息,结合附加视觉信息的图形用户接口的表示在显示器上被渲染。 These updates and the transparency information use, additional visual information in conjunction with a graphical user interface representation is rendered on a display. 用户输入通过第一连接被转发给服务器以呈现给用户接口应用。 User input is forwarded to the server for presentation to the user interface application through the first connection. 用户输入代表用户与图形用户接口的表示的交互。 User input on behalf of a user's interaction with the representation of the graphical user interface.

[0007]参考本说明书的其余部分和附图,可实现本发明的特性和优点的进一步理解。 [0007] The rest of this specification with reference to the accompanying drawings and can achieve further understanding of the nature and advantages of the present invention.

附图说明 BRIEF DESCRIPTION

[0008]图1是示出在其中可实现本发明实施例的系统示例的简图。 [0008] FIG. 1 shows a schematic view in which the system may be implemented according to exemplary embodiments of the present invention.

[0009]图2是示出用于本发明具体实施例的各种系统部件的框图。 [0009] FIG. 2 is a block diagram illustrating various system components for specific embodiments of the present invention.

[0010]图3是示出本发明具体实施例的操作的流程图。 [0010] FIG. 3 is a flowchart illustrating operation of a specific embodiment of the present invention.

具体实施方式 Detailed ways

[0011]现在将详细地参考包括由发明人设想、用于实现本发明的最佳形式的本发明的具体实施例。 [0011] Reference will now be made in detail contemplated by the invention comprise specific embodiments of the present invention is the best form of the invention is used. 这些具体实施例的示例在附图中得以示出。 Examples of these specific embodiments is illustrated in the accompanying drawings. 虽然结合这些具体实施例,本发明得以描述,但应当理解并非意欲将本发明限制为描述的实施例。 Although in conjunction with these specific embodiments, the present invention has been described, it is to be understood that the present invention is not intended to be limited to the embodiments described. 相反,意欲覆盖可包括在由附加的权利要求限定的本发明的精神和范围内的替代、修改和等同物。 Instead, it is intended to cover may be included within the spirit and scope of the invention as defined by the appended claims alternatives, modifications, and equivalents thereof. 在以下描述中,为了提供本发明的彻底理解阐述了具体的细节。 In the following description, in order to provide a thorough understanding of the present invention, specific details are set forth. 可在没有这些具体细节的一些或全部的情况下实践本发明。 The present invention may be practiced in the absence of some or all of these specific details. 并且已知的特征可能不详细地描述以避免不必要地模糊本发明。 And known features may not described in detail to avoid unnecessarily obscuring the present invention.

[0012]根据本发明的各个实施例,在一个或多个服务器上运行的虚拟用户接口在远程客户端设备上得以呈现。 [0012] According to various embodiments of the present invention, virtual user running on one or more server interface is presented on remote client devices. 这种实施例可表征为集中(pool)、管理并清除虚拟用户接口会话的服务。 Embodiment may be characterized as centralized (pool), and clears the service management virtual user interface session this embodiment. 本发明的实施例参考与控制关联电视的操作(例如,菜单等级)的机顶盒相关联的用户接口而得以描述。 Operation of the embodiment with reference to the television associated with the control of the present invention (e.g., menu level) associated with the set-top box user interfaces is described. 然而,应当注意提及这种实施例并非意欲限制。 However, it should be noted that reference to such embodiments not intended to limit embodiments. 更确切地,设想本发明的这样的实施例,在这些实施例中与各种各样应用的任一种相关联的接口可以以资源方面在客户端处需要非常少的方式得以呈现,并且可被配置为很大程度上独立于客户端设备硬件。 Rather, it is contemplated such an embodiment of the present invention, the interface with any of a variety of applications may be associated with resources requires very little in the way of these examples to be presented at the client, and may be It is configured largely independent of the client device hardware.

[0013]图1示出了在其中本发明的实施例可得以实现的网络环境的简图。 [0013] FIG. 1 shows a schematic embodiment in which the embodiment of the present invention may be implemented in a network environment. 网络102可表示使用各种各样的网络技术和通信协议中的任一种来实现的(多个)专有和/或公共局域或广域网络的任何组合。 Network 102 may represent any of a wide variety of networks using technologies and communication protocols implemented (s) any combination of proprietary and / or public local or wide area network. 例如,网络102可表示一个或多个IP网络、电缆网络、电信网络、卫星电视网络、有线或无线网络,或其任何组合。 For example, network 102 may represent one or more IP networks, cable networks, telecommunications networks, satellite television networks, wired or wireless network, or any combination thereof. 在各种用户位置104中的硬件通过网络102被连接至可由一个或多个服务供应商操作和/或控制的服务器106、108和110(和关联的数据存储装置)。 Hardware in various user locations 104 are connected via a network to 102 by one or more service providers operating and / or control the server 106, 108 and 110 (and associated data storage means). 将理解服务器的每个表示可代表在其上可分布与本发明实施例相关联的各种过程和计算负载的多个服务器。 Each server will be appreciated represented may represent various processes which may be distributed associated with an embodiment of the present invention, a plurality of servers and computational load. 在本描述中,服务器106、108和110可代表特殊实现的多个方面,例如虚拟用户接口服务器、媒体服务器等。 In this description, the server 106, 108 and 110 may represent various aspects of the particular implementation, for example, virtual user interface servers, media servers.

[0014]用户位置硬件可包括客户端设备(例如机顶盒112)和关联的显示器(例如电视114)。 [0014] The user location hardware may include a client device (e.g., set top box 112) and the associated display (e.g., television 114). 可替换地,显示器设备116可包括充足的集成计算资源以根据本发明与远程服务器交互,例如某些新一代数字电视。 Alternatively, the display apparatus 116 may include sufficient integrated computing resources to interact with the server according to the present invention is a remote, such as some new generation of digital television. 根据一些实施例,用户位置可以是例如在酒店环境(hospitality context)中的客房,具有是酒店私有网络基础设施一部分的居间网络。 According to some embodiments, the user location may be, for example, in a hotel environment (hospitality context) in the room, a private network having a hotel intervening part of the network infrastructure. 可替换地,用户位置可以是订购例如由线缆或卫星系统运营商提供的数字电视和/或信息业务的私人住宅和/或公司。 Alternatively, the user's location can be ordered such as provided by cable or satellite digital television system operators and / or private residences information services and / or company. 根据另一个替换方式,用户位置可以是与诸如因特网这样的公共网络相连接的私人住宅和/或公司。 According to another alternative embodiment, the user location may be a private house such a public network such as the Internet connected and / or companies. 参考这些示例,本领域技术人员将理解可能实施例的全部范围。 Referring to these examples, those skilled in the art will appreciate the full range of possible embodiments.

[0015]根据实施例的一个特殊分类,机顶盒的用户接口运行在系统中远离机顶盒的虚拟用户接口(UI)服务器上。 [0015] According to a particular embodiment of the classification, the user interface runs away from the set-top box STB in the system on a virtual user interface (UI) server. 如上所述,该虚拟UI服务器实际上可以是多个服务器。 As noted above, the virtual UI server may actually be multiple servers. 可使用诸如虚拟网络计算协议(VNC)、远程桌面协议(RDP)或Apple(苹果)远程桌面协议(ARD)这样的各种协议中的任一种,通过专用通信信道将该用户接口的增量图形更新转发至机顶盒。 Any of a variety of protocols may be used such Protocol Virtual Network Computing (VNC), such as a Remote Desktop Protocol (RDP), or Apple (apple) remote desktop protocol (ARD) in increments over a dedicated communication channel to the user interface graphics update forwarded to the set-top box. 更一般地,在实施例的该分类中本发明的实施例可使用任何已知或专有协议得以实现,通过该协议接口的增量图形更新可被发送给客户端,并且在该协议中从客户端接收的用户输入驱使图形更新。 More generally, in this embodiment, the classification embodiments of the present invention may use any known or proprietary protocol is achieved by incrementally updating the graphical interface of the protocol may be sent to the client, and the protocol from the client receives user input driven graphical updates.

[0016]正如将理解的那样,传统的VNC系统典型地包括客户端、服务器和被称为RFB(即远程帧缓冲器)的相对简单的通信协议。 [0016] As will be appreciated, a conventional VNC system typically includes a client, server, and is called the RFB (i.e., remote frame buffer) in a relatively simple communication protocol. 帧缓冲器是包括在给定时刻组成显示图像的每个像素的颜色值的存储器缓冲器。 The frame buffer is a memory buffer comprising a color value of each pixel constituting the display image at a given time. 由于帧缓冲器通过应用软件得以修改,VNC服务器向VNC客户端发送该帧缓冲器的小矩形以在关联的显示器上渲染,所述小矩形对应于自从最后的更新已经改变的区域。 Since the frame buffer is modified by application software, the VNC server sends the frame buffer VNC client to render a small rectangle on the associated display, a small rectangle corresponding to the region have changed since the last update. 客户端向VNC服务器发送用户输入以驱使这些更新。 The client sends user input to the VNC server to drive these updates.

[0017]根据涉及机顶盒的一些实施例,机顶盒既处理视频流的播放,又处理用户接口的表示。 [0017] According to some embodiments relate to a set top box, the set-top box for processing a video stream playback only, and represented by the user interface process. 正如将理解的那样,用户接口运行在远程平台上,并在该远程平台上被渲染(render)到帧缓冲器,该远程平台将该接口的图形更新发送给机顶盒。 As will be appreciated, the user interface running on the remote platform, and is rendered (the render) to the frame buffer on the remote platform, the platform pattern remote interface to send updates to the set top box. 根据本发明的各个实施例,当更新由机顶盒接收时,它们与由该机顶盒正在播放的视频相“混合”。 According to various embodiments of the present invention, when an update is received by the set-top box, with which the video being played by the set-top box with the "mix." 根据这些实施例的一些,通过识别在用户接口更新中的信息的多个部分是否是不透明的、透明的或半透明的来促进(facilitate)该混合操作,从而显示器的对应像素可以被渲染为图形、视频或者两者的结合。 According to some of these embodiments, information identifying a plurality of portions through a user interface whether updates are opaque, transparent or translucent to facilitate (Facilitate) the mixing operation, so that the display of the corresponding pixel may be rendering graphics , video, or a combination of both.

[0018]现在将参考图2描述根据本发明实施例提供虚拟用户接口的特殊结构的一个示例。 [0018] Reference will now be described with an exemplary embodiment of the present invention provides a virtual user interface of a specific structure of FIG. 根据一些实施例,描述的部件可部署在与在图1中描述的计算环境类似的计算环境中。 According to some embodiments, the components described may be deployed in computing environment depicted in FIG. 1 similar computing environments. 然而,示出了多种软件部件,而没有具体参考在其上运行这些软件部件的硬件或计算平台,因为本领域的技术人员将理解这种软件部件可以以各种各样的方式得以部署,而不脱离本发明的范围。 However, it illustrates various software components, and no specific reference to these running hardware or software components of a computing platform on which, as those skilled in the art will appreciate that such software components may be deployed in a variety of ways, without departing from the scope of the invention. 例如,在服务器侧的系统部件可驻留在相同或不同的计算平台上。 For example, system components on the server side may reside on the same or different computing platforms. 因此,对具体部件的位置或环境的具体引用的至少一些不应当被认为是限制本发明。 Thus, at least some should not be considered as limiting the invention to the specific position or the specific environment of the reference member.

[0019]现在参考图2以及图3的流程图,当客户端设备上电时,远程UI客户端210启动 [0019] Referring now to the flowchart of FIG. 2 and FIG. 3, when the client device is powered on, the remote UI client 210 starts

(301)并尝试与在虚拟UI服务器上的远程用户接口守护进程(Daemon)202建立连接,例如制作TCP连接请求。 (301) and try the remote user interface UI daemon on a virtual server (Daemon) 202 to establish a connection, such as making a TCP connection request. 远程用户接口守护进程202监听来自客户端的这种连接请求(302),并为每一个开启背景线程(304)。 This connection request (302) Remote User Interface Daemon 202 listens from the client and open a background thread for each (304). 由守护进程202发起的背景线程在它们对应的虚拟用户接口会话(下文讨论)持续的期间持续。 By the daemon 202 initiates a background thread continued sustained during their corresponding virtual user interface session (discussed below).

[0020]守护进程202(其可以是互相结合运行的多个这种守护进程之一)负责管理系统中并发用户接口会话的集合。 It is responsible for managing a collection of concurrent user interface sessions in the system of [0020] Daemon 202 (which may be combined with each one of a plurality of such daemon running). 与关联的虚拟帧缓冲器206—起,守护进程202维持在服务器上运行的可用VNC服务器实例209的列表。 Associated with the virtual frame buffer 206- from the daemon running on the server 202 to maintain a list of available VNC server instance 209. 当它接收到连接请求时,守护进程202将可用VNC服务器实例之一分配给该连接(308)(从列表中将它移除),并将会话信息返回给客户端 When it receives a connection request, Daemon 202 will assign one of the available VNC server instances to the connection (308) (removing it from the list), and the session information back to the client

(310),通过该会话信息会话可得以识别。 (310), can be identified by the session information. 根据一个具体实施例,会话信息包括IP地址和端口号,使用它们客户端可与分配的VNC服务器实例相连接。 According to a particular embodiment, the session information includes an IP address and port number, their use may be connected to the client VNC server instance allocated.

[0021] 守护进程202还创建运行在虚拟UI服务器上的用户接口壳(shell)实例208(312)。 [0021] Daemon 202 also creates a virtual UI server running on a user interface shell (the shell) Example 208 (312). 用户接口壳被创建并被渲染到与分配的VNC服务器实例相关联的帧缓冲器。 The user interface shell is created and rendered to the frame buffer associated with the VNC server instance allocated. 根据具体的实施例,该壳是生成用户接口的Adobe Flash播放器。 According to a particular embodiment, the shell is generated Adobe Flash Player user interface. 与本地运行在传统机顶盒上的播放器相对比,壳208将用户接口渲染到与分配的VNC服务器实例相关联的帧缓冲器(314),而不是在机顶盒中的帧缓冲器。 And run locally on a player than the conventional set-top box relative to the shell 208 to render the user interface associated with the VNC server instance allocated to the frame buffer (314), rather than a frame buffer in the set top box. 由于在服务器上可用的较多的计算资源,用户接口帧可以以比在传统机顶盒上高很多的帧速率得以渲染(在特定实现中几乎快100倍)。 Since the available resources on the server computing more, than the user interface frames may be much higher in the traditional set-top box to render a frame rate (nearly 100 times faster in a particular implementation).

[0022] 在描述的基于VNC的实现中,每个VNC服务器实例209是与X Windows服务器211(例如,Xvnc4)耦合的VNC服务器。 [0022] In described implementations VNC-based, each VNC server instance 209 is an X Windows server 211 (e.g., Xvnc4) VNC server coupled. XWindows服务器为X Windows客户端应用提供诸如用户接口应用(208)这样的渲染API,将每个这种应用渲染至与耦合的VNC服务器共享的帧缓冲器。 XWindows application server provides a user interface such as a (208) as X Windows client applications such rendering API, rendering each such application to share with a VNC server coupled to a frame buffer. 在服务器(其可以是一个或多个服务器)上运行的每个VNC服务器实例209连续地监视它的关联帧缓冲器,并向VNC服务器实例被分配给的VNC客户端发送图形更新。 A server (which may be one or more servers) running on each VNC server instance 209 continuously monitors its associated frame buffer, and transmits graphical updates to the VNC server instance allocated VNC client. 根据具体实施例,这些更新响应于来自客户端的更新请求而得以发送,并且不必要与被渲染到缓冲器的帧相对应。 According to a particular embodiment, these updates in response to the update request from the client is transmitted, and do not necessarily correspond with the rendered frame to the buffer. 更确切地,它们更典型地是自从最后的更新被发送给客户端后已改变的像素的某种集合(典型地是接口的一个或多个矩形部分)。 Rather, they are more typically some set of pixels since the last update is sent to the client has been changed (typically one or more rectangular portions of the interface). 根据具体实施例,由客户端来决定是请求帧缓冲器的全部内容还是仅请求增量更新。 According to a particular embodiment, the client is determined by the content request entire frame buffer or only an incremental update request. 在后一种情况下,VNC服务器实例回想客户端最后请求更新时虚拟帧缓冲器的状态,并仅发送自从最后请求后已改变的信息。 In the latter case, the VNC server instance recall the last client request updates virtual frame buffer state, and transmits only the information since the last request has been changed. 此外,VNC服务器可以不立即响应客户端的更新请求。 Further, VNC server may not respond immediately update request of the client. 即,VNC服务器可暂时地等待来聚集发送给客户端的更新,以提供更有带宽效率的响应。 That is, the VNC server may wait briefly to aggregate updates sent to the client, in response to provide a more bandwidth efficiency.

[0023]如上所述,当客户端设备上电时,远程UI客户端210启动并与守护进程202建立连接。 [0023] As described above, when the client device is powered on, the remote UI client 210 starts to establish a connection with Daemon 202. 远程UI客户端210然后在客户端设备(例如机顶盒)上启动VNC客户端212 (例如DirectVNC)的实现,它是从分配的VNC服务器实例请求接口更新的应用。 Remote UI Client 210 then starts to achieve a VNC client 212 (e.g. DirectVNC) on the client device (e.g. a set top box), which is a request from the application interface update VNC server instance allocated. 使用会话信息,VNC客户端212与分配给客户端的VNC服务器实例建立直接VNC连接(316),通过该连接接口更新随后得以传送。 Using the session information, VNC client 212 with the VNC server instance allocated to the client to establish a direct VNC connection (316), through which the connection interface is then updated to be transmitted.

[0024]应当再次注意,虽然在一些实施例中客户端设备可以是机顶盒,但是设想了在其中在此描述的功能被集成于最终渲染接口的硬件的实现,例如具有充足处理和存储器资源的数字电视。 [0024] It should be noted again that, although in some embodiments the client device may be a set top box, it is contemplated that the functions described herein is integrated in the final rendering hardware interfaces implemented wherein e.g. having sufficient processing and memory resources figures TV. 可替换地,在此描述的功能的至少一些可在与电视、机顶盒或两者一起运行分离的、独立的设备中得以实现。 Alternatively, at least some of which may be implemented in a separate operation with a television, set-top box, or both, in separate devices function as described herein.

[0025] 一旦VNC连接得以建立,VNC客户端212从对应的VNC服务器实例请求它在机顶盒上渲染的更新(318)。 [0025] Once the VNC connection is established, VNC client 212 requests its set-top box rendering update (318) from the corresponding VNC server instance. 这通过在VNC客户端212和客户端设备渲染硬件214之间的连接得以完成。 This is accomplished by connecting 212 and 214 between the client device rendering hardware VNC client.

[0026]根据实施例的特殊分类,提供了用于处理透明像素信息的对VNC协议的扩展。 [0026] According to a particular class of embodiments, extensions to the VNC protocol is provided for processing the transparent pixel information. 如已知的那样,在图形系统中,帧缓冲器是保存组成显示图像的像素颜色值的存储器的通常使用的区域。 As it is known, in a graphics system, the frame buffer is a commonly used storage area constituting the display memory pixel color values ​​of the image. 取决于帧缓冲器的目的,在帧缓冲器中存储像素的方式可采用许多不同的格式。 Depending on the purpose of the frame buffer, the frame buffer storing pixel way may take many different formats. 例如,用于存储视频帧的帧缓冲器可以以平面(planar )YUV420格式存储像素,而用于驱动显示器设备的帧缓冲器可以以打包(Packed)RGB格式存储像素。 For example, a frame buffer for storing video frames may be flat (planar) YUV420 format storing pixel, and for driving the display device may be packed frame buffer (Packed) RGB format for storing pixels. 这些不同的格式由帧缓冲器的颜色空间和在该颜色空间中像素的预定义的二进制表示来确定。 These different formats are determined by the color space of the frame buffer and the pixel in a predefined color space, the binary representation. 例如,RGB颜色空间可以存储在16比特字内的像素,其中5比特分配给红色和蓝色分量,6比特分配给绿色分量。 E.g., RGB color space pixel may be stored in 16-bit words, wherein 5 bits are allocated to red and blue components and 6 bits allocated to the green component. 可替换地,可使用24比特,其中为三种颜色分量的每个分配8个比特。 Alternatively, use may be 24 bits, where 8 bits of each allocated to the three color components. 然而,由于24比特(3个字节)在具有32比特自然(native)字大小的计算机系统中运行不方便,通常以32比特字存储RGB像素,其中为每个颜色分量分配8比特,剩下的另外8比特未使用。 However, since 24 bits (3 bytes) is inconvenient to run in a computer system having a 32-bit natural (Native) word size, typically 32-bit word stored RGB pixel, where each color component is assigned 8 bits, and the remaining another 8 unused bits. 用来存储像素的比特数有时被称为帧缓冲器的颜色深度。 The number of bits used to store a pixel is sometimes referred to as color depth of the frame buffer.

[0027]在一些图形系统中,特别是在其中设想将使用其他图像源(例如视频帧)合成显示图像的那些系统,与每个像素一起,帧缓冲器还可以存储透明信息。 [0027] In some graphics systems, especially envisaged in which other image sources (e.g., video frames) of the display image synthesis systems that, together with each pixel, the frame buffer may also store transparency information. 该透明信息有时被称为“阿尔法(α)通道”或“阿尔法(α)部件”。 This transparency information is sometimes referred to as "alpha ([alpha]) channel" or "alpha ([alpha]) member." 像素的阿尔法分量确定像素的透明度,从完全不透明到完全不可见。 Pixel alpha component of the pixel is determined transparency, from completely opaque to completely invisible. 使用对本领域技术人员来说公知的标准图像合成规则,包括与阿尔法信息一起的像素数据的帧缓冲器可用来合成多个图像。 Using well-known standard image compositing rules to the skilled person, including a frame buffer with pixel data along with alpha information can be used to synthesize a plurality of images. 其一个可能应用可以是“混合”视频帧和图形层,使得(被渲染到图形层的)图形用户接口可以以半透明的方式覆盖在视频上。 A possible application which may be a "hybrid" video frame and the graphics layer, such that (rendered into the graphical layer) graphical user interface may be a semi-transparent manner overlaid on the video.

[0028]在RGB帧缓冲器中,其中在32比特字中使用24比特存储像素信息,未使用的8比特通常被用来为每个像素存储阿尔法通道信息。 [0028], wherein the pixel information is stored in a 24-bit word in the 32 bit RGB frame buffer, 8 unused bits are typically used to store the alpha channel information for each pixel. 这种像素格式有时被称为ARGB: 8888,然而包括阿尔法通道的其他像素格式也是可能的。 Such a pixel format is sometimes referred to as ARGB: 8888, although other pixel formats including an alpha channel are also possible.

[0029]根据本发明的各种实施例,X Windows服务器被配置为渲染使用24比特在32比特字中保存RGB信息的、存储RGB格式像素的帧缓冲器。 [0029] According to various embodiments of the present invention, X Windows server is configured to save the 24-bit RGB information in a 32-bit word used for rendering, the frame buffer stores pixel RGB format. 标准X Windows客户端应用将通过XWindows API渲染该RGB帧缓冲器。 Standard X Windows client application will render the RGB frame buffer by XWindows API. 然而,诸如需要渲染透明信息的那些专门的应用能够直接访问X Windows帧缓冲器(使用对XWindows的标准XSHM扩展)。 However, those specialized applications such as rendering transparent information needs to be able to directly access the frame buffer X Windows (using standard XSHM extension of the XWindows). 这些应用可使用在每个32比特像素中未使用的8比特来为每个像素存储阿尔法分量。 These applications may use the unused 32 bits in each pixel is 8 bits to store the alpha component for each pixel. 不幸地,不采用这种方法的标准XWindows客户端将不必要在每个像素中正确地设置阿尔法分量(因为XWindows不直接支持它),因此将“继承”每个像素的任何预先存在的阿尔法分量。 Unfortunately, this method is not a standard XWindows client unnecessarily correctly set the alpha component in each pixel (since it does not directly support XWindows), thus "inherit" any pre-existing alpha component for each pixel . 出于这个原因,必须修改标准的X Windows服务器,使得标准的客户端应用将渲染每个像素的阿尔法分量,产生完全不透明的像素。 For this reason, it is necessary to modify the standard X Windows server, making the standard client application will render the alpha component for each pixel, resulting in completely opaque pixels. 该改变确保标准的X Windows客户端应用将以不透明的方式渲染,而更专门的客户端应用可通过直接为每个像素设置阿尔法分量以半透明的方式来渲染。 The change that the standard X Windows client applications will render an opaque manner, whereas more specialized client applications can render directly by setting the alpha component for each pixel of a semi-transparent manner.

[0030]因此,本发明的实施例采用与VNC服务器耦合的、如上所述修改的XWindows服务器。 [0030] Thus, embodiments of the present invention employ coupled to the VNC server, modified as described above XWindows server. VNC服务器监视与X Windows服务器共享的帧缓冲器,并使用VNC协议向在网络中的其它地方的VNC客户端提供图形更新。 VNC server monitoring with X Windows server share a frame buffer, and using the VNC protocol to provide updates to the VNC graphical client elsewhere in the network. VNC协议的固有部分是客户端和服务器之间图形更新即将使用的像素格式的协商。 VNC is an inherent part of the agreement negotiated between the pixel format graphics update client and server that will be used. VNC协议定义了几个不同的像素格式,没有一个包括阿尔法分量,即标准的VNC协议不支持透明信息。 VNC protocol defines several different pixel formats, none of the alpha component comprises, i.e., the standard VNC protocol does not support transparency information. 因此,本发明的这些实施例扩展VNC协议以支持附加的像素格式。 Thus, these embodiments of the present invention extend the VNC protocol to support additional pixel formats.

[0031] VNC协议的扩展可通过创建所谓的伪编码格式来实现。 Extended [0031] VNC protocol may be implemented by creating so-called pseudo encoding formats. 客户端然后将通过请求对应于该扩展的伪编码来发信号通知它支持给定的扩展。 The client is then signaled by requesting a pseudo encoding corresponding to that extension which supports a given extension. 不支持该扩展的服务器将简单地忽视该请求,而适当增强的服务器将通过向客户端发送与该扩展关联的专用确认消息来响应。 The server does not support the extension will simply ignore the request, the server sends the appropriate enhanced dedicated acknowledgment message associated with the extension by the response to the client.

[0032]根据本发明的这些实施例,VNC协议通过增加被称为“保持阿尔法通道(PreserveAlpha Channel)”的伪编码消息得以扩展。 [0032] By increasing referred to as "held alpha channel (PreserveAlpha Channel)" according to the embodiments of the present invention, VNC protocol message is encoded dummy extension. 当客户端通过该伪编码请求服务器等待(abide)时,服务器通过向客户端发送专用确认消息来响应。 When a client requests the server waits (abide) by this pseudo encoding, the server sends a dedicated acknowledgment message in response to the client. 客户端然后请求使用24比特在32比特字中存储RGB数据的RGB像素格式,服务器确保所有图形更新保持使用32比特字中剩余8比特的阿尔法分量数据,以传送每个像素的阿尔法分量。 The client then requests an RGB pixel format using 24-bit 32-bit word stored in the RGB data, the server ensures that all graphical updates using the 32-bit word held in the remaining 8-bit alpha component data, to transfer the alpha component for each pixel. 特别地,发送图形更新的“ZRLE”方法将不把32比特的像素压缩为24比特,这将导致阿尔法分量数据的丢失。 In particular, the transmission of graphics update "ZRLE" method will not compress the 32-bit pixel is 24 bits, which results in the loss of the alpha component data.

[0033]重提图3,根据一些实施例,VNC客户端212还向VNC服务器实例转发从机顶盒接收的各种类型的输入(例如,通过遥控、指示器或鼠标或键盘与虚拟接口的用户交互)(320),VNC服务器实例将它们通过X Windows服务器API传递给用户接口壳208。 [0033] FIG revert to 3, according to some embodiments, VNC client 212 also forwards various types of input received from the set top box to the VNC server instance (e.g., the user interaction by remote control, keyboard or mouse pointer or the virtual interface ) (320), VNC server instance is transmitted to them via the user interface 208 case X Windows server API. 一旦接收到这些用户输入事件,用户接口应用208对这些事件作出反应,这可能引起对用户接口作出图形更新并因此图形更新被渲染到与分配的X Windows服务器相关联的帧缓冲器。 Upon receiving the user input events, user interface application 208 to respond to these events, which may cause the frame buffer interface to a graphical update to the user and therefore rendered graphics update is assigned to the X Windows server is associated.

[0034]当客户端设备是运行的时,用户接口壳和分配给机顶盒的VNC服务器实例持续,甚至当机顶盒正在播放其他媒体并且用户没有观看或与虚拟用户接口交互时。 [0034] When a client device is running, the user interface shell and assigned to the set-top box VNC server instances continue, even when the set-top boxes and other media is playing or who did not see the user interface to interact with the virtual time. 被建立以发起用户接口会话的连接,例如TCP套接字,也维持会话的持续时间,以指示分配给该客户端(例如VNC服务器实例)的资源仍然是可用的(即使在该连接上传输很少或不传输)。 Connection is established to initiate conversation user interface, such as TCP socket, but also to maintain the duration of the session, indicating assigned to the client (eg VNC server instance) the resource is still available (even on the very transport connection little or no transmission). 当该会话结束时(例如客户端设备下电),该TCP连接终止(322),并且远程UI守护进程在服务器侧发起涉及终止用户接口壳(324)和回收VNC服务器实例(326)的整理。 When the session ends (e.g., the client device is powered), the TCP connection is terminated (322), and the remote UI daemon finishing process initiated at the server side involving termination of the user interface shell (324) and recovering the VNC server instance (326).

[0035]值得注意得是,在使用机顶盒实现的实施例中,在用户接口中由用户选择的、在关联的电视上由机顶盒播放的实际媒体内容典型地通过在图2中未示出的其他信道(例如在远程媒体服务器和在机顶盒上运行的对应中间件之间的TCP或UDP连接)传输给该机顶盒。 [0035] It is noteworthy too that, in the embodiment using the set-top box, the user interface selected by the user, on the associated television broadcast by the set-top box the actual media content typically by other in FIG. 2 (not shown) channel (e.g., a TCP or UDP connection between a remote media server and a corresponding middleware operating on the set top box) transmitted to the set top box.

[0036]根据实施例的具体分类,用户接口壳功能性从机顶盒到后端服务器(back endserver)的移动涉及在机顶盒上原有的初始壳中的功能性的分叉。 [0036] According to a particular class of embodiments, the user interface shell functionality from the set top box to the backend server moving (back endserver) relates to bifurcated top box functionality on the original initial shell. 即,在以前的原有用户接口运行在机顶盒上的解决方案中,它与机顶盒的渲染硬件(即将接口拖曳到帧缓冲器中)和驻留中间件(例如215)对接,以控制媒体播放。 That is, before the original user interface running the solution on the set-top box, the set-top box with its rendering hardware (the interface boxes coming into the frame buffer) and a resident middleware (e.g., 215) abutting to control media playback. 根据本发明的特殊实现及如图2所示,用户接口被移动到后端,对渲染硬件的接口有效地由VNC客户端替代,对中间件的接口由运行在机顶盒上、处理与中间件215的通信的瘦居间“代理”代替。 According to the present invention and the particular implementation, the user interface is moved to the rear end in FIG. 2, an alternative for effectively rendering hardware interface to the VNC client, the middleware interface running on the set-top box, processing middleware 215 communication thin intervening "proxy" instead. 该“中间件远程提取层”(例如中间件网络代理216)通过网络连接(例如TCP套接字)接收来自用户接口壳的通信,并将它们转换为由中间件识别的进程间通信,例如得以建立并在机顶盒运行期间持续的、经由Unix套接字的通信。 Receive communications from the user interface shell "Middleware Remote Abstraction Layer" (e.g., Middleware Network Proxy 216) via a network connection (e.g., TCP sockets), and converts them into the process identified by the communication between the middleware, for example to establishment and ongoing communication via Unix socket during a set-top box operation. 在壳和提取层之间的套接字例如可使用从守护进程202传递给壳208的客户端的IP地址来建立。 Between the socket housing and the extraction layer may be used, for example, transmitted from daemon 202 to the IP address of the client 208 to establish the housing.

[0037]根据具体实施例,中间件远程提取层可表征为有效地作为对下层平台的媒体能力的提取层的事件传递和获取系统,即,它提供在下层硬件平台和在远程用户接口中控制它的逻辑之间的必要转换。 [0037] According to a particular embodiment, the Middleware Remote Abstraction Layer may be characterized as effective as the ability of the media abstraction layer underlying platform event delivery and retrieval system, i.e., it provides the underlying hardware platform and the user interface in the remote control it is necessary to convert between logical. 由中间件远程提取层处理的调用(call)例如包括确定由于用户交互(例如调到新的电视频道)需要对媒体播放设备状态作出的变化的信息。 Including, for example determined by the Middleware Remote Abstraction Layer processing of calls (call) result of user interaction (such as transferred to the new TV channel) requires information on changes in state media playback device made. 例如,当用户按压其遥控上的“上一个频道”时,这产生由运行在远程服务器上的用户接口壳使用的远程按键按压事件。 For example, when the user presses its "on a channel" on the remote control, which produces a remote key press event by using the user interface shell running on the remote server. 在用户接口壳中的逻辑然后确定在指南中下一个频道是什么并向用户呈现对应的细节。 Logic in the user interface shell and then determine the next channel in the guide is presented to the user what the corresponding details. 一旦选择那个频道,用户接口可通过中间件调用套接字发送具有命令的消息以开始选择频道的播放。 Once selected the channel, the user interface can send a message with the command of the socket by calling the middleware to start playing selected channel. 中间件远程提取层因此仅需要提供可以以相同或类似方式在任何硬件上呈现的通用接口,从而揭示该平台的媒体播放能力。 Middleware Remote Abstraction Layer thus only needs to provide a generic interface may be presented in the same or similar manner on any hardware, the media player to reveal the capabilities of the platform. 从中间件到用户接口壳的通知事件可包括任何种类的硬件或播放状态更新。 Event notification from the middleware to the user interface shell may include any sort of hardware or playback status updates. 这种事件的示例包括当前片段(clip)已结束的通知、当前片段的时间位置、错误状态信息或涉及目标平台的硬件能力或媒体的动态状态的任何事。 Examples of such events include notification of the end of the current clip (clip), anything dynamic state of the time position of the current segment, error status information or involve the target platform hardware capabilities or the media.

[0038]根据实施例的特定集合,由中间件远程提取层提供的提取机制采用一组本地端口,并且通过这些端口可向本地独立的软件提供终点服务。 [0038] According to a particular set of embodiments, the extraction mechanism provided by the Middleware Remote Abstraction Layer employs a set of local ports, and may provide end point services to local independent software through these ports. 根据一个这种实施例,中间件远程提取层分别维持两个与中间件回调本地套接字相连接的Unix套接字和中间件事件传递套接字。 According to one such embodiment, the Middleware Remote Abstraction Layer maintains two respectively intermediate and Unix sockets callback local socket and the middleware event delivery socket connected. 中间件远程提取层还维持两个网络端口,使得远程服务能够通过远程协议访问该系统,就像它们是本地程序一样。 Middleware Remote Abstraction Layer also maintains two network ports, making the remote service can access the system via a remote protocol, as if they were local program. 根据一些实施例,中间件远程提取层还可以被用于促进机顶盒的遥控,例如用于测试或调试目的。 According to some embodiments, the Middleware Remote Abstraction Layer may also be used to facilitate the set top box remote control, for example, for testing or debugging purposes.

[0039]应当理解,中间件远程提取层不是根据本发明的虚拟用户接口的所有实现的必需部件。 [0039] It should be understood that all the necessary components to achieve the Middleware Remote Abstraction Layer is not in accordance with the present invention, virtual user interface. 即,如上所述,中间件远程提取层被提供为在其中原始设计运行在机顶盒上的大部分软件功能被移动至后端服务器的解决方案的一部分。 That is, as described above, Middleware Remote Abstraction Layer is provided as part of the original design in which the solution runs most of the software functions on the set top box is moved to the back-end server. 倘若该提取层允许这种解决方案,而不是必须修改在机顶盒上现有的中间件。 If the abstraction layer allows such a solution, rather than having to modify the existing middleware on the set top box. 然而,根据本发明的各种实施例呈现其他类型的虚拟用户接口(取决于它们的功能)可以不需要这种居间代理(例如不需要控制媒体播放的应用)或者不应当具有这种控制能力。 However, in accordance with various embodiments of the present invention exhibit other types of virtual user interfaces (depending on their functions) may not require such an intervening proxy (e.g., media playback applications need not be controlled) or should not have such control capabilities.

[0040]更一般地,设想了本发明的实施例,其中运行在虚拟UI服务器上的用户接口壳可用任何被配置为通过X Windows服务器渲染其自身到帧缓冲器的任何应用来代替。 [0040] More generally, embodiments of the invention are contemplated in which the user interface shell operating on the virtual UI server using any itself is configured to render to a frame buffer by any application server instead of X Windows. 根据实施例的特殊分类,这种应用优选地不包括过多的任何视频内容,因为如果通过VNC连接传送,在远程客户端上视频的渲染可能不完全满足要求。 According to a particular class of embodiments, such applications preferably do not include any excessive video content, because if the transmission is connected via the VNC, the video is rendered on a remote client might not be entirely satisfactory. 即,正确渲染视频所需的帧速率一般地将不匹配用VNC类似方法进行的增量更新的速率,从而导致各种不想要的视频假象(video artifact),诸如“破裂”或者部分清染的视频帧。 That is, the frame rate required to render correctly the video will not generally match the rate of increment is updated by VNC similar manner, resulting in various undesirable video artifacts (video artifact), such as a "burst" or partially dyed clear video frame. 用于传送这种视频内容的适当选择是于2009年5月27日申请的、美国专利申请号为N0.1 2/473086的“虚拟桌面服务(VIRTUALDESKTOP SERVICES)”(代理卷号为STSNP012),为了所有目的,其全部公开内容通过引用结合于此。 Appropriate choice for delivering this video content is on May 27, 2009 application, US Patent Application No. N0.1 2/473086 "virtual desktop services (VIRTUALDESKTOP SERVICES)" (Attorney Docket No. STSNP012), for all purposes, the entire disclosure of which is incorporated herein by reference.

[0041]鉴于前述内容,设想了实施例,其中根据本发明生成的虚拟用户接口包括音频内容。 [0041] In view of the foregoing, embodiments are contemplated, wherein the virtual user interface of the present invention comprises generating audio content. 根据一些这种实施例,除了上述对接口的图形更新外,与接口相关联的音频流也被发送给客户端。 According to some such embodiments, in addition to the graphical updates of the interface to the outside, an interface associated with the audio stream is also sent to the client. 这可以例如使用VNC协议的适当扩展来实现。 This may be achieved for example using an appropriate extension of the VNC protocol. 可替换地,辅助音频信道也可由单独的协议来提供(例如,脉冲音频)。 Alternatively, a secondary audio channel may also be provided by a separate protocol (e.g. Pulse Audio).

[0042]根据本发明的一些实施例远程运行任意应用的能力有效地将嵌入式客户端设备转换为能够享有更广得多的范围的软件应用的第三方开发的开放式平台。 An open platform for third-party developers [0042] effectively convert the embedded client device capabilities in accordance with some embodiments remotely run any application embodiment of the present invention is able to enjoy a much broader range of software applications. 即,与电缆或卫星电视系统或者与其他专有数字电视系统相关联的机顶盒的嵌入式、专有计算环境的本性呈现出对第三方开发者开发用于这种平台的应用的障碍。 That is, the set-top box with embedded cable or satellite TV system or with other proprietary digital television system associated with the proprietary nature of computing environment presents obstacles for third-party developers to develop applications for this platform. 相反地,本发明的实施例能够通过在更多个标准服务器平台上实现任意应用的远程操作和传输来创建这种机会,从而使传统地仅在桌面计算系统上可用的应用在嵌入式计算平台上可用。 Rather, embodiments of the present invention can be to create such opportunities by remote control and transmission of arbitrary applications on more standard server platforms, so that conventionally available applications on a desktop computing platform, a computing system embedded only on available.

[0043]本发明的实施例可用于将广范围的功能性(例如光栅化(rasterizat1n)、字体清染、图形处理等)移动到后端服务。 [0043] Embodiments of the invention may be used for a wide range of functionality (e.g., rasterization (rasterizat1n), dyeing clear font, graphics, etc.) to a back-end service. 客户端设备的任务因此可显著地减少,例如转发用户输入、渲染提供的视觉更新和向接口揭示API的媒体播放能力。 Thus the task of the client device may be significantly reduced, for example, forwarding user input, rendering provided visual updates, and the media playback capability to reveal the API interface. 任何给定应用的响应性、渲染能力和逻辑能力可因此变得设备中立。 Responsive any given application, rendering capabilities, and logic capabilities apparatus can thus become neutral. 通过除去对图形渲染系统的依赖并使用具有小资源基础的瘦客户端来代替它,移植到(porting to)其他平台的依赖也可得以消除。 By removing dependency on a graphics rendering system having a small and thin client resource base to replace it, the transplant dependent (porting to) other platforms may also be eliminated. 该瘦客户端可被配置为不需要支持库,并且仅需要最小标准特征,例如网络套接字和要更新的帧缓冲器。 The thin client can be configured so that no support libraries, and only requires a minimum standard features, e.g., network sockets, and a frame buffer to update. 由于服务器结构(例如x86平台)典型地将运行具有所有必需库的充分突出的(featured)操作系统来运行任何新式的软件,因此中立于目标平台的新的软件特征和部件也可以快速和容易地得以集成。 Since the server architecture (e.g., an x86 platform) will typically run with all required libraries sufficiently protrude (featured) operating system to run any modern software, new software features and components remain the target platform may also be quickly and easily It is integrated. 因此,设想本发明的实施例完全消除了必须移植、为其提供库或调试几乎任何人们可能希望在几乎任何客户端设备上提供给终端用户的软件,消除了包括专有机顶盒环境等。 Accordingly, the present contemplated embodiment of the invention completely eliminates the need to transfer, provide almost any library or debug it may be desirable to provide the software to the end user on virtually any client device, including proprietary set top box environments eliminates the like.

[0044]虽然本发明参考其特定实施例得以特别地展示和描述,但本领域的技术人员将理解可对公开的实施例的形式和细节作出改变,而不脱离本发明的精神或范围。 [0044] While the present invention with reference to particular specific embodiments shown and described, those skilled in the art will appreciate that changes may be made in form and detail to the disclosed embodiments without departing from the spirit or scope of the invention. 并且,尽管参考各个实施例在此讨论了本发明的各个优点、方面和目的,将理解本发明的范围不应当由对这些优点、方面和目的的引用限制。 Also, although the embodiments discussed herein with reference to various embodiments of the present invention, various advantages, aspects, and objects will be appreciated that the scope of the invention should not be, aspects and objects referenced by the limit these advantages. 更确切地,本发明的范围应当参考所附权利要求得以确定。 Rather, the scope of the invention should be determined with reference to the appended claims.

Claims (9)

1.一种用于在网络上提供虚拟用户接口的系统,包括: 一个或多个服务器,其被部署在网络上,并被配置为管理多个虚拟帧缓冲器和多个用户接口应用的操作,所述一个或多个服务器进一步被配置为将所述用户接口应用的每一个与对应的其中一个虚拟帧缓冲器相关联,每个用户接口应用被配置为将对应的图形用户接口渲染到所述对应的虚拟帧缓冲器,所述一个或多个服务器进一步被配置为通过网络与多个客户端的每一个建立第一专用、持久、双向的连接以实施虚拟用户接口会话,将所述虚拟帧缓冲器之一分配给所述客户端的每一个,并且通过所述第一连接将来自每个虚拟帧缓冲器的所述图形用户接口的更新转发给所述对应的客户端,所述更新中的至少一些包括用于在所述对应的客户端处混合所述图形用户接口和附加视觉信息的透明信息,所述一个或 CLAIMS 1. A system for providing virtual user interfaces over a network, comprising: one or more servers deployed on the network and configured to manage operation of a plurality of virtual frame buffers and a plurality of user interface applications said one or more servers are further configured to corresponding to each one of the user interface application where a virtual frame buffer associated with each user interface application is configured to render a corresponding graphical user interface to the said corresponding virtual frame buffer, the one or more servers are further configured to establish a first private network through each of the plurality of clients, persistent, bi-directional connection to conduct a virtual user interface session, the virtual frame said one of the buffers is assigned to each client, and through the first connection from each virtual frame buffer to update the graphical user interface to be forwarded to the corresponding client, the update of mixing at least some of which include transparency information for the graphical user interface and the additional visual information at the client corresponding to the one or 多个服务器进一步被配置为通过所述第一连接从每个客户端接收用户输入,并将所述用户输入提供给所述对应的用户接口应用,所述用户输入代表和每个客户端相关联的用户与所述对应的图形用户接口的表示的交互;和具有关联显示器的所述多个客户端,每个客户端被配置为通过所述第一连接从所述对应的虚拟帧缓冲器接收图形用户接口更新,并使用所述更新和所述透明信息在所述关联显示器上渲染具有所述附加视觉信息的所述对应的图形用户接口的表示,每个客户端进一步被配置为通过所述第一连接将所述用户输入发送给所述一个或多个服务器,其中所述对应的虚拟帧缓冲器与多个服务器实例中的一个服务器实例相关联,以及在所述虚拟用户接口会话完成时回收所关联的服务器实例,以及其中所述客户端中的一个或多个进一步被配置为使用中间件 A plurality of server is further configured to receive user input from each client via the first connection and provide the user input to the corresponding user interface application, the user input on behalf of each client and the associated corresponding to user interaction with the graphical user interface representation; and said plurality of clients with associated displays, each client being configured to receive from the corresponding virtual frame buffer via the first connection a graphical user interface update, and using the graphical user corresponding to the updated information and the rendering transparent with the additional visual information on the associated display representation of the interface, each client is further configured by said when the user input a first connection to transmit the one or more servers, wherein the virtual frame buffer corresponding to the plurality of servers in a server instance associated with the instance, and is completed in the virtual user interface session examples of the associated recovery server, and wherein the one or more of the clients is further configured to use the middleware 提供媒体播放控制功能,并且其中所述客户端中的所述一个或多个包括被配置为通过所述网络上的第二连接促进在所述对应的用户接口应用和所述中间件之间的通信的中间件提取层。 To provide media playback control functions, and wherein said client comprises one or more configured to facilitate the connection through the second network between the corresponding user interface application and the middleware communication middleware abstraction layer.
2.根据权利要求1所述的系统,其中所述客户端中的一个或多个在被配置为在所述关联显示器上提供交互电视业务的机顶盒中被实现。 2. The system according to claim 1, wherein said client is implemented in one or more configured to provide interactive television services on the associated display of the set top box.
3.根据权利要求2所述的系统,其中所述图形用户接口包括用于导航所述交互电视业务的交互菜单。 3. System according to claim 2, wherein said graphical user interface comprises interactive menu for navigating the interactive television services.
4.根据权利要求1所述的系统,其中所述客户端中的一个或多个被集成在具有所述关联显示器的单一设备中。 4. The system of claim 1, wherein said client is a single or a plurality of devices of the integrated associated display.
5.根据权利要求1所述的系统,其中所述第一连接包括虚拟网络计算(VNC)连接,并且其中所述一个或多个服务器被配置为响应于从所述对应的客户端接收的伪编码消息在所述更新的至少一些中包括所述透明信息。 5. The system according to claim 1, wherein said first connection comprises a virtual network computing (VNC) connection, and wherein said one or more servers are configured to respond to a received client from the corresponding dummy encoding information message including the transparent in at least some of the updates.
6.根据权利要求1所述的系统,所述一个或多个服务器进一步被配置为响应于来自所述客户端的对应的连接请求通过网络与所述客户端的每一个建立第二连接,并且响应于所述第二连接的终止,终止所述对应的虚拟接口会话。 6. The system according to claim 1, wherein the one or more servers are further configured to respond to the corresponding connection from the client a second request to establish a connection with each of the network client, and in response to the second connection is terminated, terminating the corresponding virtual interface session.
7.根据权利要求1所述的系统,其中每个更新仅包括自从在前更新后已改变的所述对应的图形用户接口的一个或多个部分。 7. The system according to claim 1, wherein each update comprises only one or more portions of the first update since the change of the corresponding graphic user interface.
8.一种提供虚拟用户接口的方法,包括: 管理多个虚拟帧缓冲器的操作; 响应于来自远程客户端的连接请求,将用户接口应用与分配的其中一个虚拟帧缓冲器相关联,所述用户接口应用被配置为将图形用户接口渲染到所述分配的虚拟帧缓冲器; 建立服务器和所述远程客户端之间的第一专用、持久、双向的连接以实施虚拟用户接Π会话; 通过所述第一连接将所述图形用户接口的更新从所述分配的虚拟帧缓冲器转发给所述远程客户端,所述更新中的至少一些包括用于在所述远程客户端处混合所述图形用户接口和附加视觉信息的透明信息;和通过所述第一连接从所述远程客户端接收用户输入,并将所述用户输入提供给所述用户接口应用,所述用户输入代表和所述远程客户端相关联的用户与所述图形用户接口的表示的交互, 其中所述分配的虚拟帧缓冲器与多 A method for providing a virtual user interface, comprising: operating a plurality of virtual frame buffer management; in response to a connection request from a remote client, the user interface application and assigned one of the virtual frame buffers, the user interface application is configured to render a graphical user interface to the allocated virtual frame buffer; established between a first dedicated server and the remote client, persistent, bi-directional connection to conduct a virtual user session access Π; by the first connection updating the graphical user interface are forwarded from the allocated virtual frame buffer to the remote client, said update comprising at least some of the mixing for at the remote client at graphical user interface transparency information and the additional visual information; and a receiver through the first connection from the remote client user input, and providing the user input to the user interface application, the user input and the representatives represents the remote client user interaction associated with the graphical user interface, wherein the allocated virtual frame buffers with multiple 服务器实例中的一个服务器实例相关联,以及在所述虚拟用户接口会话完成时回收所关联的服务器实例,以及其中所述远程客户端进一步被配置为使用中间件提供媒体播放控制功能,并且其中所述远程客户端包括被配置为通过网络上的第二连接促进在对应的用户接口应用和所述中间件之间的通信的中间件提取层。 Server instance associated with the server instance associated with instances of a server, and recovering when the virtual user interface session is completed, and wherein the remote client is further configured to provide media playback control functions using middleware, and wherein said remote client includes a middleware abstraction layer configured to facilitate communication between the corresponding user interface application and the middleware via a second connection over the network.
9.一种用于与虚拟用户接口交互的方法,包括: 向远程服务器发送连接请求,所述服务器被配置为管理多个虚拟帧缓冲器的操作,并且响应于所述连接请求,将用户接口应用与分配的其中一个虚拟帧缓冲器相关联,所述用户接口应用被配置为将图形用户接口渲染到所述分配的虚拟帧缓冲器; 建立客户端和所述服务器之间的第一专用、持久、双向的连接以实施虚拟用户接口会话; 通过所述第一连接从所述分配的虚拟帧缓冲器接收所述图形用户接口的更新,所述更新中的至少一些包括用于混合所述图形用户接口和附加视觉信息的透明信息; 使用所述更新和所述透明信息在显示器上渲染结合所述附加视觉信息的所述图形用户接口的表示;和通过所述第一连接向所述服务器转发用户输入以呈现给所述用户接口应用,所述用户输入代表用户与所述图形用户接 A method for virtual interaction with a user interface, comprising: transmitting a connection request to a remote server, the server is configured to manage operation of a plurality of virtual frame buffers and, in response to the connection request, the user interface wherein the application and distribution of a virtual frame buffers, the user interface application is configured to render a graphical user interface to the allocated virtual frame buffer; establishing a first dedicated between the client and the server, persistent, bi-directional connection to conduct a virtual user interface session; receiving updates the graphical user interface from the allocated virtual frame buffer via the first connection, at least some of said updating comprises mixing the pattern for user interface transparency information and the additional visual information; using the updated rendering information and the transparency in combination with the additional visual information on a display of the graphical user interface represented; and forwarding to the server via the first connection presented to the user input to a user interface application, the user input representative of a user with the graphical user access 的表示的交互, 其中所述分配的虚拟帧缓冲器与多个服务器实例中的一个服务器实例相关联,以及在所述虚拟用户接口会话完成时回收所关联的服务器实例,以及其中所述客户端进一步被配置为使用中间件提供媒体播放控制功能,并且其中所述客户端包括被配置为通过网络上的第二连接促进在对应的用户接口应用和所述中间件之间的通信的中间件提取层。 Interaction representation, wherein the server instance associated with the allocated virtual frame buffer to the plurality of servers in a server instance associated with the instance, and when the recovered virtual user interface session is completed, and wherein the client It is further configured to provide media playback control functions using middleware, and wherein said client comprises a middleware is configured to facilitate communication between the extraction in the corresponding user interface application and the middleware via a second connection over the network Floor.
CN201110041831.XA 2010-01-15 2011-01-14 Virtual User Interface CN102158474B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US12/688,343 US9229734B2 (en) 2010-01-15 2010-01-15 Hospitality media system employing virtual user interfaces
US12/688343 2010-01-15

Publications (2)

Publication Number Publication Date
CN102158474A CN102158474A (en) 2011-08-17
CN102158474B true CN102158474B (en) 2016-12-14

Family

ID=

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999017549A1 (en) * 1997-09-26 1999-04-08 Sun Microsystems, Inc. Interactive graphical user interface for television set-top box
US5926647A (en) * 1996-10-11 1999-07-20 Divicom Inc. Processing system with dynamic alteration of a color look-up table
CN101568914A (en) * 2007-06-12 2009-10-28 梁庚荣 Two-tier architecture for remote access service

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5926647A (en) * 1996-10-11 1999-07-20 Divicom Inc. Processing system with dynamic alteration of a color look-up table
WO1999017549A1 (en) * 1997-09-26 1999-04-08 Sun Microsystems, Inc. Interactive graphical user interface for television set-top box
CN101568914A (en) * 2007-06-12 2009-10-28 梁庚荣 Two-tier architecture for remote access service

Similar Documents

Publication Publication Date Title
KR101503386B1 (en) Negotiating capabilities between a wireless sink and a wireless source device
US8122475B2 (en) Remote control for video media servers
CN1310487C (en) Method and apparatus for providing virtual desktop system architecture
CN102203760B (en) Remote computing platforms providing high-fidelity display and interactivity for clients
US7173635B2 (en) Remote graphical user interface support using a graphics processing unit
CN103403695B (en) A method for using a remote desktop client facilitate a common interface to the system and remote desktop session
CN101981558B (en) Systems and methods for managing multimedia operations in remote sessions
CN101263485B (en) Remoting redirection layer for graphics device interface
CN100418055C (en) Method and system for sharing user interface window small parts
US20120173755A1 (en) Multi-User Display Proxy Server
US7590750B2 (en) Systems and methods for multimedia remoting over terminal server connections
US10108386B2 (en) Content provisioning for wireless back channel
US7076735B2 (en) System and method for network transmission of graphical data through a distributed application
US8799362B2 (en) Scalable high-performance interactive real-time media architectures for virtual desktop environments
CN101401087B (en) Efficient encoding of alternative graphic sets
US9878240B2 (en) Add-on management methods
CN1308895C (en) Systems and methods for generating visual representations of graphical data and digital document processing
US8606952B2 (en) Method and system for optimizing bandwidth usage in remote visualization
KR101495464B1 (en) User input device for wireless back channel
CN100585550C (en) Systems and methods for projecting content from computing devices
Jurgelionis et al. Platform for distributed 3D gaming
JP4959706B2 (en) Portable remote device having an expandable feature set
CN101351785B (en) Multi-user terminal services accelerator
US7716273B2 (en) Systems and methods for projecting content from computing devices
US7614075B2 (en) Dynamically generating video streams for user interfaces